-
Notifications
You must be signed in to change notification settings - Fork 7.8k
Show citations at the end of each turn #7350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @scidomino, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request implements the functionality to display citations at the end of each turn in the Gemini CLI. This enhancement improves the user experience by providing clear source attribution for generated content, thereby increasing transparency and trustworthiness of the information provided.
Highlights
- Citation Event Handling: A new
GeminiEventType.Citation
has been introduced to specifically handle and display citation information originating from the model's responses. - CLI Output Integration: The non-interactive command-line interface has been updated to output citation events directly to
stdout
, ensuring that source attributions are visible during command-line usage. - UI Stream Processing: The UI's Gemini stream hook now processes and renders citation events as informative messages, providing a clear and user-friendly visual representation of the sources.
- Core Logic for Citation Extraction: Robust logic has been implemented within the core
Turn
class to extract, format, and aggregate citation metadata from Gemini API responses. This ensures that only valid citations with URIs are displayed and presented in a sorted, readable format. - Comprehensive Testing: New unit tests have been added to comprehensively validate the correct behavior of citation extraction and event generation across various scenarios, including single, multiple, and invalid citations.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command>
or @gemini-code-assist <command>
. Below is a summary of the supported commands.
Feature | Command | Description |
---|---|---|
Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/
folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces functionality to display citations at the end of each turn. The changes span across the core logic, the interactive UI, and the non-interactive CLI to handle and display citation events. New tests have been added to verify the citation handling logic.
My main feedback is regarding code duplication. A getCitations
helper function has been added in two different files with different implementations. This should be consolidated into a single function in the utility file to improve maintainability and avoid confusion.
09d905a
to
0eb703c
Compare
Code Coverage Summary
CLI Package - Full Text Report
Core Package - Full Text Report
For detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
0eb703c
to
4dfa074
Compare
4dfa074
to
f68b4fc
Compare
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a new feature to display citations for generated text at the end of each turn. The feature is controlled by a new showCitations
setting, which is disabled by default. The implementation is well-structured, with changes spanning from the core logic in @google/gemini-cli-core
to the UI hooks and components in @google/gemini-cli
. The core Turn
class now collects citations and emits a dedicated Citation
event upon completion. The useGeminiStream
hook handles this new event, displaying the citations as an info message if the feature is enabled. The addition of comprehensive unit tests for the new citation logic in turn.test.ts
is a great inclusion and covers important edge cases. Overall, the changes are solid and well-implemented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be clear. Supporting collapsing citations by default can be a follow up pr.
Approved after the other hopefully minor comments are discussed
@@ -45,6 +45,7 @@ const MIGRATION_MAP: Record<string, string> = { | |||
hideFooter: 'ui.hideFooter', | |||
showMemoryUsage: 'ui.showMemoryUsage', | |||
showLineNumbers: 'ui.showLineNumbers', | |||
showCitations: 'ui.showCitations', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this can be omitted from the migration map as you are post migration. @galz10 can verify that this doesn't break things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I spoke with Gal. This is needed for it to be editable int he UI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fyi @galz10 can we fix this? Would prefer we didn't have to add things here just to make them editable in the UI.
if (!settings?.merged?.ui?.showCitations) { | ||
return; | ||
} | ||
if (pendingHistoryItemRef.current) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the case finishing the pending history item when there is a citation makes me a bit nervous... partly because of how restricted the static rendering is. possibly bad idea is if we can we instead integrate citations with the appropriate pending history item.
Reasons: might have wacky race conditions if the pending history item is still being added to or if there is a newer item if the query to get the citation was slow. we'll have to leverage the refreshStatic hack to make this work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only return citations at the end of a turn. Never in the middle. So in practice I think this would never happen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested this with function calls and it works as expected. |
TLDR
Show citations at the end of each turn. Disabled by default (with
showCitations
option insettings.json
).Deeper Dive
We want this to be enabled by default for OAuth users that aren't on free tier. That will be done in a follow up PR.
Reviewer Test Plan
Add the following to your
settings.json
:"showCitations": true,
Then ask:
Without looking it up Recite a the frog prince fairy tale
It doesn't always work on the first try. You may have to ask for 4 or 5 fairy tales to get some citations.
Testing Matrix
Linked issues / bugs
Contributes: #6260