Skip to content

Commit b1badea

Browse files
dlarocquedlarocque
committed
feat(ai): add support for grounding with google search
1 parent 40be2db commit b1badea

16 files changed

+562
-20
lines changed

‎.changeset/five-kids-grow.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'firebase': minor
3+
'@firebase/ai': minor
4+
---
5+
6+
Add support for Grounding with Google Search.

‎common/api-review/ai.api.md

Lines changed: 40 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -485,6 +485,15 @@ export interface GoogleAIGenerateContentResponse {
485485
usageMetadata?: UsageMetadata;
486486
}
487487

488+
// @public
489+
export interface GoogleSearch {
490+
}
491+
492+
// @public
493+
export interface GoogleSearchTool {
494+
googleSearch: GoogleSearch;
495+
}
496+
488497
// @public @deprecated (undocumented)
489498
export interface GroundingAttribution {
490499
// (undocumented)
@@ -497,16 +506,30 @@ export interface GroundingAttribution {
497506
web?: WebAttribution;
498507
}
499508

509+
// @public
510+
export interface GroundingChunk {
511+
web?: WebGroundingChunk;
512+
}
513+
500514
// @public
501515
export interface GroundingMetadata {
502516
// @deprecated (undocumented)
503517
groundingAttributions: GroundingAttribution[];
504-
// (undocumented)
518+
groundingChunks?: GroundingChunk[];
519+
groundingSupports?: GroundingSupport[];
520+
// @deprecated (undocumented)
505521
retrievalQueries?: string[];
506-
// (undocumented)
522+
searchEntryPoint?: SearchEntrypoint;
507523
webSearchQueries?: string[];
508524
}
509525

526+
// @public
527+
export interface GroundingSupport {
528+
confidenceScores?: number[];
529+
groundingChunkIndices?: number[];
530+
segment?: Segment;
531+
}
532+
510533
// @public
511534
export enum HarmBlockMethod {
512535
PROBABILITY = "PROBABILITY",
@@ -852,14 +875,17 @@ export enum SchemaType {
852875
STRING = "string"
853876
}
854877

855-
// @public (undocumented)
878+
// @public
879+
export interface SearchEntrypoint {
880+
renderedContent: string;
881+
}
882+
883+
// @public
856884
export interface Segment {
857-
// (undocumented)
858885
endIndex: number;
859-
// (undocumented)
860886
partIndex: number;
861-
// (undocumented)
862887
startIndex: number;
888+
text: string;
863889
}
864890

865891
// @public
@@ -896,7 +922,7 @@ export interface TextPart {
896922
}
897923

898924
// @public
899-
export type Tool = FunctionDeclarationsTool;
925+
export type Tool = FunctionDeclarationsTool | GoogleSearchTool;
900926

901927
// @public
902928
export interface ToolConfig {
@@ -956,5 +982,12 @@ export interface WebAttribution {
956982
uri: string;
957983
}
958984

985+
// @public
986+
export interface WebGroundingChunk {
987+
domain?: string;
988+
title?: string;
989+
uri?: string;
990+
}
991+
959992

960993
```

‎docs-devsite/_toc.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,18 @@ toc:
7676
path: /docs/reference/js/ai.generativemodel.md
7777
- title: GoogleAIBackend
7878
path: /docs/reference/js/ai.googleaibackend.md
79+
- title: GoogleSearch
80+
path: /docs/reference/js/ai.googlesearch.md
81+
- title: GoogleSearchTool
82+
path: /docs/reference/js/ai.googlesearchtool.md
7983
- title: GroundingAttribution
8084
path: /docs/reference/js/ai.groundingattribution.md
85+
- title: GroundingChunk
86+
path: /docs/reference/js/ai.groundingchunk.md
8187
- title: GroundingMetadata
8288
path: /docs/reference/js/ai.groundingmetadata.md
89+
- title: GroundingSupport
90+
path: /docs/reference/js/ai.groundingsupport.md
8391
- title: ImagenGCSImage
8492
path: /docs/reference/js/ai.imagengcsimage.md
8593
- title: ImagenGenerationConfig
@@ -130,6 +138,8 @@ toc:
130138
path: /docs/reference/js/ai.schemarequest.md
131139
- title: SchemaShared
132140
path: /docs/reference/js/ai.schemashared.md
141+
- title: SearchEntrypoint
142+
path: /docs/reference/js/ai.searchentrypoint.md
133143
- title: Segment
134144
path: /docs/reference/js/ai.segment.md
135145
- title: StartChatParams
@@ -150,6 +160,8 @@ toc:
150160
path: /docs/reference/js/ai.videometadata.md
151161
- title: WebAttribution
152162
path: /docs/reference/js/ai.webattribution.md
163+
- title: WebGroundingChunk
164+
path: /docs/reference/js/ai.webgroundingchunk.md
153165
- title: analytics
154166
path: /docs/reference/js/analytics.md
155167
section:

‎docs-devsite/ai.googlesearch.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GoogleSearch interface
13+
Configuration for the [GoogleSearchTool](./ai.googlesearchtool.md#googlesearchtool_interface)<!-- -->.
14+
15+
Currently, this interface is empty and serves as a placeholder for future configuration options.
16+
17+
<b>Signature:</b>
18+
19+
```typescript
20+
export declare interface GoogleSearch
21+
```

‎docs-devsite/ai.googlesearchtool.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GoogleSearchTool interface
13+
A tool that allows the generative model to connect to Google Search to access and incorporate up-to-date information from the web into its responses.
14+
15+
When this tool is used, the model's responses may include "Grounded Results" which are subject to the Grounding with Google Search terms outlined in the [Service Specific Terms](https://cloud.google.com/terms/service-terms)<!-- -->.
16+
17+
<b>Signature:</b>
18+
19+
```typescript
20+
export declare interface GoogleSearchTool
21+
```
22+
23+
## Properties
24+
25+
| Property | Type | Description |
26+
| --- | --- | --- |
27+
| [googleSearch](./ai.googlesearchtool.md#googlesearchtoolgooglesearch) | [GoogleSearch](./ai.googlesearch.md#googlesearch_interface) | Specifies the Google Search configuration. Currently, this is an empty object, but it's reserved for future configuration options. |
28+
29+
## GoogleSearchTool.googleSearch
30+
31+
Specifies the Google Search configuration. Currently, this is an empty object, but it's reserved for future configuration options.
32+
33+
<b>Signature:</b>
34+
35+
```typescript
36+
googleSearch: GoogleSearch;
37+
```

‎docs-devsite/ai.groundingchunk.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GroundingChunk interface
13+
Represents a chunk of retrieved data that supports a claim in the model's response. This is part of the grounding information provided when grounding is enabled.
14+
15+
<b>Signature:</b>
16+
17+
```typescript
18+
export interface GroundingChunk
19+
```
20+
21+
## Properties
22+
23+
| Property | Type | Description |
24+
| --- | --- | --- |
25+
| [web](./ai.groundingchunk.md#groundingchunkweb) | [WebGroundingChunk](./ai.webgroundingchunk.md#webgroundingchunk_interface) | Contains details if the grounding chunk is from a web source. |
26+
27+
## GroundingChunk.web
28+
29+
Contains details if the grounding chunk is from a web source.
30+
31+
<b>Signature:</b>
32+
33+
```typescript
34+
web?: WebGroundingChunk;
35+
```

‎docs-devsite/ai.groundingmetadata.md

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,11 @@ export interface GroundingMetadata
2323
| Property | Type | Description |
2424
| --- | --- | --- |
2525
| [groundingAttributions](./ai.groundingmetadata.md#groundingmetadatagroundingattributions) | [GroundingAttribution](./ai.groundingattribution.md#groundingattribution_interface)<!-- -->\[\] | |
26+
| [groundingChunks](./ai.groundingmetadata.md#groundingmetadatagroundingchunks) | [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface)<!-- -->\[\] | A list of [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects. Each chunk represents a piece of retrieved content (e.g. from a web page). that the model used to ground its response. |
27+
| [groundingSupports](./ai.groundingmetadata.md#groundingmetadatagroundingsupports) | [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface)<!-- -->\[\] | A list of [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface) objects. Each object details how specific segments of the model's response are supported by the <code>groundingChunks</code>. |
2628
| [retrievalQueries](./ai.groundingmetadata.md#groundingmetadataretrievalqueries) | string\[\] | |
27-
| [webSearchQueries](./ai.groundingmetadata.md#groundingmetadatawebsearchqueries) | string\[\] | |
29+
| [searchEntryPoint](./ai.groundingmetadata.md#groundingmetadatasearchentrypoint) | [SearchEntrypoint](./ai.searchentrypoint.md#searchentrypoint_interface) | Google search entry for the following web searches. An HTML/CSS snippet that can be embedded in a web page to display a Google Search Entry point for follow-up web searches related to the model's response. |
30+
| [webSearchQueries](./ai.groundingmetadata.md#groundingmetadatawebsearchqueries) | string\[\] | A list of web search queries that the model performed to gather the grounding information. These can be used to allow users to explore the search results themselves. |
2831

2932
## GroundingMetadata.groundingAttributions
3033

@@ -38,16 +41,53 @@ export interface GroundingMetadata
3841
groundingAttributions: GroundingAttribution[];
3942
```
4043

44+
## GroundingMetadata.groundingChunks
45+
46+
A list of [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects. Each chunk represents a piece of retrieved content (e.g. from a web page). that the model used to ground its response.
47+
48+
<b>Signature:</b>
49+
50+
```typescript
51+
groundingChunks?: GroundingChunk[];
52+
```
53+
54+
## GroundingMetadata.groundingSupports
55+
56+
A list of [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface) objects. Each object details how specific segments of the model's response are supported by the `groundingChunks`<!-- -->.
57+
58+
<b>Signature:</b>
59+
60+
```typescript
61+
groundingSupports?: GroundingSupport[];
62+
```
63+
4164
## GroundingMetadata.retrievalQueries
4265

66+
> Warning: This API is now obsolete.
67+
>
68+
> Use [GroundingMetadata.groundingSupports](./ai.groundingmetadata.md#groundingmetadatagroundingsupports) instead.
69+
>
70+
4371
<b>Signature:</b>
4472

4573
```typescript
4674
retrievalQueries?: string[];
4775
```
4876

77+
## GroundingMetadata.searchEntryPoint
78+
79+
Google search entry for the following web searches. An HTML/CSS snippet that can be embedded in a web page to display a Google Search Entry point for follow-up web searches related to the model's response.
80+
81+
<b>Signature:</b>
82+
83+
```typescript
84+
searchEntryPoint?: SearchEntrypoint;
85+
```
86+
4987
## GroundingMetadata.webSearchQueries
5088

89+
A list of web search queries that the model performed to gather the grounding information. These can be used to allow users to explore the search results themselves.
90+
5191
<b>Signature:</b>
5292

5393
```typescript

‎docs-devsite/ai.groundingsupport.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GroundingSupport interface
13+
Provides information about how a specific segment of the model's response is supported by the retrieved grounding chunks.
14+
15+
<b>Signature:</b>
16+
17+
```typescript
18+
export interface GroundingSupport
19+
```
20+
21+
## Properties
22+
23+
| Property | Type | Description |
24+
| --- | --- | --- |
25+
| [confidenceScores](./ai.groundingsupport.md#groundingsupportconfidencescores) | number\[\] | A list of confidence scores, corresponding to each index in <code>groundingChunkIndices</code>. Each score indicates the model's confidence that the correspondingly indexed grounding chunk supports the claim in the response segment. Scores range from 0.0 to 1.0, where 1.0 is the highest confidence. This list will have the same number of elements as <code>groundingChunkIndices</code>. |
26+
| [groundingChunkIndices](./ai.groundingsupport.md#groundingsupportgroundingchunkindices) | number\[\] | A list of indices that refer to specific [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects within the [GroundingMetadata.groundingChunks](./ai.groundingmetadata.md#groundingmetadatagroundingchunks) array. These referenced chunks are the sources that support the claim made in the associated <code>segment</code> of the response. For example, an array <code>[1, 3, 4]</code> means that <code>groundingChunks[1]</code>, <code>groundingChunks[3]</code>, and <code>groundingChunks[4]</code> are the retrieved content supporting this part of the response. |
27+
| [segment](./ai.groundingsupport.md#groundingsupportsegment) | [Segment](./ai.segment.md#segment_interface) | Specifies the segment of the model's response content that this grounding support pertains to. |
28+
29+
## GroundingSupport.confidenceScores
30+
31+
A list of confidence scores, corresponding to each index in `groundingChunkIndices`<!-- -->. Each score indicates the model's confidence that the correspondingly indexed grounding chunk supports the claim in the response segment. Scores range from 0.0 to 1.0, where 1.0 is the highest confidence. This list will have the same number of elements as `groundingChunkIndices`<!-- -->.
32+
33+
<b>Signature:</b>
34+
35+
```typescript
36+
confidenceScores?: number[];
37+
```
38+
39+
## GroundingSupport.groundingChunkIndices
40+
41+
A list of indices that refer to specific [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects within the [GroundingMetadata.groundingChunks](./ai.groundingmetadata.md#groundingmetadatagroundingchunks) array. These referenced chunks are the sources that support the claim made in the associated `segment` of the response. For example, an array `[1, 3, 4]` means that `groundingChunks[1]`<!-- -->, `groundingChunks[3]`<!-- -->, and `groundingChunks[4]` are the retrieved content supporting this part of the response.
42+
43+
<b>Signature:</b>
44+
45+
```typescript
46+
groundingChunkIndices?: number[];
47+
```
48+
49+
## GroundingSupport.segment
50+
51+
Specifies the segment of the model's response content that this grounding support pertains to.
52+
53+
<b>Signature:</b>
54+
55+
```typescript
56+
segment?: Segment;
57+
```

‎docs-devsite/ai.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,12 @@ The Firebase AI Web SDK.
9595
| [GenerateContentStreamResult](./ai.generatecontentstreamresult.md#generatecontentstreamresult_interface) | Result object returned from [GenerativeModel.generateContentStream()](./ai.generativemodel.md#generativemodelgeneratecontentstream) call. Iterate over <code>stream</code> to get chunks as they come in and/or use the <code>response</code> promise to get the aggregated response when the stream is done. |
9696
| [GenerationConfig](./ai.generationconfig.md#generationconfig_interface) | Config options for content-related requests |
9797
| [GenerativeContentBlob](./ai.generativecontentblob.md#generativecontentblob_interface) | Interface for sending an image. |
98+
| [GoogleSearch](./ai.googlesearch.md#googlesearch_interface) | Configuration for the [GoogleSearchTool](./ai.googlesearchtool.md#googlesearchtool_interface)<!-- -->. |
99+
| [GoogleSearchTool](./ai.googlesearchtool.md#googlesearchtool_interface) | A tool that allows the generative model to connect to Google Search to access and incorporate up-to-date information from the web into its responses.<!-- -->When this tool is used, the model's responses may include "Grounded Results" which are subject to the Grounding with Google Search terms outlined in the [Service Specific Terms](https://cloud.google.com/terms/service-terms)<!-- -->. |
98100
| [GroundingAttribution](./ai.groundingattribution.md#groundingattribution_interface) | |
101+
| [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) | Represents a chunk of retrieved data that supports a claim in the model's response. This is part of the grounding information provided when grounding is enabled. |
99102
| [GroundingMetadata](./ai.groundingmetadata.md#groundingmetadata_interface) | Metadata returned to client when grounding is enabled. |
103+
| [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface) | Provides information about how a specific segment of the model's response is supported by the retrieved grounding chunks. |
100104
| [ImagenGCSImage](./ai.imagengcsimage.md#imagengcsimage_interface) | An image generated by Imagen, stored in a Cloud Storage for Firebase bucket.<!-- -->This feature is not available yet. |
101105
| [ImagenGenerationConfig](./ai.imagengenerationconfig.md#imagengenerationconfig_interface) | <b><i>(Public Preview)</i></b> Configuration options for generating images with Imagen.<!-- -->See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images-imagen) for more details. |
102106
| [ImagenGenerationResponse](./ai.imagengenerationresponse.md#imagengenerationresponse_interface) | <b><i>(Public Preview)</i></b> The response from a request to generate images with Imagen. |
@@ -116,14 +120,16 @@ The Firebase AI Web SDK.
116120
| [SchemaParams](./ai.schemaparams.md#schemaparams_interface) | Params passed to [Schema](./ai.schema.md#schema_class) static methods to create specific [Schema](./ai.schema.md#schema_class) classes. |
117121
| [SchemaRequest](./ai.schemarequest.md#schemarequest_interface) | Final format for [Schema](./ai.schema.md#schema_class) params passed to backend requests. |
118122
| [SchemaShared](./ai.schemashared.md#schemashared_interface) | Basic [Schema](./ai.schema.md#schema_class) properties shared across several Schema-related types. |
119-
| [Segment](./ai.segment.md#segment_interface) | |
123+
| [SearchEntrypoint](./ai.searchentrypoint.md#searchentrypoint_interface) | Google search entry point. |
124+
| [Segment](./ai.segment.md#segment_interface) | Represents a specific segment within a [Content](./ai.content.md#content_interface) object, often used to pinpoint the exact location of text or data that grounding information refers to. |
120125
| [StartChatParams](./ai.startchatparams.md#startchatparams_interface) | Params for [GenerativeModel.startChat()](./ai.generativemodel.md#generativemodelstartchat)<!-- -->. |
121126
| [TextPart](./ai.textpart.md#textpart_interface) | Content part interface if the part represents a text string. |
122127
| [ToolConfig](./ai.toolconfig.md#toolconfig_interface) | Tool config. This config is shared for all tools provided in the request. |
123128
| [UsageMetadata](./ai.usagemetadata.md#usagemetadata_interface) | Usage metadata about a [GenerateContentResponse](./ai.generatecontentresponse.md#generatecontentresponse_interface)<!-- -->. |
124129
| [VertexAIOptions](./ai.vertexaioptions.md#vertexaioptions_interface) | Options when initializing the Firebase AI SDK. |
125130
| [VideoMetadata](./ai.videometadata.md#videometadata_interface) | Describes the input video content. |
126131
| [WebAttribution](./ai.webattribution.md#webattribution_interface) | |
132+
| [WebGroundingChunk](./ai.webgroundingchunk.md#webgroundingchunk_interface) | A grounding chunk from the web. |
127133

128134
## Variables
129135

@@ -400,7 +406,7 @@ Defines a tool that model can call to access external knowledge.
400406
<b>Signature:</b>
401407

402408
```typescript
403-
export declare type Tool = FunctionDeclarationsTool;
409+
export declare type Tool = FunctionDeclarationsTool | GoogleSearchTool;
404410
```
405411

406412
## TypedSchema

0 commit comments

Comments
 (0)