Skip to content
This repository was archived by the owner on Jul 20, 2023. It is now read-only.

Commit 6200e38

Browse files
gcf-owl-bot[bot]sofislgcf-merge-on-green[bot]alexander-fenster
authored
fix: allow passing gax instance to client constructor (#365)
* feat: add UpsertDatapoints and RemoveDatapoints rpcs to IndexService in aiplatform v1beta1 index_service.proto PiperOrigin-RevId: 469481692 Source-Link: googleapis/googleapis@624cc45 Source-Link: https://github.com/googleapis/googleapis-gen/commit/fd6880812cab6b3434a7e1d339a99907698f9508 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZmQ2ODgwODEyY2FiNmIzNDM0YTdlMWQzMzlhOTk5MDc2OThmOTUwOCJ9 * feat: add UpsertDatapoints and RemoveDatapoints rpcs to IndexService in aiplatform v1 index_service.proto PiperOrigin-RevId: 469481982 Source-Link: googleapis/googleapis@e4fe55a Source-Link: https://github.com/googleapis/googleapis-gen/commit/1ff3c40d85fe8f13538b0f2264d60f806d744fd2 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMWZmM2M0MGQ4NWZlOGYxMzUzOGIwZjIyNjRkNjBmODA2ZDc0NGZkMiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add read_mask to ListPipelineJobsRequest in aiplatform v1 pipeline_service feat: add input_artifacts to PipelineJob.runtime_config in aiplatform v1 pipeline_job PiperOrigin-RevId: 469843667 Source-Link: googleapis/googleapis@eb382ed Source-Link: https://github.com/googleapis/googleapis-gen/commit/7af72d71f16060f229b77ae7edc74da5cfb8ee79 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiN2FmNzJkNzFmMTYwNjBmMjI5Yjc3YWU3ZWRjNzRkYTVjZmI4ZWU3OSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: accept google-gax instance as a parameter Please see the documentation of the client constructor for details. PiperOrigin-RevId: 470332808 Source-Link: googleapis/googleapis@d4a2367 Source-Link: https://github.com/googleapis/googleapis-gen/commit/e97a1ac204ead4fe7341f91e72db7c6ac6016341 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZTk3YTFhYzIwNGVhZDRmZTczNDFmOTFlNzJkYjdjNmFjNjAxNjM0MSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add model_monitoring_stats_anomalies,model_monitoring_status to BatchPredictionJob in aiplatform v1beta1 batch_prediction_job.proto PiperOrigin-RevId: 470845818 Source-Link: googleapis/googleapis@284389e Source-Link: https://github.com/googleapis/googleapis-gen/commit/29deb79b04aea169a568e7c544868243d5161dde Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjlkZWI3OWIwNGFlYTE2OWE1NjhlN2M1NDQ4NjgyNDNkNTE2MWRkZSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix: use _gaxModule when accessing gax for bundling PiperOrigin-RevId: 470911839 Source-Link: googleapis/googleapis@3527566 Source-Link: https://github.com/googleapis/googleapis-gen/commit/f16a1d224f00a630ea43d6a9a1a31f566f45cdea Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZjE2YTFkMjI0ZjAwYTYzMGVhNDNkNmE5YTFhMzFmNTY2ZjQ1Y2RlYSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: sofisl <[email protected]> Co-authored-by: gcf-merge-on-green[bot] <60162190+gcf-merge-on-green[bot]@users.noreply.github.com> Co-authored-by: Alexander Fenster <[email protected]>
1 parent c0464ad commit 6200e38

File tree

70 files changed

+22729
-13610
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+22729
-13610
lines changed

protos/google/cloud/aiplatform/v1/dataset_service.proto

+1-2
Original file line numberDiff line numberDiff line change
@@ -399,8 +399,7 @@ message ListSavedQueriesRequest {
399399

400400
// Response message for [DatasetService.ListSavedQueries][google.cloud.aiplatform.v1.DatasetService.ListSavedQueries].
401401
message ListSavedQueriesResponse {
402-
// A list of SavedQueries that matches the specified filter in the
403-
// request.
402+
// A list of SavedQueries that match the specified filter in the request.
404403
repeated SavedQuery saved_queries = 1;
405404

406405
// The standard List next-page token.

protos/google/cloud/aiplatform/v1/explanation.proto

+3-3
Original file line numberDiff line numberDiff line change
@@ -182,8 +182,8 @@ message ExplanationSpec {
182182
// Required. Parameters that configure explaining of the Model's predictions.
183183
ExplanationParameters parameters = 1 [(google.api.field_behavior) = REQUIRED];
184184

185-
// Required. Metadata describing the Model's input and output for explanation.
186-
ExplanationMetadata metadata = 2 [(google.api.field_behavior) = REQUIRED];
185+
// Optional. Metadata describing the Model's input and output for explanation.
186+
ExplanationMetadata metadata = 2 [(google.api.field_behavior) = OPTIONAL];
187187
}
188188

189189
// Parameters to configure explaining for Model's predictions.
@@ -224,7 +224,7 @@ message ExplanationParameters {
224224
// explaining.
225225
//
226226
// If not populated, returns attributions for [top_k][google.cloud.aiplatform.v1.ExplanationParameters.top_k] indices of outputs.
227-
// If neither top_k nor output_indeices is populated, returns the argmax
227+
// If neither top_k nor output_indices is populated, returns the argmax
228228
// index of the outputs.
229229
//
230230
// Only applicable to Models that predict multiple outputs (e,g, multi-class

protos/google/cloud/aiplatform/v1/index.proto

+76
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,21 @@ message Index {
3838
pattern: "projects/{project}/locations/{location}/indexes/{index}"
3939
};
4040

41+
// The update method of an Index.
42+
enum IndexUpdateMethod {
43+
// Should not be used.
44+
INDEX_UPDATE_METHOD_UNSPECIFIED = 0;
45+
46+
// BatchUpdate: user can call UpdateIndex with files on Cloud Storage of
47+
// datapoints to update.
48+
BATCH_UPDATE = 1;
49+
50+
// StreamUpdate: user can call UpsertDatapoints/DeleteDatapoints to update
51+
// the Index and the updates will be applied in corresponding
52+
// DeployedIndexes in nearly real-time.
53+
STREAM_UPDATE = 2;
54+
}
55+
4156
// Output only. The resource name of the Index.
4257
string name = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
4358

@@ -93,4 +108,65 @@ message Index {
93108
// in the Index. Result of any successfully completed Operation on the Index
94109
// is reflected in it.
95110
google.protobuf.Timestamp update_time = 11 [(google.api.field_behavior) = OUTPUT_ONLY];
111+
112+
// Output only. Stats of the index resource.
113+
IndexStats index_stats = 14 [(google.api.field_behavior) = OUTPUT_ONLY];
114+
115+
// Immutable. The update method to use with this Index. If not set, BATCH_UPDATE will be
116+
// used by default.
117+
IndexUpdateMethod index_update_method = 16 [(google.api.field_behavior) = IMMUTABLE];
118+
}
119+
120+
// A datapoint of Index.
121+
message IndexDatapoint {
122+
// Restriction of a datapoint which describe its attributes(tokens) from each
123+
// of several attribute categories(namespaces).
124+
message Restriction {
125+
// The namespace of this restriction. eg: color.
126+
string namespace = 1;
127+
128+
// The attributes to allow in this namespace. eg: 'red'
129+
repeated string allow_list = 2;
130+
131+
// The attributes to deny in this namespace. eg: 'blue'
132+
repeated string deny_list = 3;
133+
}
134+
135+
// Crowding tag is a constraint on a neighbor list produced by nearest
136+
// neighbor search requiring that no more than some value k' of the k
137+
// neighbors returned have the same value of crowding_attribute.
138+
message CrowdingTag {
139+
// The attribute value used for crowding. The maximum number of neighbors
140+
// to return per crowding attribute value
141+
// (per_crowding_attribute_num_neighbors) is configured per-query. This
142+
// field is ignored if per_crowding_attribute_num_neighbors is larger than
143+
// the total number of neighbors to return for a given query.
144+
string crowding_attribute = 1;
145+
}
146+
147+
// Required. Unique identifier of the datapoint.
148+
string datapoint_id = 1 [(google.api.field_behavior) = REQUIRED];
149+
150+
// Required. Feature embedding vector. An array of numbers with the length of
151+
// [NearestNeighborSearchConfig.dimensions].
152+
repeated float feature_vector = 2 [(google.api.field_behavior) = REQUIRED];
153+
154+
// Optional. List of Restrict of the datapoint, used to perform "restricted searches"
155+
// where boolean rule are used to filter the subset of the database eligible
156+
// for matching.
157+
// See: https://cloud.google.com/vertex-ai/docs/matching-engine/filtering
158+
repeated Restriction restricts = 4 [(google.api.field_behavior) = OPTIONAL];
159+
160+
// Optional. CrowdingTag of the datapoint, the number of neighbors to return in each
161+
// crowding can be configured during query.
162+
CrowdingTag crowding_tag = 5 [(google.api.field_behavior) = OPTIONAL];
163+
}
164+
165+
// Stats of the Index.
166+
message IndexStats {
167+
// Output only. The number of vectors in the Index.
168+
int64 vectors_count = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
169+
170+
// Output only. The number of shards in the Index.
171+
int32 shards_count = 2 [(google.api.field_behavior) = OUTPUT_ONLY];
96172
}

protos/google/cloud/aiplatform/v1/index_service.proto

+58
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,22 @@ service IndexService {
9393
metadata_type: "DeleteOperationMetadata"
9494
};
9595
}
96+
97+
// Add/update Datapoints into an Index.
98+
rpc UpsertDatapoints(UpsertDatapointsRequest) returns (UpsertDatapointsResponse) {
99+
option (google.api.http) = {
100+
post: "/v1/{index=projects/*/locations/*/indexes/*}:upsertDatapoints"
101+
body: "*"
102+
};
103+
}
104+
105+
// Remove Datapoints from an Index.
106+
rpc RemoveDatapoints(RemoveDatapointsRequest) returns (RemoveDatapointsResponse) {
107+
option (google.api.http) = {
108+
post: "/v1/{index=projects/*/locations/*/indexes/*}:removeDatapoints"
109+
body: "*"
110+
};
111+
}
96112
}
97113

98114
// Request message for [IndexService.CreateIndex][google.cloud.aiplatform.v1.IndexService.CreateIndex].
@@ -201,6 +217,48 @@ message DeleteIndexRequest {
201217
];
202218
}
203219

220+
// Request message for [IndexService.UpsertDatapoints][google.cloud.aiplatform.v1.IndexService.UpsertDatapoints]
221+
message UpsertDatapointsRequest {
222+
// Required. The name of the Index resource to be updated.
223+
// Format:
224+
// `projects/{project}/locations/{location}/indexes/{index}`
225+
string index = 1 [
226+
(google.api.field_behavior) = REQUIRED,
227+
(google.api.resource_reference) = {
228+
type: "aiplatform.googleapis.com/Index"
229+
}
230+
];
231+
232+
// A list of datapoints to be created/updated.
233+
repeated IndexDatapoint datapoints = 2;
234+
}
235+
236+
// Response message for [IndexService.UpsertDatapoints][google.cloud.aiplatform.v1.IndexService.UpsertDatapoints]
237+
message UpsertDatapointsResponse {
238+
239+
}
240+
241+
// Request message for [IndexService.RemoveDatapoints][google.cloud.aiplatform.v1.IndexService.RemoveDatapoints]
242+
message RemoveDatapointsRequest {
243+
// Required. The name of the Index resource to be updated.
244+
// Format:
245+
// `projects/{project}/locations/{location}/indexes/{index}`
246+
string index = 1 [
247+
(google.api.field_behavior) = REQUIRED,
248+
(google.api.resource_reference) = {
249+
type: "aiplatform.googleapis.com/Index"
250+
}
251+
];
252+
253+
// A list of datapoint ids to be deleted.
254+
repeated string datapoint_ids = 2;
255+
}
256+
257+
// Response message for [IndexService.RemoveDatapoints][google.cloud.aiplatform.v1.IndexService.RemoveDatapoints]
258+
message RemoveDatapointsResponse {
259+
260+
}
261+
204262
// Runtime operation metadata with regard to Matching Engine Index.
205263
message NearestNeighborSearchOperationMetadata {
206264
message RecordError {

protos/google/cloud/aiplatform/v1/model_monitoring.proto

+2-4
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,8 @@ option java_package = "com.google.cloud.aiplatform.v1";
2727
option php_namespace = "Google\\Cloud\\AIPlatform\\V1";
2828
option ruby_package = "Google::Cloud::AIPlatform::V1";
2929

30-
// Next ID: 8
30+
// The objective configuration for model monitoring, including the information
31+
// needed to detect anomalies for one particular model.
3132
message ModelMonitoringObjectiveConfig {
3233
// Training Dataset information.
3334
message TrainingDataset {
@@ -162,7 +163,6 @@ message ModelMonitoringObjectiveConfig {
162163
ExplanationConfig explanation_config = 5;
163164
}
164165

165-
// Next ID: 3
166166
message ModelMonitoringAlertConfig {
167167
// The config for email alert.
168168
message EmailAlertConfig {
@@ -184,7 +184,6 @@ message ModelMonitoringAlertConfig {
184184
}
185185

186186
// The config for feature monitoring threshold.
187-
// Next ID: 3
188187
message ThresholdConfig {
189188
oneof threshold {
190189
// Specify a threshold value that can trigger the alert.
@@ -201,7 +200,6 @@ message ThresholdConfig {
201200

202201
// Sampling Strategy for logging, can be for both training and prediction
203202
// dataset.
204-
// Next ID: 2
205203
message SamplingStrategy {
206204
// Requests are randomly selected.
207205
message RandomSampleConfig {

protos/google/cloud/aiplatform/v1/model_service.proto

+1-1
Original file line numberDiff line numberDiff line change
@@ -439,7 +439,7 @@ message MergeVersionAliasesRequest {
439439

440440
// Required. The set of version aliases to merge.
441441
// The alias should be at most 128 characters, and match
442-
// `[a-z][a-z0-9-]{0,126}[a-z-0-9]`.
442+
// `[a-z][a-zA-Z0-9-]{0,126}[a-z-0-9]`.
443443
// Add the `-` prefix to an alias means removing that alias from the version.
444444
// `-` is NOT counted in the 128 characters. Example: `-golden` means removing
445445
// the `golden` alias from the version.

protos/google/cloud/aiplatform/v1/pipeline_job.proto

+16
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,18 @@ message PipelineJob {
5050

5151
// The runtime config of a PipelineJob.
5252
message RuntimeConfig {
53+
// The type of an input artifact.
54+
message InputArtifact {
55+
oneof kind {
56+
// Artifact resource id from MLMD. Which is the last portion of an
57+
// artifact resource
58+
// name(projects/{project}/locations/{location}/metadataStores/default/artifacts/{artifact_id}).
59+
// The artifact must stay within the same project, location and default
60+
// metadatastore as the pipeline.
61+
string artifact_id = 1;
62+
}
63+
}
64+
5365
// Deprecated. Use [RuntimeConfig.parameter_values][google.cloud.aiplatform.v1.PipelineJob.RuntimeConfig.parameter_values] instead. The runtime
5466
// parameters of the PipelineJob. The parameters will be passed into
5567
// [PipelineJob.pipeline_spec][google.cloud.aiplatform.v1.PipelineJob.pipeline_spec] to replace the placeholders at runtime.
@@ -81,6 +93,10 @@ message PipelineJob {
8193
// will stop scheduling any new tasks when a task has failed. Any scheduled
8294
// tasks will continue to completion.
8395
PipelineFailurePolicy failure_policy = 4;
96+
97+
// The runtime artifacts of the PipelineJob. The key will be the input
98+
// artifact name and the value would be one of the InputArtifact.
99+
map<string, InputArtifact> input_artifacts = 5;
84100
}
85101

86102
// Output only. The resource name of the PipelineJob.

protos/google/cloud/aiplatform/v1/pipeline_service.proto

+3
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,9 @@ message ListPipelineJobsRequest {
366366
// * `end_time`
367367
// * `start_time`
368368
string order_by = 6;
369+
370+
// Mask specifying which fields to read.
371+
google.protobuf.FieldMask read_mask = 7;
369372
}
370373

371374
// Response message for [PipelineService.ListPipelineJobs][google.cloud.aiplatform.v1.PipelineService.ListPipelineJobs]

protos/google/cloud/aiplatform/v1/saved_query.proto

+2-1
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ message SavedQuery {
5959

6060
// Required. Problem type of the SavedQuery.
6161
// Allowed values:
62+
//
6263
// * IMAGE_CLASSIFICATION_SINGLE_LABEL
6364
// * IMAGE_CLASSIFICATION_MULTI_LABEL
6465
// * IMAGE_BOUNDING_POLY
@@ -74,7 +75,7 @@ message SavedQuery {
7475
// Output only. Number of AnnotationSpecs in the context of the SavedQuery.
7576
int32 annotation_spec_count = 10 [(google.api.field_behavior) = OUTPUT_ONLY];
7677

77-
// Used to perform consistent read-modify-write updates. If not set, a blind
78+
// Used to perform a consistent read-modify-write update. If not set, a blind
7879
// "overwrite" update happens.
7980
string etag = 8;
8081

protos/google/cloud/aiplatform/v1/vizier_service.proto

+1-1
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,7 @@ message SuggestTrialsRequest {
285285
}
286286
];
287287

288-
// Required. The number of suggestions requested.
288+
// Required. The number of suggestions requested. It must be positive.
289289
int32 suggestion_count = 2 [(google.api.field_behavior) = REQUIRED];
290290

291291
// Required. The identifier of the client that is requesting the suggestion.

protos/google/cloud/aiplatform/v1beta1/batch_prediction_job.proto

+7
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import "google/cloud/aiplatform/v1beta1/io.proto";
2525
import "google/cloud/aiplatform/v1beta1/job_state.proto";
2626
import "google/cloud/aiplatform/v1beta1/machine_resources.proto";
2727
import "google/cloud/aiplatform/v1beta1/manual_batch_tuning_parameters.proto";
28+
import "google/cloud/aiplatform/v1beta1/model_deployment_monitoring_job.proto";
2829
import "google/cloud/aiplatform/v1beta1/model_monitoring.proto";
2930
import "google/cloud/aiplatform/v1beta1/unmanaged_container_model.proto";
3031
import "google/protobuf/struct.proto";
@@ -304,4 +305,10 @@ message BatchPredictionJob {
304305
// the input and output to the batch prediction job, as well as the provided
305306
// training dataset.
306307
ModelMonitoringConfig model_monitoring_config = 26;
308+
309+
// Get batch prediction job monitoring statistics.
310+
repeated ModelMonitoringStatsAnomalies model_monitoring_stats_anomalies = 31;
311+
312+
// Output only. The running status of the model monitoring pipeline.
313+
google.rpc.Status model_monitoring_status = 32 [(google.api.field_behavior) = OUTPUT_ONLY];
307314
}

protos/google/cloud/aiplatform/v1beta1/dataset_service.proto

+1-2
Original file line numberDiff line numberDiff line change
@@ -399,8 +399,7 @@ message ListSavedQueriesRequest {
399399

400400
// Response message for [DatasetService.ListSavedQueries][google.cloud.aiplatform.v1beta1.DatasetService.ListSavedQueries].
401401
message ListSavedQueriesResponse {
402-
// A list of SavedQueries that matches the specified filter in the
403-
// request.
402+
// A list of SavedQueries that match the specified filter in the request.
404403
repeated SavedQuery saved_queries = 1;
405404

406405
// The standard List next-page token.

protos/google/cloud/aiplatform/v1beta1/explanation.proto

+3-3
Original file line numberDiff line numberDiff line change
@@ -183,8 +183,8 @@ message ExplanationSpec {
183183
// Required. Parameters that configure explaining of the Model's predictions.
184184
ExplanationParameters parameters = 1 [(google.api.field_behavior) = REQUIRED];
185185

186-
// Required. Metadata describing the Model's input and output for explanation.
187-
ExplanationMetadata metadata = 2 [(google.api.field_behavior) = REQUIRED];
186+
// Optional. Metadata describing the Model's input and output for explanation.
187+
ExplanationMetadata metadata = 2 [(google.api.field_behavior) = OPTIONAL];
188188
}
189189

190190
// Parameters to configure explaining for Model's predictions.
@@ -229,7 +229,7 @@ message ExplanationParameters {
229229
// explaining.
230230
//
231231
// If not populated, returns attributions for [top_k][google.cloud.aiplatform.v1beta1.ExplanationParameters.top_k] indices of outputs.
232-
// If neither top_k nor output_indeices is populated, returns the argmax
232+
// If neither top_k nor output_indices is populated, returns the argmax
233233
// index of the outputs.
234234
//
235235
// Only applicable to Models that predict multiple outputs (e,g, multi-class

0 commit comments

Comments
 (0)