Skip to content

test: Add a sample generate_config yaml file. #2337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 11, 2024
Merged

test: Add a sample generate_config yaml file. #2337

Changes from 4 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3515982
test: Add hermetic build test config.
blakeli0 Jan 4, 2024
4ffdfc4
chore: Add versions path
blakeli0 Jan 4, 2024
dbf8d97
chore: Remove proto_only.
blakeli0 Jan 4, 2024
30d0fc3
Rename APIs to products, remove name from api configs.
blakeli0 Jan 5, 2024
9b40631
Add grpc and protobuf versions. Rename products to libraries, service…
blakeli0 Jan 11, 2024
e418836
Remove versions.txt path.
blakeli0 Jan 11, 2024
16ade90
Rename the config file.
blakeli0 Jan 11, 2024
23d8b91
Merge branch 'main' into add-hermetic-build-config
blakeli0 Jan 11, 2024
d1177aa
Move the config file.
blakeli0 Jan 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions library_generation/test/resources/google-cloud-java_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#Required.
gapic_generator_version: 2.30.0
#Required.
googleapis-commitish: 4512234113a18c1fda1fb0d0ceac8f4b4efe9801
#Required.
owlbot-cli-image: sha256:623647ee79ac605858d09e60c1382a716c125fb776f69301b72de1cd35d49409
#Required.
synthtool-commitish: 59fe44fde9866a26e7ee4e4450fd79f67f8cf599
#Required.
python-version: 3.11.2
#Optional. The root folder name of generated client libraries. If empty, modules will be created under current folder, useful for single module
destination-path: google-cloud-java
#Optional. This is a relative path to destination-path above. We will create a new versions.txt file if it is not specified
versions_path: versions.txt
#Required.
products:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since gapic-generator-java updates don't reflect immediately in a single PR on google-cloud-java but rather gradually updating the libraries, would it make sense to have products with an optional gapic-generator-java version that could override the repo level config?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's acceptable for now, but we should remove it and have the whole repo generated with a single version of generator all the time once we switch to the new process.

#Required. Can be used for populating the folder name java-{api_shortName}. This is also the destination-name in new-client.py.
- api_shortname: asset
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you write an example of java-bigtable, which combines two clients: bigtable and bigtable-admin?

#Optional. The default value is the title of service yaml
name-pretty: Cloud Asset
#Required.
library_type: GAPIC_AUTO
#Optional. The default value is com.google.cloud
group_id: com.google.cloud
#Optional. The default value is google.cloud.{api_shortname}
artifact_id: google.cloud.asset
#Optiona. The default value is true.
requires-billing: true
#Optional. The default value is documentation.summary from service yaml
api_description:
#Optional.
product_documentation:
#Optional.
client_documentation:
#Optional.
rest_documentation:
#Optional.
rpc_documentation:
#Required.
services:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if there's a new version defined in googleapis?
Do we want to dynamically get the versioned path or add a service in this file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new service will be added. Are new versions of APIs automatically updated now? I guess yes with proper owlbot.yaml config? if so, we may want to automatically update it with a separate process, or replace owlbot.yaml's functionality with this file. That's probably why @suztomo mentioned this idea in the meeting.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are new versions of APIs automatically updated now?

Yes, OwlBot cli copies all versions of the services using .OwlBot.yaml.

I'm thinking the wrapper file (the one reading the config file and generate libraries) can find versioned directory within proto_path.

proto_path in new-client.py is not a versioned directory, i.e., google/cloud/alloydb, while proto_path in generate_library.sh is a versioned directory, i.e., google/cloud/alloydb/v1.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to explicit list them, and/or support a wildcard configuration like google/cloud/asset/**. Because some product like Bigtable needs to include google/bigtable/admin/v2 and google/bigtable/v2, if we only supply google/bigtable, we will have to find all the services recursively and smartly, there might be some folders are excluded as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the wildcard/regex is fine.
For example: google/bigtable/v[1-9].* and google/bigtable/admin/v[1-9].*

#Required. This is a relative path to googleapis/googleapis. We'll parse all the parameters needed by generate_library.sh from BUILD.bazel in this folder.
- proto_path: google/cloud/asset/v1
- proto_path: google/cloud/asset/v1p1beta1
- proto_path: google/cloud/asset/v1p2beta1
- proto_path: google/cloud/asset/v1p5beta1
- proto_path: google/cloud/asset/v1p7beta1
- api_shortname: speech
library_type: GAPIC_AUTO
services:
- proto_path: google/cloud/asset/v1
proto_only: false