📖 (docs) draft catalogd docs with new metas endpoint #1841
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # Catalogd web server | ||
|
|
||
| [Catalogd](https://github.com/operator-framework/operator-controller/tree/main/catalogd), the OLM v1 component for making catalog contents available on cluster, includes | ||
| a web server that serves catalog contents to clients via HTTP(S) endpoints. | ||
|
|
||
| The endpoints to retrieve information about installable clusterextentions can be composed from the `.status.urls.base` of a `ClusterCatalog` resource with the selected access API path. | ||
|
|
||
| Currently, there are two API endpoints: | ||
|
|
||
| 1. `api/v1/all` endpoint that provides access to the FBC metadata in entirety. | ||
|
|
||
| As an example, to access the full FBC via the v1 API endpoint (indicated by path `api/v1/all`) where `.status.urls.base` is | ||
|
|
||
| ```yaml | ||
| urls: | ||
| base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio | ||
| ``` | ||
|
|
||
| the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/all` | ||
|
|
||
| 2. `api/v1/metas` endpoint that allows clients to retrieve filtered portions of the FBC. | ||
|
|
||
| The metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?<parameter>[&<parameter>...]`. | ||
|
|
||
| As an example, to access only the [package schema](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#olmpackage-1) blobs of the FBC via the `api/v1/metas` endpoint where `.status.urls.base` is | ||
|
|
||
| ```yaml | ||
| urls: | ||
| base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio | ||
| ``` | ||
|
|
||
| the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/metas?schema=olm.package` | ||
|
|
||
| For more examples of valid queries that can be made to the `api/v1/metas` service endpoint, please see [Catalog Queries](../howto/catalog-queries.md). | ||
|
|
||
| !!! note | ||
|
|
||
| The values of the `.status.urls` field in a `ClusterCatalog` resource are arbitrary string values and can change at any time. | ||
| While there are no guarantees on the exact value of this field, it will always contain catalog-specific API endpoints for use | ||
| by clients to make a request from within the cluster. | ||
|
|
||
| ## Interacting With the Server | ||
|
|
||
| ### Supported HTTP Methods | ||
|
|
||
| The HTTP request methods supported by the catalogd web server are: | ||
|
|
||
| - [GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET) | ||
| - [HEAD](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD) | ||
|
|
||
| ### Response Format | ||
|
|
||
| Responses are encoded as a [JSON Lines](https://jsonlines.org/) stream of [File-Based Catalog](https://olm.operatorframework.io/docs/reference/file-based-catalogs) (FBC) [Meta](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#schema) objects delimited by newlines. | ||
|
|
||
| ??? example "Example JSON-encoded FBC snippet" | ||
|
There was a problem hiding this comment. same here, this doesn't display correctly, together with the two ``` blocks below There was a problem hiding this comment. This is fine with our rendering system ... just not the github review front-end. There was a problem hiding this comment. good to know, thanks. @anik120 feel free to resolve, as I can't do it with my access level :-) |
||
|
|
||
| ```json | ||
| { | ||
| "schema": "olm.package", | ||
| "name": "cockroachdb", | ||
| "defaultChannel": "stable-v6.x", | ||
| } | ||
| { | ||
| "schema": "olm.channel", | ||
| "name": "stable-v6.x", | ||
| "package": "cockroachdb", | ||
| "entries": [ | ||
| { | ||
| "name": "cockroachdb.v6.0.0", | ||
| "skipRange": "<6.0.0" | ||
| } | ||
| ] | ||
| } | ||
| { | ||
| "schema": "olm.bundle", | ||
| "name": "cockroachdb.v6.0.0", | ||
| "package": "cockroachdb", | ||
| "image": "quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba", | ||
| "properties": [ | ||
| { | ||
| "type": "olm.package", | ||
| "value": { | ||
| "packageName": "cockroachdb", | ||
| "version": "6.0.0" | ||
| } | ||
| }, | ||
| ], | ||
| } | ||
| ``` | ||
|
|
||
| Corresponding JSON lines response: | ||
| ```jsonlines | ||
| {"schema":"olm.package","name":"cockroachdb","defaultChannel":"stable-v6.x"} | ||
| {"schema":"olm.channel","name":"stable-v6.x","package":"cockroachdb","entries":[{"name":"cockroachdb.v6.0.0","skipRange":"<6.0.0"}]} | ||
| {"schema":"olm.bundle","name":"cockroachdb.v6.0.0","package":"cockroachdb","image":"quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba","properties":[{"type":"olm.package","value":{"packageName":"cockroachdb","version":"6.0.0"}}]} | ||
| ``` | ||
|
|
||
| ### Compression Support | ||
|
|
||
| The `catalogd` web server supports gzip compression of responses, which can significantly reduce associated network traffic. In order to signal that the client handles compressed responses, the client must include `Accept-Encoding: gzip` as a header in the HTTP request. | ||
|
|
||
| The web server will include a `Content-Encoding: gzip` header in compressed responses. | ||
|
|
||
| !!! note | ||
|
|
||
| Only catalogs whose uncompressed response body would result in a response size greater than 1400 bytes will be compressed. | ||
|
|
||
| ### Cache Header Support | ||
|
|
||
| For clients interested in caching the information returned from the `catalogd` web server, the `Last-Modified` header is set | ||
| on responses and the `If-Modified-Since` header is supported for requests. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Catalog queries | ||
|
|
||
| After you [add a catalog of extensions](../tutorials/add-catalog.md) to your cluster, you must port forward your catalog as a service. | ||
| Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster. | ||
| * You have installed the `jq` CLI tool. | ||
|
|
||
| !!! note | ||
| By default, Catalogd is installed with TLS enabled for the catalog webserver. | ||
| The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag. | ||
|
|
||
| !!! note | ||
| While using the `/api/v1/metas` endpoint shown in the below examples, it is important to note that the metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?<parameter>[&<parameter>...]`. e.g. `schema=<schema_name>&package=<package_name>`, `schema=<schema_name>&name=<name>`, and `package=<package_name>&name=<name>` are all valid parameter combinations. However `schema=<schema_name>&version=<version_string>` is not a valid parameter combination, since version is not a first class FBC meta field. | ||
|
|
||
| You also need to port forward the catalog server service: | ||
|
|
||
| ``` terminal | ||
| kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443 | ||
| ``` | ||
|
|
||
| Now you can use the `curl` command with `jq` to query catalogs that are installed on your cluster. | ||
|
|
||
| ## Package queries | ||
|
|
||
| * Available packages in a catalog: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' | ||
| ``` | ||
|
|
||
| * Packages that support `AllNamespaces` install mode and do not use webhooks: | ||
| ``` terminal | ||
| jq -cs '[.[] | select(.schema == "olm.bundle" and (.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true)) and .spec.webhookdefinitions == null) | .package] | unique[]' | ||
| ``` | ||
|
|
||
| * Package metadata: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name>' | ||
| ``` | ||
|
|
||
| `<package_name>` | ||
| : Name of the package from the catalog you are querying. | ||
|
|
||
| * Blobs that belong to a package (that are not schema=olm.package): | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?package=<package_name>' | ||
| ``` | ||
|
|
||
| `<package_name>` | ||
| : Name of the package from the catalog you are querying. | ||
|
|
||
| Note: the `olm.package` schema blob does not have the `package` field set. In other words, to get all the blobs that belong to a package, along with the olm.package blob for that package, a combination of both of the above queries need to be used. | ||
|
|
||
| ## Channel queries | ||
|
|
||
| * Channels in a package: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=<package_name>' | ||
| ``` | ||
|
|
||
| `<package_name>` | ||
| : Name of the package from the catalog you are querying. | ||
|
|
||
| * Versions in a channel: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=zoperator&name=alpha' | jq -s '.[] | .entries | .[] | .name' | ||
| ``` | ||
|
|
||
| `<package_name>` | ||
| : Name of the package from the catalog you are querying. | ||
|
|
||
| `<channel_name>` | ||
| : Name of the channel for a given package. | ||
|
|
||
| ## Bundle queries | ||
|
|
||
| * Bundles in a package: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&package=<package_name>' | ||
| ``` | ||
|
|
||
| `<package_name>` | ||
| : Name of the package from the catalog you are querying. | ||
|
|
||
| * Bundle dependencies and available APIs: | ||
| ``` terminal | ||
| curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&name=<bundle_name>' | jq -s '.[] | .properties[] | select(.type=="olm.gvk")' | ||
| ``` | ||
|
|
||
| `<bundle_name>` | ||
| : Name of the bundle for a given package. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| hide: | ||
| - toc | ||
| --- | ||
|
|
||
| # Explore Available Content | ||
|
|
||
| After you [add a catalog of extensions](add-catalog.md) to your cluster, you must port forward your catalog as a service. | ||
| Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster. | ||
| * You have installed the `jq` CLI tool. | ||
|
|
||
| !!! note | ||
| By default, Catalogd is installed with TLS enabled for the catalog webserver. | ||
| The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag. | ||
|
|
||
| ## Procedure | ||
|
|
||
| 1. Port forward the catalog server service: | ||
|
|
||
| ``` terminal | ||
| kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443 | ||
| ``` | ||
|
|
||
| 2. Return a list of all the extensions in a catalog via the v1 API endpoint: | ||
| ``` terminal | ||
| curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' | jq -s '.[] | .name' | ||
| ``` | ||
|
|
||
| ??? success | ||
| ``` text title="Example output" | ||
| "ack-acm-controller" | ||
| "ack-acmpca-controller" | ||
| "ack-apigatewayv2-controller" | ||
| "ack-applicationautoscaling-controller" | ||
| "ack-cloudfront-controller" | ||
| "ack-cloudtrail-controller" | ||
| "ack-cloudwatch-controller" | ||
| "ack-cloudwatchlogs-controller" | ||
| "ack-dynamodb-controller" | ||
| "ack-ec2-controller" | ||
| "ack-ecr-controller" | ||
| "ack-ecs-controller" | ||
| "ack-efs-controller" | ||
| "ack-eks-controller" | ||
| "ack-elasticache-controller" | ||
| "ack-emrcontainers-controller" | ||
| "ack-eventbridge-controller" | ||
| "ack-iam-controller" | ||
| "ack-kafka-controller" | ||
| "ack-keyspaces-controller" | ||
| "ack-kinesis-controller" | ||
| "ack-kms-controller" | ||
| "ack-lambda-controller" | ||
| "ack-memorydb-controller" | ||
| "ack-mq-controller" | ||
| "ack-networkfirewall-controller" | ||
| "ack-opensearchservice-controller" | ||
| "ack-pipes-controller" | ||
| "ack-prometheusservice-controller" | ||
| "ack-rds-controller" | ||
| "ack-route53-controller" | ||
| "ack-route53resolver-controller" | ||
| "ack-s3-controller" | ||
| "ack-sagemaker-controller" | ||
| "ack-secretsmanager-controller" | ||
| "ack-sfn-controller" | ||
| "ack-sns-controller" | ||
| "ack-sqs-controller" | ||
| "aerospike-kubernetes-operator" | ||
| "airflow-helm-operator" | ||
| "aiven-operator" | ||
| "akka-cluster-operator" | ||
| "alvearie-imaging-ingestion" | ||
| "anchore-engine" | ||
| "apch-operator" | ||
| "api-operator" | ||
| "api-testing-operator" | ||
| "apicast-community-operator" | ||
| "apicurio-registry" | ||
| "apimatic-kubernetes-operator" | ||
| "app-director-operator" | ||
| "appdynamics-operator" | ||
| "application-services-metering-operator" | ||
| "appranix" | ||
| "aqua" | ||
| "argocd-operator" | ||
| ... | ||
| ``` | ||
|
|
||
| !!! important | ||
| Currently, OLM 1.0 does not support the installation of extensions that use webhooks or that target a single or specified set of namespaces. | ||
|
|
||
| 3. Return list of packages which support `AllNamespaces` install mode, do not use webhooks, and where the channel head version uses `olm.csv.metadata` format: | ||
|
|
||
| ``` terminal | ||
| curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle | jq -cs '[.[] | select(.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true) and .spec.webhookdefinitions == null) | .package] | unique[]' | ||
| ``` | ||
|
|
||
| ??? success | ||
| ``` text title="Example output" | ||
| "ack-acm-controller" | ||
| "ack-acmpca-controller" | ||
| "ack-apigateway-controller" | ||
| "ack-apigatewayv2-controller" | ||
| "ack-applicationautoscaling-controller" | ||
| "ack-athena-controller" | ||
| "ack-cloudfront-controller" | ||
| "ack-cloudtrail-controller" | ||
| "ack-cloudwatch-controller" | ||
| "ack-cloudwatchlogs-controller" | ||
| "ack-documentdb-controller" | ||
| "ack-dynamodb-controller" | ||
| "ack-ec2-controller" | ||
| "ack-ecr-controller" | ||
| "ack-ecs-controller" | ||
| ... | ||
| ``` | ||
|
|
||
| 4. Inspect the contents of an extension's metadata: | ||
|
|
||
| ``` terminal | ||
| curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name> | ||
| ``` | ||
|
|
||
| `package_name` | ||
| : Specifies the name of the package you want to inspect. | ||
|
|
||
| ??? success | ||
| ``` text title="Example output" | ||
| { | ||
| "defaultChannel": "stable-v6.x", | ||
| "icon": { | ||
| "base64data": "PHN2ZyB4bWxucz0ia... | ||
| "mediatype": "image/svg+xml" | ||
| }, | ||
| "name": "cockroachdb", | ||
| "schema": "olm.package" | ||
| } | ||
| ``` | ||
|
|
||
| ### Additional resources | ||
|
|
||
| * [Catalog queries](../howto/catalog-queries.md) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't have a problem here. The
!!! notenotation is supported by the mkdocs pipeline.For e.g. see existing rendered docs here, immediately before the section 3 header: "Granting User Access to API Resources".