Skip to content

Commit e88bcf9

Browse files
anik120anik120
committed
(docs) draft catalogd docs with new metas endpoint
Closes operator-framework#1782 Signed-off-by: Anik Bhattacharjee <[email protected]>
1 parent 029d19f commit e88bcf9

File tree

3 files changed

+351
-0
lines changed

3 files changed

+351
-0
lines changed

‎docs/draft/api-reference/catalogd-webserver-metas-endpoint.md

+111
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Catalogd web server
2+
3+
[Catalogd](https://github.com/operator-framework/operator-controller/tree/main/catalogd), the OLM v1 component for making catalog contents available on cluster, includes
4+
a web server that serves catalog contents to clients via HTTP(S) endpoints.
5+
6+
The endpoints to retrieve information about installable clusterextentions can be composed from the `.status.urls.base` of a `ClusterCatalog` resource with the selected access API path.
7+
8+
Currently, there are two API endpoints:
9+
10+
1. `api/v1/all` endpoint that provides access to the FBC metadata in entirety.
11+
12+
As an example, to access the full FBC via the v1 API endpoint (indicated by path `api/v1/all`) where `.status.urls.base` is
13+
14+
```yaml
15+
urls:
16+
base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio
17+
```
18+
19+
the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/all`
20+
21+
2. `api/v1/metas` endpoint that allows clients to retrieve filtered portions of the FBC.
22+
23+
The metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?<parameter>[&<parameter>...]`.
24+
25+
As an example, to access only the [package schema](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#olmpackage-1) blobs of the FBC via the `api/v1/metas` endpoint where `.status.urls.base` is
26+
27+
```yaml
28+
urls:
29+
base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio
30+
```
31+
32+
the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/metas?schema=olm.package`
33+
34+
For more examples of valid queries that can be made to the `api/v1/metas` service endpoint, please see [Catalog Queries](../howto/catalog-queries.md).
35+
36+
!!! note
37+
38+
The values of the `.status.urls` field in a `ClusterCatalog` resource are arbitrary string values and can change at any time.
39+
While there are no guarantees on the exact value of this field, it will always contain catalog-specific API endpoints for use
40+
by clients to make a request from within the cluster.
41+
42+
## Interacting With the Server
43+
44+
### Supported HTTP Methods
45+
46+
The HTTP request methods supported by the catalogd web server are:
47+
48+
- [GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET)
49+
- [HEAD](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD)
50+
51+
### Response Format
52+
53+
Responses are encoded as a [JSON Lines](https://jsonlines.org/) stream of [File-Based Catalog](https://olm.operatorframework.io/docs/reference/file-based-catalogs) (FBC) [Meta](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#schema) objects delimited by newlines.
54+
55+
??? example "Example JSON-encoded FBC snippet"
56+
57+
```json
58+
{
59+
"schema": "olm.package",
60+
"name": "cockroachdb",
61+
"defaultChannel": "stable-v6.x",
62+
}
63+
{
64+
"schema": "olm.channel",
65+
"name": "stable-v6.x",
66+
"package": "cockroachdb",
67+
"entries": [
68+
{
69+
"name": "cockroachdb.v6.0.0",
70+
"skipRange": "<6.0.0"
71+
}
72+
]
73+
}
74+
{
75+
"schema": "olm.bundle",
76+
"name": "cockroachdb.v6.0.0",
77+
"package": "cockroachdb",
78+
"image": "quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba",
79+
"properties": [
80+
{
81+
"type": "olm.package",
82+
"value": {
83+
"packageName": "cockroachdb",
84+
"version": "6.0.0"
85+
}
86+
},
87+
],
88+
}
89+
```
90+
91+
Corresponding JSON lines response:
92+
```jsonlines
93+
{"schema":"olm.package","name":"cockroachdb","defaultChannel":"stable-v6.x"}
94+
{"schema":"olm.channel","name":"stable-v6.x","package":"cockroachdb","entries":[{"name":"cockroachdb.v6.0.0","skipRange":"<6.0.0"}]}
95+
{"schema":"olm.bundle","name":"cockroachdb.v6.0.0","package":"cockroachdb","image":"quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba","properties":[{"type":"olm.package","value":{"packageName":"cockroachdb","version":"6.0.0"}}]}
96+
```
97+
98+
### Compression Support
99+
100+
The `catalogd` web server supports gzip compression of responses, which can significantly reduce associated network traffic. In order to signal that the client handles compressed responses, the client must include `Accept-Encoding: gzip` as a header in the HTTP request.
101+
102+
The web server will include a `Content-Encoding: gzip` header in compressed responses.
103+
104+
!!! note
105+
106+
Only catalogs whose uncompressed response body would result in a response size greater than 1400 bytes will be compressed.
107+
108+
### Cache Header Support
109+
110+
For clients interested in caching the information returned from the `catalogd` web server, the `Last-Modified` header is set
111+
on responses and the `If-Modified-Since` header is supported for requests.

‎docs/draft/howto/catalog-queries-metas-endpoint.md

+93
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Catalog queries
2+
3+
After you [add a catalog of extensions](../tutorials/add-catalog.md) to your cluster, you must port forward your catalog as a service.
4+
Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install.
5+
6+
## Prerequisites
7+
8+
* You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster.
9+
* You have installed the `jq` CLI tool.
10+
11+
!!! note
12+
By default, Catalogd is installed with TLS enabled for the catalog webserver.
13+
The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag.
14+
15+
!!! note
16+
While using the `/api/v1/metas` endpoint shown in the below examples, it is important to note that the metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?<parameter>[&<parameter>...]`. e.g. `schema=<schema_name>&package=<package_name>`, `schema=<schema_name>&name=<name>`, and `package=<package_name>&name=<name>` are all valid parameter combinations. However `schema=<schema_name>&version=<version_string>` is not a valid parameter combination, since version is not a first class FBC meta field.
17+
18+
You also need to port forward the catalog server service:
19+
20+
``` terminal
21+
kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443
22+
```
23+
24+
Now you can use the `curl` command with `jq` to query catalogs that are installed on your cluster.
25+
26+
## Package queries
27+
28+
* Available packages in a catalog:
29+
``` terminal
30+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package'
31+
```
32+
33+
* Packages that support `AllNamespaces` install mode and do not use webhooks:
34+
``` terminal
35+
jq -cs '[.[] | select(.schema == "olm.bundle" and (.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true)) and .spec.webhookdefinitions == null) | .package] | unique[]'
36+
```
37+
38+
* Package metadata:
39+
``` terminal
40+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name>'
41+
```
42+
43+
`<package_name>`
44+
: Name of the package from the catalog you are querying.
45+
46+
* Blobs that belong to a package (that are not schema=olm.package):
47+
``` terminal
48+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?package=<package_name>'
49+
```
50+
51+
`<package_name>`
52+
: Name of the package from the catalog you are querying.
53+
54+
Note: the `olm.package` schema blob does not have the `package` field set. In other words, to get all the blobs that belong to a package, along with the olm.package blob for that package, a combination of both of the above queries need to be used.
55+
56+
## Channel queries
57+
58+
* Channels in a package:
59+
``` terminal
60+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=<package_name>'
61+
```
62+
63+
`<package_name>`
64+
: Name of the package from the catalog you are querying.
65+
66+
* Versions in a channel:
67+
``` terminal
68+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=zoperator&name=alpha' | jq -s '.[] | .entries | .[] | .name'
69+
```
70+
71+
`<package_name>`
72+
: Name of the package from the catalog you are querying.
73+
74+
`<channel_name>`
75+
: Name of the channel for a given package.
76+
77+
## Bundle queries
78+
79+
* Bundles in a package:
80+
``` terminal
81+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&package=<package_name>'
82+
```
83+
84+
`<package_name>`
85+
: Name of the package from the catalog you are querying.
86+
87+
* Bundle dependencies and available APIs:
88+
``` terminal
89+
curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&name=<bundle_name>' | jq -s '.[] | .properties[] | select(.type=="olm.gvk")'
90+
```
91+
92+
`<bundle_name>`
93+
: Name of the bundle for a given package.

‎docs/draft/tutorials/explore-available-content-metas-endpoint.md

+147
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
---
2+
hide:
3+
- toc
4+
---
5+
6+
# Explore Available Content
7+
8+
After you [add a catalog of extensions](add-catalog.md) to your cluster, you must port forward your catalog as a service.
9+
Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install.
10+
11+
## Prerequisites
12+
13+
* You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster.
14+
* You have installed the `jq` CLI tool.
15+
16+
!!! note
17+
By default, Catalogd is installed with TLS enabled for the catalog webserver.
18+
The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag.
19+
20+
## Procedure
21+
22+
1. Port forward the catalog server service:
23+
24+
``` terminal
25+
kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443
26+
```
27+
28+
2. Return a list of all the extensions in a catalog via the v1 API endpoint:
29+
``` terminal
30+
curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' | jq -s '.[] | .name'
31+
```
32+
33+
??? success
34+
``` text title="Example output"
35+
"ack-acm-controller"
36+
"ack-acmpca-controller"
37+
"ack-apigatewayv2-controller"
38+
"ack-applicationautoscaling-controller"
39+
"ack-cloudfront-controller"
40+
"ack-cloudtrail-controller"
41+
"ack-cloudwatch-controller"
42+
"ack-cloudwatchlogs-controller"
43+
"ack-dynamodb-controller"
44+
"ack-ec2-controller"
45+
"ack-ecr-controller"
46+
"ack-ecs-controller"
47+
"ack-efs-controller"
48+
"ack-eks-controller"
49+
"ack-elasticache-controller"
50+
"ack-emrcontainers-controller"
51+
"ack-eventbridge-controller"
52+
"ack-iam-controller"
53+
"ack-kafka-controller"
54+
"ack-keyspaces-controller"
55+
"ack-kinesis-controller"
56+
"ack-kms-controller"
57+
"ack-lambda-controller"
58+
"ack-memorydb-controller"
59+
"ack-mq-controller"
60+
"ack-networkfirewall-controller"
61+
"ack-opensearchservice-controller"
62+
"ack-pipes-controller"
63+
"ack-prometheusservice-controller"
64+
"ack-rds-controller"
65+
"ack-route53-controller"
66+
"ack-route53resolver-controller"
67+
"ack-s3-controller"
68+
"ack-sagemaker-controller"
69+
"ack-secretsmanager-controller"
70+
"ack-sfn-controller"
71+
"ack-sns-controller"
72+
"ack-sqs-controller"
73+
"aerospike-kubernetes-operator"
74+
"airflow-helm-operator"
75+
"aiven-operator"
76+
"akka-cluster-operator"
77+
"alvearie-imaging-ingestion"
78+
"anchore-engine"
79+
"apch-operator"
80+
"api-operator"
81+
"api-testing-operator"
82+
"apicast-community-operator"
83+
"apicurio-registry"
84+
"apimatic-kubernetes-operator"
85+
"app-director-operator"
86+
"appdynamics-operator"
87+
"application-services-metering-operator"
88+
"appranix"
89+
"aqua"
90+
"argocd-operator"
91+
...
92+
```
93+
94+
!!! important
95+
Currently, OLM 1.0 does not support the installation of extensions that use webhooks or that target a single or specified set of namespaces.
96+
97+
3. Return list of packages where the channel head version use `olm.csv.metadata` format, support `AllNamespaces` install mode and do not use webhooks:
98+
99+
``` terminal
100+
curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle | jq -cs '[.[] | select(.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true) and .spec.webhookdefinitions == null) | .package] | unique[]'
101+
```
102+
103+
??? success
104+
``` text title="Example output"
105+
"ack-acm-controller"
106+
"ack-acmpca-controller"
107+
"ack-apigateway-controller"
108+
"ack-apigatewayv2-controller"
109+
"ack-applicationautoscaling-controller"
110+
"ack-athena-controller"
111+
"ack-cloudfront-controller"
112+
"ack-cloudtrail-controller"
113+
"ack-cloudwatch-controller"
114+
"ack-cloudwatchlogs-controller"
115+
"ack-documentdb-controller"
116+
"ack-dynamodb-controller"
117+
"ack-ec2-controller"
118+
"ack-ecr-controller"
119+
"ack-ecs-controller"
120+
...
121+
```
122+
123+
4. Inspect the contents of an extension's metadata:
124+
125+
``` terminal
126+
curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name>
127+
```
128+
129+
`package_name`
130+
: Specifies the name of the package you want to inspect.
131+
132+
??? success
133+
``` text title="Example output"
134+
{
135+
"defaultChannel": "stable-v6.x",
136+
"icon": {
137+
"base64data": "PHN2ZyB4bWxucz0ia...
138+
"mediatype": "image/svg+xml"
139+
},
140+
"name": "cockroachdb",
141+
"schema": "olm.package"
142+
}
143+
```
144+
145+
### Additional resources
146+
147+
* [Catalog queries](../howto/catalog-queries.md)

0 commit comments

Comments
 (0)