(docs) update catalogd docs to include new metas endpoint

Closes operator-framework#1782

Signed-off-by: Anik Bhattacharjee <[email protected]>

Author: anik120

docs/api-reference/catalogd-webserver.md

@@ -1,9 +1,14 @@
 # Catalogd web server
 
 [Catalogd](https://github.com/operator-framework/operator-controller/tree/main/catalogd), the OLM v1 component for making catalog contents available on cluster, includes
-a web server that serves catalog contents to clients via an HTTP(S) endpoint.
+a web server that serves catalog contents to clients via HTTP(S) endpoints.
+
+The endpoints to retrieve information about installable clusterextentions can be composed from the `.status.urls.base` of a `ClusterCatalog` resource with the selected access API path.
+
+Currently, there are two API endpoints: 
+
+1. `api/v1/all` endpoint that provides access to the FBC metadata in entirety. 
 
-The endpoint to retrieve this information can be composed from the `.status.urls.base` of a `ClusterCatalog` resource with the selected access API path.
 As an example, to access the full FBC via the v1 API endpoint (indicated by path `api/v1/all`) where `.status.urls.base` is
 
 ```yaml
@@ -13,6 +18,21 @@ As an example, to access the full FBC via the v1 API endpoint (indicated by path
 
 the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/all`
 
+2. `api/v1/metas` endpoint that allows clients to retrive filtered portions of the FBC. 
+
+The metas endpoint accepts additional parameters following the pattern `/api/v1/metas?<parameters>`, and accepts parameters that corresponds to any combination of the fields of the [FBC Meta structure](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#schema) (except for `Blob`).
+
+As an example, to access only the [package schema](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#olmpackage-1) blobs of the FBC via the `api/v1/metas` endpoint where `.status.urls.base` is
+
+```yaml
+    urls:
+        base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio
+```
+
+the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/metas?schema=olm.package`
+
+For more examples of valid queries that can be made to the `api/v1/metas` service endpoint, please see [Catalog Queries](../howto/catalog-queries.md).
+
 !!! note
 
     The values of the `.status.urls` field in a `ClusterCatalog` resource are arbitrary string values and can change at any time.

docs/howto/catalog-queries.md

@@ -12,6 +12,10 @@ Then you can query the catalog by using `curl` commands and the `jq` CLI tool to
     By default, Catalogd is installed with TLS enabled for the catalog webserver.
     The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag.
 
+!!! note 
+    While using the `/api/v1/metas` endpoint shown in the below examples, it is important to note that the endpoint accepts parameters following the pattern `/api/v1/metas?<parameters>`. Valid parameter set is a any combination of the fields of the [FBC Meta Structure](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#schema) (except for `Blob`).
+    e.g. `schema=<schema_name>&package=<package_name>`, `schema=<schema_name>&name=<name>`, and `package=<package_name>&name=<name>` are all valid parameter combinations. However `schema=<schema_name>&version=<version_string>` is not a valid parameter combination, since version is not a first class FBC meta field. 
+    
 You also need to port forward the catalog server service:
 
 ``` terminal
@@ -20,17 +24,11 @@ kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443
 
 Now you can use the `curl` command with `jq` to query catalogs that are installed on your cluster.
 
-``` terminal title="Query syntax"
-curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | <query>
-```
-`<query>`
-: One of the `jq` queries from this document
-
 ## Package queries
 
 * Available packages in a catalog:
     ``` terminal
-    jq -s '.[] | select( .schema == "olm.package")'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package'
     ```
 
 * Packages that support `AllNamespaces` install mode and do not use webhooks:
@@ -40,44 +38,35 @@ curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | <query>
 
 * Package metadata:
     ``` terminal
-    jq -s '.[] | select( .schema == "olm.package") | select( .name == "<package_name>")'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name>'
     ```
 
     `<package_name>`
     : Name of the package from the catalog you are querying.
 
-* Catalog blobs in a package:
+* Blobs that belong to a package (that are not schema=olm.package):
     ``` terminal
-    jq -s '.[] | select( .package == "<package_name>")'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?package=<package_name>'
     ```
 
     `<package_name>`
     : Name of the package from the catalog you are querying.
 
+Note: the `olm.package` schema blob does not have the `package` field set. In other words, to get all the blobs that belong to a package, along with the olm.package blob for that package, a combination of both of the above queries need to be used. 
+
 ## Channel queries
 
 * Channels in a package:
     ``` terminal
-    jq -s '.[] | select( .schema == "olm.channel" ) | select( .package == "<package_name>") | .name'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=<package_name>'
     ```
 
     `<package_name>`
     : Name of the package from the catalog you are querying.
 
 * Versions in a channel:
     ``` terminal
-    jq -s '.[] | select( .package == "<package_name>" ) | select( .schema == "olm.channel" ) | select( .name == "<channel_name>" ) | .entries | .[] | .name'
-    ```
-
-    `<package_name>`
-    : Name of the package from the catalog you are querying.
-
-    `<channel_name>`
-    : Name of the channel for a given package.
-
-* Latest version in a channel and upgrade path:
-    ``` terminal
-    jq -s '.[] | select( .schema == "olm.channel" ) | select ( .name == "<channel_name>") | select( .package == "<package_name>")'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=<package_name>' | jq -s '.[] | select( .name == "<channel_name>" ) | .entries | .[] | .name'
     ```
 
     `<package_name>`
@@ -90,19 +79,16 @@ curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | <query>
 
 * Bundles in a package:
     ``` terminal
-    jq -s '.[] | select( .schema == "olm.bundle" ) | select( .package == "<package_name>") | .name'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&package=<package_name>'
     ```
 
     `<package_name>`
     : Name of the package from the catalog you are querying.
 
 * Bundle dependencies and available APIs:
     ``` terminal
-    jq -s '.[] | select( .schema == "olm.bundle" ) | select ( .name == "<bundle_name>") | select( .package == "<package_name>")'
+    curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&name=<bundle_name>' | jq -s '.[] | .properties | .[] | select(.type=="olm.gvk")'
     ```
 
-    `<package_name>`
-    : Name of the package from the catalog you are querying.
-
     `<bundle_name>`
     : Name of the bundle for a given package.

docs/tutorials/explore-available-content.md

@@ -27,7 +27,7 @@ Then you can query the catalog by using `curl` commands and the `jq` CLI tool to
 
 2. Return a list of all the extensions in a catalog via the v1 API endpoint:
     ``` terminal
-    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | jq -s '.[] | select(.schema == "olm.package") | .name'
+    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' | jq -s '.[] | .name'
     ```
 
     ??? success
@@ -97,38 +97,33 @@ Then you can query the catalog by using `curl` commands and the `jq` CLI tool to
 3. Return list of packages that support `AllNamespaces` install mode and do not use webhooks:
 
     ``` terminal
-    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | jq -cs '[.[] | select(.schema == "olm.bundle" and (.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true)) and .spec.webhookdefinitions == null) | .package] | unique[]'
+    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle | jq -cs '[.[] | select(.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true) and .spec.webhookdefinitions == null) | .package] | unique[]'
     ```
 
     ??? success
         ``` text title="Example output"
-        {"package":"ack-acm-controller","version":"0.0.12"}
-        {"package":"ack-acmpca-controller","version":"0.0.5"}
-        {"package":"ack-apigatewayv2-controller","version":"1.0.7"}
-        {"package":"ack-applicationautoscaling-controller","version":"1.0.11"}
-        {"package":"ack-cloudfront-controller","version":"0.0.9"}
-        {"package":"ack-cloudtrail-controller","version":"1.0.8"}
-        {"package":"ack-cloudwatch-controller","version":"0.0.3"}
-        {"package":"ack-cloudwatchlogs-controller","version":"0.0.4"}
-        {"package":"ack-dynamodb-controller","version":"1.2.9"}
-        {"package":"ack-ec2-controller","version":"1.2.4"}
-        {"package":"ack-ecr-controller","version":"1.0.12"}
-        {"package":"ack-ecs-controller","version":"0.0.4"}
-        {"package":"ack-efs-controller","version":"0.0.5"}
-        {"package":"ack-eks-controller","version":"1.3.3"}
-        {"package":"ack-elasticache-controller","version":"0.0.29"}
-        {"package":"ack-emrcontainers-controller","version":"1.0.8"}
-        {"package":"ack-eventbridge-controller","version":"1.0.6"}
-        {"package":"ack-iam-controller","version":"1.3.6"}
-        {"package":"ack-kafka-controller","version":"0.0.4"}
-        {"package":"ack-keyspaces-controller","version":"0.0.11"}
+        "ack-acm-controller"
+        "ack-acmpca-controller"
+        "ack-apigateway-controller"
+        "ack-apigatewayv2-controller"
+        "ack-applicationautoscaling-controller"
+        "ack-athena-controller"
+        "ack-cloudfront-controller"
+        "ack-cloudtrail-controller"
+        "ack-cloudwatch-controller"
+        "ack-cloudwatchlogs-controller"
+        "ack-documentdb-controller"
+        "ack-dynamodb-controller"
+        "ack-ec2-controller"
+        "ack-ecr-controller"
+        "ack-ecs-controller"
         ...
         ```
 
 4. Inspect the contents of an extension's metadata:
 
     ``` terminal
-    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/all | jq -s '.[] | select( .schema == "olm.package") | select( .name == "<package_name>")'
+    curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=<package_name>
     ```
 
     `package_name`