Strawperson proposal for URI-based tag resolution

[handrews]

Tag resolution is difficult to do with URIs because tags are ordered in the OpenAPI Object, and the order is significant.

This change takes the approach of adding an OpenAPI Document URI to determine which OpenAPI Object is used to look up the named tag.

In the Operation Object, this is a parallel field called tagRefs, that groups tags under URIs. In the Tag Object, the parentRef field provides the context for the existing parent field.

• schema changes are included in this pull request
• schema changes are needed for this pull request but not done yet
• no schema changes are needed for this pull request

[baywet]

@lornajane would you mind giving it an initial review when you have a couple of minutes please?

[baywet]

@handrews to follow up on the discussion from the call. Here is what I hope is a better explanation of my thoughts on the topic.

Problem: which document should we use to lookup the tags when there are multiple documents involved? Solution: let's use URI/something that points to the document. And have prescriptive rules about what to do when the document "reference" is absent. That makes perfect sense to me, I'm not debating that.

What I was trying to express in my confused wording is that OAI has lots of different ways of resolving "references" (internal or external):

• most of them use the $ref syntax
• Schema does that + superset with $id & co
• tags and security schemes today rely on the name only (which is problematic)

With the currently proposed changes, we're effectively introducing two new ways of resolving references just for the tags:

• from the operation object: read this new tag ref to know where the document is and what tags to get from it.
• from the tag object: read the parent tag + parent document fields

What I was trying to suggest is to align those two things instead:

• operation object: works just the same because people don't like union types ;-)
• document object: gets a tag refs as well.
• tag object (in document tags): remains unchanged

This way if you have an external parent to a tag, it simply needs to be in the tag refs. And you don't get the ordering issue we talked about.

It's always easier to communicate ideas with examples:

provinces.yaml

# ...
tags:
  - name: quebec
  - name: ontario
  - name: BC
# ...

cities.yaml

tagRefs:
  - provinces.yaml
    - quebec
    - ontario
    - BC
tags:
  - name: montreal
    parent: quebec
  - name: toronto
    parent: ontario
  - name: vancouver
    parent: BC

[karenetheridge]

One of the alternatives I thought of here is to use an array tuple to refer to a tag when it was defined in another document: e.g.

$self: https://example.com/main_api
...
tags:
  - name: car
  - name: truck
paths:
  /foo:
    get:
      tags:
        - car
        - truck
        - [ https://example.com/animal_api, cat ]
        - [ https://example.com/animal_api, dog ]

$self: https://example.com/animal_api
...
tags:
  - name: cat
  - name: dog

The other thought I had was to refer to tags using a URI with a plain-name fragment, but using that directly in the place of a tag name could be ambiguous (unless we prohibited the # character from appearing in a tag name):

$self: https://example.com/main_api
...
paths:
  /foo:
    get:
      tags:
        - car
        - '#tag-truck'    # this is the same as just 'truck' as the local document is used
        - 'https://example.com/animal_api#tag-cat'
        - 'https://example.com/animal_api#tag-dog'

edit: I withdraw this proposal and I will write up a separate proposal in its own document.

[karenetheridge]

I would also like to get a clearer understanding of what exactly the problem is here that we're trying to solve? Is it just that we can refer to a tag by name and not see clearly what document to find it in? Or that we want to duplicate tag names across multiple documents without that being an error?

In other words -- can someone provide an example of something they want to do, that isn't possible to express today?

[handrews]

@karenetheridge this is not possible today:

openapi: 3.2.0
$self: https://example.com/openapi/entry
tags:
- name: foo
paths:
  /something
    $ref: shared#/components/pathItems/Something

openapi: 3.2.0
$self: https://example.com/openapi/shared
tags:
- name: bar
components:
  pathItems:
    Something:
      get:
        tags:
        - foo
        - bar

In this example, what we want is for the foo tag to resolve to the entry document (which is RECOMMENDED as of 3.1.1 / 3.0.4) and the bar tag to resolve to shared document (which we have no way of doing that is even RECOMMENDED).

---

One of the alternatives I thought of here is to use an array tuple to refer to a tag when it was defined in another document

This gets into the problems with union types and strongly typed languages.

The other thought I had was to refer to tags using a URI with a plain-name fragment

Throwing in new fragment syntax/semantics would be a big deal, adding complexity to an already complex area. Technically we could do it, but my strong preference would be to avoid specifying plain-name fragment behavior in 3.x so that we can reserve that for a coherent approach in 4.0. This would also impact the media type registration work happening in parallel.

[handrews]

@baywet Thanks, that makes things more clear. That would work if we're willing to say that a given tag in a document can only resolve to a single, specific document (either the implicit-RECOMMENDED entry document if it is not in tagRefs, or the document given in tagRefs if the tag is found in tagRefs). In other words, in your example quebec in cities.yaml always resolves to provinces.yaml. It is not possible to have a quebec tag on one operation that resolves to provinces.yaml, and another that resolves elsewhere.

This is probably a reasonable restriction to gain a simpler syntax, so if other folks prefer it, I could definitely support this. It does not prevent anything that can be done today, and having the same tag resolve differently is probably not a great practice so I wouldn't feel too bad about not supporting it.

[karenetheridge]

In this example, what we want is for the foo tag to resolve to the entry document (which is RECOMMENDED as of 3.1.1 / 3.0.4) and the bar tag to resolve to shared document (which we have no way of doing that is even RECOMMENDED).

Okay, so the real problem here is that the use of multiple documents to form a description is under-specified, and isn't really specific to tags at all. The same issue exists for operationId (do we look for the referenced operation in the current document, the entry document, or something else?) and also for /paths entries (can we combine paths definitions from multiple documents together in a single description?) So why don't we solve this problem more universally?

I'm in the middle of implementing multiple document support right now, and I'm finding that it's pretty straightforward to allow definitions of all the above types to come from all documents -- it just means there is more than one phase of document verification: one where a single document is parsed in isolation, and the second where multiple already-parsed documents are combined together to form a single description. In that second phase, there are more potential error sources -- where tag names, operationIds, and path templates can all be duplicated between documents. But at runtime (taking an HTTP request as input, and applying it to the openapi description to parse it) is just the same as with a single document.

What complications am I not seeing here? sincere question.

[lornajane]

I think there are complications here that I was not seeing and I'm still not sure that we're making things clearer (at least so far - all the discussion definitely helps so thanks everyone who already chimed in!). Tags are a top-level element for an OpenAPI description so the tags available are the ones either declared or included by reference there. The same goes for every other element - Operations don't arrive by inference, they have to be in the paths section or with an explicit reference to include them from somewhere else. I think if you're referring to an operation in a document, it's reasonable to refer to a tag as well.

I can see the use case for pulling in any tags that were referred to in an operation but not included in the entry document - but I'd expect tooling to close that gap and make a better developer experience, not to have the specification explain how that logic should be done.

[handrews]

@lornajane

I can see the use case for pulling in any tags that were referred to in an operation but not included in the entry document - but I'd expect tooling to close that gap and make a better developer experience, not to have the specification explain how that logic should be done.

When I am processing an operation and find an entry in the Operation Object's tags list, I need to figure out where to look to find any matching Tag Object, and when to give up and just treat it as a tag without a Tag Object.

It's not clear to me what you mean by "pulling in" tags, as that doesnt seem to match any requirement that we have at the moment. Could you elaborate on that?

[lornajane]

Perhaps my "pulling in" is your "where to look". Either the tag is in the tags list, or it isn't. Getting the tag from somewhere else is not what I expect. Or I totally didn't understand your point (sorry! I have definitely been confused about this idea for a while and I know it's come up a few times).

[handrews]

@lornajane there are basically two places to look:

• In the entry document's OpenAPI Object's tags array. This was the only option in 2.0
• In the document where the tag is used (in an Operation Object or, with the parent field, a Tag Object). This is seen as intuitive to many if not most users who started with 3.1, but there was never any guidance about it.

The same problem exists for Security Scheme component names used in Security Requirement Objects.

TL;DR: We had to RECOMMEND the 2.0 entry document behavior for compatibility, but the 3.1 same-document behavior is more suitable to 3.1+ processing. The inability to do what new 3.1 users expect is the problem to solve here.

---

Due to the historical behavior of 2.0, which most tooling vendors carried over into 3.0 as 3.0 did not provide any clear guidance on the processing model (the 2.0 language was removed and not replaced with anything), in 3.0.4 and 3.1.1, we RECOMMENDED resolving tag names and security scheme component names to the entry document.

This recommendation has the advantage of doing what the most-established (and least likely to change no matter what we say) tools are already doing. It also was preferred because of the two options, referencing the entry document can't be solved by a URI reference, as the entry document might be different for different uses of a shared referenced document (JSON Schema solves this "can't know the URI of the entry doc" problem with $dynamicRef).

However, many people expect tags and security scheme components to resolve to the current document. Adding a URI reference mechanism allows this to be explicitly supported, so people aren't relying on implementation-defined behavior.

To solve this problem for Security Requirement Objects (PR #4388), I followed the precedent of the Discriminator Object's mapping field, which also resolves as either a component name or a URI. Unfortunately, it is not as straightforward to extend this solution to tags due to the ordering and the relatively fragile nature of JSON Pointers involving array indicies.

[lornajane]

Thanks, that's a good explanation.

However I don't think that the idea of finding tags in other documents should be included in the specification. It's a reasonable feature request, but people/tools can figure out if/how to bring in tags from dependent documents if they need to, it's out of scope for the specification to dictate how the correct information gets to the right place IMO.

[handrews]

However I don't think that the idea of finding tags in other documents should be included in the specification.

I don't understand this. It's already how many tools work, so it's already "in the specification." It's already implemented. It's far too late to close the door here.

[handrews]

@karenetheridge

So why don't we solve this problem more universally?

In what way are we not doing this, within the confines of the existing specification text?

operationId has special text that limits how we can deal with it.

For Security Requirement Objects and tags, I am (as I have explained above) trying to follow the precedent of the mapping field of the Discriminator Object, which is the closest analogue.

Please tell me what else we could do that:

• Does not break our compatibility rules
• Does not create new URI fragment rules
• Does not introduce unnecessary further divergence from the established mechanisms (the various URI-based keywords, the dual-function mapping, and the unique operationId) that already exist and can't be removed?

[handrews]

@lornajane

This is how it is already in the specification (added and discussed in PR #3856, backported to 3.0.4 in #3906):

For resolving component and tag name connections from a referenced (non-entry) document, it is RECOMMENDED that tools resolve from the entry document, rather than the current document. This allows Security Scheme Objects and Tag Objects to be defined next to the API’s deployment information (the top-level array of Server Objects), and treated as an interface for referenced documents to access.

[handrews]

@baywet on further thought, I want to emphasize that the most common use case here will be ensuring that tags in a shared document resolve from that same document:

tags:
- name: x
- name: y
/foo:
  get:
    tagRefs:
      "#":
        - x
        - y

With your approach, it is necessary to replicate the tags up to a new top-level tagRefs far from the point of use. With the example I'm giving here, it is barely more effort than using the existing tags field.

[handrews]

Closing- will submit a proposal.