PYTHON-5061 - Add an API to extend the bson TypeRegistry

[NoahStapp]

No description provided.

[ShaneHarvey]

Are there thread safety concerns here? What happens when add_codec is called when the registry is being used for bson encoding/decoding? What about with the C extensions and/or free threading mode?

[NoahStapp]

If add_codec is called while the registry is in use, the new codec added would only be available for encoding/decoding after the add_codec call completes. Yes, this is technically not thread-safe, but is extending a TypeRegistry while already encoding/decoding something we actually expect users to do?

If it is, avoiding that race condition would require us to detect if a TypeRegistry is already in use for encoding/decoding and block add_codec until the processing completes.

[ShaneHarvey]

I propose we keep the Registry immutable. After all, CodecOptions is intended to be immmutable and the registry is part of CodecOptions. This new api should return a new registry instance and keep the old one unchanged.

[NoahStapp]

Is there a reason CodecOptions is intended to be immutable besides the race condition outlined above? Returning a new registry instance would then require users to create a new CodecOptions with that registry, along with all of the other settings they may have configured in the original. That seems significantly more cumbersome to me.

[ShaneHarvey]

Mutating is problematic. Consider one section of your app is using CodecOptions, and then another unrelated thread changes some of the settings (eg via add_codec or some other option), that could violate the assumptions of the other code and lead to TypeErrors.

I think I would even prefer an api like this:

orig_codecs = coll.codec_options.type_registry.codecs
new_registry = TypeRegistry([new_codec]+orig_codecs)
new_coll = coll.with_options(codec_options=coll.codec_options.with_options(type_registry=new_registry))

All we need to add is a type_registry.codecs property. Sure this is cumbersome but it does the job, follows our existing conventions, and should be rarely used.

[NoahStapp]

You're saying instead of adding an API to add codecs, just add an API to return all of the current codecs? Ignoring the double underscore on TypeRegistry.__type_codecs, doesn't that already exist and make this change unneeded in the first place?

[ShaneHarvey]

Yes but __type_codecs is private. We need to expose it in a public api (as well as fallback_encoder).

[NoahStapp]

Yes, the change is making those two attributes publicly available. I like that approach.

[NoahStapp]

@ShaneHarvey made a good point that this can be done with fewer potentially far-reaching changes by simply making the following pattern available publicly:

orig_codecs = coll.codec_options.type_registry.codecs
fallback_encoder = coll.codec_options.type_registry.fallback_encoder
new_registry = TypeRegistry([new_codec]+orig_codecs, fallback_encoder)
new_coll = coll.with_options(codec_options=coll.codec_options.with_options(type_registry=new_registry))

[blink1073]

Nit, we should use :attr:https://www.sphinx-doc.org/en/master/usage/domains/python.html#role-py-attr