Brainstorming fields()

[Tinche]

Hello. Due to... circumstances... (mostly python/mypy#5144) looks like if we want proper fields() support in Mypy, we'll have to implement it ourselves in the attrs plugin. I volunteer to do this, and have a version ready in a branch.

Now, if we're already implementing special logic for this in the Mypy plugin we might as well discuss potential improvements.

Let me start with an example, to set the tone.

Right now, fields() returns a namedtuple of attr.Attribute. This is quite sufficient for introspection (so, enough for cattrs) but it's a little weak for implementing simple ORMs/ODMs. Which could be why we haven't seen many of those based on attrs.

I'm working on open sourcing our internal Mongo ODM, where classes are defined using attrs. Two interesting use cases come to mind.

Example 1: loading projected data.

from attrs import fields as f, define

@define
class Model:
    _id: ObjectId
    username: str

res = await mongo_client.find(Model, {"_id": ...}, projection=[f(Model)._id, f(Model).username])

This works in runtime, and will work statically once the plugin implements fields. One improvement that comes to mind here would be having the __attrs_attrs__ field available under a different name. For example, we could do something like SQLAlchemy and have it under Model.c. The query would then be:

res = await mongo_client.find(Model, {"_id": ...}, projection=[Model.c._id, Model.c.username])

This can work today in runtime, but not statically. Maybe we could think of a way to make it declarative and then support it in Mypy?

Might be nice enough to use for the query condition as well.

res = await mongo_client.find(Model, {Model.c._id: ...}, projection=[Model.c._id, Model.c.username])

I guess a shitty thing about the .c approach is that you cannot have an attribute named c, and it's essentially arbitrary. So maybe it should be configurable? A good thing about it is that it's easier on the eyes and no overhead of a function call.

Example 2: using a different Attribute class

I don't have this particular problem in Mongo since queries in Mongo are just bson documents, but if you tried replicating some of SQLAlchemy you'd run into it. You might want to add functionality to attr.Attribute to be able to write things like:

condition = f(User).username == "Tin"

This would just produce a False, instead of something else you might want to use later.

If we could supply our own attribute class to fields we might be able to deal with this issue. So fields might look like:

def fields(cls, *, attr_cls=None): ...  # Return a `NamedTuple[attr_cls, ...]` if provided instead

And in the plugin I could ensure that it works statically too. We'd probably cache it somewhere on the class. We could make attr_cls implement a classmethod like from_attribute.

Then in the hypothetical SQLAlchemy clone, they would have their own fields defined as fields = partial(attrs.fields, attr_cls=SQLAlchemyAttribute).

[Tinche]

Another potential use case has to do with nested evolves. Imagine if there was a special Attribute variant that supported this:

fields(Model).a.b.c

and produced an instance that could give you the exact path (['a', 'b', 'c']) if examined. It could even check, by implementing a __getattr__ hook and looking at it's type. It would have to be a special variant for its own fields to not clash with the attributes.