Post-Init-Only arguments

[hynek]

So data classes have the concept of init-only variables:

@dataclass
class C:
    i: int
    j: int = None
    database: InitVar[DatabaseType] = None

    def __post_init__(self, database):
        if self.j is None and database is not None:
            self.j = database.lookup('j')

c = C(10, database=my_database)

Thanks to converters I think the need in attrs is much lower, but we should keep an eye on the popularity of it.

I think I could live with something like:

@attr.s
class C:
   x = attr.ib()
   cache = attr.post_init_arg(default=None)

   def __attrs_post_init__(self, cache):
       # ...

I don’t think it should be shoehorned into attr.ib().

---

To be clear: the example above is totally possible with attrs, it's just a bit more clunky.

[Tinche]

But ah, if you declare it like that static tools will think it's a field by default, no?

My gut feeling is additional args to init don't really deserve to be in the class body like that.

[hynek]

Fuck yeah, you’re totally right.

It should be an argument to @attr.s.

[RonnyPfannschmidt]

i would like to point out that "named constructors" are commonly used for such setups allowing more clear spelling

@attr.s
class C:
    i = attr.ib()
    j: int = attr.ib()

    @classmethod
    def from_database(cls, i, database):
           j = database.lookup('j')
          return cls(i, j)

so depending on the actual needs and intended use-cases it may be entirely sensible to just leave it to the syntax

[hynek]

Yeah, I love named classmethods as named constructors. That’s one of the reasons why I’m not all in on the idea. But I’d like to watch how it’s embraced. It doesn’t help to die alone on a hill even if you’re right.

[joeblackwaslike]

I think this would be amazing and it's one of the cool things about dataclasses that attr's doesn't have

[pbourguignon]

I just came across this discussion while at a loss trying to figure out a right way to use attrs in a non-attrs hierarchy.

I think that the following simple addition would strictly extend the domain of application of attrs without impacting anyone who does not need it.

A new option (say, extra_args) to @attr.s, as in:

@attr.s(extra_args=True)
class Child(NonAttrsClass):
    a = attr.ib()
    b = attr.ib()

    def __attrs_post_init__(self, base_attr):
        super().__init__(self, base_attr)

would cause the following __init__ to be generated:

def __init__(self, a, b, *args, **kwargs):
    ...deal with a and b...
    self.__attrs_post_init__(*args, **kwargs)

This would already cover a great deal of situations where attrs falls short of providing a satisfactory solution.

One could even argue that the option is superfluous: instead, __init__ could just raise if any of args or kwargs is non-empty and __attrs_post_init__ is missing.

[wsanchez]

This is solvable with a custom __init__ as per #393:

@attr.s(extra_args=True)
class Child(NonAttrsClass):
    a = attr.ib()
    b = attr.ib()

    def __init__(self, a, b, c):
        self. __attrs_init__(self, a, b)
        # ... do something with c ...

Which also has the advantage of not having an opinion in attrs about how to solve this: if the default behavior doesn't work for you, write your own constructor.

[pbourguignon]

Thanks for the prompt reply, this is bringing me much further already. Much appreciated!

This solution has one drawback however: it makes me repeat myself. Each attribute is now listed three times (declaration, __init__ signature, __attrs_init__ signature) instead of one, although attrs knows what it needs already. The above proposal builds upon a feature of Python syntax that overcomes just that in a very neat manner.

Also, I don't see the above proposal as an opinionated solution to the specific problem of non-attrs class hierarchies (in particular, it does not allow to invoke the base class constructor before attrs-generated __init__ runs), but rather as a way to pass more information to the constructor than just attribute initialization values, without attrs taking any responsibility for this additional bit. Or to put it differently, it could be a very generic way of passing arguments to __attrs_post_init__ without bloating the class namespace.

[wsanchez]

Yeah… I'm just wonder out loud about how many variations of how one could change __init__'s handling of args we want to build in versus just allowing one to take control for any and all corner cases.

[wsanchez]

Back to @pbourguignon's proposal, would it be feasible to just notice extra arguments in the signature for __attrs_post_init__? As in:

@attr.s()
class Child(NonAttrsClass):
    a: int
    b: str

    def __attrs_post_init__(self, x: bytes, y: bool = False):
        super().__init__(self, x, y=y)

Which could be use used thusly:

child = Child(1, "foo", b"")

assert child.a == 1
assert child.b == "foo"
assert child.x == b""
assert not child.y

[pbourguignon]

I think there is a possibility to keep it stupid simple by generally changing the attrs-generated __init__ as follows:

def __init__(self, attr1, attr2, *args, **kwargs):
    ...do what it already does...
    if hasattr(self, __attrs_post_init__):
        self.__attrs_post_init__(*args, **kwargs)  # This raises already on invalid/missing arguments
    elif len(args) + len(kwargs) > 0:
        raise ArgumentError("Extra arguments passed to __init__, but not used.")

This seems generic enough so I wouldn't expect further variation to be called for. It still does not cover the case where a base class needs to be initialized before attrs-generated __init__ is executed, but that one just calls for a custom __init__.

On the other hand, the solution you suggest has the added benefit of __init__ signature being complete. But that comes at some additional development cost.

[pbourguignon]

As for your question:

would it be feasible to just notice extra arguments in the signature for __attrs_post_init__

I'd say inspect has all you need for that.

[wsanchez]

On the other hand, the solution you suggest has the added benefit of __init__ signature being complete.

I think that's pretty important for typing.

[pbourguignon]

On the other hand, the solution you suggest has the added benefit of init signature being complete.

I think that's pretty important for typing.

Yes, it is, and definitely calls for the more elaborate solution you outlined.
I think I'll give it a go.

[sinback]

@pbourguignon just wondering, did you make any progress on this? I would definitely use this feature if it existed! The __post_init__ hook is a really compelling part of the dataclass implementation -- but right now I'm stuck writing grosser classes by hand thanks to another limitation of dataclasses that is solved in attrs...

[pbourguignon]

@sinback Sorry, no, not yet. I settled on a custom init solution for my current needs.

[indigoviolet]

@hynek - two years after the "on hold" status - have your thoughts changed about this feature? I was recently looking for it.

[hynek]

I'm squarely in the "named constructor" / "classmethod constructor". I find the amount of complexity this would introduce (especially taking into account subclassing) just isn't worth it.

[QuentinSoubeyran]

For those interested, I had the same problem where __attrs_post_init__ needed a context. So I wrote some helpers and a @post_init decorator (source code) that generates the new __init__() by merging the parameters of __post_init__() and __init__(), as suggested by wsanchez.

The decorator saves writing a custom __init__() and properly handles type annotations and defaults of __init__() and __post_init__(). It does not handle inheritance; based on dataclasses handling of InitVar, it might be sufficient to traverse the reversed mro and merge the __post_init__ signature on top the the current result, but I had no need for that.

This might not the best solution for my particular use-case, but it might be useful to others.

[sagittarian]

I'm squarely in the "named constructor" / "classmethod constructor". I find the amount of complexity this would introduce (especially taking into account subclassing) just isn't worth it.

I agree that this approach seems like a clean solution in many cases. The problem I have with this approach is that most of my arguments need to be passed from the classmethod constructor to the default __init__, so I have to either repeat all the arguments (which defeats most of the benefit of using attrs), or use **kwargs, which loses type checking.

[hynek]

I don't disagree, I just haven't seen a proposal where the outcome would justify the induced complexity. :-/

[RonnyPfannschmidt]

I'd like to note that even with the handwritten approach init only arguments have horrendous maintenance costs over time

In pytest the overly tricky collection node constructors are a major showstopper for direly needed refactorings we want to land since well before 2016

[A5rocks]

I'm not entirely sure how attrs could even best support inheritance -- it seems dataclasses just kinda throws its hands up and hopes for the best

>>> from dataclasses import dataclass, InitVar
>>> @dataclass
... class C:
...     x: InitVar[int]
...     def __post_init__(self, x):
...         print(x)
...
>>> @dataclass
... class D(C):
...     x: int
...
>>> D(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 4, in __init__
TypeError: C.__post_init__() missing 1 required positional argument: 'x'