extend Enum api

[Borda]

🚀 Feature

it may be nice to have some general calls over our enums such as Distributed mentioned by @carmocca in #5300 (comment)

Any logic to compare different DistributedTypes should be encapsulated by the enum itself.

• PoC: Accelerator refactor #5743 (comment)
• PoC: Accelerator refactor #5743 (comment)
• Ensure accelerator is valid if running interactively #5970 (comment)

Motivation

easier handling some cumulative behaviours, such is_ddp which cover DDP and DDP_SPAWN

Pitch

class DistributedType(LightningEnum):
    DP = 'dp'
    ...

    def is_distributed(self):
        return self in (DistributedType.DDP, DistributedType.DDP_SPAWN, DistributedType.DDP2)

[awaelchli]

I would like to do a comment/counter-pitch here.
If we had a strict, finite set of training types the enum with the is_distributed etc. functionality would be a great solution.
However, users can pass in their own plugins and they will not fall into the fixed categories
(DistributedType.DDP, DistributedType.DDP_SPAWN, DistributedType.DDP2)

I suggest that we keep these enum types strictly internal and only use them for argument parsing part where the options are strictly limited.

[carmocca]

Maybe I am missing something but isn't that already a problem? see (random example):

https://github.com/PyTorchLightning/pytorch-lightning/blob/6b990f3fa5646cfbf2e2945daf568a27921b8362/pytorch_lightning/trainer/connectors/accelerator_connector.py#L518

This issue is just about moving those checks from one place to the other (connector to Enum), not if those checks should exist at all

[awaelchli]

Yes, should be fine to move them.
Just wanted to bring awareness of the current limitation

[carmocca]

This might not be relevant after #6090, wdyt @kaushikb11 ?

[kaushikb11]

Interesting, this is something that could be looked into. Like the idea of having it separate from the Accelerator Connector.

[carmocca]

Closing as we are trying to reduce our dependence on Enums, since they limit customization: #10422