Support passing Accelerator objects to the accelerator flag with devices=x

[kaushikb11]

🐛 Bug

Expected behavior

As we are walking down the path to support #10410, it is important to support this behavior.

trainer = Trainer(accelerator=GPUAccelerator(CustomStrategy()), devices=4)

# should be equivalent to

trainer = Trainer(accelerator="gpu", devices=4, strategy=CustomStrategy())

Environment

• PyTorch Lightning Version (e.g., 1.3.0):
• PyTorch Version (e.g., 1.8)
• Python version:
• OS (e.g., Linux):
• CUDA/cuDNN version:
• GPU models and configuration:
• How you installed PyTorch (conda, pip, source):
• If compiling from source, the output of torch.__config__.show():
• Any other relevant information:

Additional context

cc @tchaton @justusschock @kaushikb11 @awaelchli @Borda

[ananthsub]

I disagree with this proposal. #10416 is going the opposite route: the strategy should own the accelerator, not the other way around. @kaushikb11 it'd be good for us to sync up along with @awaelchli and @four4fish to ensure we're all aligned

[kaushikb11]

@ananthsub Noted! The example was considering the current state.

The core part, it needs to support is

trainer = Trainer(accelerator=GPUAccelerator(), devices=4)

# should be equivalent to

trainer = Trainer(accelerator="gpu", devices=4)

Also, this will be a small step towards supporting pluggable custom accelerators (which is not possible right now). Will write a RFC for it tomorrow.

trainer = Trainer(accelerator=NewSOTAAccelerator(), devices=4)

[awaelchli]

I agree this is a bug. This works today:

trainer = Trainer(accelerator=NewSOTAAccelerator(), gpus=4)

Whether gpus or devices are supplied should not matter.

[ananthsub]

@kaushikb11 what remains to be done here following #12030 ?

[kaushikb11]

@kaushikb11 what remains to be done here following #12030 ?

Yup, this issue has been resolved.