Deprecate the BasePlugin

[ananthsub]

🚀 Feature

Deprecate the BasePlugin interface
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/plugins/base_plugin.py

cc @four4fish

Motivation

We are auditing the Lightning components and APIs to assess opportunities for improvements:

• https://docs.google.com/document/d/1xHU7-iQSpp9KJTjI3As2EM0mfNHHr37WZYpDpwLkivA/edit#
• Review Lightning architecture & API #7740

The BasePlugin initially started out as a unifying interface for the training type and precision plugins in lightning. however, both the training type and precision plugins have since evolved to be much broader in scope. almost all the new functionality has been added directly to these classes. it no longer makes sense to have a joint interface across these two classes, especially as we seek to evolve these independently.

Rather, the trend we're seeing is to have much narrower interfaces for specific functionality. Examples here are:

• Cluster Environments (don't follow the plugins base class)
• Checkpoint IO Plugin (don't follow the plugins base class)

Upcoming:

• Collectives (which won't follow the plugins base class): Consolidate collective functions #7534

Longer term, a promising direction is that the training type plugin (or whatever its evolution is) acts as a composition of components implementing these interfaces

Pitch

• Remove all subclass dependencies of the base plugin
• Deprecate the base plugin in v1.5
• Remove this method in v1.7

@awaelchli @tchaton @SeanNaren @justusschock as the plugin API is still in beta, does the 2 minor version cadence for deprecations still apply?

Alternatives

Additional context

---

If you enjoy Lightning, check out our other projects! ⚡

• Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

• Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning

• Bolts: Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

• Lightning Transformers: Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

[justusschock]

@ananthsub I think we can do it as you proposed in your pitch. The idea was more to have an unified interface for those, but maybe we should allow them to differ.

Re the deprecation progress: I think the approach you mentioned in your pitch makes sense, since this is still in beta (likely not much longer though) but on the other hand it is one of the framework's core components so even the beta parts should be stable :)

[awaelchli]

Sounds good! I forgot this even exists 😅