Fix spawn plugins not deleting temp checkpoint (#10935)

Author: awaelchli

CHANGELOG.md

@@ -19,6 +19,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Fixed an issue with item assignment on the logger on rank > 0 for those who support it ([#10917](https://github.com/PyTorchLightning/pytorch-lightning/pull/10917))
 - Fixed importing `torch_xla.debug` for `torch-xla<1.8` ([#10836](https://github.com/PyTorchLightning/pytorch-lightning/pull/10836))
 
+- Fixed an issue with `DDPSpawnPlugin` and related plugins leaving a temporary checkpoint behind ([#10934](https://github.com/PyTorchLightning/pytorch-lightning/pull/10934))
+
 
 ## [1.5.4] - 2021-11-30
 

pytorch_lightning/plugins/training_type/ddp_spawn.py

@@ -333,6 +333,7 @@ def __recover_child_process_weights(self, best_path, last_path):
         if last_path is not None and self.lightning_module.trainer.state.fn == TrainerFn.FITTING:
             ckpt = pl_load(last_path, map_location=lambda storage, loc: storage)
             self.lightning_module.load_state_dict(ckpt)
+            self.checkpoint_io.remove_checkpoint(last_path)
 
     def barrier(self, *args, **kwargs) -> None:
         if not distributed_available():