Multi process when saving to netcdf

[peterdudfield]

Detailed Description

Use threading/ mulit cpu when saving batch to netcdf

Context

This will speed up saving to netcdf

Possible Implementation

loop here

[JackKelly]

Yeah, I agree... I'm planning to implement this when I rip out the pytorch dataloader, as part of #86 and #213. #202 is particularly relevant.

(nowcasting_dataset should still use mutliple worker processes, thanks to the pytorch dataloader that's still in nowcasting_dataset)

[peterdudfield]

Ah perhaps I just mean really specifically, when Batch.save_netcdf() is called, it uses multi process

[JackKelly]

Good point! That should happen "naturally" as part of #202 (and, in 202, we can use multiple processes per modality)

[peterdudfield]

Yea I agree, ive put it in a PR anyway, as it was pretty small and simple. And if it makes a difference then its all good