Use 'large crop' of satellite image to provide a wider geographical context

[JackKelly]

Some of @jacobbieker's models include a low-res satellite image of a wider geo region. And this trick is used in the MetNet paper, too. And a wider view of the satellite imagery is probably necessary for forecasting PV power for entire GSP regions, too.

@jacobbieker, how large are your 'large' crops?

[jacobbieker]

They are around 768km in physical size, 256x256 with 3km pixels for a center prediction area of between 48 and 96km square, depending on the model.

[jacobbieker]

For context, MeNet uses 1024km squares to predict the center 64x64km. They do this for 8 hours ahead though, since we are more focused on shorter timescales, smaller crops should be sufficient.

[JackKelly]

Cool, thank you! And, do you down-sample the 'large crop' as a pre-processing step? Or do you give the full-res 'large crop' to a CNN front-end, and let the CNN down-sample the image?

[jacobbieker]

I do it in the CNN, here: https://github.com/openclimatefix/metnet/blob/main/metnet/layers/Preprocessor.py You could do it beforehand and it'd probably save some computation, but this meant I didn't need to change the inputs vs other models. The preprocessing stage doesn't have any CNN parts to it, just pooling and space2depth layers. There is a CNN downsampler though which then works on the downscaled imagery: https://github.com/openclimatefix/metnet/blob/75e743423d84b1661296bafa83e9188a26ff26f8/metnet/metnet.py#L10

[JackKelly]

OK, cool, thanks! In which case - instead of having separate, pre-processed 'centre crop' and 'large crop' features in each example - maybe we should just always have the satellite imagery as be a large crop (~768km x 768km) and let the model decide what to do with the image?

[jacobbieker]

Yeah! I think that's a good way of doing it, gives us a lot of flexibility then

[JackKelly]

I think this is implemented now, I think (unless I've misunderstood?). Closing.

Related issue: openclimatefix/nowcasting_dataloader#27