Avoid reading input files to memory

[jku]

The API uses bytes as type for all file contents. This is fine for certs and signatures, but the input files can be arbitrarily large: the API should not force reading input files to memory all at once.

[woodruffw]

I agree that we should avoid reading inputs fully into memory, although I think we're actually limited by our dependencies in terms of actually doing this. For example, we use cryptography's EllipticCurvePrivateKey.sign, which only takes a bytes as input (not a file-like object).

We could maybe work/hack around that by using mmap and bytearray, but that requires investigation (and a confirmation from upstream that they intend to support non-bytes).

[jku]

oh I did not realize that -- this is disappointing from cryptography (I suppose the actual process still works with chunks at a time).

If it's an upstream issue then I'm fine with closing this as well: I expected this to be an easy fix not an issue with a dependency.

[woodruffw]

I'm okay with keeping it open for now! I think we should investigate the feasibility of using bytearray, and I can chat with the cryptography maintainers to see if they have any insights here.

[woodruffw]

One of the cryptography maintainers pointed out to me that ECDSA signing is defined by signing over the digest of the input rather than the full input, and that cryptography's API's actually supports pre-hashing the input and passing that in instead of the full bytes.

As a result, this should actually be possible: we can stream the input into the digest function instead, and then pass Prehashed(SHA256()) during signing instead of having the signing operating do the hash internally!

Relevant APIs: https://cryptography.io/en/latest/hazmat/primitives/asymmetric/utils/#cryptography.hazmat.primitives.asymmetric.utils.Prehashed

[woodruffw]

Doing this in #329.