More resilient gpg getting

[yosifkit]

• move to "high-availability" subset.
• do keys one at a time to ensure every key imports

[tianon]

👍 LGTM

To add a little more context for why this PR is happening, gpg --recv-keys key1 key2 will only fail (non-zero exit code) if both key1 and key2 failed to fetch, so every now and then we get an intermittent keyserver issue that causes one or two keys to fail but the command succeeds so it's baked into our cache with missing keys and the build fails repeatedly after that. This is the reason for the other part of this where we switch to just high-availability servers and fetch individual keys in a loop. 🤘

[md5]

Is there any reason to have GPG_KEYS in a separate ENV instead of just putting it into the RUN statement?

[tianon]

One reason is that it makes updating it via "update.sh" simpler (like we do here). Another is that it adds an extra layer of simple accountability making it easy for others to verify the value (ie, scan the list and make sure it matches what upstream publishes, if/when they do actually publish the list).

[tianon]

Are there good reasons not to do so? (to play a little devil's advocate 😄)

[md5]

Those are good reasons. I didn't scroll all the way down to the update.sh changes 🙈

I guess I'm just still in a scarcity mindset regarding the number of layers in images.

[tianon]

😄 Please don't stop making helpful, polite comments like this -- having to be more long-winded about reasons for doing things is ++++ and usually leads to either more information about the why behind the changes being publicly posted (which is good) or an improved process in general (which is even better). 👍

[yosifkit]

Step 6 : RUN set -xe    && for key in $GPG_KEYS; do         gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key";  done
 ---> Running in b918241a27d6
+ gpg --keyserver ha.pool.sks-keyservers.net --recv-keys 6E4F6AB321FDC07F2C332E3AC2BF0BC433CFC8B3
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key 33CFC8B3 from hkp server ha.pool.sks-keyservers.net
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key 33CFC8B3: public key "Ferenc Kovacs <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
+ gpg --keyserver ha.pool.sks-keyservers.net --recv-keys 0BD78B5F97500D450838F95DFE857D9A90D90EC1
gpg: requesting key 90D90EC1 from hkp server ha.pool.sks-keyservers.net
gpg: key 90D90EC1: public key "Julien Pauli <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
 ---> b1cfedd17311

[tianon]

Pretty, 😍!

[tianon]

not ha.

[tianon]

Those missing ha.s are hilariously why the Travis build failed. 👍

+ gpg --keyserver pool.sks-keyservers.net --recv-keys F38252826ACD957EF380D39F2F7956BC5DA04B5D
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key 5DA04B5D from hkp server pool.sks-keyservers.net
gpgkeys: key F38252826ACD957EF380D39F2F7956BC5DA04B5D can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
INFO[0180] The command [/bin/sh -c set -xe  && for key in $GPG_KEYS; do         gpg --keyserver pool.sks-keyservers.net --recv-keys "$key";     done] returned a non-zero code: 2 

[tianon]

LGTM