The gopkg.in/yaml.v3 dependency is a build risk

[rix0rrr]

Hi, I'm from the AWS CDK team.

Your dependency on gopkg.in/yaml.v3 involves an external site: gopkg.in. As of today, any attempt to fetch a package from this site fails with the error:

go: gopkg.in/[email protected]: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /go/pkg/mod/cache/vcs/9241c28341fcedca6a799ab7a465dd6924dc5d94044cbfabb75778817250adfc: exit status 128:
    error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway
    fatal: The remote end hung up unexpectedly

This site seems to be run by a single person called Gustavo Niemeyer. It doesn't look like there is an SLA on this website, and there is no reasonable expectation of this person investing timely effort into fixing their site. In the mean time, any build involving aws-lambda-go will fail.

Currently for us, since our PR validation build (and release pipeline) does a go build involving aws-lambda-go, it looks like we cannot merge any PR and not release any version of CDK because of this.

It seems problematic that we are at the mercy of the volunteer efforts by a single person in order to successfully complete a build involving this AWS-vended library. I would ask you to find replacements for any dependencies that come from gopkg.in. Their presence constitutes an availability risk.

[bmoffatt]

I don't have any objection to removing this indirect dependency, it's brought in for us by github.com/urfave/cli/v2 and it's use in cmd/build-lambda-zip isn't particularly complex.

That said, I wouldn't recommend this as the correct general guidance for aws projects. gopkg.in/yaml is a really common dependency, and also happens to also be direct dependency for more than a dozen other aws open source projects (ref yaml.v2, ref yaml.v3). This package is also far from the only project to have a vanity import url that can result in the type of uptime issues that ya'lls PR workflows saw. The ecosystem solution for ensuring high uptime for build dependencies is to hit a module proxy rather than resolving to origin. This has been the default behavior since go1.13 (ref).

To fix your PR workflows, check first if something like GOPROXY=direct was set to bypass the default proxy. This configuration may not be necessary for your usecase.

[bmoffatt]

Dependency has been removed with https://github.com/aws/aws-lambda-go/releases/tag/v1.30.0