cmd/go: get -d k8s.io/kubernetes does not say what has happened

[ysmolski]

What version of Go are you using (go version)?

$ go version
go version devel +d21c7b7282 Wed May 8 19:40:38 2019 +0000 darwin/amd64

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/thorn/Library/Caches/go-build"
GOENV="/Users/thorn/Library/Preferences/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/thorn/p"
GOPROXY="direct"
GOROOT="/Users/thorn/g"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/Users/thorn/g/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/km/50gy6_q557v6_7vxbf9b29v00000gn/T/go-build891423326=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Both of these commands:

$ cd ~ && go get -d k8s.io/kubernetes
or
$ cd $GOPATH && go get -d k8s.io/kubernetes

What did you expect to see?

Some message of what has happened.

What did you see instead?

Something happened and no helpful message indicated what. I expected to see the repo from k8s.io/kubernetes to appear in my $GOPATH, but that never happened. And since it is the breaking change in go get I was triple confused, since there was not output at all.

I also tried to see verbose output to get a clue what happened, but the output was not helpful either:

$ go get -d k8s.io/kubernetes
Fetching https://k8s.io/kubernetes?go-get=1
reading from https://k8s.io/kubernetes?go-get=1: status code 200
get "k8s.io/kubernetes": found meta tag get.metaImport{Prefix:"k8s.io/kubernetes", VCS:"git", RepoRoot:"https://github.com/kubernetes/kubernetes"} at //k8s.io/kubernetes?go-get=1

[ysmolski]

I am sorry. I totally missed that go 1.13 is indeed making a breaking change to the behaviour of go get, but my confusion is still there. For that particular repo there is no output from go get -d.

[bradfitz]

/cc @bcmills @jayconrod

[jayconrod]

Did go get actually fail? That is, does it exit with a non-zero status?

In your environment, it looks like you're running in module mode. When successful, go get should not print anything, except possibly some finding, extracting, downloading messages. Modules will be downloaded to $GOPATH/pkg/mod, not $GOPATH/src.

If go get is failing silently and not downloading modules to the cache, that would be a problem. I wasn't able to reproduce this with a recent master though. Do you have any other network or proxy settings that might prevent go get from working?

[josharian]

I run into this regularly. Speaking for myself, ‘go get -d’ succeeds...but my intent was to get the code in question so that I could play with it and read it locally. I was confused the first few times when I didn’t find it it my GOPATH. Now I sigh, and re-run with GO111MODULE=off.

Is there a convenient way in module mode to just grab the code to mess around with it? (Note that ‘git clone’ is not an answer, because ‘go get’ understands things that are difficult to resolve manually—meta tags for custom domains, VCS resolution.)

If not, maybe this issue is a feature request for that, somewhat analogous to the feature request for a global install command. (I’m on a phone so I don’t have the issue number handy.)

[ysmolski]

I second what Josh said. I want to bring your attention to the interface. As for my case, the problem with go get interface is that it's behavior has dramatically changed, and yet there are no clues in the output of it, nor in verbose version of the output. It is a bad interface. What is the point of -v option in the get command?

I understand that many people here already get used to the new behavior of go get. But what about the rest of community, some of them might get confused too. Can we improve their experience?

[ysmolski]

@josharian meant #30515. This is related too: #31529

[bcmills]

Is there a convenient way in module mode to just grab the code to mess around with it?

That's arguably one of the use-cases for go mod vendor, assuming that you don't want to retain (or upstream) any local changes.

[bcmills]

the problem with go get interface is that it's behavior has dramatically changed, and yet there are no clues in the output of it, nor in verbose version of the output.

That's true, but I wonder how quickly folks will get used to it: go get in module mode in general does not download dependencies to GOPATH.

I suppose that we could print some sort of informational message after a go get — for example, if we are in module mode, outside of any module, and did not install any binaries as a result of the command. (On the other hand, the go command generally follows the Unix tradition of not printing anything on success. We've diverged from that somewhat in module mode already, but I hope that we can dial that back down at some point.)

[thepudds]

(Note that ‘git clone’ is not an answer, because ‘go get’ understands things that are difficult to resolve manually—meta tags for custom domains, VCS resolution.)

There is also #18387 ("cmd/go: provide a way to resolve an import path to a vcs, url, etc").

If the go command provided a simple way to do that, I would guess someone from the broader community would build a basic utility (perhaps called something like gopath) that enabled something like gopath get <importpath>[@version] that dropped the source code into GOPATH in the old standard location.

Alternatively, I suppose one could try to build something like that utility even now by doing something like:

1. create a temporary module
2. optional: create a temporary GOPATH
3. do an actual go get <importpath><@version> from within the temporary module
4. copy the code out of GOPATH/pkg/mod to the old standard GOPATH location.

(Perhaps a bonus would be an option for first determining the current version of the dependency in the current module via go list so that you don't need to type a version yourself if you want something other than @latest).

[bcmills]

If the go command provided a simple way to do that, I would guess someone from the broader community would build a basic utility (perhaps called something like gopath) that enabled something like gopath get <importpath> that dropped the source code into GOPATH in the old standard location.

Does @rogpeppe's gohack not already do something like that? (It seems like the obvious behavior for gohack get when executed outside of a module, but perhaps I'm missing something.)

[thepudds]

gohack puts it HOME/gohack/.

I think gohack also relies on the code being in GOPATH/pkg/mod (and doesn't get it on its own), but not 100% sure.

But gohack could change, of course.

[josharian]

That's arguably one of the use-cases for go mod vendor, assuming that you don't want to retain (or upstream) any local changes.

That assumes you're working on a particular project. I'm usually kicking the tires/exploring something, not necessarily within a project. I could invent one, I suppose, and then make a Go file and add an import...but that seems like a lot of mechanism for something that used to be a one-liner that I used all the time.

Does @rogpeppe's gohack not already do something like that? (It seems like the obvious behavior for gohack get when executed outside of a module, but perhaps I'm missing something.)

$ GO111MODULE=on gohack get github.com/rogpeppe/gohack
cannot determine main module: cannot read current go.mod file: open : no such file or directory

Right now, it appears that gohack doesn't function outside a module. One could, of course, add it.

But again, this used to be a frequent use built-in one-liner. I'm not sure that kicking it to external tools is the right idea.

[lfaoro]

Instead of breaking the accustomed behavior of go get https://repo to download the source in $GOPATH/src and having to resort setting a variable in order to restore it $ GO111MODULE=off go get https://repo, I suggest we provide a flag for our users which will let them easily download any public Go source as they are accustomed to.

• GO111MODULE flag might disappear from future releases.
• it's not very friendly towards the users, having them resort to use this flag

from go help get:

The -d flag instructs get to download the source code needed to build
the named packages, including downloading necessary dependencies,
but not to build and install them.

Before it downloaded to $GOPATH/src now it downloads to $GOPATH/pkg/mod -- although, now I can't contribute to the downloaded code, have to manually clone.

Even using go mod by default, allow go get -d to download the source in $GOPATH/src as everybody is already accustomed to.

There's thousands of Go repos out there, with README.md docs instructing users to:

• step 1: go get -d https://repo
• step 2: cd $GOPATH/src/repo
• step 3: make install

All those instructions are now broken when a user has GO111MODULE=on. In the end devs figure it out anyway, they'll use the GO111MODULE=off setting before go get or just clone the repo as usual; although it's not as convenient as it was, would be very nice if we could restore this particular convenience, which we had with Go.

[bcmills]

I expected to see the repo from k8s.io/kubernetes to appear in my $GOPATH

Given how long modules have been in use, I suspect we've missed the window of opportunity to smooth out this migration.

I think the remaining rough edge in the user experience is covered by #31529.