build: bootstrap.bash builds a lot of useless stuff

[bradfitz]

While cross-compiling ARM for faster builders (#17105) in a mode similar to src/bootstrap.bash, I see many useless things being built:

net/http (http2, hpack, httplex, ...)
A bunch of packages never needed by cmd/compile, cmd/link, etc.

It'd be nice to have a mode which only built the bare minimum to build the cross-compiled tools.

[bradfitz]

For example, I'm thinking of something like this, behind a flag or environment variable perhaps:

diff --git a/src/make.bash b/src/make.bash
index 1a1412a..7db2123 100755
--- a/src/make.bash
+++ b/src/make.bash
@@ -165,7 +165,7 @@ if [ "$GOHOSTARCH" != "$GOARCH" -o "$GOHOSTOS" != "$GOOS" ]; then
        # CC_FOR_TARGET is recorded as the default compiler for the go tool. When building for the host, however,
        # use the host compiler, CC, from `cmd/dist/dist env` instead.
        CC=$CC GOOS=$GOHOSTOS GOARCH=$GOHOSTARCH \
-               "$GOTOOLDIR"/go_bootstrap install -gcflags "$GO_GCFLAGS" -ldflags "$GO_LDFLAGS" -v std cmd
+               "$GOTOOLDIR"/go_bootstrap install -gcflags "$GO_GCFLAGS" -ldflags "$GO_LDFLAGS" -v cmd/compile cmd/link cmd/asm
        echo
 fi

That seems to cut down user CPU from 3.5 minutes to 2.5 minutes, and sys CPU from 14.5s to 11.5s.

Note sure if cmd/compile cmd/link cmd/asm is sufficient, though.

/cc @ianlancetaylor @minux @mdempsky

[minux]

I think cmd/go is required too?

[bradfitz]

@minux, doesn't seem to be. cmd/go is needed by bootstrap.bash (for avoidable reasons), but isn't needed by make.bash, which the builders use.

The cross-compiling builder does this instead of using bootstrap.bash:

        remoteErr, err := kubeBC.Exec("/bin/bash", buildlet.ExecOpts{
                SystemLevel: true,
                Args: []string{
                        "-c",
                        "cd $WORKDIR/go/src && " +
                                "./make.bash && " +
                                "cd .. && " +
                                "mv bin/*_*/* bin && " +
                                "rmdir bin/*_* && " +
                                "rm -rf pkg/linux_amd64 pkg/tool/linux_amd64 pkg/bootstrap pkg/obj",
                },
                Output: st,
                ExtraEnv: []string{
                        "GOROOT_BOOTSTRAP=/go1.4",
                        "CGO_ENABLED=1",
                        "CC_FOR_TARGET=arm-linux-gnueabihf-gcc",
                        "GOOS=" + goos,
                        "GOARCH=" + goarch,
                        "GOARM=7", // harmless if GOARCH != "arm"                                                                      
                },
                Debug: true,
        })

And I see no reason why the linux_amd64 host needs cmd/go, any networking packages (we should not use the network during the make.bash build),cmd/vet`, etc.

[minux]

Interesting. If I'm reading it correctly, make.bash is using $GOROOT_BOOTSTRAP/bin/go to build the bootstrapping cmd/dist and compiler. Am I missing something?

[bradfitz]

/cc @mdempsky for thoughts.

[bradfitz]

I added a minimal output mode in f0f62fc for the Go builders. I'm happy enough with that for now.