Cross-compilation produces different binaries

[derekperkins]

1. What version of Go are you using (go version)?
   1.6
2. What operating system and processor architecture are you using (go env)?
   darwin amd64 & linux amd64
3. What did you do?
   I cross-compiled a 20MB binary on darwin for linux, which I run inside a docker scratch container. I also compiled the same program with the same dependencies on a Ubuntu 14.04 machine.
4. What did you expect to see?
   I expected the binary to be the same and execute the same inside Docker.
5. What did you see instead?
   The binary cross-compiled on darwin worked just fine inside the scratch container, but the Ubuntu compiled binary exited saying that a file was not found. When running the "broken" binary in a docker container with a debian base image, it then ran.

I am not using cgo or anything similar. It's a fairly large $work application, so I can't easily reproduce it, but this behavior has been tested across multiple machines and environments. It seems as though something is being statically linked into the darwin binary that is not being linked in the Ubuntu compiled binary.

[davecheney]

Can you please run ldd $BINARY in both the "broken" and the "working" versions. My hunch is this is a dynamically linked program and the dynamic loader, ld.so is in different locations on different systems.

When you says "exited saying that a file was not found" do you mean ./$BINARY acted as if $BINARY was not there?

[davecheney]

Btw, by default, unless you have taken explicit action, cross compiling will produce a static binary, compiling locally on linux for linux will produce a dynamically linked binary.

[bradfitz]

Sounds like the docker container doesn't have libc and the binary is not static.

[derekperkins]

@davecheney & @bradfitz - you're correct, it was an issue with it dynamically looking for a library. I wasn't aware that the stdlib (I think net/http) used cgo by default on Linux. I misread the error message as it saying that "$BINARY was not there", when the error was actually emitted by Go itself.
no such file or directory is in syscall/zerrors....

After finding out that Linux code isn't actually statically linked, I found a lot of threads and projects that got bit by this same error. It would have been much easier to track down if there had been better logging, letting me know that a dependency wasn't found, rather than a generic no such file or directory.

Thanks for helping me track that down!

[derekperkins]

For reference, #9344