cmd/link: compress debug info on windows

[dajoo75]

What did you do?

Compiled a basic hello world program on 64-bit Windows 10 with go tip (+b459e00), go 1.8.1 and go 1.10.3 and compared the binary sizes.

               hello_go1.8.1.exe  |  hello_go1.10.3.exe  |  hello_+b459e00.exe
binary size    1 633 280 bytes    |  2 057 728 bytes     |  2 848 768 bytes
                                  |                      |
.rdata                     N/A    |             N/A      |    752 128 bytes
.text          1 068 032 bytes    |  1 181 696 bytes     |    575 488 bytes
.debug_info      260 608 bytes    |    396 800 bytes     |    457 216 bytes
.debug_loc                 N/A    |             N/A      |    404 480 bytes
.symtab           98 304 bytes    |    141 312 bytes     |    142 336 bytes
.debug_frame      70 144 bytes    |     75 264 bytes     |     76 288 bytes
.debug_line       68 519 bytes    |     72 192 bytes     |    139 264 bytes
.debug_pubtypes   30 720 bytes    |     42 496 bytes     |     44 544 bytes
.debug_pubnames   17 408 bytes    |     40 960 bytes     |     41 472 bytes
.data             15 360 bytes    |     77 824 bytes     |     78 336 bytes
.debug_ranges              N/A    |     25 600 bytes     |    133 632 bytes
.idata             1 536 bytes    |      1 024 bytes     |      1 024 bytes
.debug_aranges       512 bytes    |              N/A     |              N/A
.debug_abbrev        512 bytes    |        512 bytes     |        512 bytes
.debug_gdb_scripts         N/A    |        512 bytes     |        512 bytes

What did you expect to see?

The binary size to be about the same at tip (+b459e00) as the previous release (go1.10.3 at the time of writing this). And if possible, the binary size to have gone down.

Other related issues: #6853, #11799

What did you see instead?

The binary size at tip (+b459e00) has increased by 38 percent since the previous release (go1.10.3 at the time of writing this).

Does this issue reproduce with the latest release (go1.10.3)?

Yes

System details

go version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000 windows/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="C:\Users\loki\AppData\Local\go-build"
GOEXE=".exe"
GOHOSTARCH="amd64"
GOHOSTOS="windows"
GOOS="windows"
GOPATH="C:\home\projects\go"
GOPROXY=""
GORACE=""
GOROOT="C:\home\installs\go\go.tip"
GOTMPDIR=""
GOTOOLDIR="C:\home\installs\go\go.tip\pkg\tool\windows_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\loki\AppData\Local\Temp\go-build103655438=/tmp/go-build -gno-record-gcc-switches"
VGOMODROOT=""
GOROOT/bin/go version: go version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000 windows/amd64
GOROOT/bin/go tool compile -V: compile version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000
gdb --version: GNU gdb (GDB) 8.0

[ALTree]

As you word it, this would technically be a dup of #6853.

What I think it's worth noting is that CL 118276 (cmd/link: compress DWARF sections in ELF binaries) only helped with ELF binaries. So I think it would be better if you renamed this issue to point out this specific issue (i.e., make this issue a TODO reminder to compress debug info on windows, too).

In theory we had #11799 (cmd/link: compress debug info) for the general problem, but it got closed after the ELF work.

[dajoo75]

@ALTree I've renamed the issue to cmd/link: compress debug info on windows as you suggested.

Others more in the know about Windows, the PE file format and the debug story around Windows (gdb, delve etc.) have to chime in on the correct way forward.

[dajoo75]

I tried to get this working on Windows (made some small changes to cmd/link and, debug/pe) and hello world.exe went from 2.8MB to 1.9MB. I almost got delve to start but didn't have time to finish it.

There doesn't seem to be an insurmountable amount of work left to do to get compression working on Windows as well (since most of the work seems to be done already in #11799) but I may be wrong (the devil is in the details and all that).

I would feel better if the original authors of #11799 looked at this as well (together with Alex) since they know what they are doing :).

cc @heschik, @alexbrainman, @ianlancetaylor, @aclements

[aarzilli]

@dajoo75 what approach are you taking? Elf supports compression but PE doesn't (based on a ctrl-f search in the relevant MSDN page), so .zdebug?

[dajoo75]

Yes .zdebug. That seems to be the way the Go team have solved this. Dwarf sections in a PE file. Package debug/pe has a DWARF-function that delve uses. I’m decompressing the sections inside debug/pe just like the debug/elf solution. The consumer, e.g. delve, only sees uncompressed data, at least that was the idea... but I didn’t have time to finish it. /Daniel

…

18 juni 2018 kl. 08:17 skrev Alessandro Arzilli ***@***.***>: @dajoo75 what approach are you taking? Elf supports compression but PE doesn't (based on a ctrl-f search in the relevant MSDN page), so .zdebug? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[aarzilli]

Ah, I didn't notice the switch to zdebug. In that case: https://github.com/derekparker/delve/pull/1237

[alexbrainman]

Thank you @dajoo75 for creating the issue.

If I understand correctly, CL 118276 added compression for ELF DWARF sections. I would try and add some code to make .zdebug_... sections appear in Windows executables, and see where it gets us. I doubt I would have time before weekend. So others are welcome to this.

I also wonder if debuggers used by our users will support this. Lucky we have @aarzilli here, so he can fix Delve. Not so easy to fix gdb. We will see.

Alex

[aarzilli]

Not so easy to fix gdb. We will see.

I think gdb was the original user of the zdebug convention, I imagine that should be fine. The problem is lldb.

[alexbrainman]

The problem is lldb.

Does lldb work on Windows?

Alex