cmd/go: document if the go test and build cache is threadsafe

[alvaroaleman]

What version of Go are you using (go version)?

$ go version
go version go1.15rc1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

go help cache

What did you expect to see?

A sentence telling me if the go build and test cache is safe for parallel use by multiple invocation of the go binary

What did you see instead?

[davecheney]

I don’t think it is

[ALTree]

I know for sure that cleaning the build cache (go cache -clean) while building something is not safe. This is discussed in #31948, which also functions as a TODO for possibly making it safe.

[ALTree]

cc @bcmills @jayconrod

[alvaroaleman]

I know for sure that cleaning the build cache (go cache -clean) while building something is not safe.

Okay but what about multiple parallel go build or go test invocations?

[jayconrod]

Okay but what about multiple parallel go build or go test invocations?

This is safe on most file systems. Files are added to the cache atomically, and a global file lock is held when files are evicted. If the file system does not support file locking or atomic renaming, then it may not be safe.

[bcmills]

FWIW, the build and test cache does not use file locking, and I don't think it relies on atomic renaming either. (Instead, it writes the expected lengths in metadata files, and overwrites and appends to partial files in-place rather than truncating or deleting them.)

[bcmills]

I take that back: we use file-locking in one place in the test cache, to store the expiration date for stale tests.

go/src/cmd/go/internal/test/test.go

Lines 609 to 613 in 6a167c7

	// Read testcache expiration time, if present.
	// (We implement go clean -testcache by writing an expiration date
	// instead of searching out and deleting test result cache entries.)
	if dir := cache.DefaultDir(); dir != "off" {
	if data, _ := lockedfile.Read(filepath.Join(dir, "testexpire.txt")); len(data) > 0 && data[len(data)-1] == '\n' {

[networkimprov]

@alvaroaleman are you hoping to do simultaneous go build/go test runs of the same project, or different ones, or partially overlapping ones?

[alvaroaleman]

@networkimprov the question came up in the context of "We prepopulate and then use one cache per machine that runs many concurrent builds"

[networkimprov]

If prepopulated, is your cache read-only?

[alvaroaleman]

If prepopulated, is your cache read-only?

Its not my cache and I believe its not

[seankhliao]

Duplicate of #26677