GoPeopleRedoModuleVersions

By admin
I’ll start with an illustration.

; cd /tmp ; git clone https://github.com/golangci/golangci-lint ; cd golangci-lint ; git checkout ab3c3cd6 ; cd cmd/golangci-lint ; go build [succeeds with no error] ; go clean -modcache ;

GOPROXY
ORG

=direct go build […] verifying github.com/butuzov/[email protected]: checksum mismatch downloaded: h1:QXLHriOCzRI8VN9JPhfDcaaxg3TMFD46n1Pq6Wf5zEw=

go.sum
PERSON

: h1:w5Ks4tnfeFDZskGJ2x1GAkx5gaQV+kdU3NKNr3NEBzY= SECURITY ERROR This download does NOT match an earlier download recorded in

go.sum
ORG

. The bits may have been replaced on the origin server, or an attacker may have intercepted the download attempt. For more information, see ‘go help module-auth’.

(The particular Git commit is the current one; I’m specifying it because this whole situation will hopefully change in the future.)

Experienced Go developers know what is going on here; it’s a variant of the

half
CARDINAL

missing import. At some point the developer of the

ireturn
FAC

module released a v0.2.1, then changed their mind and re-released a different thing as v0.2.1. During the time in the middle (sort of), golangci-lint updated to ‘[email protected]’, saved the checksum in its

go.sum
ORG

, and caused the ‘v0.2.1’ module to be fetched through the (default) Go module proxy (possibly as part of running CI or

dependabot
PERSON

tests), which cached it. Now anyone who fetches ‘[email protected]’ through the default Go module proxy gets

one
CARDINAL

version, which is the version golangci-lint requires, and anyone who fetches the real version directly gets a different version, which the Go tooling refuses to accept.

(Or perhaps the

first
ORDINAL

version of [email protected] was cached in the Go module proxy before golangci-lint even noticed that it had been updated, and everything was done with and against that cached copy.)

You can say that this isn’t supposed to happen (

the Go Module Reference
WORK_OF_ART

talks about how a ‘version’ is supposed to identify an immutable thing, emphasis mine, for example). Unfortunately we live in the real world, where it does, as we see here. Possibly the Go documentation doesn’t write strongly enough that once you’ve released a given version you can never change what it is, even if you only released it for a day or even an hour. But even then people would likely keep on doing it.

(Note that the window in the middle can be very small. All you need is

one
CARDINAL

person or automated system to fetch the

first
ORDINAL

version through the Go module proxy in order to cause the Go module proxy to freeze on the

first
ORDINAL

version of the release. You might have published the

first
ORDINAL

version for

only a few minutes
TIME

, but if it’s the wrong few minutes, things get stuck. This is likely counterintuitive, since we seem to have a general feeling that we can fix mistakes if we act sufficient quickly, hastily grabbing our mistake back.)

When this happens all of the results are bad. For example, the Git version of golangci-lint has been depending on a module you could only get from the cache of the Go module proxy for

over ten days
DATE

now, and probably no one has realized (and the Go module cache doesn’t promise to cache all module versions forever). Also, the real version of v0.2.1 isn’t actually being used by anyone who uses the Go module proxy; it may be released in its upstream repository, but on the module proxy it’s hidden by the previous v0.2.1, and its developer may be none the wiser about this. I doubt any of the parties involved intended any of these effects, and I think that part of the issue is that these problems are hard to notice by default.

I strongly believe that

one
CARDINAL

thing that would help this overall situation is if every Go project with

CI
ORG

periodically built itself directly against all dependency modules, bypassing both any Go module proxy and the local Go module cache. This would at least detect missing or changed dependencies (direct and indirect), and get people to resolve the situation

one
CARDINAL

way or another. If you have a Go project or routinely (re)build Go things that you depend on, I suggest that you consider doing this periodically. Otherwise someday you may get an unpleasant surprise.

(To be clear, there is no automatic solution possible for this. Go has the

go.sum
ORG

database of module checksums and module authentication in general for very good reasons and you never want to automatically override its view of things.

One
CARDINAL

way or another the projects involved need to take manual steps to resolve the situation; here that might be falling back to the prior version of the ‘ireturn’ module, which I believe is consistent.)