What's coming in Go 1.15
LWN.net needs you! Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing |
Go 1.15, the 16th major version of the Go programming language, is due out on August 1. It will be a release with fewer changes than usual, but many of the major changes are behind-the-scenes or in the tooling: for example, there is a new linker, which will speed up build times and reduce the size of binaries. In addition, there are performance improvements to the language's runtime, changes to the architectures supported, and some updates to the standard library. Overall, it should be a solid upgrade for the language.
Since the release of Go 1.0, the Go team has consistently shipped improvements to the tooling and the standard library with each version, but has always been conservative about language changes. Many other languages ship significant language features every release, but Go has only shipped a few minor ones in the versions since 1.0.
This is a conscious design choice: since the 1.0 release, the emphasis from the team has been stability and simplicity. The Go 1 compatibility promise guarantees that all programs written for Go 1.0 will continue to run correctly, unchanged, for all 1.x versions. Go programmers usually see this as a good thing — their programs continue to "just work", but generally get consistently faster.
In the upcoming 1.15 version, changes to the language specification are basically non-existent as expected; the improvements are in the tooling, the performance of the compiler, and in the standard library. As tech lead Russ Cox noted, the core developers are planning to be extra-conservative in 1.15 given the pandemic:
[...] Go 1.15 is going to be a smaller release than usual, and that's okay.
On May 1, Go 1.15 entered feature freeze, and the Go team plans to make the final release on August 1, keeping to the regular sixth-month release cycle.
The Go development model is rather different than that of most open-source languages. The language was designed at Google and most of the core developers work there (so ongoing development is effectively sponsored by Google). The language has a permissive, BSD-style license, and development is done in the open, with general discussion on the golang-dev mailing list. Changes or new features are proposed and discussed in the GitHub repository's issues, and code review is done via comments on the Gerrit code changes (called "changelists" or "CLs").
A new linker
One of the largest tooling changes in 1.15 is the completely rewritten linker. The new linker design document, authored by Go core contributor Austin Clements in September 2019, details the motivation for the rewrite and the improvements it will bring. There are three major structural changes in the new linker:
- Moving work from the linker to the compiler: this enables parallelization, as compiles are done in parallel across multiple CPUs (or machines), but the link step almost always has to be done in serial at the end of the build. Additionally, the results of the compiler are cached by the Go tooling.
- Improving key data structures, primarily by avoiding strings. The current linker uses a big symbol table indexed by string; the new design avoids strings as much as possible by using a symbol-numbering technique.
- Avoiding loading all input object files into memory at once: this makes the new linker use less memory for large programs, and allocate less memory overall (the current linker spends over 20% of its time in the garbage collector).
Now that Ken Thompson, author of the original linker, has retired, there's also the matter of maintainability. As Clements put it:
Given the sweeping long-term changes, this work is being done on a branch (dev.link) that is merged into master only at stable points. Than McIntosh, who is working on the new linker, described what has already been done for 1.15: most of the structural improvements in the design document have been completed, including the new object file format and tighter symbol representation. Builds are already faster and use less memory than in 1.14, but some features (for example, using the DWARF 5 debugging format) will have to wait for 1.16.
Clements added more detail on the parallelization efforts, as well as the gradual way the work is being phased in:
For now, the linker still converts the output back to the old in-memory representation for the last part of the linking. Presumably in a future version of Go these last steps will be moved into the new linker and the conversion phase will be removed entirely, reducing link time and memory usage further.
Smaller binaries
Related are several improvements that reduce the size of executables built with Go 1.15. As Brad Fitzpatrick showed, the new linker eliminates a lot more unused code, bringing Fitzpatrick's (rather artificial) test program down from 8.2MB in Go 1.14 to 3.9MB in 1.15. For more realistic programs, binary sizes go down by 3.5% or as much as 22%. A web server program that I run went down from 21.4MB to 20.3MB, a reduction of 4.9%.
The biggest contributors to this are the unused code elimination in the new linker, along with several targeted improvements, such as Clements's CL 230544, which reduces the number of stack and register maps included in the executable. These maps are used by Go's garbage collector (GC) to determine what objects are alive, but are now only needed at call sites, instead of for every instruction. This change reduces the size of the go binary by 5.7%; it also speeds up compiles and links by a significant amount.
Due to Go's ability to inspect types at runtime (using the reflect package), Go binaries contain a significant amount of type information. CL 231397 by Cherry Zhang only includes a symbol's type information in the output if it's converted to an interface (only values converted to an interface can be used with reflection). This change reduces the size of a hello-world program by 7.2%.
There are a few other minor improvements to binary size, such as Brad Fitzpatrick's CL 228111, which avoids including both the TLS client and server code in the output if only one of them is used, reducing the size of a TLS dial hello world program by 3.2%.
Performance improvements
Go 1.15 introduces many minor performance improvements, but two of the more notable ones are from prolific non-Google contributor Josh Bleecher Snyder. CL 216401 avoids allocating memory when converting small integers to an interface value, giving a 2% improvement in compile-to-assembly times. Converting to interface values is like "boxing" in other languages; the optimization is similar in spirit to Python's small integer caching, though it happens in Go far less often due to static typing.
The second of Snyder's changes is CL 226367 in the internals of the compiler and runtime, which allows the compiler to use more x86 registers for the garbage collector's write-barrier calls. Go uses a write barrier (kind of like a lock) to maintain data integrity on the heap when the GC is running concurrently with user code (this detailed analysis of Go's GC has more information). This results in slightly smaller binaries and a 1% improvement in compile times.
Michael Knysze significantly increased throughput of memory allocation for large blocks by redesigning the memory allocator's "mcentral" data structure to reduce lock contention. The new allocation code is more than twice as fast for blocks of 12KB or larger.
Tooling and ports
The Go "modules" feature (Go's dependency management system) was first introduced in Go 1.11, and support for a module mirror or "proxy" was added in 1.13. Version 1.15 adds support for a fallback proxy, allowing the go tool to fall back to a secondary host if the first one fails when downloading module source code. Fallbacks are specified using the GOMODCACHE environment variable's new "|" separator.
Go 1.15 removes two older ports: darwin/386 and darwin/arm, which provided 32-bit binaries on macOS and other Apple operating systems. Fitzpatrick notes that macOS Catalina doesn't support running 32-bit apps, so removing those ports will help free up macOS build machines as well as shrinking the compiler slightly. These ports were announced as deprecated in the Go 1.14 release, and will be removed in Go 1.15.
On the other hand, the linux/arm64 port was upgraded to a "first class port", which means that broken builds for linux/arm64 will block releases; official binaries as well as install documentation are provided by the Go team. As Fitzpatrick noted, Linux 64-bit Arm is now at least as important as 32-bit Arm, which is already a first-class port.
On Windows, Go 1.15 now generates executables that use address-space layout randomization (ASLR) by default. ASLR uses position-independent code to randomize the addresses of various data areas on startup, making it harder for attackers to predict target addresses and create memory-corruption exploits.
Standard library additions
Go's standard library is large and fairly stable; in Go 1.15 only relatively minor features were added.
The standard library's testing package is quite minimalist — the Go philosophy is to avoid domain-specific languages for writing tests and assertions, and instead to just write plain Go, which the developer already knows. But the core developers found creating a temporary directory useful enough to approve adding a TempDir() method that lazily creates a temporary directory for the current test and deletes it automatically when the test is finished.
The net/url package adds a new URL.Redacted() method that returns the URL as a string, but with the password redacted (replaced by xxxxx). URLs with passwords such as https://username:password@example.com/ are not usually used in browsers anymore, but are still surprisingly common in scripts and tools. Redacted() can be used to log URLs more securely, in line with RFC 3986's guidelines to not render the part after the : as clear text.
A new time/tzdata package was added to allow embedding a static copy of the time zone database in executables. Because it adds about 800KB to the executable, it's opt-in: either by importing the time/tzdata package, or by compiling with the timetzdata build tag. The embedded database can make time zone database access more consistent and reliable on some systems (particularly Windows), and it may also be useful in virtualized environments like Docker containers and the Go playground.
Parting thoughts
Go uses GitHub issues to track all bugs and feature requests, so you can scan the list of closed issues in the Go 1.15 milestone for further exploration of what's in the release. The 1.15 final release is still over 2 months away, but you can easily test your own code against the latest version using the gotip tool, or wait for the binary beta release — scheduled for June 1. Bugs found now will almost certainly be fixed before the 1.15 final release.
Index entries for this article | |
---|---|
GuestArticles | Hoyt, Ben |
(Log in to post comments)
What's coming in Go 1.15
Posted May 12, 2020 18:56 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
What's coming in Go 1.15
Posted May 12, 2020 19:50 UTC (Tue) by mgk (subscriber, #74833) [Link]
What's coming in Go 1.15
Posted May 13, 2020 4:42 UTC (Wed) by felixfix (subscriber, #242) [Link]
What's coming in Go 1.15
Posted May 13, 2020 20:50 UTC (Wed) by benhoyt (subscriber, #138463) [Link]
What's coming in Go 1.15
Posted May 13, 2020 21:06 UTC (Wed) by felixfix (subscriber, #242) [Link]
What's coming in Go 1.15
Posted May 14, 2020 6:34 UTC (Thu) by epa (subscriber, #39769) [Link]
I’m about to eat breakfast and it will be my first meal since dinner last night.
What's coming in Go 1.15
Posted May 13, 2020 21:12 UTC (Wed) by jake (editor, #205) [Link]
Or I did :)
fixed now ...
jake
What's coming in Go 1.15
Posted May 13, 2020 12:32 UTC (Wed) by mdlayher (subscriber, #99548) [Link]
https://www.reddit.com/r/golang/comments/ginp22/whats_com...
What's coming in Go 1.15
Posted May 13, 2020 20:41 UTC (Wed) by benhoyt (subscriber, #138463) [Link]
What's coming in Go 1.15
Posted May 13, 2020 12:39 UTC (Wed) by mirabilos (subscriber, #84359) [Link]
There’s already another language called Go!
If you can’t call it Issue9, call it Google golang instead.
Thanks, in the name of all small software projects afraid of being steamrolled over by gigants like Google.
What's coming in Go 1.15
Posted May 13, 2020 13:37 UTC (Wed) by mdlayher (subscriber, #99548) [Link]
What's coming in Go 1.15
Posted May 13, 2020 14:32 UTC (Wed) by mirabilos (subscriber, #84359) [Link]
And no, it’s never time to let that rest. What if Google decides to occupy the name of _your_ project next?
What's coming in Go 1.15
Posted May 13, 2020 15:05 UTC (Wed) by tao (subscriber, #17563) [Link]
What's coming in Go 1.15
Posted May 13, 2020 15:21 UTC (Wed) by mirabilos (subscriber, #84359) [Link]
What's coming in Go 1.15
Posted May 13, 2020 19:09 UTC (Wed) by dottedmag (subscriber, #18590) [Link]
What's coming in Go 1.15
Posted May 14, 2020 4:28 UTC (Thu) by felixfix (subscriber, #242) [Link]
What's coming in Go 1.15
Posted May 14, 2020 8:29 UTC (Thu) by ptman (subscriber, #57271) [Link]
What's coming in Go 1.15
Posted May 14, 2020 11:19 UTC (Thu) by tao (subscriber, #17563) [Link]
If you don't feel at home with git (which I can understand; it requires a very different mindset) at least use something like SVN.
What's coming in Go 1.15
Posted May 14, 2020 16:09 UTC (Thu) by mirabilos (subscriber, #84359) [Link]
I do use git, but I’ve been working with CVS for two decades and am the current de-facto maintainer.
(I don’t know the other git thing, but IIRC wasn’t it just short for “GNU It” or something, a file manager or so?)
Oh, and RCS is nice for ad-hōc single-file version control, such as my DNS zone file. It’s also used in at least two BSDs’ etckeeper equivalent.
What's coming in Go 1.15
Posted May 14, 2020 16:57 UTC (Thu) by josh (subscriber, #17465) [Link]
GNU Interactive Tools (Debian package "gnuit"), a file manager that people liked to use, similar to Midnight Commander. There were bitter flamewars about git (the version control system) "stealing" the name of git (which also installed /usr/bin/git), up to the point where Debian used to ship a script that would invoke GNU Interactive Tools if you had it installed and just typed "git", but would run the version control system if you ran git with arguments. There were also arguments about popularity, much like those that happened when the node.js binary "node" conflicted with a HAM radio tool called "node".
Eventually, GNU Interactive Tools renamed that one binary to "gitfm", the excessively magic DWIM script went away, and people got over it. It's also worth observing that the "gnuit" package hasn't had any updates (other than binNMUs) since oldoldstable. Similarly, I would suspect that far more people first heard of the "Go!" (with exclamation point) language because of the gripe against Google's "Go" language than were originally (or are now) users of the "Go!" language.
What's coming in Go 1.15
Posted May 14, 2020 17:04 UTC (Thu) by josh (subscriber, #17465) [Link]
What's coming in Go 1.15
Posted May 15, 2020 11:01 UTC (Fri) by cladisch (✭ supporter ✭, #50193) [Link]
Then you might be interested in SRC, which puts a modern UI on top of RCS: http://www.catb.org/~esr/src/
(Here, "modern UI" means a CLI like svn/hg/git.)
What's coming in Go 1.15
Posted May 15, 2020 16:07 UTC (Fri) by mirabilos (subscriber, #84359) [Link]
Thanks anyway!
What's coming in Go 1.15
Posted May 14, 2020 8:59 UTC (Thu) by Karellen (subscriber, #67644) [Link]
Although the name "git" was used by another software project before Linus used it, at least that other project was not also a revision control system. You could at least distinguish between them by search for "git version control" or "git [whatever the other program did]".[0]
On the other hand, calling your programming language "go" when, not only is there another software project called "go" out there, but that other project is also a programming language, does seem like a particularly egregious instance of name-sniping. Especially when it comes from the 900lb gorilla that is Google.
"We don't care; we don't have to. We're Google."
[0] Seriously, I used to know what it did, and now I'm blanking on it. So I tried to find out what it was, and came up short. Search engines are swamped with references to the Linus' git (and github), and Wikipedia's page for git, and the disambiguation page for git, don't mention it. Which is somewhat ironic, given that the Wikipedia page for golang does have a link to a page for the original Go! programming language.
What's coming in Go 1.15
Posted May 14, 2020 11:30 UTC (Thu) by tao (subscriber, #17563) [Link]
I guess it all comes down to how well known the previous language was.
Name clashes happen, probably far more often than you'd expect. Google releasing something called "Go" isn't exactly a stretch. If nothing it's surprising that Google hasn't named something "go" earlier...
At work I've been hacking away for about a year on a set of tools under an umbrella name that I was informed of last week was the name of another set of tools in the exact same domain.
What's coming in Go 1.15
Posted May 14, 2020 16:14 UTC (Thu) by mirabilos (subscriber, #84359) [Link]
From the official golang history site, there’s a link to a pre-history site which mentions versions before 0.9, whose oldest entry is this:
r56 (released 2011/03/16)
The r56 release was the first stable release and corresponds to
[132]weekly.2011-03-07.1. The numbering starts at 56 because before this release,
what we now consider weekly snapshots were called releases.
The bugreport was entered on 2009-11-11, about 1½ years before even that. Plenty of time to change it.
Note that one of the earliest answers was quick to distinguish as “Google go”, and the website is “golang”, so use that if you can’t use Issue9 (which, by the way, would have been a <em>great</em> name, considering its Plan 9 history).
What's coming in Go 1.15
Posted May 14, 2020 11:43 UTC (Thu) by leromarinvit (subscriber, #56850) [Link]
Seriously, I used to know what it did, and now I'm blanking on it. So I tried to find out what it was, and came up short. Search engines are swamped with references to the Linus' git (and github), and Wikipedia's page for git, and the disambiguation page for git, don't mention it.
Doesn't that kind of refute your point that [y]ou could at least distinguish between them by search for "git version control" or "git [whatever the other program did]"? Anyway, I guess you mean gnuit, which includes a file manager that apparently used to be called "git" (and is now called "gitfm"). I only found it because I remembered that the git (VCS) package used to be called git-core in Debian and derivatives, because "git" was already taken. Debian's snapshot archive still has entries for the old package. But without that knowledge, I probably wouldn't have found it. Neither Google nor Duck Duck Go turn up gnuit for me when searching for "git file manager".
What's coming in Go 1.15
Posted May 15, 2020 9:04 UTC (Fri) by Karellen (subscriber, #67644) [Link]
Doesn't that kind of refute your point
Not really? I don't think that there are going to be many instances where someone is going to be searching for a particular software package, knowing only its name, and not having a clue what it does. The only reason I was searching some software called "git" without knowing anything about what it did was to try and make a clever point on an internet forum unrelated to what it actually did. I think that 99.99% of the time that people search for software, it's because they want to use the software for a purpose, so they probably have a good idea of what it does.
I did at least acknowledge the irony of the situation. :-)
Anyway, yes, it was "gnuit" that I was trying to find. Thanks! Hopefully that will stick in my brain for the next time this topic comes up somewhere...
What's coming in Go 1.15
Posted May 14, 2020 12:26 UTC (Thu) by rschroev (subscriber, #4164) [Link]
> [0] Seriously, I used to know what it did, and now I'm blanking on it. So I tried to find out what it was, and came up short. Search engines are swamped with references to the Linus' git (and github), and Wikipedia's page for git, and the disambiguation page for git, don't mention it. Which is somewhat ironic, given that the Wikipedia page for golang does have a link to a page for the original Go! programming language.
I think you mean GNU Interactive Tools [1], now known as gnuit but formerly as git: "a set of interactive text-mode tools, closely integrated with the shell. It contains an extensible file system browser, an ascii/hex file viewer, a process viewer/killer and some other related utilities and shell scripts."
I remember encountering that package in old Debian versions when trying to find Linus' git but finding GNU Interactive Tools instead (git wasn't yet packaged for Debian at the time, or maybe under a different name). It's very hard to find now if you don't know exactly what to look for. Looks pretty dead too: the newest entry in the changelog is from 2009. I found it by searching for 'git' in the Packages file of on old Debian release in the Debian archives. It got renamed from git to gnuit back in 2007, I assume to try to remedy the name clash with Linus' git, making it even harder to find unless you know the new name.
What's coming in Go 1.15
Posted May 14, 2020 17:12 UTC (Thu) by josh (subscriber, #17465) [Link]
The language called "Go!" (with exclamation point) hasn't seen activity in what appears to be more than a decade. The author later tossed the sources on github five years ago, for what appears to be archival purposes, as it hasn't seen a single commit since then; they appear to have moved on to other programming language projects. The wikipedia article at https://en.wikipedia.org/wiki/Go!_(programming_language) probably wouldn't exist (for notability reasons) if not for the high-profile conflict with Google's language "Go" (no exclamation point).
The language seems to be, for all intents and purposes, historical. And I don't think by any stretch of the imagination its popularity was hurt by the naming conflict with Google.