[HN Gopher] Show HN: Go Plan9 Memo
___________________________________________________________________
Show HN: Go Plan9 Memo
A quick dive into the Plan9 assembly I picked up while developing
my SIMD package for Go, and how it led to a 450% performance boost
in calculations.
Author : pehringer
Score : 213 points
Date : 2024-10-18 14:36 UTC (8 hours ago)
(HTM) web link (pehringer.info)
(TXT) w3m dump (pehringer.info)
| pphysch wrote:
| This is neat. Why do you think this is not already part of the
| standard library?
| pehringer wrote:
| I think the Go std lib is focused on portability over raw
| performance
| twoodfin wrote:
| I think the more interesting architecture / priorities question
| is why the Go compiler can't yet(?) auto-vectorize?
| lexh wrote:
| A bit over my head, but I enjoyed the way the writing brings us
| along for the ride.
|
| This can't be the first pass someone has made at something like
| this, right? There must be literal dozens of SIMD thirsty Gophers
| around. Would a more common pattern be to use CGO?
| dpifke wrote:
| (Rejected) proposal to add something like this to the standard
| library gives some context:
| https://github.com/golang/go/issues/53171
| chrchang523 wrote:
| The problem with cgo is the high function-call overhead; you
| only want to use it for fairly big chunks of work. Calling an
| assembly function from Go is a lot cheaper.
|
| https://pkg.go.dev/github.com/grailbio/base/simd has some work
| I've done in this vein.
| Andoryuuta wrote:
| I think people certainly have been trying for a while. In fact,
| I recall being on a (Skype?) call with my brother almost a
| decade ago while he was trying to write an SIMD library in Go.
| If I remember correctly, at that time, a bunch of the AVX
| instructions weren't even encodable in Go's Plan9 assembler -
| so we had to manually encode them as bytes [0].
|
| The most complete library I've seen (though admittedly never
| used) uses CGO _partially_, with a neat hack to avoid the
| overhead that it comes with [1].
|
| [0]:
| https://github.com/slimsag/rand/blob/f1e8d464c0021a391d5cd64...
|
| [1]: https://github.com/alivanz/go-simd/
| tempfile wrote:
| Is Plan9 in this context related at all to Bell Labs's Plan9?
| Seems too similar to be coincidence.
| wrs wrote:
| Yes, some of the same people created Go.
| saclark11 wrote:
| Yes. Two of Go's creators, Rob Pike and Ken Thompson, were also
| authors of Plan 9 at Bell Labs.
| kibwen wrote:
| Go itself is from the same people who worked on Plan9 (which is
| why I find the choice of name in the OP particularly
| confusing).
| pjmlp wrote:
| Go is the evolution of Limbo from Inferno, which was designed
| based on the failure of Alef on Plan 9, combined with a minor
| set of Oberon-2 influences.
| 201984 wrote:
| From https://go.dev/doc/asm:
|
| > The assembler is based on the input style of the Plan 9
| assemblers, which is documented in detail elsewhere. If you
| plan to write assembly language, you should read that document
| although much of it is Plan 9-specific.
|
| That first sentence links to https://9p.io/sys/doc/asm.html. So
| yes, definitely not a coincidence.
| rob74 wrote:
| In case it's not obvious, the "Plan 9" in the Go Assembler's name
| comes from https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs,
| and the reason for that is of course that two of the "Go
| founders" (Rob Pike and Ken Thompson) are Bell Labs alumni. Some
| more background on the Go assembler: https://go.dev/doc/asm
| colechristensen wrote:
| The same people used the same name for two computer related
| projects.
|
| That's certainly an ... interesting choice.
| lagniappe wrote:
| Definitely don't look into Nix then
| lloeki wrote:
| It is, essentially, the same project. Plan9 is the umbrella
| term for the whole operating system.
|
| https://plan9.io/sys/doc/comp.html
|
| IIRC before Go was self compiling, it was compiled using 9c,
| and its architecture inherits from 9c.
|
| EDIT: some nearby threads got the details better!
| floren wrote:
| > IIRC before Go was self compiling, it was compiled using
| 9c, and its architecture inherits from 9c.
|
| Back in those days I actually found that, with a few small
| tweaks, I could compile the Plan 9 operating system using
| Go's C suite. We didn't pursue it further but this was one
| of the options we looked into for cross-compiling Plan 9
| from other operating systems.
| stonogo wrote:
| Since to Go C suite was ported Plan 9 compilers, I'm not
| sure why this would be a surprise. Since I'm obviously
| missing something would you share your thoughts on what
| challenges you expected?
| floren wrote:
| It wasn't any surprise, nor did I intend to imply it was
| surprising, just relaying an anecdote. There were a few
| differences that required small tweaks to the source, but
| we got the Plan 9 kernel compiled with the Go C compilers
| in a day.
| colechristensen wrote:
| Interesting trivia about the connection to plan9 the
| operating system.
|
| >Go uses its own internal assembly language called Plan9.
|
| Plan9 is the name of the OS. You wouldn't name a
| programming language "Linux", even if Linus created it and
| it was super related or not at all related.
| seryoiupfurds wrote:
| It's not "plan9 assembly language" as in "the assembly
| language named plan9". Read it as "the otherwise unnamed
| custom assembly language used in the plan9 operating
| system".
|
| The article simply misspoke by saying that the assembly
| language is "called plan9".
| debugnik wrote:
| No way, the article consistently refers to the assembler
| syntax as "Plan9" throughout the text and title and they
| talk about "x86 Plan9" and "arm Plan9".
|
| Considering there is no introduction at all to this
| beyond "I discovered it's called Plan9", I'm assuming the
| author really thinks this is a language widely named
| "Plan9".
| KerrAvon wrote:
| They seem to be badly mistaken
| sph wrote:
| That seems to be the case:
| https://news.ycombinator.com/item?id=41880346
| colechristensen wrote:
| EDIT: it seems the author is just mistaken, the Go assembler
| is just referred to by everyone else as "Go Assembler" and
| nothing to do with plan9, other than the various connections
| to its origin.
| p_l wrote:
| The reason for that is not just being alumni, but also that Go
| implementation started on top of Plan 9's compiler
| infrastructure that was later rewritten in pure Go
| nzach wrote:
| > I am not sure why the Go team went down this route. Maybe it
| simplifies the compiler by having this bespoke assembly format?
|
| This was to allow for easy cross-compilation.
|
| You should probably clarify that you are talking about 'golang
| plan9 assembly format', and not about the plan9 OS. This confused
| me a lot. (https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs)
|
| I didn't know you could easily hand write 'golang assembly' for
| your functions. This seems really useful in some cases.
|
| But I do wonder, why the std lib doesn't offer this ?
|
| EDIT: I found the anwser:
| https://github.com/golang/go/issues/53171
|
| Here is a good explanation provided by Ian Lance Taylor:
|
| > This proposal is focused on code generation rather than
| language, and the details of how it changes the language are
| difficult to understand. Go is intended to be a simple language
| that it is easy to understand. Introducing complex semantics for
| performance reasons is not a direction that the language is going
| to take.
| pehringer wrote:
| If you want to see some Plan9 in the std library check out the
| math package!
|
| https://github.com/golang/go/tree/master/src/math
| sph wrote:
| Plan 9 has nothing to do with assembly language nor the variant
| Go uses. See other comments in this thread.
|
| It would be like calling the AT&T style of assembly used by GNU
| binutils a "Linux"
| saclark11 wrote:
| > Overall, pretty weird stuff. I am not sure why the Go team went
| down this route. Maybe it simplifies the compiler by having this
| bespoke assembly format?
|
| Rob Pike spoke on the design of Go's assembler at a talk in 2016
| [1][2]. I think it basically came down to the observation that
| most assembly language is roughly the same, so why not build a
| common assembly language that "lets you talk to the machine at
| the lowest level and yet not have to learn a new syntax." It also
| enables them to automatically generate a working assembler given
| an instruction manual PDF for a new architecture as input.
|
| [1]: https://www.youtube.com/watch?v=KINIAgRpkDA [2]:
| https://go.dev/talks/2016/asm.slide#1
| alphazard wrote:
| And it worked. Go established cross-compilation as table-stakes
| for new programming languages, at a time when very few were
| doing it well, if at all.
| p_l wrote:
| Go essentially copied the design from Plan9 compilers, which
| it was originally based on. It's one of the many things it
| inherited from Plan9 environment.
| kjksf wrote:
| It's more "reused" because Plan9 compilers were designed /
| co-implemented by Pike / Thomson, 2 out of 3 original
| designers of Go.
|
| For those interested, here's Thomson's paper about Plan9 C
| compilers: https://9p.io/sys/doc/compiler.html and
| https://doc.cat-v.org/bell_labs/new_c_compilers/
| derefr wrote:
| I would love to see a deep dive on what features /
| architectural paradigms the Golang runtime shares with
| Plan9. Has anything like that been written?
|
| One that always sticks out to me personally is the use in
| Go of the term "dial" instead of "connect" for network
| connection establishment. This is, AFAICT, another
| Pike+Thompson-ism, as it can be seen previously in the
| form of the Plan9 dial(3) syscall --
| https://9fans.github.io/plan9port/man/man3/dial.html .
|
| ---
|
| A tangent: I have wondered before whether Pike and
| Thompson drafted the design for the language that would
| become Golang long before working at Google, initially to
| replace C specifically in the context of being the
| lingua-franca for systems programming _on Plan 9_. And
| that, therefore -- at least in the designer 's minds --
| Golang would have always had Plan9 as its secret
| "flagship target" that it should have a 1:1 zero-
| impedance abstraction mapping onto. Even if they never
| bothered to actually make a Plan9 Golang runtime.
|
| You could test this hypothesis by implementing an actual
| Golang runtime for Plan9+, and then comparing it to the
| Golang runtimes for other OSes -- if Plan9 were the
| "intended home" for Golang programs, then you'd expect
| the Golang runtime to be very "thin" on Plan9.
|
| (To put that another way: imagine the Golang runtime as
| something like WINE -- a virtualization layer that
| implements things that _could_ be syscalls / OS library
| code, in the form of client-side runtime shim code. A
| "WINE implementation for Windows" would be an extremely
| thin shim, as every shim call would just point to a
| 1:1-API-matched piece of OS-provided code. My hypothesis
| here is that "Golang for Plan9" is the same kind of thing
| as "WINE for Windows.")
|
| + I was saying this as a thought experiment, not thinking
| there would actually be a Plan9 implementation of the
| Golang runtime... but there is!
| (https://go.dev/wiki/Plan9) So someone can actually check
| this :)
| mbivert wrote:
| > I would love to see a deep dive on what features /
| architectural paradigms the Golang runtime shares with
| Plan9. Has anything like that been written?
|
| If it has, then it's most likely available on
| https://cat-v.org/. Even if it hasn't, cat-v.org is a
| great starting point.
|
| Besides, close to your line of thought, and assuming you
| didn't knew about this already, Pike & al previously
| worked on Limbo[0], a "predecessor" of Go, used to wrote
| Inferno[1], a Plan9-like OS, which could be hosted on
| arbitrary OSes via a bespoke virtual machine called
| "Dis".
|
| So there were indeed a few previous "drafts" for Go. I'd
| doubt that Go has been designed "for" Plan9 though.
|
| [0]: https://en.wikipedia.org/wiki/Limbo_(programming_lan
| guage)
|
| [1]:
| https://en.wikipedia.org/wiki/Inferno_(operating_system)
| exitb wrote:
| There's also libthread[1] which implements concurrency
| model similar to goroutines (at least earlier, as they
| appear to be no longer just cooperatively scheduled?).
| That manual also mentions Alef and Newsqueak as
| influential predecessors.
|
| [1] http://man.9front.org/2/thread
| gavindean90 wrote:
| It sort of was and Rob blogged about it. It is based
| really on Newsqueak and plan9 C which is very different
| from Unix C
| pram wrote:
| I'd guess dial is probably a Bell Labs-ism
| cloudfudge wrote:
| Yes, this is a great leap forward in my opinion. I had to do
| a project at a previous job where I wrote an agent that ran
| on x86, MIPS and ARM, and doing it in Go was a no-brainer.
| The other teams who had a bunch of C code that was a
| nightmare to cross-compile were so jealous they eventually
| moved a lot of things to Go.
|
| I've been doing this for 35 years and cross compiling
| anything nontrivial was always a toolchain nightmare.
| Discovering a world where all I had to do was set
| GOARCH=mips64 (and possibly GOOS=darwin if I wanted mac
| binaries) before invoking the compiler is so magical I was
| extremely skeptical when I first read about it.
| bombela wrote:
| As long as you don't have C libraries to cross compile /
| link against of course ;)
| akira2501 wrote:
| sqlite is the only thing that makes me sad I have
| CGO_ENABLED=0.
| thombles wrote:
| This non-cgo port might help - I started using it
| recently and it's fine but I'm not exactly a demanding
| user https://pkg.go.dev/modernc.org/sqlite
| somat wrote:
| A fair enough assessment, it be that way, however I will
| note that a large reason that C exists in the first place
| was to have a machine independent language to write
| programs in.
| pjmlp wrote:
| For the UNIX authors, that was already possible a decade
| before.
| kloop wrote:
| > however I will note that a large reason that C exists
| in the first place was to have a machine independent
| language to write programs in.
|
| That's fair, but what we call a monstrosity by modern
| standards is much simpler than porting the assembly
|
| There were cross plaform languages before C, but they
| never really took off for system development the wat C
| did (OSs, for example were commonly written in pure
| assembly)
| dsv3099i wrote:
| mips64!? That's a blast from the past. It must be some kind
| of legacy hw that's getting current software updates in
| some kind of really niche use case. Or academia. :)
|
| Like previous you, I have to admit I'm skeptical but would
| be happy to be wrong.
| wahern wrote:
| MIPS64 is still very much alive:
| https://store.ui.com/us/en/products/er-4
|
| "EdgeOS" is based on Linux, and people run vanilla Linux
| distributions on those boxes, as well as OpenBSD and
| NetBSD.
|
| I wonder how long Marvell will continue selling those
| Octeon MIPS64 chips, though. Marvell (then Cavium)
| switched to ARM nearly a decade ago (2016) for newer
| chips in the Octeon series. I think Loongson sells more
| modern MIPS64 (or at least MIPS64-like) chips, but they
| don't seem to be commercially available outside China.
| cloudfudge wrote:
| > mips64 .. must be some kind of legacy hw that's getting
| current software updates
|
| Hundreds of thousands of linux-based smartnic cards,
| actually. Fun stuff. Those particular ones were EOLd and
| have been replaced with ARM but the MIPS based ones will
| live on in the datacenters until they die, I'm sure.
|
| > Like previous you, I have to admit I'm skeptical but
| would be happy to be wrong
|
| Seriously, you are going to be delighted to be wrong. On
| your linux machine, go write a go program and write
| "GOOS=darwin GOARCH=arm64 go build ..." and you will have
| yourself an ARM mac binary. Or for going the other way,
| use GOOS=linux GOARCH=amd64. It really is that simple.
| pjmlp wrote:
| The idea isn't new, for example Amsterdam Compiler Kit from
| 1980's.
| sweeter wrote:
| This is by far one of the best parts of Go. Its all around
| simple and painless to use. anyone designing a language
| should study what Go did well and what they didn't.
| Yoric wrote:
| FWIW, that's also more or less what SpiderMonkey has been doing
| for ~25 years (and I presume other JavaScript VMs).
| mxey wrote:
| > After doing some preliminary research I discovered that Go uses
| its own internal assembly language called Plan9.
|
| Is the language actually called that?
| nzach wrote:
| This is a fair question. Initially I just assumed this was
| true. Because someone who did research on this topic would not
| get something like this wrong. And besides that, if you know a
| little about the project this name could make some sense.
|
| But the more I look into it, the more I think this is just a
| LLM hallucination.
|
| The doc about the 'assembly' format doesn't give a proper name.
| It just calls 'go assembler'.
|
| And I think the source of this hallucination was this first
| paragraph:
|
| > The assembler is based on the input style of the Plan 9
| assemblers, which is documented in detail elsewhere. If you
| plan to write assembly language, you should read that document
| although much of it is Plan 9-specific. The current document
| provides a summary of the syntax and the differences with what
| is explained in that document, and describes the peculiarities
| that apply when writing assembly code to interact with Go.
| mseepgood wrote:
| No, it doesn't have a name. Plan 9 is an operating system, and
| this style of assembly language syntax originates from the
| assembler used on this operating system. Its like saying "The
| GNU Compiler Collection uses its own internal assembly language
| called Unix."
| dekhn wrote:
| Well, linux is just a mechanism to make more copies of GCC,
| so that tracks.
| anthk wrote:
| Kinda like GNU (Unix reimplementation) was a fancy platform
| to run Emacs on top.
| dekhn wrote:
| basically: https://www.gnu.org/gnu/rms-lisp.en.html
| rsc wrote:
| No. It's just Go assembly. (It happens to be a Plan 9-derived
| syntax, but we call it Go assembly.) See
| https://go.dev/doc/asm.
| spiffistan wrote:
| There was a controversy when Go came out about the naming due to
| another language also being called Go, and the top voted
| alternative name was Plan9, and as an homage they may have used
| that internally instead.
| stonogo wrote:
| The top voted alternative was "Issue 9" which served as a
| reference to Plan 9 and happened to be the actual issue number
| in the Go project on Google Code opened by the guy who's
| programming language (named "Go!") was already out there.
| spiffistan wrote:
| You're right, my bad
| umvi wrote:
| > Basically, I wanted a set of functions that would allow me to
| perform simd operations on slices.
|
| How is this actually used though? I searched the whole article,
| and no where are any operations being performed on slices.
|
| Edit: I found what I was looking for in the linked docs:
| https://pkg.go.dev/github.com/pehringer/simd#pkg-index
|
| So basically if you have 2 slices you want to add together,
| instead of a for loop you could do it in parallel with simd
| using: simd.AddInt32(slice1, slice2, result)
| pehringer wrote:
| I should have added some examples in the docs / repo :(
| checkout the benchmark code at the bottom of this file for an
| example:
| https://github.com/pehringer/simd/blob/main/simd_test.go
| joeegan2202 wrote:
| A couple points on Go assembly:
|
| 1. On amd64 those ints are actually 64bit. If you used int32 then
| they would be be word aligned in the parameter list. However,
| there is a gotcha with that. The return values will always start
| at a dword aligned offset on 64bit system.
|
| 2. NOSPLIT is defined in "textflag.h" which Go's compiler
| automatically provides. However, NOSPLIT is, from everything I've
| read, only respected on runtime.XX functions, so it's not doing
| anything there, and it's also not necessary. NOSPLIT tells the
| compiler not to insert code to check if the stack needs to split
| because it's going to overflow, which is technically unnecessary
| if the function doesn't need any stack space. It's basically only
| there on the function that checks for stack splits, to prevent
| that code from being injected into itself.
| pehringer wrote:
| Thank you for the explanation! That makes a lot more sense :)
| pixelesque wrote:
| Very minor nit, doesn't change anything about the article
| otherwise, but the SIMD acronym stands for *Single* Instruction,
| Multiple Data conventionally.
| sedatk wrote:
| > 4 represents "NOSPLIT" which we need for some reason
|
| For those who are curious: "In the general case, the frame size
| [the parameter after NOSPLIT] is followed by an argument size,
| separated by a minus sign. (It's not a subtraction, just
| idiosyncratic syntax.) The frame size $24-8 states that the
| function has a 24-byte frame and is called with 8 bytes of
| argument, which live on the caller's frame. If NOSPLIT is not
| specified for the TEXT, the argument size must be provided. For
| assembly functions with Go prototypes, go vet will check that the
| argument size is correct."
|
| Source: https://go.dev/doc/asm
| cmovq wrote:
| > However, (as far as I can tell) Go's compiler does not utilizes
| simd
|
| Can someone confirm this? Not even for straightforward copies?
| kristianp wrote:
| Relevant to which processors go supports, is this section (1).
| Base x64 support includes SSE and SSE2. I don't know if the go
| compiler produces it, though. Unlike extremely complex compilers
| like gcc, where performance is the top priority, the go compiler
| favours simplicity in a Wirthian(2) fashion favouring a simple,
| fast compiler.
|
| (1) https://go.dev/wiki/MinimumRequirements#amd64
|
| (2) https://irreal.org/blog/?p=7075
|
| https://smartgo.blog/2024/01/06/niklaus-wirth/
___________________________________________________________________
(page generated 2024-10-18 23:00 UTC)