[HN Gopher] Runtime code generation and execution in Go
       ___________________________________________________________________
        
       Runtime code generation and execution in Go
        
       Author : ingve
       Score  : 141 points
       Date   : 2024-05-28 05:24 UTC (1 days ago)
        
 (HTM) web link (mathetake.github.io)
 (TXT) w3m dump (mathetake.github.io)
        
       | fl0ki wrote:
       | I didn't know Go reserved some registers for its own use. For
       | libraries that don't use cgo but do use assembly [1], does this
       | mean they have to generate that assembly with a compiler flag
       | that avoids reserved registers?
       | 
       | [1] A common optimization technique, like in the superb
       | https://github.com/klauspost/compress
        
         | jhoechtl wrote:
         | Pusha popa and never yield back in between should suffice?
        
           | fl0ki wrote:
           | Non-CGO goroutines are preemptible via signals so there may
           | still be more to it than that. If anyone knows the precise
           | mechanism used I'd be very keen to hear it.
        
       | evacchi wrote:
       | @ncruces and I are also contributors to wazero, so ask away if
       | you have questions :)
        
       | metadat wrote:
       | Really cool article. This sort of stuff is mind expanding for me!
       | 
       | I wonder what it would take to make a version that works on
       | Windows. Will you indulge me?
        
         | ncruces wrote:
         | wazero works fine on Windows.
         | 
         | In fact the exact same assembly works across Windows, macOS,
         | Linux and FreeBSD: the calling convention used by the compiler
         | is entirely custom, and all "host calls" are provided by
         | wazero, so assembly ports fine across OSes.
        
           | metadat wrote:
           | Perhaps I was unclear (or I'm confused now, haha):
           | 
           | > The following is the tiny demo of the runtime code
           | generation and execution in Go. I assume you are on a Unix-
           | like system like Linux or macOS on an AArch64 machine.
           | 
           | Is there a way to get tinydemo working on Windows?
        
             | ncruces wrote:
             | Oh, OK.
             | 
             | You need to replace calls to syscall.Mmap with
             | windows.VirtualAlloc [1], syscall.Mprotect with
             | windows.VirtualProtect, etc.
             | 
             | You don't even need to change the ASM preamble.
             | 
             | [1]:
             | https://pkg.go.dev/golang.org/x/sys/windows#VirtualAlloc
        
               | evacchi wrote:
               | and for reference see how it's done in wazero https://git
               | hub.com/tetratelabs/wazero/blob/c397a402ad17e495a...
        
       | kgeist wrote:
       | I wonder how safe the their (wazero's) approach is from the
       | security point of view. According to the article (and I know it
       | myself too after studying the Go runtime), the Go runtime is
       | quite finicky when it comes to execution of foreign code because
       | it can easily interfer with the goroutine scheduler, GC, etc.
       | (unless you use CGO which deals with it, but they don't use it).
       | As the article explains, it's easy to introduce random crashes
       | by, say, innocently using a "wrong" register. Also, the Go
       | runtime can change considerably from version to version (say,
       | when they introduced preempted scheduling based on signals, or
       | when they moved away from segmented stacks etc.), and so today
       | Wazero may work OK but with a new Go version it may unexpectedly
       | crash or corrupt memory in some subtle ways. Considering that the
       | whole point of WASM is sandboxing, reading this article didn't
       | make me feel very confident about the project.
        
         | ncruces wrote:
         | If it gives you some confidence, wazero (the Go Wasm runtime)
         | is used by the Go team (in CI) to test their own Go-to-Wasm
         | compiler.
         | 
         | So if they break wazero, they'll know pretty soon from failing
         | tests of their own Wasm/WASI toolchain.
         | 
         | https://go.dev/blog/wasi
         | 
         | https://github.com/golang/go/blob/go1.22.3/misc/wasm/go_wasi...
         | 
         | As for sandboxing, wazero tends to take sandboxing rather more
         | seriously than other similar runtimes.
         | 
         | Memory sandboxing is implemented through explicit bounds checks
         | (rather than memory mapping and guard pages), which allows
         | denser deployments and requires less runtime support.
         | 
         | And, by default, the WASI implementation doesn't expose any
         | capabilities (not even access to the system clock). You can
         | also interject any relevant WASI host calls, and do your own
         | filtering.
         | 
         | And if the compiler is not your cup of tea, you can also use
         | the (much slower) interpreter.
        
           | kgeist wrote:
           | That's good to hear that the Go team themselves test wazero.
           | Thank you for clarifying.
        
       ___________________________________________________________________
       (page generated 2024-05-29 23:01 UTC)