[HN Gopher] WASI Support in Go
___________________________________________________________________
WASI Support in Go
Author : spacey
Score : 125 points
Date : 2023-09-13 16:27 UTC (6 hours ago)
(HTM) web link (go.dev)
(TXT) w3m dump (go.dev)
| kungfufrog wrote:
| Can someone hit me with the value proposition of all this WASI
| stuff and WASM and ELI5? (I get the browser use-case)
|
| My understanding is as follows: WASM - a
| portable, platform-independent virtual machine for executing a
| "web assembly" WASI - an extension to the virtual
| machine that adds APIs for interacting with the system and breaks
| all the WASM sandboxing (presumably NOT platform-independent?)
|
| Is the point of this addition to Go that I can now target "WASM
| implementations that have WASI" with Go source code compiled to
| WASM?
|
| Why would someone want to do that? Just for edge functions in
| cloud workers?
| candiddevmike wrote:
| I still don't understand the benefit of running a Go app on a
| cloud provider using this. Anyone want to help me?
|
| Is it an edge play for using something like Cloudflare Workers?
|
| Is it cheaper vs standard serverless/container deployments? Go
| apps can already scale to 0 for these use cases.
| dilyevsky wrote:
| It's basically what jvm should have been but never delivered:
|
| 1. Could be an edge/iot play (tho not with standard go
| toolchain bc those binaries are huge and slow)
|
| 2. Trusted environment makes it interesting for all kinds of
| sbom-related usecases. Think weapons tech, space, etc
|
| 3. On the container side things are less clear to me but may
| offer better startup time, reduced footprint etc
|
| 4. Finally, for plugabble software it's most interesting option
| personally. It's basically 2.) but with focus on DX over
| security
| alecthomas wrote:
| I believe the primary benefit is the sandboxing, but I imagine
| there are secondary benefits such as that the host platform can
| be any CPU architecture or OS.
| viccuad wrote:
| It consolidates Go compiled to WASI as an alternative of doing
| containers. Linux containers don't "Run anywhere." as docker.io
| says. You need a specific architecture _and_ kernel features,
| which is not obvious from afar.
|
| There's also other benefits. Example: the team I work on
| compiled Kyverno, a CNCF K8s policy engine written in Go, to a
| WASI target. We are building Kubewarden, a CNCF policy engine
| where policies are Wasm binaries shipped in OCI registries. We
| strive to build "a Universal Policy Engine". Now, we have an
| experimental Kubewarden policy `kyverno-dsl-policy` that allows
| you to reuse Kyverno DSL with us. We also provide WaPC as a
| target, more performant and secure, hence normal SDKs for Go,
| Rust, C#, swift, typescript... In addition to supporting Rego,
| again compiled to Wasm.
|
| IMHO you only benefit from the real sandboxing from WaPC, as
| WASI's posix-like interface allows you to attack the host.
|
| The next step for the official Go compiler is to export the
| function symbols, to allow for WaPC.
| Thaxll wrote:
| As if WASI does not need specific architecture and kernel
| features?
| carbotaniuman wrote:
| WASI runners are just an application. I guess you need your
| kernel to run prograns and have syscalls, but that is a low
| bar.
| Thaxll wrote:
| Docker needs the 3.10 kernel which is 10 years old.
| mbertschler wrote:
| But it needs Linux, while wasm runtimes are available for
| all OSes natively
| rockwotj wrote:
| How is the Universal Policy Engine different than Open Policy
| Agent?
| viccuad wrote:
| Just gave a talk on Monday about it in containerdays.io,
| but the video is not in youtube yet!
|
| In a nutshell, with Kubewarden we strive to build the
| universal policy engine by:
|
| - Provide all personas (policy consumer, policy developer,
| policy distributor, engine admin, engine
| developer/integrator, etc) with current and future
| industry-standard workflows, not only a subset of personas,
| nor more than needed knowledge for those personas. It's a
| bold statement, and if it would be universal it should
| indeed cater to everyone.
|
| - This is achieved with policies as code, which are Wasm
| modules: Wasm policies allows us to support Rego DSL
| (OPA/Gatekeeper), YAML, SDKs for Wasm-compiled languages,
| and now an experimental Kyverno DSL policy by compiling it
| to WASM with WASI. Great for using your language and tools
| of preference.
|
| - Wasm modules have first class support In OCI registries,
| just like container images: Use same tools that you know as
| artifact distributor: SBOMs, signing and verifying with
| cosign, airgap, slsa.dev, etc.
|
| - Policies can be evaluated out-of-cluster: great for
| CI/CD, dev loop, integration tests, etc.
|
| - Modular architecture informed by using Wasm policies: OCI
| registry, policy-server, k8s controller, out-of-cluster cli
| (kwctl), etc. This also helps in adopting future industry-
| standard workflows.
|
| - Usual features of a Policy engine (mutating, context-
| aware, recurring scanner of already in-cluster resources,
| etc). Plus ample room for new features thanks to the
| architecture. E.g: possibility to run the policy-server
| directly in the k8s apiserver (one colleague already
| presented that in Kubecon), possibility to evaluate out-of-
| cluster policies outside of clusters like OPA just by
| running the policy-server standalone, more DSLs compiled to
| Wasm, more languages, etc.
|
| - Vendor neutral, CNCF project, open source, developed in
| the open.
| umvi wrote:
| I feel like we need better WASM performance in go before we get
| WASI. In my experience go wasm performance is pretty bad, usually
| significantly worse than vanilla JS.
|
| Rust (or really anything LLVM backed) is still probably the best
| WASM language in terms of performance and support, but .NET
| (don't forget to turn on AOT) is starting to get really good too
| (except for the fact that .NET compiler barfs out a bazillion
| files that the browser needs vs. 1 self contained .js or .wasm
| file which sucks if you are trying to build a self contained
| library like OpenCV.js)
| dgb23 wrote:
| > usually significantly worse than vanilla JS
|
| Can to elaborate?
|
| This is can be true if you're interacting with browser API's
| such as the DOM frequently, because there's an overhead.
|
| But I've seen several projects where (non-GC) WASM has improved
| performance significantly for specific tasks. You won't get
| native performance obviously.
| achille-roussel wrote:
| This is a good callout, although we probably won't be able to
| significantly improve the performance of Go compiled to WASM
| until WebAssembly evolves and introduces support for threads or
| stack switching so we can define goroutines based on those;
| right now the main reason Go compiled to WASM isn't in the
| ballpark of native perf is due to the stack switching emulation
| we have to do to get the cooperative scheduling of goroutines
| to work. We'll need WASM runtimes to offer more advanced
| primitives that we can rely on to implement those features of
| the Go language and produce much higher-quality code.
| dilyevsky wrote:
| I wish they added some way of exporting wasm funcs as well so
| that they could be called from host. Tinygo wasm target supports
| both "exports" and "imports". The other thing that is worse
| compared to tinygo is the generated binary size seems to be about
| 10x larger. From my brief look at the transpiled .wat printout a
| lot of included funcs aren't being called anywhere...
| rockwotj wrote:
| FWIW we simulate exports by instead having `main` call an
| imported function that blocks until it's ready to return with
| the needed data.
|
| So instead of:
|
| `host -> call foo on guest -> return to host`
|
| `host -> call guest main -> call foo on host -> host returns
| when ready -> guest calls foo when done`
|
| FWIW the scheduler (so goroutines) don't work in go if you're
| not calling from an main, so anytime you call a custom export
| then try to use a goroutine you'll get panics.
|
| 10x size is about the blowup we see as well. It's also likely
| to be slower (some of the Tinygo authors said ~20% slowdown
| compared to tinygo) probably due to the simpler/smaller runtime
| and LLVM being better at optimizing.
| jbrandhorst wrote:
| Exports are something we'd like to work on but it turns out
| it's pretty complicated to reconcile the Go runtime with the
| WebAssembly environment, especially when there's only a single
| thread. We'll get to exports as soon as we can but it may
| require Wasm threads to be stable first.
| jsd1982 wrote:
| The go-compiled wasm is also extremely slow compared to
| tinygo's wasm. Doing simple things like `fmt.Printf()` degraded
| performance significantly.
| alecthomas wrote:
| I wonder if fmt.Printf() in particular is slow due to the CGo
| transition? Assuming there even is a CGo transition when
| compiled to WASM...
| jbrandhorst wrote:
| There's no CGO involved when compiling to Wasm. The
| sometimes slow performance is due to the hoops the compiled
| code has to jump through to support the Go runtime and
| goroutine preemption on a single thread.
| alecthomas wrote:
| I understand there's no CGo specifically, but I'm
| wondering if the Go runtime when running under WASM still
| has to manage switching out the goroutine stacks for
| "WASM stacks" when it's calling out through the WASM VM.
|
| Edit: from this comment [1] it sounds like this is the
| case.
|
| [1] https://news.ycombinator.com/item?id=37501552
| viccuad wrote:
| I'm also waiting for that :) See
| https://github.com/golang/go/issues/42372
| dilyevsky wrote:
| Yeah there is an escape hatch via js package but it's
| annoying
___________________________________________________________________
(page generated 2023-09-13 23:01 UTC)