[HN Gopher] Show HN: Sqinn-Go is a Golang library for accessing ...
___________________________________________________________________
Show HN: Sqinn-Go is a Golang library for accessing SQLite
databases in pure Go
Author : cvilsmeier
Score : 57 points
Date : 2023-10-04 18:36 UTC (4 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| eatonphil wrote:
| If you _really_ want to use SQLite without anything that isn 't
| Go (since this project involves forking and communicating with a
| non-Go SQLite in a separate process over pipes), there's a Go
| translation of SQLite's C source. :)
|
| https://gitlab.com/cznic/sqlite
|
| https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...
| [deleted]
| hamandcheese wrote:
| Would it be possible to link a pure go codebase to pre-compiled
| sqlite binaries and not need to worry about cgo when cross
| compiling?
| scosman wrote:
| What's the case against cgo for SQLite? Just the usual cgo
| performance overhead?
|
| It seems like a pretty good cgo use case: a decent amount of
| work, which is typically slow enough that cgo overhead isn't
| perf critical (because DB usually means disk reads), a super
| robust and well tested C library with a super well maintained
| cgo wrapper (mattn).
| infogulch wrote:
| mattn's sqlite3 is probably the most ideal use case for cgo
| imaginable. But it can still be annoying to set up cgo to
| build across many platforms.
|
| I think Go should just pull a Zig and just embed a full blown
| C compiler into go build.
| pram wrote:
| Didn't it? Isn't that what 5c, 6c, and 8c were.
| scosman wrote:
| Exactly. Good to know I didn't miss a trade off in the mix.
|
| If I'm choosing to use a C framework (SQLite) I'm okay
| signing up for the environment costs. Prefer that over
| abstractions in an intermediate layer that might not be
| maintained in a few years.
| kunley wrote:
| Yeah, once I was on windows and couldn't get cgo sqlite
| working on a variety of mingw and alike compilers... felt
| like 90s.
|
| Using this cznic's pure-Go sqlite saved the day.
| cvilsmeier wrote:
| Sqinn author here. Nothing against CGO, but I develop/deploy
| on Win/Linux, and cross-compiling CGO is very painful.
| Regarding performance: To my own surprise, Sqinn out-performs
| mattn (and others) for normal workloads, see
| https://github.com/cvilsmeier/sqinn-go-bench
| eatonphil wrote:
| I think it's a somewhat unfair (though who cares if it's
| unfair) comparison because you aren't using the
| database/sql interface and mattn does.
|
| If you drop that interface, you get much better
| performance.
|
| See: https://github.com/eatonphil/gosqlite for example.
|
| Edit: Nevermind, you did include crawshaw (which doesn't
| use database/sql) in your benchmark!
| parhamn wrote:
| > Just the usual cgo performance overhead?
|
| No, the performance is certainly orders of magnitude faster
| than serializing over std streams on a subprocess (c ffi
| calls in cgo are 10s of nanoseconds).
|
| But one of the big draws of golang is the write-once-compile-
| anywhere toolchain and calls cgo makes that harder.
| Groxx wrote:
| To be a bit more specific here: pure Go binaries are
| trivial to cross-compile and they Just Work(tm) basically
| all the time. `GOOS="darwin" GOARCH="arm64" go build .` and
| you're done. Just iterate over the combinations you care
| about, they'll all work.
|
| As soon as you _or a library_ touches cgo, you have to deal
| with finding and setting up C cross-compilation tooling for
| your target(s), dynamic linking details, _tons_ of stuff
| that may have nothing to do with your code or be an area
| you 're an expert in as a Go developer.
| irusensei wrote:
| Golang works on Plan9. It can even bootstrap itself. A
| few months ago I was trying to setup some server software
| on 9Front for giggles and while most stuff worked I
| couldn't past the Sqlite CGO dependencies.
| ncruces wrote:
| If you still have that itch to scratch, you can try:
| https://github.com/ncruces/go-sqlite3
|
| You'll need to use the sqlite3_nolock build tag;
| concurrent writes will quickly corrupt your database.
| SetMaxOpenConns(1) is your friend.
|
| But it should work. I'm interested if it doesn't.
| Feedback appreciated.
| ComputerGuru wrote:
| But compiling the non-go assistant process is not going to
| be any easier than cgo, right?
| codetrotter wrote:
| In the mentioned https://gitlab.com/cznic/sqlite there
| would not be any assistant process, right?
| jerf wrote:
| No, but that has the disadvantage of being C compiled
| into Go, then being compiled into native executable.
|
| I'm actually surprised by how readable this came out;
| props to the Go->C compiler author. But you can guess
| that pushing this sort of thing through the Go compiler
| is going to cause some slowdowns due to sheer paradigm
| mismatch: https://gitlab.com/cznic/sqlite/-/blob/master/l
| ib/sqlite_lin...
| throwaway894345 wrote:
| I don't think the paradigm is particularly mismatched,
| right? If you translate C to Go, it would be pretty much
| best case for Go (neither language likes lots of small
| allocations). But Go lacks a lot of optimizations that
| most C compilers support, like return value optimization,
| aggressive inlining, etc. C also has lighter-weight
| function calls and so on that you pay for in Go.
|
| Maybe this is what you mean by paradigm mismatch, but
| usually I would think of something like translating an
| allocation-heavy Java app into Go as paradigm mismatch.
| parhamn wrote:
| Well its a different access pattern. As the underlying
| library points out:
|
| > It is used in programming environments that do not
| allow calling C API functions.
|
| Also I guess which one is easier will be subjective. The
| steps are sorta similar:
|
| Step 1) install sqlite or squinn on the base system (the
| latter might be harder)
|
| Step 2) if sqllite use cgo, if squinn just use go (the
| former might be harder but more performant)
| llimllib wrote:
| the other pain point I know of is that it's hard to cross-
| compile. You can do it with zig[1], but it's still not
| pleasant.
|
| 1: https://zig.news/kristoff/building-sqlite-with-cgo-for-
| every...
| tptacek wrote:
| Or with musl-cross:
|
| https://github.com/FiloSottile/homebrew-musl-cross
|
| It works pretty well! It's a thing you might keep in your
| back pocket to test builds from your ARM dev machine on a
| dev host, and then let the CI/CD system build the real
| version later.
| arp242 wrote:
| And Zig only covers a pretty limited range of platforms to
| start with.
| kunley wrote:
| Yep, confirming that cznic's pure-Go modernc.org/sqlite works
| great.
| tedunangst wrote:
| The logic of easy cross compiles doesn't really hold up for go
| translated SQLite. It depends on a huge pile of per platform
| support code, of varying quality. If you're only going to
| target known working platforms, may as well use cgo and a known
| working cross compiler.
| ncruces wrote:
| Does compiling to WASM, using a cgo free WASM runtime, and
| replacing the OS layer (VFS) with portable Go code count?
|
| That's the elevator pitch (so far) for:
| https://github.com/ncruces/go-sqlite3/tree/main
| tedunangst wrote:
| Yeah, that's cool. :) although probably a bit slower
| running through an interpreter.
|
| Modernc/libc takes a rather different approach. It's got
| some pretty funny files in it. https://gitlab.com/cznic/lib
| c/-/blob/master/musl_openbsd_arm...
| ncruces wrote:
| It _is_ slower.
|
| The WASM runtime https://wazero.io has a compiler on
| amd64 and arm64 (on Linux, macOS, Windows, and FreeBSD),
| but the current compiler while very fast (at compiling),
| is very naive (generates less than optimal code).
|
| An optimizing compiler is currently being developed, and
| should be released in the coming months. I'm optimistic
| that this compiler _will_ cover the performance gap
| between WASM and modernc.
| eatonphil wrote:
| I don't think cznic's argument is about cross compiling but
| just being able to avoid cgo. Since some people really don't
| like cgo.
| tgv wrote:
| It's not 100% Go, though. It forks a process that manages the
| sqlite file. Since the communication with it is written in go, it
| avoids using cgo.
| [deleted]
___________________________________________________________________
(page generated 2023-10-04 23:00 UTC)