[HN Gopher] Show HN: voidDB - A transactional key-value DB writt...
___________________________________________________________________
Show HN: voidDB - A transactional key-value DB written in Go for
64-bit Linux
Author : joel-ling
Score : 59 points
Date : 2025-01-31 13:45 UTC (18 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| joel-ling wrote:
| Seemingly faster and more compact than lmdb-go, bbolt, Badger,
| and goleveldb at <2K LOC. goos: linux
| goarch: arm64 pkg: github.com/voidDB/voidDB/test
| BenchmarkVoidPut-2 131072 14933 ns/op
| BenchmarkVoidGet-2 131072 1060 ns/op
| BenchmarkVoidGetNext-2 131072 245.8 ns/op
| BenchmarkLMDBPut-2 131072 22414 ns/op
| BenchmarkLMDBGet-2 131072 1826 ns/op
| BenchmarkLMDBGetNext-2 131072 602.2 ns/op
| BenchmarkBoltPut-2 131072 66984 ns/op
| BenchmarkBoltGet-2 131072 2552 ns/op
| BenchmarkBoltGetNext-2 131072 254.6 ns/op
| BenchmarkLevelPut-2 131072 44182 ns/op
| BenchmarkLevelGet-2 131072 30949 ns/op
| BenchmarkLevelGetNext-2 131072 3441 ns/op
| BenchmarkBadgerPut-2 131072 15182 ns/op
| BenchmarkBadgerGet-2 131072 33114 ns/op
| BenchmarkBadgerGetNext-2 131072 12895 ns/op
| BenchmarkNothing-2 131072 0.3239 ns/op
|
| Check out the docs at https://pkg.go.dev/github.com/voidDB/voidDB
| KAdot wrote:
| The benchmark doesn't accurately represent the real-world
| database performance because the dataset is too small (roughly
| half a gigabyte based on [1]?), meaning it fits into the page
| cache bypassing disk I/O.
|
| [1]:
| https://github.com/voidDB/voidDB/blob/master/test/bench_test...
| chronode wrote:
| This is a cool effort but their claims are wildly misleading
| as their benchmarks aren't fair across the board:
|
| For example:
|
| - BenchmarkVoidPut runs a single O(n) sized transaction
|
| - BenchmarkLMDBPut runs a single O(n) sized transaction
|
| - BenchmarkBadgetPut runs O(n) O(n) sized transactions (!!!)
|
| I made a local change so that all Put benchmarks ran O(n)
| O(1) sized transactions and the results were quite different:
| Void was the slowest, followed by LMDB, Bold, LevelDB, then
| Badger.
|
| I'd also wager the LMDB author would also (lovingly!) tell us
| we're holding it wrong
| arandomusername wrote:
| LMDB is the only one here in C, so the interop is probably
| what makes LMDB so slow in here
| fasteo wrote:
| // voidDB is a cherished toy, a journey into the Unknown, a
| heroic struggle, // and a work of love. It is the "Twee!"
| of a bird; a tree falling in the // forest; yet another
| programmer pouring their drop into the proverbial [bit] //
| bucket. Above all, it is a shrine unto simple, readable, and
| functional // code; an assertion that the dichotomy between
| such aesthetics and practical // performance is mere
| illusion.
|
| [1]
| https://github.com/voidDB/voidDB/blob/ac104867749e2c698b3d61...
| srameshc wrote:
| I admire the author who took up the challenge of writing a KV
| DB and cherishing self's efforts. I hope someday I could do
| something similar and feel accomplished.
| joel-ling wrote:
| Five years ago, I was new to KV DBs and Golang. You can!
| kunley wrote:
| Love the poetry in the comment prose! Endeavour so much needed
| in the industry. It can only come out of the joy of coding, how
| else could it be born...
| jakewins wrote:
| It always made me happy to hack nearby the Neo4j page cache
| "Muninn", because it meant I'd pass by Chris Vest's poem and
| giant ascii Raven: https://github.com/neo4j/neo4j/blob/releas
| e/5.26.0/community...
| joel-ling wrote:
| Beautiful reminder of the often forgotten aspect of
| technical work.
|
| Huginn: thought, perception, comprehension, ...
|
| Muninn: care, urge, wonder, curiosity, interest, ...
|
| https://en.wikipedia.org/wiki/Huginn_and_Muninn#Etymology
| joel-ling wrote:
| Thank you for highlighting. These contain references to some
| philosophies I relate to and find comforting, including those
| most prominently articulated by Ernest Becker and Alan Watts.
| sushidev wrote:
| Faster than rocksdb?
| dangoodmanUT wrote:
| probably not if it's LMDB-based (at least for writes)
| sirjaz wrote:
| Since there are no strict OS dependency I can see, has anyone
| tried to compile this and run it on Windows?
| 38 wrote:
| seems pretty bloated
|
| https://github.com/voidDB/voidDB/blob/master/go.sum
| seanw444 wrote:
| A lot of it is indirect dependencies, and most of the direct
| dependencies in the go.mod are just for benchmarks, as far as I
| can tell.
| joel-ling wrote:
| That is correct. Remove directories "test/" and
| "cursor/test/", then run "go mod tidy"; you should see only
| one dependency in go.mod: require
| golang.org/x/sys v0.29.0
|
| Could have used "syscall" in the standard library instead of
| "golang.org/x/sys/unix" if not for these:
| https://pkg.go.dev/golang.org/x/sys/unix#F_OFD_GETLK
|
| ... and this: https://go.googlesource.com/proposal/+/refs/hea
| ds/master/des...
| maxmcd wrote:
| Only "x/sys/unix" outside of tests/benchmarks
| ~/go/src/github.com/voidDB/voidDB git:(master) $ go
| run github.com/tailscale/depaware@latest
| github.com/voidDB/voidDB dependencies: (generated by
| github.com/tailscale/depaware)
| github.com/voidDB/voidDB/common
| from github.com/voidDB/voidDB+
| github.com/voidDB/voidDB/cursor
| from github.com/voidDB/voidDB
| github.com/voidDB/voidDB/free
| from github.com/voidDB/voidDB
| github.com/voidDB/voidDB/node
| from github.com/voidDB/voidDB+
| github.com/voidDB/voidDB/reader
| from github.com/voidDB/voidDB
| golang.org/x/sys/unix
| from github.com/voidDB/voidDB+ bytes
| from github.com/voidDB/voidDB+ cmp
| from internal/fmtsort+ encoding/binary
| from github.com/voidDB/voidDB/common+ errors
| from bytes+ D fmt
| from golang.org/x/sys/unix hash
| from github.com/voidDB/voidDB+ hash/fnv
| from github.com/voidDB/voidDB io
| from bytes+ io/fs
| from internal/filepathlite+ iter
| from reflect+ math
| from encoding/binary+ math/bits
| from golang.org/x/sys/unix+ os
| from fmt+ path
| from io/fs reflect
| from encoding/binary+ slices
| from encoding/binary+ LD sort
| from golang.org/x/sys/unix strconv
| from fmt+ LD strings
| from golang.org/x/sys/unix sync
| from encoding/binary+ sync/atomic
| from internal/bisect+ syscall
| from github.com/voidDB/voidDB/cursor+ time
| from github.com/voidDB/voidDB+ unicode
| from bytes+ W unicode/utf16
| from internal/poll+ unicode/utf8
| from bytes+
| joel-ling wrote:
| Thank you for showing substantive evidence in defence of
| voidDB; it is very vindicating.
| ncruces wrote:
| It doesn't take much to split your go.mod file require
| blocks into needed dependencies, and test dependencies,
| with a few comments explaining the why of each dependency.
|
| Go tooling will preserve your go.mod comments and
| structure.
|
| You can also consider moving integration tests and
| benchmarks to another module in the same repo.
| binary132 wrote:
| I can't help but think this would be more valuable as a C (or
| even Rust or Zig!) library with a Go API. Then it could be used
| elsewhere too.
| natas wrote:
| it's screaming for a REPL...
| alexpadula wrote:
| Very cool! keep it up :)
___________________________________________________________________
(page generated 2025-02-01 08:01 UTC)