[HN Gopher] Show HN: Brioche - A new Nix-like package manager
___________________________________________________________________
Show HN: Brioche - A new Nix-like package manager
This is a project I've wanted to write for a long time now. I
really love the ideas from Nix and I still have a ton of respect
for the project, but Nix-the-language never felt intuitive to me
and I wanted something with more approachable tooling (although
this was circa 2016, so I'm sure Nix has improved a lot since then
too-- that was before Flakes were around!) Anyway, I started on
the current iteration of Brioche about 6 months ago, and I finally
cut an initial release. I'd still consider this a "technical
preview" version (performance especially is pretty painful, so
that'll be a focus of mine in the coming weeks). But it's finally
at a point where it does work end-to-end and folks can take it for
a test drive!
Author : kylewlacy
Score : 72 points
Date : 2024-06-03 16:00 UTC (7 hours ago)
(HTM) web link (brioche.dev)
(TXT) w3m dump (brioche.dev)
| tombert wrote:
| Flakes definitely help with the giant megarepo annoyances of
| NixOS, though they're still a little irritating. If you are
| writing in a languages that doesn't hasn't had its packages
| directly integrated into the build system (like Python's has), it
| can be really irritating to do anything with them, since the `nix
| build` command disables network access and you therefore cannot
| use regular package manager.
|
| I'm doing a project in Julia, and I'm using Nix Flakes to do it,
| but it's been immensely annoying to actually get that working
| [1]. As a result, I've had avoid using the `nix build` command
| entirely (though the flakes are still useful for the `nix
| develop` command).
|
| All that said, do you plan on having Brioche work with
| reproducible builds, and if so do you have a plan to make what I
| mentioned a bit less irritating?
|
| [1] I know Julia2Nix exists, and I have never managed to actually
| get that working on any platform.
| kylewlacy wrote:
| This was one of my bigger pain points with Nix as well: there
| was a lot of "reinventing the world" just to avoid network
| access. With Brioche, I stuck with disabling network access by
| default, but there's an escape hatch to specifically opt-in to
| networking (by calling `.unsafe({ networking: true })` on a
| process recipe). My thoughts are that Cargo, NPM, Poetry, etc.
| have all done a great job building amazing tooling to download
| and verify resources from the network against a lockfile, and I
| wanted to be able to leverage as much of that as possible. So,
| for example, `npm clean-install` will give you more-or-less the
| same guarantees that Nix does, so my thought was as long as
| it's encapsulated properly, I'd rather lean on existing tooling
| (that's also why I used the term "unsafe", you need to make
| sure you only access the network with tools that do sufficient
| verification!)
|
| I've generally stayed away from using the term "reproducible
| build" when talking about Brioche, because I don't feel like it
| fits the reproducible-builds.org definition (though I don't
| think Nix does either). But, if a build is cached locally or in
| the registry, then you're guaranteed to get the same result,
| since it'll just re-use the cache
|
| The sandboxing also gives pretty strong guarantees around
| hermetic builds[1]. So I think you could do reproducible builds
| _within_ Brioche (and I'd like to add tools to make this even
| easier), but I'd say Brioche itself doesn't give you
| reproducible builds out of the box
|
| [1]: https://bazel.build/basics/hermeticity
| tombert wrote:
| That's actually really great to hear! I might need to play
| with this tonight then.
|
| It looks like the project files aren't radically dissimilar
| to Flakes, so I think you're really on the right track for
| making something that could be really useful for a lot of
| people. Great work!
| forgotpwd16 wrote:
| >`nix build` command disables network access
|
| Only if building with sandbox enabled. Can disable it if
| network access is required. Seems someone opened an issue
| asking[1] for granular permissions (explicit network
| restriction) but has been marked as stale. In same issue
| someone else has made comment providing an hybrid approach.
|
| Since you mentioned Julia, it's possible to build Julia
| environments (with arbitrary packages) using the
| `.withPackages` function. E.g. `julia.withPackages ["Plots"]`.
|
| [1]: https://github.com/NixOS/nix/issues/4584
| tombert wrote:
| I didn't realize that. I will give that a look.
| Aerbil313 wrote:
| Beat me to it! It's my dream to write a Nix replacement in a more
| approachable language. I can't state enough how the choice of a
| bespoke DSL _that doesn 't immediately make sense to most
| developers_ keeps Nix/Nixos community from growing. And Nix is 18
| years old now, it's long overdue for a rewrite[1]. Don't
| implement channels to start with. Focus on full reproducibility.
| (...getting into sci-fi territory...) Build in QEMU and KVM
| virtualization to be able to build and _run_ most pieces of
| software ever published. No, Docker is a horribly leaky
| abstraction. Run a torrent network by default as cache.nixos.org
| equivalent, because you can 't keep up with the demand with
| centralized solutions, even nixpkgs gave up on it despite their
| free credits on AWS.
|
| 1: I believe all software without exception needs a full rewrite
| _at least_ every 10 years.
| tombert wrote:
| I think Flakes kind of help with the centralized problems. At
| least individual flakes can be run or installed separately,
| they don't have to be merged into the core nixpkgs repo, and
| they (at least in theory) should be giving the same build
| reproducibility as you'd get from installing directly from
| nixpkgs.
|
| I agree with the torrent idea though.
| phlip9 wrote:
| Congrats on the release! I love the focus on devex w/ typescript
| and autocomplete. That's probably one of my biggest pain points
| with Nix -- writing any non-trivial package always requires a
| ripgrep adventure through nixpkgs. Finding the right poorly
| documented and poorly discoverable derivation attributes is
| always such a chore.
|
| What are your plans for cross-compilation or heavy package
| customization? One of nixpkgs coolest party tricks imo is that
| you can just change the stdenv and get a musl static binary or
| cross-compiled binary.
| kylewlacy wrote:
| > What are your plans for cross-compilation or heavy package
| customization? One of nixpkgs coolest party tricks imo is that
| you can just change the stdenv and get a musl static binary or
| cross-compiled binary.
|
| So in general, I don't think I'm going to have anything quite
| as powerful as Nix's overrides. But I'm hoping most of the use-
| cases for it will be covered by some simpler options:
|
| - Since build definitions are functions, package authors can
| just take arguments for the things they want downstream users
| to be able to customize (e.g. `configure` flags, optional
| features and plugins, etc.)
|
| - I haven't built it yet, but I think adding support for
| dependency overrides would be fairly easy, a la Cargo.
| Basically, you'd just fork or clone the package you want to
| tweak, make your tweaks, then set an "overrides" option to use
| it instead. I know that's not a super satisfying answer, but
| that should help cover a lot of use cases
|
| - For toolchains specifically, I have an idea in mind for this
| as well (also not implemented at all). At a high level, the
| idea is that packages could use "dynamic bindings", which you
| can then override for downstream recipes (this would require
| some new runtime features in Brioche itself). The toolchain
| itself would effectively be a dynamic binding, letting you pick
| a different recipe (so you could swap glibc for musl, or gcc
| for clang, etc). Cross-compilation would also be built on this
| same feature
| charlotte-fyi wrote:
| Very exciting. The ideas behind Nix are so good and everything
| else so bad. It seems like a lot of people are trying to solve
| this by building abstractions on top of Nix, but I'm really
| skeptical that is the solution. As crazy as a full rewrite is, I
| hope someone succeeds!
| tombert wrote:
| Agreed. NixOS is a marvel of engineering to me, and kind of
| hard to go back from once you get used to it. Automatic
| snapshotting on every configuration change, the entire system
| state being configurable through text files and therefore never
| being ambiguous, being able to temporarily install stuff
| without it polluting your path for forever by using nix-shells,
| clearly being able to see and define stuff like boot parameters
| and kernel modules are just insanely wonderful things, all
| while still using (I think) a vanilla kernel and really no
| runtime overhead, allowing you to make an insanely lean system
| without ever being unsure if you're missing something or having
| to worry about extra crap that you installed by accident
| lingering forever. In my mind about as close to an "objectively
| better" way to handle an OS (at least for people who are
| technical) as one can ask for. I have no desire to go back to
| any other distro for my server.
|
| But the Nix language itself is really quite annoying. I mean,
| I've more or less gotten used to its annoyances, and I do think
| that some of the DSLs it has are excellent (I _really_ like the
| Nginx and systemd configuration stuff, for example), and a lot
| of the configs are just `services.myservice.enable = true`
| which is fine, but a lot of the time I 'm kind of confused
| about what syntax is allowed and how loops work and the like.
| It's not horrible or anything, just a bit annoying because I'll
| occasionally have to do a nixos-rebuild like three or four
| times because I messed up some subtle syntax, and it's
| _especially_ annoying if I have to go dig at the root Nix
| package to find out what I did wrong [1].
|
| I think decentralizing stuff in the form of flakes might be
| able to help with this, if for no other reason the area in
| which you'd be forced to look for configuration stuff could be
| reduced, but I do think NixOS would benefit from some
| rearchitecture.
|
| [1] Which happened yesterday with an ethernet card
| configuration:
| https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modu...
| andrewla wrote:
| I'm very excited by all the attempts to replace Nix, but I don't
| think I'll be exploring this much deeper.
|
| In my opinion the issue with Nix is that the data model is not
| crisply defined -- it's there, but hidden under a lot of goop
| that is the Nix language itself and the various assumptions and
| baggage that goes with it.
|
| What I want is a primarily declarative syntax supporting a rich
| set of data structures, ideally a non-Turing set of primitives,
| with a much more intuitive way of gluing things together. So
| basically bash (or even sh) with a well-defined way of
| transmitting environment variables and setting up the
| environment.
|
| The idea of importing a language that has broader support
| (typescript) as an alternative to the Nix language seems
| appealing at first, but typescript is such a high-dependency
| system that it's hard to get excited about it.
| infogulch wrote:
| Agreed on all counts, especially the central issue with nix and
| the properties that I'd want out of a replacement. I think CUE
| ( https://cuelang.org/ ) is a perfect language for this.
| nine_k wrote:
| Cue is fun, but already the unification example
| (https://cuelang.org/docs/tour/basics/duplicate-fields/)
| shouts "footguns" to me :(
|
| TS is certainly excessively powerful though.
| infogulch wrote:
| I don't think this would be a problem in practice. By
| default structs are "open" in cue, i.e. as long as matching
| fields unify, disparate fields just merge. There are also
| "closed" structs that allow creating a definition that
| fails to unify with fields not explicitly listed.
| https://cuelang.org/docs/tour/types/closed/
| from-nibly wrote:
| What foot guns are you seeing specifically?
| likium wrote:
| Starlark is widely used by Bazel and Buck. That would fulfill
| similar properties and build system folks have one fewer thing
| to learn.
| leoh wrote:
| flox is the best thing I know of for articulating binary
| dependencies (language runtimes, etc.), which is probably the
| sweet spot for nix anyways at the moment (i.e. as opposed to
| trying to build everything "With the One [Tool] to [Replace]
| Them All"). flox uses nix for its backend, but has a simple
| TOML syntax and is properly humble about what it can do -- but
| killer at it -- as opposed to promising the world.
|
| https://flox.dev
| charlotte-fyi wrote:
| I think that declarative configuration languages scratch the
| itch of a lot of developer brains, but in practice it doesn't
| really matter and using more familiar languages is a huge
| benefit to devex that shouldn't be overlooked if the goal is to
| be mainstream. Yes, it requires more discipline on the reviewer
| front not to create abstractions that are too
| powerful/footgun-y, but it's totally possible to write clear,
| declarative code in a "real" programming language without
| artificial constraints.
| __MatrixMan__ wrote:
| I'm not so worried about my disciplined coworker who just
| wants to help. If we were all reviewing his code I'd agree
| with you.
|
| The people I want to help are those who are unknowingly
| reviewing malicious commits, and I think that declarative
| configuration languages have a part to play there.
| charlotte-fyi wrote:
| This is solved in tools like Pulumi by having a declarative
| and auditable build artifact as an intermediate step that
| can be diffed. This seems to solve a lot of the security
| issues (and is generally a good idea anyway).
| NegativeLatency wrote:
| I would still prefer to debug terraform (which is a fair
| bit more declarative) rather than pulumi
| zamalek wrote:
| I'm also exploring a post-Nix package manager, wip:
| https://github.com/porkg/porkg
|
| My mantra has been to avoid "getting bored" and inventing DSLs
| for the longest time. I initially sought to use Nickel-lang, but
| it was missing some features that would make it an ideal
| candidate for this. I started writing my own (you may find this
| in the history) before realizing "WTF are you doing writing
| another shitty DSL?" I have subsequently decided that shell
| scripts (or anything you can shebang) are good enough, i.e.
| pkgbuild inspiration.
|
| I also plan to avoid making a derivation the source of
| reproducibility. Instead, a lockfile will offer that. This should
| alleviate the issue whereby updating the like of glibc cascades
| into an entire rebuild.
|
| Any *OS and home-manager would need to bring in a configuration
| language. I think Cue really makes the most sense, but that's
| still a long way off.
|
| Nice to see you used JS instead of yet another DSL :)
| Aerbil313 wrote:
| I suggest the post author and you look into nu as the package
| build language. It's uniquely suited for this task, being a
| cross-platform shell[1] with real programming language
| features, data types, and using built-in uutils instead of
| relying on platform-dependent coreutils. It also limits
| mutability[2], favoring a more functional than imperative
| approach, which would at least theoretically help with
| reproducibility. A guy from Determinate Systems recently made
| an experiment on integrating nushell to nix and the results
| were positive. Take a look:
| https://determinate.systems/posts/nuenv/
|
| 1: https://www.nushell.sh/
|
| 2: https://www.nushell.sh/book/thinking_in_nu.html#variables-
| ar...
| kylewlacy wrote:
| Nushell has definitely been on my radar for a while! but I
| missed DetSys was playing around with it, I'll have to give
| that a look
|
| One neat thing about Brioche is that Bash isn't really baked
| into it the same way as the Nix ecosystem. In a lot of the
| docs and examples, I used the `std.runBash` Brioche function
| for running scripts, but that's a fairly simple 11-line
| function[^1]. I wanted to make it just as easy to add
| `runZsh`, `runFish`, `runNushell`, etc. functions that work
| just as well as Bash (and they could either be put in the std
| library or in their own packages). So hopefully, there will
| be a Brioche `runNushell` function in the near future :)
|
| [^1]: https://github.com/brioche-dev/brioche-
| packages/blob/9fd5109...
| kylewlacy wrote:
| ooh, porkg looks pretty interesting, will definitely keep an
| eye on the project!
|
| Definitely agreed about the lockfile ideas! I went with a
| fairly similar design, although I haven't really escaped the
| "rebuild the world" situation yet and I've rebuilt a _lot_ of
| copies of gcc from source by now! (that's also partially
| because I set up the packages repo as a workspace[1], which I
| felt was easier to iterate on in the early days... I might
| eventually split each package into their own projects or
| separate repos so they can keep lockfiles independent of each
| other)
|
| and yeah, I definitely felt the temptation to write my own DSL
| but I stayed strong! I just knew that it'd be a huge uphill
| battle, especially because I wanted to provide good editor
| support (IMO implementing a language is (relatively) easy, but
| implementing a language with good error messages and LSP
| support is crazy hard)
|
| [^1]: https://brioche.dev/docs/core-concepts/workspaces/
| __MatrixMan__ wrote:
| Some of the oddities of the nix language are pretty useful for
| its domain. Recursive attribute sets, for instance, save a lot of
| headaches if you're trying to have only a single source of truth.
| Do you feel like these translate to typescript nicely?
|
| As somebody who knows nix but doesn't know typescript, I found
| myself looking for a rosetta stone page where I could look at two
| chunks of code that do the same thing, but in separate languages.
| This would let me use the familiar language to scrutinize the
| other.
|
| I wouldn't normally ask for such a thing, but if you're putting
| "Nix-like" in the title then maybe it might be worth adding a
| comparison page to the docs.
| charlotte-fyi wrote:
| The syntax isn't as clean but you can add property accessors to
| objects in JavaScript that accomplishes the same thing.
| kylewlacy wrote:
| I've been asked about overrides and attributes a lot! That was
| one of the sacrifices I had to make to go with a more
| traditional language, so it's definitely a negative point
| compared to Nix. That said, I'm hoping to have some conventions
| and features that will cover _most_ of the use-cases that
| overrides give you, but that's definitely still future work at
| this point and I probably won't be able to cover everything
| overrides do
|
| And yep, I think having a "Brioche for Nix users" guide makes a
| lot of sense, although it's not the first time that question so
| I'll probably stand up a first-pass version of it sooner rather
| than later (my Nix skills are also pretty rusty-- I'll need to
| brush up a bit first before I write it!)
| leoh wrote:
| I am not writing typescript -- heaven forbid -- so I can manage
| cargo, so I can write Rust.
|
| Dropping the snark, though -- most times I have seen folks
| attempt to fold other build systems into some polyglot common
| one, it seems to create a lot of problems -- for at the end of
| the day, build tools (whether npm, cargo, gradle, docker, etc.)
| each have their issues. When these issues arise, they require
| understanding the original build tool. Bazel, for example, is a
| "polyglot" system which many try to use to manage npm. In my
| experience, when folks use bazel but have never used npm
| directly, they often get lost, because they now have to deal with
| two abstractions (bazel, npm) simultaneously neither have a way
| for bazel to do the thing they need, nor an understanding of how
| to get bazel to do that thing (nor npm itself to do the thing),
| also because they have never worked with npm directly.
|
| Same goes for cargo, incidentally. Switching over to that build
| tool -- like any major build tool, it has an insane amount of
| well-honed documentation and thousands of answered questions
| online (not to mention LLMs trained somewhat well on it). Users
| when they need to do something with brioche or have an issue with
| it will not necessarily have the benefit of the massive
| collaborative support available for cargo.
|
| Incidentally, I don't think that nix is free of this problem,
| though it does attempt to manage these issues -- sometimes
| reasonably successfully -- by virtue of keeping interfaces to
| other build tools in simple, direct ways.
|
| Even then, however, the best way I have seen nix used is as a
| system for declaring all binary dependencies or other
| dependencies that are either not manageable or especially
| competently managed by other build tools -- which nix allows for
| in a virtual environment by shipping a shell.nix or flake.
|
| In other words -- bringing in the right version of java, gradle,
| node, etc. But then just directing users to the specific, native
| build tools that often have a massive number of contributors.
|
| The closest thing to a tool that does this (guides developers to
| other build tools) is flox, which uses TOML, but uses nix for its
| backend via C API. I haven't had a chance to use it much yet, but
| it looks very promising https://flox.dev.
|
| ===
|
| Okay, all this rambling aside -- building a tool like this for
| yourself or a small team, especially when you have the time to do
| it, it works well, and you're having fun... well, there's an
| insane amount to be said for that; and it will doubtlessly lead
| you to learning a ton about other build tools and likely offer
| you deep insights that are only possible when attempting to build
| something like brioche. So, on that count, major kudos to you.
___________________________________________________________________
(page generated 2024-06-03 23:01 UTC)