[HN Gopher] Show HN: Brioche - A new Nix-like package manager
___________________________________________________________________
Show HN: Brioche - A new Nix-like package manager
This is a project I've wanted to write for a long time now. I
really love the ideas from Nix and I still have a ton of respect
for the project, but Nix-the-language never felt intuitive to me
and I wanted something with more approachable tooling (although
this was circa 2016, so I'm sure Nix has improved a lot since then
too-- that was before Flakes were around!) Anyway, I started on
the current iteration of Brioche about 6 months ago, and I finally
cut an initial release. I'd still consider this a "technical
preview" version (performance especially is pretty painful, so
that'll be a focus of mine in the coming weeks). But it's finally
at a point where it does work end-to-end and folks can take it for
a test drive!
Author : kylewlacy
Score : 150 points
Date : 2024-06-03 16:00 UTC (1 days ago)
(HTM) web link (brioche.dev)
(TXT) w3m dump (brioche.dev)
| tombert wrote:
| Flakes definitely help with the giant megarepo annoyances of
| NixOS, though they're still a little irritating. If you are
| writing in a languages that doesn't hasn't had its packages
| directly integrated into the build system (like Python's has), it
| can be really irritating to do anything with them, since the `nix
| build` command disables network access and you therefore cannot
| use regular package manager.
|
| I'm doing a project in Julia, and I'm using Nix Flakes to do it,
| but it's been immensely annoying to actually get that working
| [1]. As a result, I've had avoid using the `nix build` command
| entirely (though the flakes are still useful for the `nix
| develop` command).
|
| All that said, do you plan on having Brioche work with
| reproducible builds, and if so do you have a plan to make what I
| mentioned a bit less irritating?
|
| [1] I know Julia2Nix exists, and I have never managed to actually
| get that working on any platform.
| kylewlacy wrote:
| This was one of my bigger pain points with Nix as well: there
| was a lot of "reinventing the world" just to avoid network
| access. With Brioche, I stuck with disabling network access by
| default, but there's an escape hatch to specifically opt-in to
| networking (by calling `.unsafe({ networking: true })` on a
| process recipe). My thoughts are that Cargo, NPM, Poetry, etc.
| have all done a great job building amazing tooling to download
| and verify resources from the network against a lockfile, and I
| wanted to be able to leverage as much of that as possible. So,
| for example, `npm clean-install` will give you more-or-less the
| same guarantees that Nix does, so my thought was as long as
| it's encapsulated properly, I'd rather lean on existing tooling
| (that's also why I used the term "unsafe", you need to make
| sure you only access the network with tools that do sufficient
| verification!)
|
| I've generally stayed away from using the term "reproducible
| build" when talking about Brioche, because I don't feel like it
| fits the reproducible-builds.org definition (though I don't
| think Nix does either). But, if a build is cached locally or in
| the registry, then you're guaranteed to get the same result,
| since it'll just re-use the cache
|
| The sandboxing also gives pretty strong guarantees around
| hermetic builds[1]. So I think you could do reproducible builds
| _within_ Brioche (and I'd like to add tools to make this even
| easier), but I'd say Brioche itself doesn't give you
| reproducible builds out of the box
|
| [1]: https://bazel.build/basics/hermeticity
| tombert wrote:
| That's actually really great to hear! I might need to play
| with this tonight then.
|
| It looks like the project files aren't radically dissimilar
| to Flakes, so I think you're really on the right track for
| making something that could be really useful for a lot of
| people. Great work!
| fire_lake wrote:
| > have all done a great job building amazing tooling to
| download and verify resources from the network against a
| lockfile
|
| This is a bad assumption
| IshKebab wrote:
| Why? Are you referring to the fact that it's opt-in; you
| have to use `cargo build --locked` or `npm ci`?
| bavarianbob wrote:
| I interpreted the parent to mean that it might not be
| fair to assume existing package managers have done a
| _great_ job at downloading and verifying, especially
| verifying, resources from the network. There are
| businesses that exist attempting to solve this problem
| like socket.io. Safely installing the correct
| dependencies for a project is still not a guarantee from
| any of the major package managers.
| YuukiRey wrote:
| If the underlying package eco system is a mess then it'll
| also be a mess in Nix. To this day Nix doesn't really have
| a good answer to Python or JavaScript packaging.
|
| If the underlying package manager is good (Rust, Go) then
| there's just no need to reinvent the wheel.
| forgotpwd16 wrote:
| >`nix build` command disables network access
|
| Only if building with sandbox enabled. Can disable it if
| network access is required. Seems someone opened an issue
| asking[1] for granular permissions (explicit network
| restriction) but has been marked as stale. In same issue
| someone else has made comment providing an hybrid approach.
|
| Since you mentioned Julia, it's possible to build Julia
| environments (with arbitrary packages) using the
| `.withPackages` function. E.g. `julia.withPackages ["Plots"]`.
|
| [1]: https://github.com/NixOS/nix/issues/4584
| tombert wrote:
| I didn't realize that. I will give that a look.
| Filligree wrote:
| Instead of disabling it completely, you can set it to
| 'relaxed'. That still allows network access, but maintains
| the other isolation guarantees.
|
| Obviously this introduces a potential impurity. I use it
| for installing NeoForge into a minecraft server derivation;
| the downloaded files _could_ change behind my back, but so
| long as the version doesn 't change it should still be
| compatible. It hasn't yet caused trouble.
|
| NixOS has a lot of escape hatches once you look around, and
| you shouldn't be afraid to use them. The downside of impure
| derivations aren't quite as bad as not-using-derivations,
| anyway.
|
| Another option is to declare it as a fixed-output
| derivation, in which case network access is enabled by
| default. This doesn't work for most installers, though; at
| a minimum you'll need to delete logfiles that might contain
| timestamps.
| Rucadi wrote:
| If you mark a derivation as impure with __impure=true; and
| activate "ca-derivations flakes nix-command" as experimental
| features, you can access the internet.
|
| I agree sometimes is a pain tho.
|
| But most of the time, If you need internet access, if you can
| specify the "hash" of the result file, anything that you do to
| achieve that file can have internet access.
| Aerbil313 wrote:
| Beat me to it! It's my dream to write a Nix replacement in a more
| approachable language. I can't state enough how the choice of a
| bespoke DSL _that doesn 't immediately make sense to most
| developers_ keeps Nix/Nixos community from growing. And Nix is 18
| years old now, it's long overdue for a rewrite[1]. Don't
| implement channels to start with. Focus on full reproducibility.
| (...getting into sci-fi territory...) Build in QEMU and KVM
| virtualization to be able to build and _run_ most pieces of
| software ever published. No, Docker is a horribly leaky
| abstraction. Run a torrent network by default as cache.nixos.org
| equivalent, because you can 't keep up with the demand with
| centralized solutions, even nixpkgs gave up on it despite their
| free credits on AWS.
|
| 1: I believe all software without exception needs a full rewrite
| _at least_ every 10 years.
| tombert wrote:
| I think Flakes kind of help with the centralized problems. At
| least individual flakes can be run or installed separately,
| they don't have to be merged into the core nixpkgs repo, and
| they (at least in theory) should be giving the same build
| reproducibility as you'd get from installing directly from
| nixpkgs.
|
| I agree with the torrent idea though.
| phlip9 wrote:
| Congrats on the release! I love the focus on devex w/ typescript
| and autocomplete. That's probably one of my biggest pain points
| with Nix -- writing any non-trivial package always requires a
| ripgrep adventure through nixpkgs. Finding the right poorly
| documented and poorly discoverable derivation attributes is
| always such a chore.
|
| What are your plans for cross-compilation or heavy package
| customization? One of nixpkgs coolest party tricks imo is that
| you can just change the stdenv and get a musl static binary or
| cross-compiled binary.
| kylewlacy wrote:
| > What are your plans for cross-compilation or heavy package
| customization? One of nixpkgs coolest party tricks imo is that
| you can just change the stdenv and get a musl static binary or
| cross-compiled binary.
|
| So in general, I don't think I'm going to have anything quite
| as powerful as Nix's overrides. But I'm hoping most of the use-
| cases for it will be covered by some simpler options:
|
| - Since build definitions are functions, package authors can
| just take arguments for the things they want downstream users
| to be able to customize (e.g. `configure` flags, optional
| features and plugins, etc.)
|
| - I haven't built it yet, but I think adding support for
| dependency overrides would be fairly easy, a la Cargo.
| Basically, you'd just fork or clone the package you want to
| tweak, make your tweaks, then set an "overrides" option to use
| it instead. I know that's not a super satisfying answer, but
| that should help cover a lot of use cases
|
| - For toolchains specifically, I have an idea in mind for this
| as well (also not implemented at all). At a high level, the
| idea is that packages could use "dynamic bindings", which you
| can then override for downstream recipes (this would require
| some new runtime features in Brioche itself). The toolchain
| itself would effectively be a dynamic binding, letting you pick
| a different recipe (so you could swap glibc for musl, or gcc
| for clang, etc). Cross-compilation would also be built on this
| same feature
| charlotte-fyi wrote:
| Very exciting. The ideas behind Nix are so good and everything
| else so bad. It seems like a lot of people are trying to solve
| this by building abstractions on top of Nix, but I'm really
| skeptical that is the solution. As crazy as a full rewrite is, I
| hope someone succeeds!
| tombert wrote:
| Agreed. NixOS is a marvel of engineering to me, and kind of
| hard to go back from once you get used to it. Automatic
| snapshotting on every configuration change, the entire system
| state being configurable through text files and therefore never
| being ambiguous, being able to temporarily install stuff
| without it polluting your path for forever by using nix-shells,
| clearly being able to see and define stuff like boot parameters
| and kernel modules are just insanely wonderful things, all
| while still using (I think) a vanilla kernel and really no
| runtime overhead, allowing you to make an insanely lean system
| without ever being unsure if you're missing something or having
| to worry about extra crap that you installed by accident
| lingering forever. In my mind about as close to an "objectively
| better" way to handle an OS (at least for people who are
| technical) as one can ask for. I have no desire to go back to
| any other distro for my server.
|
| But the Nix language itself is really quite annoying. I mean,
| I've more or less gotten used to its annoyances, and I do think
| that some of the DSLs it has are excellent (I _really_ like the
| Nginx and systemd configuration stuff, for example), and a lot
| of the configs are just `services.myservice.enable = true`
| which is fine, but a lot of the time I 'm kind of confused
| about what syntax is allowed and how loops work and the like.
| It's not horrible or anything, just a bit annoying because I'll
| occasionally have to do a nixos-rebuild like three or four
| times because I messed up some subtle syntax, and it's
| _especially_ annoying if I have to go dig at the root Nix
| package to find out what I did wrong [1].
|
| I think decentralizing stuff in the form of flakes might be
| able to help with this, if for no other reason the area in
| which you'd be forced to look for configuration stuff could be
| reduced, but I do think NixOS would benefit from some
| rearchitecture.
|
| [1] Which happened yesterday with an ethernet card
| configuration:
| https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modu...
| whateveracct wrote:
| ever use nix repl? i found my complaints about grokkability
| disappeared once i realized i could easily introspect my build
| (and all my dependencies' builds) at any level.
|
| that + learning the like 5 or so idioms that pervade nixpkgs
| and you can use Nix quite successfully imo.
| n8henrie wrote:
| Could you share an example of how you use nix repl? I use it
| every once in a while, but usually I'm starting from scratch
| and trying to replicate an error for debugging. It almost
| sounds like you're using the `--debugger` option (isn't there
| something like that?) which I've not had much luck with.
| whateveracct wrote:
| I use it to inspect various attributes of various packages.
|
| When a build fails, I use the option to keep the output
| directory and inspect things. That's another main useful
| method.
| andrewla wrote:
| I'm very excited by all the attempts to replace Nix, but I don't
| think I'll be exploring this much deeper.
|
| In my opinion the issue with Nix is that the data model is not
| crisply defined -- it's there, but hidden under a lot of goop
| that is the Nix language itself and the various assumptions and
| baggage that goes with it.
|
| What I want is a primarily declarative syntax supporting a rich
| set of data structures, ideally a non-Turing set of primitives,
| with a much more intuitive way of gluing things together. So
| basically bash (or even sh) with a well-defined way of
| transmitting environment variables and setting up the
| environment.
|
| The idea of importing a language that has broader support
| (typescript) as an alternative to the Nix language seems
| appealing at first, but typescript is such a high-dependency
| system that it's hard to get excited about it.
| infogulch wrote:
| Agreed on all counts, especially the central issue with nix and
| the properties that I'd want out of a replacement. I think CUE
| ( https://cuelang.org/ ) is a perfect language for this.
| nine_k wrote:
| Cue is fun, but already the unification example
| (https://cuelang.org/docs/tour/basics/duplicate-fields/)
| shouts "footguns" to me :(
|
| TS is certainly excessively powerful though.
| infogulch wrote:
| I don't think this would be a problem in practice. By
| default structs are "open" in cue, i.e. as long as matching
| fields unify, disparate fields just merge. There are also
| "closed" structs that allow creating a definition that
| fails to unify with fields not explicitly listed.
| https://cuelang.org/docs/tour/types/closed/
| from-nibly wrote:
| What foot guns are you seeing specifically?
| nine_k wrote:
| Go find a place where the struct value is clearly
| defined.
|
| It may seem that any definition is final, but none is.
| Since the last definition of a struct field overwrites a
| previous one, the order in which e.g. files are
| interpreted matters. It's like writing a config by using
| Python dicts, with the added complication that creation
| of a new one and updating of an existing one are
| indistinguishable.
|
| This may be okay in practice, given some discipline.
| from-nibly wrote:
| No it cant overwrite values. It has to agree or it will
| throw an error.
|
| You can have wider types being overridden by more closed
| types. e.g. string => "this literal". But you cant have
| two disagreeing values.
|
| I use this to create kubernetes manifests at work, at a
| reasonably large scale. The inability to join multiple
| files without fear of collision is specifically one of
| the best features.
| nine_k wrote:
| Hmm, looks better this way!
| Sadzeih wrote:
| As someone who works with Kubernetes every day and has
| been curious about CUE for some time, can you describe a
| bit how you're using it to create manifests? Did it
| replace helm for you? Or is it to create CRDs?
|
| Edit: I'm struggling to see where it fits in the whole
| k8s ecosystem.
| likium wrote:
| Starlark is widely used by Bazel and Buck. That would fulfill
| similar properties and build system folks have one fewer thing
| to learn.
| hamandcheese wrote:
| Yes, and Bazel makes some very serious trade-offs in order to
| make starlark work. Notably, dependencies are referenced as
| stringly labels as a poor substitute for lazy evaluation
| (starlark itself has strict evaluation semantics).
|
| This in turn requires additional tooling to catch errors
| early, and also means that a starlark-repl for Bazel will
| never really be all that useful, since the build graph
| doesn't exist in starlark alone.
|
| In my experience, this makes Bazel a significantly harder
| build system to truly grok, tho perhaps easier to use it
| without understanding it.
|
| Contrast with nix, where the entire build graph exists as a
| nix expression. In my experience, you can gain a surprisingly
| deep understanding of nix armed only with knowledge of nix-
| the-language (and without knowing any implementation details
| of nix-the-binary-that-builds-derivations).
| leoh wrote:
| flox is the best thing I know of for articulating binary
| dependencies (language runtimes, etc.), which is probably the
| sweet spot for nix anyways at the moment (i.e. as opposed to
| trying to build everything "With the One [Tool] to [Replace]
| Them All"). flox uses nix for its backend, but has a simple
| TOML syntax and is properly humble about what it can do -- but
| killer at it -- as opposed to promising the world.
|
| https://flox.dev
| mmgutz wrote:
| There's also [devbox](https://github.com/jetify-com/devbox).
|
| Tried a lot of them, and after a while I found the nix the
| package manager on non NixOS requires too many workarounds.
| Things don't just work. For example, installing alacritty
| requires an OpenGL wrapper. Neovim can't find libraries to
| build some plugins. Basically, anything GUI had issues.
|
| In the end, `cargo install`, `go install` and download a
| release archive from github are simpler to script for most of
| the tools I use.
| andrewla wrote:
| This has been a consistent problem for me as well; Nix is
| way too concerned about purity. I think it comes from a
| desire to make package installation fast by using prebuilt
| binaries. I personally could give or take the prebuilt
| binary concept; nice to have but I'm willing to pay the
| time cost to be able to run things on non-NixOS systems.
| charlotte-fyi wrote:
| I think that declarative configuration languages scratch the
| itch of a lot of developer brains, but in practice it doesn't
| really matter and using more familiar languages is a huge
| benefit to devex that shouldn't be overlooked if the goal is to
| be mainstream. Yes, it requires more discipline on the reviewer
| front not to create abstractions that are too
| powerful/footgun-y, but it's totally possible to write clear,
| declarative code in a "real" programming language without
| artificial constraints.
| __MatrixMan__ wrote:
| I'm not so worried about my disciplined coworker who just
| wants to help. If we were all reviewing his code I'd agree
| with you.
|
| The people I want to help are those who are unknowingly
| reviewing malicious commits, and I think that declarative
| configuration languages have a part to play there.
| charlotte-fyi wrote:
| This is solved in tools like Pulumi by having a declarative
| and auditable build artifact as an intermediate step that
| can be diffed. This seems to solve a lot of the security
| issues (and is generally a good idea anyway).
| NegativeLatency wrote:
| I would still prefer to debug terraform (which is a fair
| bit more declarative) rather than pulumi
| mtndew4brkfst wrote:
| This is pretty much the same stance many people hold about
| Pulumi and I never found it compelling there either. In that
| product's case, the core languages for writing expressions
| are the languages that IMO have some of the most
| dysfunctional dependency-management stories in industry
| (Python, Node/JS/TS, Golang).
|
| Since part of the sales pitch of using a general lang is
| emphasis on code reuse, it rings kind of hollow for me there,
| and here too.
| zamalek wrote:
| I'm also exploring a post-Nix package manager, wip:
| https://github.com/porkg/porkg
|
| My mantra has been to avoid "getting bored" and inventing DSLs
| for the longest time. I initially sought to use Nickel-lang, but
| it was missing some features that would make it an ideal
| candidate for this. I started writing my own (you may find this
| in the history) before realizing "WTF are you doing writing
| another shitty DSL?" I have subsequently decided that shell
| scripts (or anything you can shebang) are good enough, i.e.
| pkgbuild inspiration.
|
| I also plan to avoid making a derivation the source of
| reproducibility. Instead, a lockfile will offer that. This should
| alleviate the issue whereby updating the like of glibc cascades
| into an entire rebuild.
|
| Any *OS and home-manager would need to bring in a configuration
| language. I think Cue really makes the most sense, but that's
| still a long way off.
|
| Nice to see you used JS instead of yet another DSL :)
| Aerbil313 wrote:
| I suggest the post author and you look into nu as the package
| build language. It's uniquely suited for this task, being a
| cross-platform shell[1] with real programming language
| features, data types, and using built-in uutils instead of
| relying on platform-dependent coreutils. It also limits
| mutability[2], favoring a more functional than imperative
| approach, which would at least theoretically help with
| reproducibility. A guy from Determinate Systems recently made
| an experiment on integrating nushell to nix and the results
| were positive. Take a look:
| https://determinate.systems/posts/nuenv/
|
| 1: https://www.nushell.sh/
|
| 2: https://www.nushell.sh/book/thinking_in_nu.html#variables-
| ar...
| kylewlacy wrote:
| Nushell has definitely been on my radar for a while! but I
| missed DetSys was playing around with it, I'll have to give
| that a look
|
| One neat thing about Brioche is that Bash isn't really baked
| into it the same way as the Nix ecosystem. In a lot of the
| docs and examples, I used the `std.runBash` Brioche function
| for running scripts, but that's a fairly simple 11-line
| function[^1]. I wanted to make it just as easy to add
| `runZsh`, `runFish`, `runNushell`, etc. functions that work
| just as well as Bash (and they could either be put in the std
| library or in their own packages). So hopefully, there will
| be a Brioche `runNushell` function in the near future :)
|
| [^1]: https://github.com/brioche-dev/brioche-
| packages/blob/9fd5109...
| kylewlacy wrote:
| ooh, porkg looks pretty interesting, will definitely keep an
| eye on the project!
|
| Definitely agreed about the lockfile ideas! I went with a
| fairly similar design, although I haven't really escaped the
| "rebuild the world" situation yet and I've rebuilt a _lot_ of
| copies of gcc from source by now! (that's also partially
| because I set up the packages repo as a workspace[1], which I
| felt was easier to iterate on in the early days... I might
| eventually split each package into their own projects or
| separate repos so they can keep lockfiles independent of each
| other)
|
| and yeah, I definitely felt the temptation to write my own DSL
| but I stayed strong! I just knew that it'd be a huge uphill
| battle, especially because I wanted to provide good editor
| support (IMO implementing a language is (relatively) easy, but
| implementing a language with good error messages and LSP
| support is crazy hard)
|
| [^1]: https://brioche.dev/docs/core-concepts/workspaces/
| chubot wrote:
| _I have subsequently decided that shell scripts (or anything
| you can shebang) are good enough, i.e. pkgbuild inspiration._
|
| If you run out of steam with shell, keep Hay in mind -- which
| is part of Oils.
|
| It upgrades shell with declarative data, like YAML but not YAML
| syntax :)
|
| https://www.oilshell.org/release/0.22.0/doc/hay.html
|
| Discussion -
| https://lobste.rs/s/phqsxk/hay_ain_t_yaml_custom_languages_f...
|
| It's not set in stone yet -- I'm planning an overhaul based on
| some feedback, so I welcome any more (e.g. on Github or Zulip)
| __MatrixMan__ wrote:
| Some of the oddities of the nix language are pretty useful for
| its domain. Recursive attribute sets, for instance, save a lot of
| headaches if you're trying to have only a single source of truth.
| Do you feel like these translate to typescript nicely?
|
| As somebody who knows nix but doesn't know typescript, I found
| myself looking for a rosetta stone page where I could look at two
| chunks of code that do the same thing, but in separate languages.
| This would let me use the familiar language to scrutinize the
| other.
|
| I wouldn't normally ask for such a thing, but if you're putting
| "Nix-like" in the title then maybe it might be worth adding a
| comparison page to the docs.
| charlotte-fyi wrote:
| The syntax isn't as clean but you can add property accessors to
| objects in JavaScript that accomplishes the same thing.
| kylewlacy wrote:
| I've been asked about overrides and attributes a lot! That was
| one of the sacrifices I had to make to go with a more
| traditional language, so it's definitely a negative point
| compared to Nix. That said, I'm hoping to have some conventions
| and features that will cover _most_ of the use-cases that
| overrides give you, but that's definitely still future work at
| this point and I probably won't be able to cover everything
| overrides do
|
| And yep, I think having a "Brioche for Nix users" guide makes a
| lot of sense, although it's not the first time that question so
| I'll probably stand up a first-pass version of it sooner rather
| than later (my Nix skills are also pretty rusty-- I'll need to
| brush up a bit first before I write it!)
| hamandcheese wrote:
| I think the lack of true laziness will be a big performance
| problem for a large build graph.
|
| On the other hand the monolithic nature of the nixpkgs package
| set is one of the authors gripes with nix, so performance at
| that scale may be a non-goal.
| kylewlacy wrote:
| I'd definitely like to have good performance even for large
| build graphs! I'm hoping the laziness exists "where it
| counts". To walk through an example, if you build your
| backend, and your backend calls the function `postgres()`,
| and that calls `openssl()`, and THAT calls `gcc()`, etc.,
| etc., each function is basically building an object to
| represent its chunk of the build graph (each function returns
| a "recipe"). Nothing gets built until that object gets
| returned from the top-level function and the runtime does
| something with it
|
| In other words, the eager part is basically constructing the
| build graph. Maybe I'm wrong but I don't that this would
| necessarily be slower than the lazy version. In practice the
| most complex build graph I've made is basically the full
| chain of Linux From Scratch builds (that's the basis for my
| toolchain currently), and I think that takes about 400-500ms
| to evaluate. It's about 160 build steps, so it's not _simple_
| but I know build graphs can also get a lot more complex, so
| I'll just have to keep an eye on performance as I start to
| get into more and more complex builds
|
| Maybe I'm missing something but intuitively I'd expect this
| approach to be fairly efficient-- as long as build scripts
| only call these functions when they're used as part of the
| build graph
| hamandcheese wrote:
| I think it really depends on your definition of "large". I
| don't think strict eval + full build graph can scale to
| something the size of nixpkgs, for example.
|
| I mentioned in another comment that this is why Bazel uses
| simple strings to form dependencies on other targets. That
| way Bazel can manage the laziness and only evaluate what is
| needed without needing to use or invent a language with
| lazy evaluation.
|
| But that is also the big downside (in my opinion) - the
| full build graph necessarily can't exist purely in starlark
| (at least for Google-scale projects) which increases
| complexity of the tool overall.
|
| Edit: I'd like to add, though, that I think it's perfectly
| fine to not scale to Google scale or nixpkgs scale! Many
| many projects could still benefit from a great build tool.
| aseipp wrote:
| Honestly, I think the "stringly-typed targets" thing
| isn't too bad, having used Buck2 quite a bit, and being a
| Nix user for 10+ years. If anything, it's a small price
| to pay for some of the other smart things you get in
| return, like the far more fine-grained action graph and
| the tooling around BUILD files like querying. One weird
| benefit of that stringly-typed bit is that the BUILD
| files you have don't even have to meaningfully evaluate
| or even _parse_ correctly, so you can still build other
| subsets of the tree even when things are broken; at
| ridiculous-scale it 's nearly impossible to guarantee
| that, and it's something Nix does worse IMO since
| evaluation of the full nixpkgs tree is slow as hell in my
| experience but a requirement because a single eval error
| in the tree stops you dead in your tracks.
|
| Also, no matter how much I might not like it as a
| language nerd, I think Starlark is simply far more
| "familiar" for your-average-bear than the Nix language
| is, which matters quite a bit? It might be more complex
| in some dimension, but the problem space is fundamentally
| complex I think. So other factors like how approachable
| the language is matters. (And at least in Buck2, you can
| use MyPy style typing annotations, thank God.)
| leoh wrote:
| I am not writing typescript -- heaven forbid -- so I can manage
| cargo, so I can write Rust.
|
| Dropping the snark, though -- most times I have seen folks
| attempt to fold other build systems into some polyglot common
| one, it seems to create a lot of problems -- for at the end of
| the day, build tools (whether npm, cargo, gradle, docker, etc.)
| each have their issues. When these issues arise, they require
| understanding the original build tool. Bazel, for example, is a
| "polyglot" system which many try to use to manage npm. In my
| experience, when folks use bazel but have never used npm
| directly, they often get lost, because they now have to deal with
| two abstractions (bazel, npm) simultaneously neither have a way
| for bazel to do the thing they need, nor an understanding of how
| to get bazel to do that thing (nor npm itself to do the thing),
| also because they have never worked with npm directly.
|
| Same goes for cargo, incidentally. Switching over to that build
| tool -- like any major build tool, it has an insane amount of
| well-honed documentation and thousands of answered questions
| online (not to mention LLMs trained somewhat well on it). Users
| when they need to do something with brioche or have an issue with
| it will not necessarily have the benefit of the massive
| collaborative support available for cargo.
|
| Incidentally, I don't think that nix is free of this problem,
| though it does attempt to manage these issues -- sometimes
| reasonably successfully -- by virtue of keeping interfaces to
| other build tools in simple, direct ways.
|
| Even then, however, the best way I have seen nix used is as a
| system for declaring all binary dependencies or other
| dependencies that are either not manageable or especially
| competently managed by other build tools -- which nix allows for
| in a virtual environment by shipping a shell.nix or flake.
|
| In other words -- bringing in the right version of java, gradle,
| node, etc. But then just directing users to the specific, native
| build tools that often have a massive number of contributors.
|
| The closest thing to a tool that does this (guides developers to
| other build tools) is flox, which uses TOML, but uses nix for its
| backend via C API. I haven't had a chance to use it much yet, but
| it looks very promising https://flox.dev.
|
| ===
|
| Okay, all this rambling aside -- building a tool like this for
| yourself or a small team, especially when you have the time to do
| it, it works well, and you're having fun... well, there's an
| insane amount to be said for that; and it will doubtlessly lead
| you to learning a ton about other build tools and likely offer
| you deep insights that are only possible when attempting to build
| something like brioche. So, on that count, major kudos to you.
| Zambyte wrote:
| I'm really happy to see more work on package managers in this
| space. I used Nix for a little bit, but I shared the frustration
| expressed by other commenters on the language itself. I have
| found my zen on GNU Guix, but more options are better :)
| notthemessiah wrote:
| Tvix aims to be a modulary implementation of Nix, which could
| create a bridge between Nix and Guix, according to a claim by
| its author:
|
| "It should be possible to write an evaluator that plugs in the
| Guile language (for compatibility with GNU Guix), to use
| arbitrary builders, and to replace the store implementation."
|
| https://tvix.dev/
| zarathustreal wrote:
| I've generally found the most interesting things are the ones
| that don't feel intuitive at first. Familiarity bias can really
| hold you back. The language semantics of Nix are basically
| Haskell's which.. is probably the One True Way to write correct
| code if we're honest with ourselves.
| hexo wrote:
| nix does not have haskells semantics
| raffraffraff wrote:
| Reading the comments makes me think I'm turning into my dad. He
| was a simple man in terms of education and career. He was a fire
| man on a stream train in 1950, a farmer, barman, handyman. Out in
| the sticks, he was everyone's go-to guy when a vet couldn't be
| found, because he kept a "livestock first aid kit" including
| penicillin & syringes. He talked to vets and read books and knew
| about the most common ailments and how to treat them. He serviced
| his own car and tractor, including stripping the tractor engine
| down in his yard without proper tools, and getting cylinders
| bored and pistons sleeved, and putting it back together. He ran
| electric cable in farm building and sheds (after chatting to an
| electrician) and he did a ton of building work around the house.
| But at a certain age, maybe around 50, all new technology was
| out. Electricity was fine but electronics were out. Mechanical
| problems were fine because you could look straight at a thing and
| figure out how it clicked together, but once it had a
| microcontroller and a panel that abstracted away the actual
| working of the machine it just became some weird arbitrary
| "thing" that blocked his view of the machine. I'm sure if he was
| in his 20s when this stuff appeared he'd be all over it, but he
| was born in 1926, so I think the mighty micro was too late for
| him. He flatly refused to use the VCR, alarm systems, any type of
| computer. Show him a rubis cube, he's interested. Show him a
| microwave oven with more than 5 buttons and he'll make a sandwich
| instead.
|
| This is me with nix. I'm ok with all the stuff that's emerged in
| my field (infra/sysadmin) since the 90s. But nix makes me
| instantly glaze over. Several times I tried, but it's like my
| brain unplugs itself. I'm sure it does some wonderful stuff at an
| extremely high price that I'm not willing to pay.
|
| Nix has a huge mindshare, which I don't think you can win back if
| Nix keeps fixing the core issues mentored in these comments. Best
| of luck with this, though.
| otabdeveloper4 wrote:
| Nix solves the same problem we used to solve back in the day
| (before Docker taught us to run everything as root and disable
| firewalls, #yolo) ad-hoc with random poorly maintained scripts.
|
| If you never had to solve this problem then it probably isn't
| for you. If you did, then being able to finally replace the old
| contraptions with something properly engineered is a breath of
| fresh air.
| Dalewyn wrote:
| >before Docker taught us to run everything as root
|
| Windows called, it wants its Administrators back.
| bayindirh wrote:
| Windows guys _still_ have these, right?
| 7bit wrote:
| Very smart
| bayindirh wrote:
| The thing is, old ways didn't have to be contraptions at all.
| If we had to run many services on a system, we instead ran
| BSD with jails, or if we have to run Linux, with chroots.
|
| These solutions, abstracted the convoluted solutions away.
| Who didn't want to form these solutions themselves (which is
| fair, because it needs intricate knowledge about close-to-OS
| stuff) built "contraptions" instead.
|
| The core solutions to core problems didn't change at all. We
| sometimes use the more modern approaches, because we feel
| lazy sometimes.
| otabdeveloper4 wrote:
| At some point you will want to manage your own software
| supply chain. A C++ compiler is the bare minimum, but then
| you'll surely want a Python environment too, and of course
| some npm packages for the frontend. Then you'll want some
| cross-compilation - for the new ARM processors, or maybe
| you want a different libc. After that you need caching and
| remote builds, because build times are a pain in the ass.
|
| Here you come to realize that Nix already comes with all
| that out of the box, with some minimal tweaking.
|
| Jails/chroots/containers are an orthogonal thing, though
| often whatever comes out of your software supply chain
| you'd want to run containerized too.
| aseipp wrote:
| Nix does not solve the problem of "running things in a
| chroot." It solves the problem of having to reliably and
| consistently build software in a repeatable, reproducible,
| programmable way, up-to and including all the fundamental
| system components, and it offers thousands and thousands of
| packages in order to make that usable for typical modern
| workflows. To a first approximation, this problem is about
| 10,000 times harder to solve than just "using chroot."
|
| It's like the difference between building a single building
| and building an entire city, in terms of scale and total
| effort. That's why it's taken Nix nearly 20 years to get
| where it is.
|
| > We sometimes use the more modern approaches, because we
| feel lazy sometimes.
|
| No, we often use them because they are actually materially
| better. And Nix is vastly better than all the prior
| attempts at this problem. If you didn't have those problems
| or don't _think_ you did, that 's fine. But it's got
| nothing to do with "laziness" whatever that means. It's got
| everything to do with the results. And the results are
| ultimately quite good.
| taink wrote:
| Reminds me of this quote from Douglas Adams:
| I've come up with a set of rules that describe our reactions to
| technologies: 1. Anything that is in the world when
| you're born is normal and ordinary and is just a natural part
| of the way the world works. 2. Anything that's invented
| between when you're fifteen and thirty-five is new and exciting
| and revolutionary and you can probably get a career in it.
| 3. Anything invented after you're thirty-five is against the
| natural order of things.
|
| It's been used to criticize anyone who rejects new technology,
| but it's just a funny quote that has some truth to it.
| stackskipton wrote:
| As I've hit age defined in 3, some of it is also "Anything
| invented after you're thirty-five is mostly same stuff you
| had before but upgraded. You are confused why everyone is
| losing their minds over it."
|
| It reminds me of current AI craze. We had a bit of this in
| 2016, it didn't go well:
| https://en.wikipedia.org/wiki/Tay_(chatbot)
| falcor84 wrote:
| >mostly same stuff you had before but upgraded. You are
| confused why everyone is losing their minds over it
|
| That's a good way of thinking about this. [Showing my age
| here], I remember being unfazed about the web in the mid
| 90s, thinking to myself "I already have Encarta which does
| hypertext with multimedia so well, why do I need this new
| thing populated by random people".
|
| But of course the web did have a lot to offer. I suppose I
| keep coming back to "Quantity has a quality all its own" -
| sometimes even a relatively small upgrade makes an existing
| solution suddenly accessible to more people and a viable
| solution to more problems.
| cmrdporcupine wrote:
| Here's what's been key for me:
|
| Absolutely separate -- in your mind -- NixOS from Nix.
|
| The latter I've seen be a very useful tool that does what
| people here talk about: cleans up dependency and build and
| installation management in a reasonably nice way.
|
| The former is a religion/world-view that requires a complete
| mindset change, accepting a whole dogma, and differs entirely
| from how you're used to using Linux. I gave up and switched
| back to Debian after I couldn't get a decent answer on how to
| set it up easily with my github keys to easily, y'know, check
| projects out. The mental gymnastics required wer silly.
|
| Nix the language seems obscure at first but it's not so bad
| after a while. It's sort of got a bit of syntax tossed in from
| the ML-ish programming languages so there's some familiarity
| there, for me, maybe.
|
| Where it falls down is like anything else like this: it
| basically requires that all the things you dep on also adhere
| to this philosophy and have nix setups already or you'll be in
| a world of pain rolling your own. I gave up on converting our
| $work stuff when it became clear that the versions of
| flatbuffers and a few other deps we depend on just weren't out
| there in the nix ecosystem and I wasn't going to volunteer to
| make them.
|
| Anyways, I've seen it be very useful. A former employer used it
| to set up their whole Julia environment such that custom
| patches and configuration would all be nicely applied. And this
| was for both developer workstations and CI and production
| environments. It's a nice step below docker for doing that kind
| of thing, and is far less intrusive than docker in many ways.
| Filligree wrote:
| > Where it falls down is like anything else like this: it
| basically requires that all the things you dep on also adhere
| to this philosophy and have nix setups already or you'll be
| in a world of pain rolling your own. I gave up on converting
| our $work stuff when it became clear that the versions of
| flatbuffers and a few other deps we depend on just weren't
| out there in the nix ecosystem and I wasn't going to
| volunteer to make them.
|
| Maybe I'm a bad NixOS user, but I tend to run most of my
| development work through steam-run, unless the necessary
| default.nix is _really_ simple. It works fine, and having the
| rest of the system be immutable is still an advantage.
| pxc wrote:
| > I gave up and switched back to Debian after I couldn't get
| a decent answer on how to set it up easily with my github
| keys to easily, y'know, check projects out. The mental
| gymnastics required wer silly.
|
| Huh? NixOS doesn't require any special configuration to use
| SSH keys or Git or even SSH keys stored on special hardware
| like Yubikeys. What went wrong here, or did I misunderstand
| your use case?
| barfbagginus wrote:
| Be like your father except with knowledge of Category Theory
|
| That's the best we can do.
| jhoechtl wrote:
| The Ops comment was well versed, educated and contributed to
| the problem.
|
| What do you want to tell us with category theory and how does
| your comment fit into the discussed problem?
| AJRF wrote:
| I think I am similar to you - hate complexity, hate shiny
| things - but, I love Nix because it solves a real problem. That
| problem being deterministic builds.
|
| It is a common and real problem that builds with "state" cause
| problems very often for people not even doing complex things.
|
| I think Nix is complex, but because it solves a real, common
| issue I grin and bear it.
| tkz1312 wrote:
| Looks awesome! Congrats on the release. I would be curious to
| hear your thoughts on how this compares to garn [1], a typescript
| front end to the existing nixpkgs ecosystem.
|
| 1: https://garn.io/
| doctor_phil wrote:
| I actually haven't tried Nix (or other nix-alternatives) yet.
| I've seen a lot of complaints about the language, but a lot of
| praise and love of the idea. I though Guix was the "main"
| competitor to nix, using a more mainstream language (Scheme). Is
| that still the case? How would Brioche compare against Guix if
| you are OK with both Scheme and Typescript as languages?
| rgoulter wrote:
| > I've seen a lot of complaints about the language, but a lot
| of praise and love of the idea
|
| I think when people say they like the idea of nix, they're
| referring to how it builds packages from programmatic package
| formulae into an isolated store (so e.g. you can easily have
| different programs running against different versions of
| dependencies), and the use cases this allows for. -- With an
| expressive language for describing packaging, you get stuff
| like declarative developer environments, or even declared Linux
| configuration with quick/easy rollback.
|
| I think complaints about the language are _partly_ from Nix
| being unusual enough that it takes time to get an intuition for
| the basics of how to write anything in it, but _largely_ from
| the Nixpkgs (the large package repository the Nix package
| manager uses) being a large codebase, which grew organically
| with many contributors, with many similar-but-different
| solutions to problems, and sometimes the codebase is a bit too
| clever.
| cmrdporcupine wrote:
| Guix is the main competitor but it's also focused on free
| software so mainly excludes things that don't fit that
| licensing philosophy. Yes there are I believe repositories that
| maintain non-free things but ... Nix already doesn't have
| declarations for a lot of things. Guix even less so.
| Gud wrote:
| I've always wondered why no one ports FreeBSDs pkg/ports package
| management system. From all the operating systems I've used it's
| the clear leader.
| amelius wrote:
| Can't we have something Python based, since it is already the
| lingua franca and seems to be the language that people are least
| frustrated about?
___________________________________________________________________
(page generated 2024-06-04 23:02 UTC)