[HN Gopher] Why We're Moving on from Nix
___________________________________________________________________
Why We're Moving on from Nix
Author : mooreds
Score : 260 points
Date : 2025-06-07 11:36 UTC (1 days ago)
(HTM) web link (blog.railway.com)
(TXT) w3m dump (blog.railway.com)
| lloydatkinson wrote:
| As someone with only a little experience with Nix, the points
| here don't really seem right?
|
| > This approach isn't clear or maintainable, especially for
| contributors unfamiliar with Nix's version management.
|
| > For languages like Node and Python, we ended up only supporting
| their latest major version.
|
| What is not maintainable about this? That they need to make a
| list of available versions? So, can this not be automated?
|
| Furthermore, why is Railway defining how a user uses Nix?
|
| Surely one of the points of Nix is that you can take a bare
| machine and have it configured with exactly what versions of
| packages you want? Why would Railway need to get in the way of
| the user and limit their versions anyway?
|
| Or did I misunderstand and they don't even expose Nix to the
| user? If so, the original question still stands: can't they
| automate that list of package versions?
| codethief wrote:
| > Or did I misunderstand and they don't even expose Nix to the
| user?
|
| That's at least my understanding, yes.
| notnmeyer wrote:
| you can not have a dockerfile in your project at all, push your
| code to them, and they'd build an image for it with nixpacks.
| you'd see nix stuff in your build logs, but it's behind the
| scenes for the most part.
| elbear wrote:
| The version limits come from the fact that the Nix cache
| doesn't maintain older versions. So, if you use an older
| version, you will have to compile from sources. It sounds like
| they didn't want to take it upon themselves and provide a cache
| with older versions, even though it doesn't sound like much
| effort.
|
| Honestly, the reasons given don't feel very solid. Maybe the
| person who introduced Nix left and the ones remaining didn't
| like it very much (the language itself is not very nice, the
| docs weren't great either in the past).
|
| Still, I'm not familiar enough with the stack they chose, but
| does it provide a level of determinism close to Nix? If not, it
| might come to bite them or make their life harder later on.
| e3bc54b2 wrote:
| Nix cache (cache.nixos.org) does in fact maintain older
| versions[0]. In fact, they maintain so much older stuff
| (binaries _and_ associated source), that they are used in
| both research[1][2] and are having issues with gigantic cache
| size[3][4].
|
| And yes, their reasoning implies NIH and just unfamiliarity
| combined with unwillingness to really understand Nix.
|
| [0]: https://discourse.nixos.org/t/how-long-is-binary-cache-
| kept-...
|
| [1]: https://hal.science/hal-04913007
|
| [2]: https://luj.fr/blog/is-nixos-truly-reproducible.html
|
| [3]: https://discourse.nixos.org/t/nixos-foundations-
| financial-su...
|
| [4]: https://discourse.nixos.org/t/the-nixos-foundations-
| call-to-...
| mplanchard wrote:
| Even if this weren't true, setting up their own binary
| cache on S3 would have been trivial. It took me half a day
| to get it set up for our CI pipeline.
| akho wrote:
| Nix cache does provide old versions. What they seem to want
| is old versions built with new versions of dependencies.
| That's also possible, but you will have to build things.
| koolala wrote:
| Patching old software with newer components? Does another
| tool really offer that automatically? This article is a
| promotion for their tool that does?
| akho wrote:
| That's what dynamic linking does in most linux
| distributions.
| cormacrelf wrote:
| From the article:
|
| > The way Nixpacks uses Nix to pull in dependencies often
| results in massive image sizes with a single /nix/store layer
| ... all Nix and related packages and libraries needed for both
| the build and runtime are here.
|
| This statement is kinda like "I'm giving up on automobiles
| because I can't make them go forward". This is one of the
| things Nix can do most reliably. It automates the detection of
| which runtime dependencies are actually referenced in the
| resulting binary, using string matching on /nix/store hashes.
| If they couldn't make it do that, they're doing something
| pretty weird or gravely wrong. I wouldn't even know where to
| start to try to stop Nix from solving this automatically!
|
| I wouldn't read too much into their experience with it. The
| stuff about versioning is a very normal problem everyone has,
| would have been more interesting if they attempted to solve it.
| mplanchard wrote:
| To be fair to the authors, this IS a problem, albeit one they
| phrased poorly, especially with building docker images via
| nix. The store winds up containing way more than you need (eg
| all of postgres, not just psql), and it can be quite
| difficult to patch individual packages. Derivations are also
| not well-pruned in my experience, leading to very bloated
| docker images relative to using a staged Dockerfile.
|
| Image size isn't something we've focused a lot on, so I
| haven't spent a ton of time on it, but searching for "nix
| docker image size" shows it to be a pretty commonly
| encountered thing.
| mrbluecoat wrote:
| > we don't have any problem with Nix itself. But there is a
| problem with how we were using it.
|
| A good example of 'use the right tool for the right job'. Nix is
| great for some use cases and awful for others. The problem is the
| Nix learning curve is so high that by the time you've grasped it
| enough to make a decision you feel you've invested too much time
| to back out now and pivot to something else so you try to
| shoehorn it to solve the original need.
| teekert wrote:
| I feel it like that as well, but in some ways Nix is a more
| normal programming paradigm than other OS's. We're just not
| used to thinking about an OS that way. Nix expressions have
| inputs (a package repo, lots of key-value pairs) and outputs (a
| Linux system). Idk perhaps in a couple of years it will be much
| more normal.
|
| Ie it is very easy for an AI to create a to-spec shell.nix
| (some Python packages, some Linux packages, some env vars, some
| path entries etc), or configuration.nix because of this
| paradigm.
|
| I do this a lot to include envs with repos that fully support
| the package. It would probably be more reproducible with flakes
| (a flake.nix is like a shell.nix but with version pinning... or
| something, I'm still climbing that learning hill).
| zxexz wrote:
| I often find it takes about 10 minutes of my time to package
| your average python package into nix, or 1 minute if its
| pyproject.toml based (uv/poetry projects!.
|
| There is a long tail though ( _cough_ weasyprint _cough_ ).
| gf000 wrote:
| That's mostly just a tooling issue/a paradigm mismatch.
| Python's package management is notoriously bad, so it sorta
| makes sense that it won't fit as nicely with Nix.
| throwaway314155 wrote:
| > Python's package management is notoriously bad, so it
| sorta makes sense that it won't fit as nicely with Nix
|
| This sort of thing is immaterial to users of Nix/Python
| who just want it to do the god damn thing it's supposed
| to do.
| viraptor wrote:
| The version selection part sounds weird. The versions in nixpkgs
| make sense when you're running/building the system. If you're
| providing runtimes/compilers as a platform, you really want
| something like what devenv does - provide the versions yourself.
| You really don't want to end up building an old system to provide
| an old nodejs - you're leaving security patches in dependencies
| behind. Devenv does it for example through
| https://github.com/cachix/nixpkgs-python "All Python versions,
| kept up-to-date on hourly basis using Nix."
|
| > Railway injects a deployment ID environment variable into all
| builds.
|
| They could've done it in the next layer after installation. Also,
| you can split packages into different layers. There's even
| automation for it if you need batches to keep the number of
| layers down.
| ris wrote:
| The main problem here is wanting to hang on to the "bespoke
| version soup" attitude that language package managers encourage
| (and is totally unsustainable). The alternative Mise doesn't
| appear to have any ability to understand version constraints
| between packages and certainly doesn't run tests for each
| installed package to ensure it works correctly with the
| surrounding versions. So you're not getting remotely the same
| thing.
| k__ wrote:
| _" the "bespoke version soup" attitude that language package
| managers encourage"_
|
| Care to elaborate what that means and what the alternative is?
| matthewbauer wrote:
| Not sure on the exact take of the OP, but:
|
| Package maintainers often think in terms of constraints like
| I need a 1.0.0 <= pkg1 < 2.0.0 and a 2.5.0 <= pkg2 < 3.0.0.
| This tends to make total sense in the micro context of a
| single package but always falls apart IMO in the macro
| context. The problem is:
|
| - constraints are not always right (say pkg1==1.9.0 actually
| breaks things)
|
| - constraints of each dependency combined ends up giving very
| little degrees of freedom in constraint solving, so that you
| can't in fact just take any pkg1 and use it
|
| - even if you can use a given version, your package may have
| a hidden dependency on one if pkg1's dependencies, that is
| only apparent once you start changing pkg1's version
|
| Constraint solving is really difficult and while it's a cool
| idea, I think Nixpkgs takes the right approach in mostly
| avoiding it. If you want a given version of a package, you
| are forced to take the whole package set with you. So while
| you can't say take a version of pkg1 from 2015 and use it
| with a version of pkg2 from 2025, you can just take the whole
| 2015 Nixpkgs and get pkg1 & pkg2 from 2015.
| 0xbadcafebee wrote:
| > Constraint solving is really difficult and while it's a
| cool idea, I think Nixpkgs takes the right approach in
| mostly avoiding it. If you want a given version of a
| package, you are forced to take the whole package set with
| you.
|
| Thank you, I was looking for an explanation of exactly why
| I hate Nix so much. It takes a complicated use case, and
| tries to "solve" it by making your use-case invalid.
|
| It's like the Soylent of software. _" It's hard to cook,
| and I don't want to take time to eat. I'll just slurp down
| a bland milkshake. Now I don't have to deal with the
| complexities of food. I've solved the problem!"_
| matthewbauer wrote:
| I mean you can do it in Nix using overlays and overrides.
| But it won't be cached for you and there's a lot of extra
| fiddling required. I think it's pretty much the same as
| how Bazel and Buck work. This is the future like it or
| now.
| lkjdsklf wrote:
| It's not an invalid use case in nixpkgs. It's kind of the
| point of package overlays.
|
| It removes the "magic" constraint solving that seemingly
| never works and pushes it to the user to make it work
| chriswarbo wrote:
| > I was looking for an explanation of exactly why I hate
| Nix so much
|
| Note that the parent said "I think Nixpkgs takes the
| right approach in mostly avoiding it". As others have
| already said, Nix != Nixpkgs.
|
| If you want to go down the "solving dependency version
| ranges" route, then Nix won't stop you. The usual
| approach is to use your normal language/ecosystem tooling
| (cabal, npm, cargo, maven, etc.) to create a "lock file";
| then convert that into something Nix can import (if it's
| JSON that might just be a Nixlang function; if it's more
| complicated then there's probably a tool to convert it,
| like cabal2nix, npm2nix, cargo2nix, etc.). I personally
| prefer to run the latter within a Nix derivation, and use
| it via "import from derivation"; but others don't like
| importing from derivations, since it breaks the
| separation between evaluation and building. Either way,
| this is a _very_ common way to use Nix.
|
| (If you want to be even more hardcore, you could have Nix
| run the language tooling too; but that tends to require a
| bunch of workarounds, since language tooling tends to be
| wildly unreproducible! e.g. see http://www.chriswarbo.net
| /projects/nixos/nix_dependencies.ht... )
| jonhohle wrote:
| There's no clear definition (in most languages, of
| major/minor/patch versioning). Amazon did this reasonably
| well when I was there, though the patch version was
| implicitly assigned and the major and minor required humans
| to follow the convention:
|
| You could not depend on a patch version directly in source.
| You could force a patch version other ways, but each
| package would depend on a specific major/minor and the
| patch version was decided at build time. It was expected
| that differences in the patch version were binary
| compatible.
|
| Minor version changes were typically were source
| compatible, but not necessarily binary compatible. You
| couldn't just arbitrarily choose a new minor version for
| deployment (well, you could, but without expecting it to go
| well).
|
| Major versions were reserved for source or logic breaking
| changes. Together the major and minor versions were
| considered the interface version.
|
| There was none of this pinning to arbitrary versions or
| hashes (though, you could absolutely lock that in at build
| time).
|
| Any concept of package (version) set was managed by
| metadata at a higher level. For something like your last
| example, we would "import" pkg2 from 2025, bringing in its
| dependency graph. The 2025 graph is known to work, so only
| packages that declare dependencies on any of those versions
| would be rebuilt. At the end of the operation you'd have a
| hybrid graph of 2015, 2025, and whatever new unique
| versions were created during the merge, and no individual
| package dependencies were ever touched.
|
| The rules were also clear. There were no arbitrary
| expressions describing version ranges.
| jpgvm wrote:
| Nix generally speaking has a global "nixpkgs" version (I'm
| greatly over-simplifying here ofc) in which there is a single
| version of each package.
|
| This is likely the source of their commit based versioning
| complaint/issue, i.e the commits in question are probably
| https://github.com/NixOS/nixpkgs versions if they aren't
| maintaining their own overlay of derivations.
|
| This is in contrast to systems that allow all of the versions
| to move independently of each other.
|
| i.e in the Nix world you don't just update one package, you
| move atomically to a new set of package versions. You can
| have full control over this by using your own derivations to
| customise the exact set of versions, in practice most folk
| using Nix aren't deep enough in it for that though.
| ris wrote:
| It's the idea that every application can near-arbitrarily
| choose a bespoke-but-exact mix of versions of every
| underlying package and assume they all work together. This is
| same attitude that leads to seemingly every application on
| planet earth needing to individually duplicate the work of
| reacting to every single dependabot update for their
| thousands of underlying packages and deal with the fallout of
| conflicts when they arise.
|
| Packages in nixpkgs follow the "managed distribution" model,
| where almost all package combinations can be expected to work
| together, remain reasonably stable (on the stable branch) for
| 6 months receiving security backports, then you do all your
| major upgrades when you jump to the next stable branch when
| it is released.
| James_K wrote:
| Put out fewer versions of things. It is entirely possible to
| write a piece of software and only change the interface of it
| at rare intervals. The best solution I can think of though
| would be to allow one version of a package to provide
| multiple versions of its interface. Suppose you want to
| increment the minor version number of your code and this
| involves changing the signatures of a number of functions,
| you could design a programming language packaging system such
| that both versions are defined in the same package, sharing
| code when needs be.
| chriswarbo wrote:
| That's why everything in Nixpkgs is defined as a function,
| which takes dependencies as arguments.
|
| It also puts a function in the result, called `override`,
| which can be called to swap out any of those arguments.
| James_K wrote:
| Which leads to the exact problems defined in this
| article. Many programs using many library versions. It
| would be much better from both a security and size
| perspective if these disparate packages could be served
| by a single shared object using versioned symbols.
| chriswarbo wrote:
| Hmm, not sure I agree. Most of those arguments get
| populated by taking the fixed-point, along with any given
| overlays; so it's easy to swap-out a library everywhere,
| just by sticking in an overlay. The exceptions mostly
| seem to be things that just don't work with the chosen
| version of some dependency; and even that's quite rare
| (e.g. it's common for Nixpkgs maintainers to patch a
| package in order to make the chosen dependency work;
| though that can causes other problems!)
| aidenn0 wrote:
| You can actually have both if you do it right. It's trivial to
| build a rust package with Nix from a Cargo.lock file, for
| example. _Nixpkgs_ is contrary to bespoke version soup, but Nix
| itself can be fine with it.
| jorams wrote:
| Bespoke version soup is unsustainable, but part of why people
| keep doing it is that it tends to work fine. It tends to work
| fine in part because OS-level libraries come from a different,
| much more conservative world, in which breaking backwards
| compatibility is something you try to avoid as much as
| possible.
|
| So they can take a stable, well-managed OS as a base, use tools
| like mise and asdf to build a bespoke version soup of tools and
| language runtimes on top, then run an app on top of that. It
| will almost never break. When it does break, they fiddle with
| versions and small fixes until it works again, then move on.
| The fact that it broke is annoying, but unimportant. Anything
| that introduces friction, requires more learning, or requires
| more work is a waste of time.
|
| Others would instead look for a solution to stop it from
| breaking ever again. This solution is allowed to introduce
| friction, require more learning, or require more work, because
| they consider the problem important. These people want Nix.
|
| Most people are in the first group, so a company like Railway
| that wants to grow ends up with a solution that fits that
| group.
| kesor wrote:
| Looks like they are trying to force versions into where there are
| none. Just like trying to force a square cube into a round hole.
|
| "Default versions" breaking things that depend on them? What is
| that? It is like using docker's ":latest" tag and being surprised
| each time that a new server falls on its face because the
| "default" image is actually a different version from the previous
| "default" image.
|
| I don't understand any of the explanations in this blog post.
| Seems like people who have zero clue about what a "version" of a
| software is.
|
| "no way of splitting up the Nix dependencies into separate
| layers" - Why? Of course you can split /nix/store into as many
| layers as you need. Do they even know how to use containers and
| how to use Nix in the first place?
|
| With the clear incompetence of these people, no wonder that their
| proposed solution smells like a decomposed fish.
|
| Classic NIH syndrome. There is going to be no surprise to see
| them meet the exact same problems they didn't solve with Nix to
| infest their new "solution".
| jbverschoor wrote:
| What works better to raise VC (which for many is the goal)
|
| A nix wrapper or a deployment platform
| BoorishBears wrote:
| Traction.
|
| I don't think any VC worth the time is going to sit around
| nitpicking how much Nix matters to their offering if they're
| making increasing amounts of money.
| jbverschoor wrote:
| Depends who's pretending to be a VC, or accelerator
| kesor wrote:
| It works best to post your "solution" on hacker news and get
| roasted by ALL the commenters who have a clue while you have
| none.
| 12_throw_away wrote:
| > What works better to raise VC (which for many is the
| goal)[?] A nix wrapper or a deployment platform
|
| Well, if I wanted VC money for this, even if I was buliding a
| simple nix wrapper, I'd still _tell the VCs_ that it was a
| deployment platform. (to be clear I have no opinion on TFA or
| any idea whether this is what they 're doing)
| zxexz wrote:
| I'm all for not using nix, especially where it doesn't make
| sense. But rebuilding a working system from scratch, for
| reasons that aren't actually a problem if they were to have
| spent even a couple hours looking at how people already solve
| those issues, seems fundamentally insane.
|
| Like others have said here, nix2container and flakes seem like
| they would address every problem they have.
|
| With regard to versioning, I have flakes written 3 years ago
| that still build with exactly the same versions and the same
| output as when it was first written.
|
| Sure does sound like that want to go to market and raise off a
| platform :D
|
| Edit: Literally just checked nixpacks' github and it
| immediately jumped out to me that they are using rustPlatform
| in nixpkgs, not oxalica's rust-overlay[0] which would have come
| up in any cursory search for the rust issues. And is one of the
| most useful and powerful overlays I've used.
|
| [0] https://github.com/oxalica/rust-overlay
| unshavedyak wrote:
| Yea, i kinda hate Nix - but it's fundamentally changed how i
| view OSs and the minimum features i want from them.
|
| Though I really want another language than Nixlang, but i've
| been stuck on Nix for years despite not liking it because the
| concrete builds are just so good. If Nix actually had a
| language i liked i'd probably go full immutable user config
| too and fully embrace nix. It's just so, so good.
|
| I'd switch in a heartbeat if someone iterated on Nix for some
| of my complaints, though. But i'm not switching away from the
| concrete system builds.. it's just amazing.
| dawnofdusk wrote:
| If you like Lisp there is GUIX which is the Nix packaging
| idea but using a language that is slightly less trash. The
| ecosystem is much worse so it would not be great for a
| desktop end user AFAIK.
| Cloudef wrote:
| I dont see why they couldnt made their own derivations instead of
| relying on nixpkgs hashes.
| droelf wrote:
| We've been working on Pixi, which uses Conda packages. You can
| control versions precisely and also easily build your own
| software into a package to ship it. Would be curious to chat if
| it could be useful as an alternative to `mise`.
| setheron wrote:
| Nix gives you a commit guarantee rather than arbitrary versions.
| You're going to put yourself in a bad time when you have edge
| cases: glibc changes or conflicting shared libraries.
|
| It sounds like it's a little bit too late, but I'm happy to
| provide some consulting on how you can get it to work
| idiomatically with Nix.
|
| Product looks cool!
| nialv7 wrote:
| nix solves the shared library incompatibility problem by being
| extremely conservative. every time anything changes,
| consequential or not - a comment got modified, documentation
| changes, a testcase got added, etc. - it will rebuild all
| dependents. and not just that, but all dependents of
| dependents, and dependents of dependents of dependents, on and
| on. this often results in massive massive rebuilds.
|
| sure you are not going to get shared library conflicts, but i
| think this solution is extremely wasteful, and can make
| development painful too - look at nixpkgs' staging process.
| smilliken wrote:
| The reason someone changes a dependency at all is because
| they expect a difference in behavior. No one would feel the
| motivation to go update a dependency if they aren't getting
| something out of it, that's a waste of effort and an
| unnecessary risk.
|
| Each person doesn't have to perform the build on their own. A
| build server will evaluate it and others will pull it from
| the cache.
|
| The greater waste that nix eliminates is the waste of human
| time spent troubleshooting something that broke in production
| because of what should have been an innocent change, and the
| lost business value from the decreased production. When you
| trust your dependencies are what you asked for, it frees the
| mind of doubt and lets you focus on troubleshooting more
| efficiently towards a problem.
|
| Aside, I spent over a decade on Debian derived distros. I
| never once had one of these distros complete an upgrade
| successfully between major versions, despite about 10
| attempts spread over those years, though thankfully always on
| the first sacrificial server attempted. They always failed
| with interesting issues, sometimes before they really got
| started, sometimes borking the system and needing a fresh
| install. With NixOS, the upgrades are so reliable they can be
| done casually during the workday in production without
| bothering to check that they were successful. I think that
| wouldn't be possible if we wanted the false efficiency of
| substituting similar but different packages to save the build
| server from building the exact specification. Anything short
| of this doesn't get us away from the "works on my machine"
| problem.
| gf000 wrote:
| Yep, anyone not getting how absolutely huge the Nix model
| is should just install the whole KDE desktop, the Gnome
| desktop, and uninstall both. Only nix can make it basically
| a no-op.
| nialv7 wrote:
| > With NixOS, the upgrades are so reliable
|
| Yeah they may be reliable _for you_. And do note this
| reliability doesn't come automatically with Nix's model, it
| is only possible because many people put a lot of effort
| into making it working correctly.
|
| If you use the unstable channels, you would know. My NixOS
| upgrades break _all_ the time. On average, probably once a
| month.
| setheron wrote:
| Nix has support for bit reproduction and will not rebuild on
| comments if you specify it.
|
| Of course lots of software isn't ready for but reproduction
| which is why Nix has taken such a pragmatic approach. (I have
| written a lot about this).
|
| It's all a series of tradeoffs. If your goal is
| reproducibility (as close as you can get), you will have a
| larger graph likely ..since you are accounting for more!
|
| Sometimes we like to believe we can have our cake and eat it
| too rather than understand life's a series of tradeoffs.
|
| When we think we are getting a silver bullet, we've likely
| just pushed that complexity somewhere else.
| nialv7 wrote:
| IIUC you are talking about CA-derivations? Yeah they may
| help but it's hard to know how much since it's not in
| production yet, despite being part of Eelco's original
| paper describing nix. So my hope isn't high.
|
| > When we think we are getting a silver bullet, we've
| likely just pushed that complexity somewhere else.
|
| True but we kind of just stopped looking. and I feel much
| of the solution space hasn't been explored.
| gf000 wrote:
| I mean, there is no other way that guarantees correctness
| across arbitrary tools/functions/builds. Like, what if I have
| a step that replaces certain comments with code?
|
| Also, the primary way to develop with Nix is to _create your
| exact, reproducible environment_ in the form of a shell, and
| then develop there using the usual, language-idiomatic
| iterative way.
|
| But now you can actually have a very specific compiler-flag
| for only a single dependency mixed with a full different libc
| working in a given shell 100%, for you and everyone else,
| instead of iterating through nodejs and npm version
| combination to start working on this new project, taking a
| couple of days..
| eddythompson80 wrote:
| > You're going to put yourself in a bad time when you have edge
| cases: glibc changes or conflicting shared libraries.
|
| I totally understand the value proposition of Nix. However I
| think saying "bad time" is a bit hyperbolic. At most it's
| "You'll be losing a pretty significant guarantee compared to
| Nix". Still probably "packed to be more likely to work
| correctly" than 95% of software out there.
| femiagbabiaka wrote:
| The new design is very similar to Dagger, interesting.
| wg0 wrote:
| > We also changed the codebase from Rust to Go because of the
| Buildkit libraries.
|
| Go is the best choice at the moment for such tools. These tools
| start a process, do lots of IO and exit.
|
| Very pragmatic choice.
| Onavo wrote:
| Go also statically links all dependencies and reinvents all the
| wheels usually provided by the system land. Cross compilation
| is trivial. It is unrivaled when it comes to deployment
| simplicity.
| xvilka wrote:
| Rust does the same.
| hu3 wrote:
| Yea and at least Go provides a giant Google engineering
| tier quality standard library so reinventing the wheel here
| doesn't hurt so much productivity.
|
| Meanwhile Rust requires a pile of variable quality
| community driven crates to do basic things.
| bigyabai wrote:
| Both languages have enormous cargo-culting issues when
| you try to do anything that isn't fizzbuzz. The bigger
| difference that I'd expect people to identify is that
| Rust generates freestanding binaries where Go software
| requires a carefully-set runtime. There are pros and cons
| to each approach.
| hu3 wrote:
| None of them generate true freestanding executables by
| default. Both Go and Rust require glibc.
|
| And Go's runtime is built-in by default. Unlike Java so
| there's nothing to "carefully set".
| bigyabai wrote:
| By that definition, I cannot name a single high-level
| programming language that generates freestanding
| binaries.
| hu3 wrote:
| Many of them can, including Rust and Go. Just not with
| default arguments. You need to pass linking arguments.
|
| This is not done by default to reduce binary sizes.
| XorNot wrote:
| By default no for Go, but it's much easier to get their
| then Rust.
|
| In Go you run this to build:
| CGO_ENABLED=0 GOOS=linux go build -a -ldflags
| '-extldflags "-static"' .
|
| I do not actually know how you get this done in Rust (you
| do something with musl and a bunch of other stuff).
| bithavoc wrote:
| Rust can cross-compile, yes, but is not as seamless. For
| example, Rust can not cross-compile Windows binaries from
| Linux without external support like MinGW.
|
| Go can cross-compile from Linux to Windows, Darwin and
| FreeBSD without requiring any external tooling.
| jchw wrote:
| Okay I'm a Nix enthusiast but you'll have to trust me when I say
| that I'm not criticizing them for moving away from Nix; it isn't
| that strong of an emotional attachment. However, I'm not really
| sure I understand some of these complaints and they really could
| use more explanation. For example:
|
| > The biggest problem with Nix is its commit-based package
| versioning. Only the latest major version of each package is
| available, with versions tied to specific commits in the nixpkgs
| repo.
|
| While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is
| highly unideal for cases where you want to be able to pull
| arbitrary versions of toolchains, but it is not the only way to
| go. For example, there is amazingly good Nix tooling for pulling
| an arbitrary version of Rust. Other Nix-based developer tools
| have shown how you can do this well.
|
| > no way of splitting up the Nix dependencies into separate
| layers
|
| That doesn't make any sense. You can literally just split them
| into separate layers in whatever arbitrary fashion you'd like.
| The built-in Nixpkgs docker tooling has some support for this
| even.
|
| > We also changed the codebase from Rust to Go because of the
| Buildkit libraries.
|
| This part is not related to Nix, but I find it interesting
| anyways. Obviously most people don't transition programming
| languages on a whim, it's generally something you do when you're
| already planning on building from scratch anyways. To me it
| almost sounds like different people worked on Railpacks vs
| Nixpacks.
|
| (I've definitely seen what happens when people not familiar with
| Nix wind up having to deal with unfinished Nix solutions within
| an organization. It is not pretty, as most people are unwilling
| to try to figure out Nix. I don't _generally_ use Nix at work out
| of fear of causing this situation.)
| cchance wrote:
| Honestly this feels more like rail... wants to make their own
| version, hence a new railX lol
| dleslie wrote:
| I don't use Nix, however this seems dismissive:
|
| > While Nixpkgs is an amazing resource, Nix != Nixpkgs.
|
| If Nixpkgs is the default and alternatives require additional
| research and effort then for most users it _is_ Nix.
|
| > That doesn't make any sense. You can literally just split
| them into separate layers in whatever arbitrary fashion you'd
| like. The built-in Nixpkgs docker tooling has some support for
| this even.
|
| Is this obvious, simple, and default behaviour?
| whateveracct wrote:
| Nixpkgs isn't Nix and in production you rarely just use
| Nixpkgs verbatim. It's trivial to overlay whatever versions
| you want (including forks), and I'd say it's expected for any
| company in production to manage their package set.
|
| We are talking about a company full of professionals. If they
| need something obvious, simple, and default to manage their
| build - the core business function that turns their text into
| deployable artifacts - maybe there is a skill culture issue.
|
| The industry is full of ineptitude though.
| eddythompson80 wrote:
| > The industry is full of ineptitude though.
|
| While I disagree with the person you're replying to, I find
| _your_ reply dismissive.
|
| I don't know the behind-the-scnene reasons for this, but I
| can very very easily apply a very similar situation to this
| from my experience.
|
| Nix is a full blown functional programming language along
| with a very rich (and poorly documented, niche, only second
| to C++ template in error comprehensibility[1]) ecosystem in
| itself. It's not like "docker" or "kubernetes" where you're
| mostly dealing with "data" files like yaml, json or
| Dockerfile. You're dealing with a complex programming
| project.
|
| With that in mind:
|
| - You have a core team with 1 or 2 people with Nix
| passion/expertise.
|
| - Those people do most of the heavy lifting in
| implementation.
|
| - They onboarding the team on to Nix
|
| - They evangelize Nix through the org/company
|
| - They mod and answer all the "#nix-discussions" channel
| questions
|
| Initially the system is fairly successful and everything is
| good. over the next 5-6 years it would accumulate a lot of
| feature asks. The original "Nix person" has long left. Most
| of the original people have moved either to other projects
| or not particularly that passionate about Nix. In fact, the
| "best" developer you have who has inherited the whole Nix
| thing has only really had to deal with all the shit parts
| of Nix and the system. They are they ones fixing issues,
| dealing with bugs, etc. All while maintaining 3 stacks, a
| Nix stack, a Go stack, and a Rust stack.
|
| Eventually that person/team that's annoyed by maintaining
| the Nix project wins. They want to own that code. They
| don't want to use Nix any more. They know what's needed,
| they want to implement it as part of their main Go stack
| that they are actively working on. They can optimize things
| for their special case without having to worry about
| "implementing it the Nix way" or "doing it upstream".
|
| They promise you (the management who is open to the idea,
| but trying to understand the ROI) feature parity + top 5
| feature asks for the initial release. You trust the team
| enough to let them do what they think is best.
|
| [1]: LLMs are really good at suggesting a solution given an
| error message. Nix errors bring them to their knees. It's
| always "Hmmm.... it appears that there is an error in your
| configuration... have you tried a `git revert`?"
| whateveracct wrote:
| I'm not being dismissive. Well, I am dismissing a lot of
| industry people's opinions. Because they're bad.
|
| Just because people decide stuff for money doesn't mean I
| can't call them bad. Not everyone is equally skilled.
|
| And your parable is exactly the issue. The unskilled and
| loud and whiny _do_ often win, and it 's a shame. I see
| it all the time.
|
| (Also you're way overstating Nix as a "full blown FP
| language." It isn't hard to learn. I learned it just be
| existing on a project with it. Within 6mo, now _I 'm_
| apparently a "Nix expert" and people now point at me as
| one of the people who "knows it" and "you can't expect
| everyone to know it like you do." idk maybe I'm some
| genius but I think it's more that I just don't have a bad
| personality.)
| eddythompson80 wrote:
| So you are being dismissive. That's what you're doing.
| You're dismissing more than just "stuff for money".
| You're dismissing anything that doesn't fall under the
| "skill" or "technical" category. All software projects
| contain a human element. I was showing an example from my
| experience on how something like that could happen.
|
| > A perfectly capably (but perhaps a bit esoteric)
| technology is picked by a smart passionate person for a
| project.
|
| > The novel technology is in 1 isolated module that's
| mostly feature complete for the first 1-3 years.
|
| > People in the team/company deal with that "thing" as a
| blackbox more and more
|
| > 5-10 years later, mostly new team maintaining the
| project. They hate the weird choice of tech. "Why is only
| this one component different???"
|
| > People understand the contract with the "black box"
| very well, but HATE the "black box". People think "We can
| implement the black box contract very easily"
| whateveracct wrote:
| Yes I am dismissing. People don't have a right to not be
| dismissed if I judge them poorly. People are allowed to
| have bad professional opinions of others.
|
| And I am dismissing the types you describe specifically.
| I dismiss them (privately amongst the likeminded) at work
| all the time too. I just put them on a list in my head
| when they start spouting these sorts of bad values.
| Guvante wrote:
| Two things can be true in your hypothetical:
|
| 1. Someone was forced to maintain Nix and want to switch
| to easier to maintain tooling 2. That someone can lack in
| technical understanding of the problems they are facing
|
| The former doesn't negate the later.
|
| The way I would put it is sometimes you choose a worse
| option because the people you have available are better
| at that option. That doesn't mean you made a mistake but
| it does mean your lack of expertise sent you down a
| different path.
|
| And of course to finalize I will re-emphasize my "didn't
| make a mistake" comment. Ivory tower isn't a good idea
| either.
|
| But someone responding "it is too bad the company that
| built packages couldn't properly use the package tooling
| they depended on" can still be true in a situation where
| a company made the correct decision of dropping that
| package tooling.
| Dylan16807 wrote:
| > maintaining 3 stacks, a Nix stack, a Go stack, and a
| Rust stack
|
| Nix is a package management system with a little bit of
| programming tucked onto the side. "Nix stack" is not the
| same type of thing as "Go/Rust stack". If you try to move
| things over to Go/Rust, you'll spend a little bit of time
| rewriting the code and a whole lot of time reinventing
| the wheel on everything else involved. You're not moving
| between implementations, you're building your own
| implementation. That's almost always a bad idea, and it's
| a much higher cost than learning the syntax.
|
| Moving from a Nix stack to a Go stack only makes
| _slightly_ more sense than moving from a docker stack to
| a Go stack. Which is to say, very little sense.
| gf000 wrote:
| But they also have to manage the Git stack on top of all
| that! /s
|
| (Though really, what's up with so many people in the
| industry being absolutely bad at git well into their
| careers?!)
| sepositus wrote:
| Wow, this is literally the story of my team. Luckily
| there's enough autonomy for us the escape the
| gravitational pull of the few remaining evangelists, but
| this is essentially what led us to this point.
| jchw wrote:
| > If Nixpkgs is the default and alternatives require
| additional research and effort then for most users it _is_
| Nix.
|
| This feels rather dismissive. They wrote a bespoke solution,
| not a weekend toy. Surely you'd agree that they have more
| than just surface-level knowledge of Nix, to be able to
| distinguish between Nix and Nixpkgs? They're _already_ doing
| non-trivial things by merging multiple commits of Nixpkgs in
| order to get different versions of different tools!
|
| > Is this obvious, simple, and default behaviour?
|
| Well, Nix doesn't do much of anything "by default", it's a
| pretty generic tool. But insofar as it matters, Yes, pretty
| much. `dockerTools.buildLayeredImage` will in fact
| automatically build a layered image, and it is the most
| "obvious" way (IMO) to build a docker image. There is also
| `dockerTools.buildImage` but there's no particular reason to
| use it unless you specifically want a flattened image. (The
| documentation available is clear enough about this. In fact,
| in practice, much of the time you'd probably actually want
| `dockerTools.streamLayeredImage` instead, which is also
| documented well enough, but that's beyond the point here.)
|
| But that's not my point. As far as I know, Nixpacks don't
| even use this functionality, I'm pretty sure they wrote their
| own OCI image building tools. And in that sense, it is not
| obvious why they can't split the Nix store and the article
| doesn't explain it.
|
| My point wasn't to be dismissive about the difficulties of
| Nix, it's that the blog post doesn't really do a good job of
| explaining things. It makes it sound like these are normal
| problems in Nix, but they are not; even the official Nixpkgs
| documentation often points to third party solutions for when
| you're working outside of Nixpkgs, since most of the Nixpkgs
| tools is geared for Nixpkgs and NixOS usage. As an example,
| take a look at this section of the Rust documentation in
| Nixpkgs:
|
| https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f.
| ..
|
| So even if you're relatively new to Nix, as long as you are
| reading the documentation you will indeed definitely be aware
| of the fact that there is more to the Nix ecosystem than just
| Nixpkgs. It may not be surface-level Nix knowledge, but it's
| certainly close.
| kfajdsl wrote:
| This isn't "most users", this is a large company building a
| product on top of Nix. I'm pretty sure most orgs using Nix at
| a _minimum_ have a custom overlay.
|
| If you identify these things as an issue, any competent
| engineer should find a variety of solutions with search
| and/or LLM assistance within an hour, since they're not super
| obscure requirements.
|
| I'm not saying Railway didn't do this and realize that these
| common solutions weren't viable for them, but it's odd to not
| mention anything they tried to get around it.
| nothrabannosir wrote:
| To emphasize this point: dleslie's comment is valid on a
| blog post "we tried Nix for a while to manage our
| dependencies, we are just building an app and want builds
| to work, and we decided to move on". For an end user, it is
| absolutely understandable to assume "nix = nixpkgs".
|
| But as kfajdsl points out: that's not what TFA is. This is
| a company building a product _on top of Nix_. Package
| management is their expertise. Anyone using Nix in that
| capacity understands the distinction between nix and
| nixpkgs. Which they certainly do--GP only remarked it was
| odd they didn 't _explain_ it, not that they didn 't
| _know_.
| trws wrote:
| > Is this obvious, simple, and default behaviour?
|
| Yes. There are two options IIRC, minimum layers and maximum
| layers (one per dep by default unless that makes too many,
| which is handled automatically) depending on what you want,
| and it's a Boolean flag. If you need more control it's more
| complicated but this one really is a strange criticism unless
| they're using non-standard wrappers for the usual nix way to
| do this.
| Nullabillity wrote:
| > Is this obvious, simple, and default behaviour?
|
| In the documentation[0] they're right next to each other, and
| `buildImage`[1] (builds a single layer) specifically calls
| out that you probably want to use `buildLayeredImage` or
| `streamLayeredImage`[2] (both produce a separate layer per
| dependency) instead.
|
| Neither should cause the final image to include build
| dependencies, _that_ sounds like they were doing something
| silly like running `nix-build` from inside a Dockerfile and
| just taking that as their final image. Which.. yes, would
| include build cruft. Oh,[3] I guess that was _exactly_ what
| they were doing after all. And mixing in Debian packages...
| for reasons, I guess.
|
| [0]: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-
| dockerTool...
|
| [1]: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-
| dockerToo...
|
| [2]: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-
| dockerToo...
|
| [3]: https://github.com/railwayapp/nixpacks/blob/205b33b51528
| 2cdf...
| gf000 wrote:
| > If Nixpkgs is the default and alternatives require
| additional research and effort then for most users it _is_
| Nix.
|
| This is not what parent commenter is getting at. Nix itself
| is a deterministic build tool. There is also a package
| manager built on top which uses a large collection of nix
| files to describe each package - this is nixpkgs.
|
| They use the same primitives, but the same way you don't just
| yolo your node/rust etc build versions to whatever your OS
| comes with and use a lock file, you also want to have more
| control over the exact versions and thus may use something
| other than what nixpkgs packages. Especially that it makes it
| easy to override any property of your dependencies, unlike
| any other tool out there.
| jrockway wrote:
| I think this is pretty well stated. I'll add that while nixpkgs
| isn't nix, nixpkgs is kind of the good part. I use NixOS and
| for the first time in my life, I'm using the latest version of
| the Linux kernel on release day. That's pretty excellent. While
| I've come to tolerate Debian Stable in my old age, it is always
| like stepping a few years into the past ;)
|
| The Nix language is something I could criticize for hours
| without getting bored, but it is what it is. It's old and they
| did the best they could and it's probably not worth changing.
| The Nix build system feels awfully primitive to me, often
| rebuilding stuff that doesn't need to be rebuilt for no good
| reason. (For example, my NixOS installer ISO has a ton of the
| build depend on the cmdline I pass to the kernel [just
| console=ttyS2,1500000n8], and so changing the speed of my
| serial port requires about 3 minutes of build time. It's goofy
| and makes me laugh, I'm not going to stop using Nix because of
| it... but it's also something that I wouldn't let happen in MY
| build.)
|
| Nix for Docker images is, in my opinion, what it's the worst
| at. A long time ago, I was writing some software in Go and
| needed to add the pg_dump binary from Postgres to my container
| image. The infrastructure team suggested using Nix, which I
| did, but our images blew up from 50MB of our compressed go
| binary to 1.5GB of God Knows What. pg_dump is 464K. I ended up
| doing things my way, with Bazel and rules_debian to install apt
| packages, and the result (on top of distroless) was much
| cleaner and more compact. My opinion with some actual Nix
| experience is that a Nix system always ends up being 1.4GB. My
| installer ISO is 1.4GB. My freshly installed machine is 1.4GB.
| That's just how it is, for whatever reason.
|
| Finally, the whole "I would like to build a large C++ project"
| situation is a well worn path. s/C++/Rust doesn't change
| anything material. There are build systems that exist to make
| the library situation more tolerable. They are all as
| complicated as Nix, but some work much better for this use
| case. Nix is trying to be a build system for building other
| people's software, supporting nixpkgs, and lands on the very
| generic side of things. Build systems that are designed for
| building _your_ software tend to do better at that job.
| Personally, I 'm happy with Bazel and probably wouldn't use
| anything else (except "go build" for go-only projects), but
| there are many, many, many other options. 99% of the time, you
| should use that instead of Nix (and write a flake so people can
| install the latest version of Your Thing with home-manager; or
| maybe I'm just the only person that uses their own software day
| to day and you don't actually need to do that...)
| internet_points wrote:
| > a Nix system always ends up being 1.4GB
|
| That's strange, I never had problems building really tiny
| docker (release) images with nix, in fact it felt easier than
| doing it with alpine. You just get exactly what you specify,
| no more.
|
| (OTOH, when _developing_ in nix, I always end up with a huge
| /nix/store and have no idea how to clean it without garbage
| collecting everything and having to wait all over)
| chriswarbo wrote:
| > I always end up with a huge /nix/store and have no idea
| how to clean it without garbage collecting everything and
| having to wait all over
|
| FYI you can avoid things getting garbage-collected by doing
| `nix-store --add-root`; that makes an "(indirect) garbage
| collector root"[0]. Especially useful if you're using
| import-from-derivation, since that imported derivation
| won't appear in the dependencies of your final build output
| (which, to be clear, is a good thing; since it lets us
| calculate a derivation, e.g. by solving dependency
| constraints or whatever, without affecting the eventual
| hash if that calculation happens to match a previous one!)
|
| [0] https://nix.dev/manual/nix/2.18/package-
| management/garbage-c...
| baobun wrote:
| I think the part that's easy to miss is that their users are
| devs who will want to specify their own dependencies and
| versions for arbitrary packages.
|
| The way nix works with the way nixpkgs is structured, pinning a
| version of any package means pinning a commit of the entire
| nixpkgs tree. Since package builds of node/python/ruby packages
| depend on stuff outside of the package dir in the tree, you
| need that mapping between versions and commits. It is also a
| leaky abstraction, so they will need to expose that to their
| users, who now may run into situations where they need to align
| various states of the nixpkgs repo when they just wanted to
| "yarn add new-fancy-nodejs-package-with-linked-native-deps".
|
| Using nix without nixpkgs may be fine for more scoped use but
| seems hard to justify for a platform like Railway.
| Nullabillity wrote:
| > Since package builds of node/python/ruby packages depend on
| stuff outside of the package dir in the tree, you need that
| mapping between versions and commits.
|
| > Using nix without nixpkgs may be fine for more scoped use
| but seems hard to justify for a platform like Railway.
|
| Nixpkgs isn't all-or-nothing. You're right that Nixpkgs
| itself rarely packages more than one version of something,
| but the standard approach for "language package managers" is
| that you use a tool like crate2nix[0] which automatically
| generates pinned derivations for all of your dependencies.[1]
| For system dependencies which aren't covered by your language
| package manager.. you're basically in the same position as
| for something like Debian: you can either pull it from
| Nixpkgs (and give up control beyond "which Nixpkgs am I on?")
| or you can write/fork your own package. Or you can pull
| specific packages from specific Nixpkgs checkouts and splice
| them into your "main" Nixpkgs version as an overlay (though
| this is definitely getting into the Weird Territory(tm)).
|
| [0]: https://github.com/nix-community/crate2nix
|
| [1]: For example: https://github.com/stackabletech/secret-
| operator/blob/30f0eb...
| gosub100 wrote:
| > Nix != Nixpkgs
|
| I've been told this when trying FreeBSD in regards to freebsd
| ports. pkg generally works fine for me, but one day I tried to
| go off the beaten path and compile vim with some custom USE
| flags (I forget what they are called in freebsd) in the ports
| section. It pulled down 20+ dependencies and each `make
| menuconfig` kept asking me "would you like any of these
| options" I selected a few that seemed reasonable, and lo-and-
| behold, package 16 out of 23 fails because "this-requires-that
| and that-needs-Fubar3.32.1 and Fubar3 is deprecated for Fubar4"
| and I just gave up. I get that the Core OS devs can't support
| all 10k+ packages, but they should also be very clear that if
| you actually try to _use_ them (i.e. enable custom features,
| not just compile stock code) there 's a high chance they won't
| work. Another option would be to yank them from the ports list
| if they don't compile and require at least some standard,
| independently-produced build to succeed before they appear in
| portsnap fetch.
| spease wrote:
| > While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs
| is highly unideal for cases where you want to be able to pull
| arbitrary versions of toolchains, but it is not the only way to
| go. For example, there is amazingly good Nix tooling for
| pulling an arbitrary version of Rust. Other Nix-based developer
| tools have shown how you can do this well.
|
| It's so exhausting that every single time a basic use issue
| comes up with nix, the response is "but there's a way to work
| around it (that you need tribal knowledge for and will require
| writing dozens to hundreds of lines of code to fix in a
| language that doesn't work like any of the mainstreams with bad
| error messages and poorly documented standard libraries)".
|
| People's problems with nix are not that it isn't turing-
| complete, it's that it often creates more problems than it
| solves by refusing to provide a simple first-class API that
| interoperates with idiomatic projects in that ecosystem that
| just works.
|
| If every project you try to use nix for devolves into centering
| around trying to fix issues with nix that you have to write
| your own modules for, why even bother using nix instead of
| mainstream tools with good documentation? Exactly what happened
| in this case. In most cases people are probably just deciding
| to use docker instead.
|
| Nix's refusal to address practical developer experience issues
| for a developer-facing product on a non-geologic timescale in
| favor of ideological pure flakes is rather frustrating.
|
| Yes, people are contributing their own time, but it's so damn
| frustrating to see so much technical effort going into
| something that's rendered practically unusable because of bad
| UX.
| __MatrixMan__ wrote:
| You can use nixpkgs as a single versioned input, and you can
| also give certain things their own distinct input to be
| tracked separately in flake.lock, but I wouldn't describe
| either as a "workaround" for the other.
|
| It is a bit of a headache to have to pick which mode you want
| for each dependency, but I'm not sure that's a headache that
| can be dispensed with via UX improvements.
|
| > why even bother using nix instead of mainstream tools with
| good documentation?
|
| nix lets you have a single source of truth for all of it.
| It's not nix vs apt or nix vs pip, but rather nix vs (pip &
| (apt|brew)).
|
| So far as I know, the only other tool that scratches that
| itch is bazel, otherwise you're kind of stuck with multiple
| overlapping packaging strategies that don't communicate,
| which is a recipe for "works on my machine".
| jchw wrote:
| > It's so exhausting that every single time a basic use issue
| comes up with nix, the response is "but there's a way to work
| around it (that you need tribal knowledge for and will
| require writing dozens to hundreds of lines of code to fix in
| a language that doesn't work like any of the mainstreams with
| bad error messages and poorly documented standard
| libraries)".
|
| > People's problems with nix are not that it isn't turing-
| complete, it's that it often creates more problems than it
| solves by refusing to provide a simple first-class API that
| interoperates with idiomatic projects in that ecosystem that
| just works.
|
| Well, first of all, this article isn't even about the UX of
| using Nix directly, it is about Nixpacks, a tool built on top
| of Nix. If Nix already solved the problems Nixpacks were
| trying to solve, they would've had no reason to write it, so
| I don't really see how this could be relevant.
|
| > If every project you try to use nix for devolves into
| centering around trying to fix issues with nix that you have
| to write your own modules for, why even bother using nix
| instead of mainstream tools with good documentation? Exactly
| what happened in this case. In most cases people are probably
| just deciding to use docker instead.
|
| Would it matter if there was good documentation if you were
| not willing to read it anyway? Both of the issues I talked
| about actually are covered in the official Nixpkgs
| documentation and have been for years. For example, here is
| some of the documentation for layered docker images:
|
| https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-
| dockerToo...
|
| Nixpkgs and NixOS are maybe not perfectly documented, but on
| the other hand, they're absolutely enormous projects. The
| documentation that does exist is already staggering. And of
| course it is! Seriously, try finding a singular project that
| has as big of a scope as Nixpkgs...
|
| Of course, what people seem to read from what I said is that
| Nix is actually perfect and there are no problems, but what
| I'm really saying is they didn't go into very much detail on
| how they tried to solve the issues they ran into. The two
| possibilities are that they didn't try very much, or that
| they did and they omitted it. The latter is certainly as
| plausible, but it leads to this confusing problem where their
| complaints don't really make much sense on their own. What I
| am talking about is not obscure bespoke tribal knowledge.
| It's pretty close to the first stuff you would learn if you
| read docs or tutorials.
|
| > Nix's refusal to address practical developer experience
| issues for a developer-facing product on a non-geologic
| timescale in favor of ideological pure flakes is rather
| frustrating.
|
| > Yes, people are contributing their own time, but it's so
| damn frustrating to see so much technical effort going into
| something that's rendered practically unusable because of bad
| UX.
|
| This really seems like it's veering far away from the
| discussion about Nixpacks and into personal grievances about
| Nix, but honestly there's thousands of build systems and
| package managers that don't work the way Nix does, I do not
| really see why Nix should compromise its ideals. for the sake
| of UX. But please don't get me wrong: I agree on the point
| that Nix has a worse UX than it could; I think the language
| is full of unneeded papercuts and confusing things and
| meanwhile the time it has taken to stabilize flakes and the
| new Nix command has really been a drag and introduced a lot
| of confusion. On the other hand, though, I'm not even sure
| it's worth wasting too many tears over this: as hard as it is
| to get started in Nix and as many things as there are that
| could be improved with Nix and Nixpkgs itself, really the
| chief pain I feel when dealing with anything involving Nix is
| not Nix itself but the world it exists in.
|
| Nix has existed for over 20 years and in those years the
| world of OS design and package managers only really started
| to move towards immutability and purity relatively recently.
| Most software still likes to have tons of mutable, impure
| shared state and trying to encapsulate this into Nix can be
| very painful. Nix has grown many limbs and improved in many
| ways to try to deal with this, but it is still far and away
| one of the biggest sources of confusion that I have with Nix.
|
| It is not a given that things have to work this way, but it
| is a natural consequence of the fact that Nix is trying to do
| something that is very much at odds with the way software has
| worked for a very long time. The impedance mismatch is
| massive and unavoidable, but I don't think the answer is that
| Nix should bend to deal with this. If someone finds what they
| feel is a better sweet spot between what Nix is today and the
| world outside of Nix, they should feel free to pursue that,
| but I'm involved with Nix because I think it has the right
| ideals just way too early.
|
| Having dealt with systems that try to be hermetic like Bazel,
| you can certainly get _some_ meaningful benefits from sitting
| at a midway point, but what we 're chasing are the benefits
| you get much closer to the end-game of hermetic systems, when
| hermeticity is enforced strongly across the entire system.
| This is about more than just being a convenient tool for
| developers and much more into the future of how systems are
| designed. And yeah, sure, when you try to build on top of
| this in today's world, it can be awkward, nobody is denying
| that, but a lot of this awkwardness is, unfortunately, a
| feature, and while clever Nix solutions may eventually exist
| for some of those problems (a lot of clever work _is_ being
| done, with concepts like dynamic derivations) I really feel
| strongly that you shouldn 't hold your breath.
|
| And sure, by all means, switch to OCI tooling if you feel
| like it works better. It may very well _actually_ be better
| for some use cases! But you literally _can not_ replace
| _most_ of what Nix is used for and can do with OCI tooling,
| it 's ultimately a very small subset of what Nix is capable
| of overall.
| hnaccount_rng wrote:
| I'm very sorry. But for a person that tries to get into Nix
| this reads like pure gaslighting. I haven't even found an
| official description of what Nix vs NixOS vs nixpkgs tries
| to achieve vs what they do not want to achieve.
|
| There is a near infinite collection of blog posts what
| people use Nix for. All full of Nix-internal jargon
| derisively critiquing alternative (that do in fact actually
| work) without ever coming out and laying the groundwork for
| what the underlying problem is _and what the solution to
| that is_.
|
| I think that I would end up liking Nix in principle. I
| really did enjoy the Spack package management system and I
| do think Nix is doing things reasonably similarly
| (everything is Nix, all inputs are securely provided,
| reusing things as much as possible).y current problem is
| taking a homelab server into "production" and all I really
| want is a way to write down a state I want the services on
| it to be in, have that in a git repo and be able to
| regenerate this server after loosing it (in a first step
| without any data backup). I'm still, after 2 months (wall
| clock, call it maybe 3 days actually getting into this
| project), not sure if that's a thing Nix tries to cover! (I
| think it does, but..)
| cpuguy83 wrote:
| I don't understand the versioning argument. New to nix, but I
| definitely have packages from a specific commit.
| eviks wrote:
| Couldn't quickly find the blog introducing nixpacks, but weren't
| these issues clearly visible from the start?
|
| > Smaller Builds > Better caching
|
| what were the benefits that overcame this, and what about those
| now?
| foooorsyth wrote:
| >The biggest problem with Nix is its commit-based package
| versioning.
|
| I am naive about Nix, but...
|
| ...isn't that like...the whole selling point of Nix? That it's
| specific about what you're getting, instead of allowing ticking
| time bombs like python:latest or npm-style glibc:^4.4.4
|
| Odd to attach yourself to Nix then blog against its USP.
| grep_name wrote:
| Eh. I've been using nixOS for years now and still find that I
| often desperately, desperately wish I could upgrade just one
| program that I need a new version of without risking that, you
| know, any individial single one of my installed packages has a
| change between the last update and now that messes up my
| workflow. Or that I could pin a version of a software that I'm
| happier with that version of without essentially rolling my own
| package repo. It is, in fact, the only package manager I'm
| aware of that makes it such a pain to do that. It's only
| because people in this thread are insisting it's doable that I
| say 'such a pain' instead of just 'impossible'.
|
| A few weeks ago I needed to update firefox for a bug fix that
| was causing a crash, but of course that meant updating all of
| nixpkgs. When I finished the switch, the new version of
| pipewire was broken in some subtle way and I had to roll it
| back and have been dealing with firefox crashing once a week
| instead. I can't imagine pitching this to my team for
| development when I'm having this kind of avoidable issue just
| with regular packages that aren't even language dependencies.
|
| To those who say 'if you want to lock your dependencies for a
| project, you can just build a nix flake from a locked file
| using the <rust | python | npm> tools' I say, why the hell
| would I want to do that? Being able to manage multiple
| ecosystems from the same configuration tool was half the draw
| of nix in the first place!
| kesor wrote:
| Creating overlays on nixpkgs is fairly trivial. There was a
| bug a couple of weeks ago with yt-dlp, which was fixed in the
| nightly version. But even today the new version is not yet
| available in nixpkgs-unstable. Did I wait all this time with
| a broken yt-dlp? No! I created a derivation that overrides
| the git commit of the original yt-dlp, and added it as an
| overlay to nixpkgs. Once nixpkgs has the newer version, I'll
| remove the overlay and customized derivation. This is a
| 5-lines-of-code change. You don't have to use flakes if you
| don't want to, it works without flakes as well.
|
| Now compare the above with how you would customize a version
| in other systems, like Debian with apt-pkgs ...
| justjake wrote:
| Bingo. Nix doesn't give you a generalizable-across-languages-
| and-ecosystems way of specifying specific versions without
| blowing up your package size, unless you hand Nix to your
| users (which we didn't want to do)
|
| Maybe we were holding it wrong, but, we ultimately made the
| call to move away for that reason (and more)
| yjftsjthsd-h wrote:
| Um? That's trivial with flakes (and I think it was doable
| without flakes, but I don't really remember/care). For one-
| offs (I'd probably do this for your firefox example but
| YMMV), just tell it the version to run: $ nix
| run nixpkgs#firefox -- --version Mozilla Firefox
| 138.0.1 $ nix run github:nixos/nixpkgs/nixos-
| unstable#firefox -- --version Mozilla Firefox 139.0.1
| $ nix run github:nixos/nixpkgs/b98a4e1746acceb92c509bc496ef3d
| 0e5ad8d4aa#firefox -- --version Mozilla Firefox 122.0.1
|
| Or, if you want to actually incorporate it into your system,
| tell the system flake to pull whatever versions you want:
| { inputs = { nixpkgs.url =
| "github:NixOS/nixpkgs/nixos-24.11"; nixpkgs-
| unstable.url = "github:nixos/nixpkgs/nixos-unstable";
| nixpkgs-b98a.url = "github:nixos/nixpkgs/b98a4e1746acceb92c50
| 9bc496ef3d0e5ad8d4aa"; }; outputs = { self,
| nixpkgs, nixpkgs-unstable, nixpkgs-b98a }: {
| nixosConfigurations.yourmachinename = nixpkgs.lib.nixosSystem
| { system = "x86_64-linux";
| specialArgs = { nixpkgs-unstable = import
| nixpkgs-unstable { system =
| "x86_64-linux"; };
| nixpkgs-b98a = import nixpkgs-b98a {
| system = "x86_64-linux"; };
| }; ---snip---
|
| and then when you pull packages say which one you want:
| packages = with pkgs; [ dillo # from stable
| nixpkgs nixpkgs-unstable.firefox # from unstable
| nixpkgs-b98a.whatever # from some exact commit of nixpkgs
| ]
|
| I assume you could do the same thing for project-level
| flakes, but TBH I don't usually do that so I don't have the
| code to hand. (In contrast with grabbing system packages from
| whatever version of nixpkgs I want, which I know works
| because I pulled the example code from the config on the
| machine I'm typing this comment on.)
| rgoulter wrote:
| > > The biggest problem with Nix is its commit-based package
| versioning. > ...isn't that like...the whole selling point of
| Nix?
|
| Not quite.
|
| That sentence is definitely the most ... discussion-worthy
| comment in the blog.
|
| To my understanding, OP wants to write a tool to make it easy
| for use cases like "use ruby 3.1 and gcc 12 and ...".
|
| The main Nix repository is nixpkgs. Nix packages are source-
| based, so the build steps are declared for each version. To
| save maintenance effort, nixpkgs typically only maintains one
| version of each program.
|
| I read OP's "commit-based package version" phrase to mean "if
| you want ruby 3.1, you need to find the latest commit in
| nixpkgs which used ruby 3.1, and use that nixpkgs revision". --
| Although, worth noting, this isn't the _only_ way to do it with
| Nix.
|
| Though, regarding 'commit-based versioning' as Nix's USP? I'd
| say that's also a reasonable description, yes. (By pinning a
| particular revision of Nix, the versions you use will be
| consistent).
| awinter-py wrote:
| tldr we are moving on from nix because we are selling an
| alternative to nix?
| lewo wrote:
| > With no way of splitting up the Nix dependencies into separate
| layers
|
| nix2container [1] is actually able to do that: you can explicitly
| build layers containing a subset of the dependencies required by
| your image. An example is provided in this section:
| https://github.com/nlewo/nix2container?tab=readme-ov-file#is...
|
| For instance, if your images use bash, you can explicitly create
| a layer containing the bash closure. This layer can then be used
| across all your images and is only rebuild and repushed if this
| bash closure is modified.
|
| > > pull in dependencies often results in massive image sizes
| with a single /nix/store layer
|
| This is the case for the basic nixpkgs.dockerTools.buildImage
| function but this is not true with nix2container, nor with
| nixpkgs.dockerTools.streamLayeredImage. Instead of writing the
| layers in the Nix store, these tools build a script to actually
| push the image by using existing store paths (which are Nix
| runtime dependencies of this script). Regarding the nix2container
| implementation, it builds a JSON file describing the Nix store
| paths for all layers and uses Skopeo to push the image (to a
| Docker deamon, a registry, podman, ...), by consuming this JSON
| file.
|
| (disclaimer: i'm the nix2container author)
|
| [1] https://github.com/nlewo/nix2container
| schlarpc wrote:
| Just wanted to say thanks for nix2container. I've been using it
| to do some deploys to AWS (ECR) and my iteration time between
| builds is down to single digit seconds.
| mplanchard wrote:
| We've had issues with docker image sizes and have been meaning
| to take some time to experiment with nix2container. Thanks for
| your work!
| rcarmo wrote:
| Hmmm. Interesting. I looked at the deployment file and... I'm
| doing a very similar thing, but with docker-compose and caddy:
| https://github.com/piku/kata/tree/compose
|
| Completely different approach to dependencies, though. For now.
| api wrote:
| Nix strikes me as an incredibly well thought out solution to a
| set of problems that should not exist.
|
| The OS should be immutable. Apps and services and
| drivers/extensions should be self contained. Things should not be
| installed "on" the OS. This entire concept is a trillion dollar
| mistake.
| chpatrick wrote:
| Well, what you're describing is NixOS.
| dnr wrote:
| Everything you said "should" is exactly what Nix and NixOS is.
| timeon wrote:
| This is not appropriate to post here but the site has nothing to
| do with railways.
| neuroelectron wrote:
| Nix seems to be being undermined internally. I wonder why that
| would be happening.
| brianjlogan wrote:
| Do you have any more observations of this happening?
| neuroelectron wrote:
| https://x.com/jonringer117/status/1927872172514840677
| Aurornis wrote:
| From the headline I thought they were making a single incremental
| change (Nix) but the article sounds like they're doing an entire
| rewrite under a new project name:
|
| > Since we transitioned away from Nix, we also transitioned away
| from the name Nixpacks in favor of Railpack. We also changed the
| codebase from Rust to Go because of the Buildkit libraries.
|
| Suddenly the move away from Nix seems less like an incremental
| change and more like one part of a complete overhaul of the
| entire project. Did they have a team changeover or something? Or
| did they just want to start over from scratch and rewrite the
| whole project?
|
| It also seems strange to switch to an entirely different
| programming language for a single library. I haven't found
| library FFI to be a huge obstacle in Rust.
| cybrexalpha wrote:
| That also jumped out to me as pretty weird. Buildkit as a
| library is nice, but there's no way it's good enough to justify
| throwing out the entire project. It feels like there was some
| big change internally, and the new team either wanted to
| rewrite and didn't know Rust, or ideologically prefer Go to
| Rust.
|
| Edit: Although looking at it, maybe not?
|
| Both the new project railpack[0] and the older one nixpacks[1]
| are both started by and mostly written by the same person[2],
| who is also the author of the article in question. So it
| doesn't look like a team change.
|
| It still feels... odd? Less that they made the change, projects
| go from Rust to Go all the time. But usually it's because of
| issues with Rust (hard to hire for, learning curve, etc.),
| describing it like this feels unusual?
|
| [0] https://github.com/railwayapp/railpack
|
| [1] https://github.com/railwayapp/nixpacks
|
| [2] https://github.com/coffee-cup
| miladyincontrol wrote:
| Not to be dismissive either, but guix might have been more apt
| for what they wanted versus nix. Not trying to suggest guix is
| superior, just possibly a better solution considering their
| gripes with packages.
| jchook wrote:
| Fascinated by so many replies of "actually Nix does this just
| fine, you just have to be an expert like me"
| isbvhodnvemrwvn wrote:
| It's "the usual" when mentioning nix anywhere.
| kesor wrote:
| When a company is writing all of their technology and business
| using, lets say JavaScript. And then they come here and post
| about them switching to some NIH home-brewed language instead
| because they couldn't understand how functions or arrays work.
| That is not a problem with the people commenting on their
| stupidity.
| IshKebab wrote:
| Yeah but if _everyone_ was saying "I don't understand how
| functions or arrays work in Javascript", that's a pretty
| solid indicator that functions and arrays are badly designed
| in Javascript and are unnecessarily hard to understand.
|
| I think in _some_ cases things are just fundamentally
| difficult and them being hard-to-understand is intrinsic. For
| example formal verification in Lean is hard to understand but
| I don 't think Lean is badly designed.
|
| But it's hard to see why package management is one of those
| things. There are soooo many ways Nix could be easier to use
| and understand. The language itself is unnecessarily esoteric
| in my experience - compared to something like Starlark for
| example.
| cmrdporcupine wrote:
| Welcome to the world of Nix.
|
| Just the latest in the line of "my totalizing world view will
| solve all your software problems" to which the answer of "this
| doesn't do what I want" is _always_ "you're holding it wrong."
| alexpotato wrote:
| In my experience as a DevOps/SRE, I feel like every time someone
| tries to have a system to manage dependencies etc, its goes one
| of two ways (I'll use Python as an example):
|
| OPTION 1
|
| "We'll have one big shared mono repo"
|
| Pros:
|
| - it's all in one place
|
| - it's "batteries included"
|
| - everyone uses the same one (so things like vulnerability issues
| are easy to fix)
|
| Cons:
|
| - someone always wants a special version
|
| - hard to do tiered rollouts so changes tend to be big bang
|
| - "But what about if we want to build a small docker version?"
|
| OPTION 2
|
| "Everyone gets their own conda/venv!"
|
| Pros:
|
| - Everyone gets exactly what they want
|
| - Don't use packages they don't need
|
| - Easy to upgrade in phases/tiers etc
|
| Cons:
|
| - "Wait, we have HOW MANY different conda environments??"
|
| - Libraries from different groups may not be tested with the same
| Python libraries
|
| - Vulnerability management is a nightmare b/c you don't even know
| were all of the different conda envs are.
|
| The above is why I'm always skeptical of "This new way will fix
| it all!".
|
| In short, "there are no solutions, only tradeoffs" gets more and
| more true the later I go in my career.
| anacrolix wrote:
| this is from the company that didn't have support for Docker
| images until 2023? come on...
___________________________________________________________________
(page generated 2025-06-08 23:01 UTC)