[HN Gopher] Improved evaluation times with pre-resolved Nix stor...
___________________________________________________________________
Improved evaluation times with pre-resolved Nix store paths
Author : biggestlou
Score : 36 points
Date : 2025-02-12 15:05 UTC (3 days ago)
(HTM) web link (determinate.systems)
(TXT) w3m dump (determinate.systems)
| biggestlou wrote:
| Blog post author here! Please feel free to ask me any questions
| directly in this forum :)
| mongol wrote:
| Can the evaluation tax be quantified in some way? In other
| words, how big is the problem that this solves?
| biggestlou wrote:
| Unfortunately, it's really hard to generalize. If the
| evaluation involves Nixpkgs, for example, Nix needs to copy
| Nixpkgs into the Nix store, which is many MB. And of course
| this needs to happen for all flake inputs. I've seen this
| evaluation take a few seconds and I've seen it take several
| minutes, but in general the more flake inputs the longer you
| can expect it to take.
| grhmc wrote:
| (DetSys Cofounder here) I've had Raspberry Pi's give up their
| last ghostly breath evaluating NixOS.
| asmor wrote:
| DetSys, once again, building a moat.
|
| I really do not like how this approach depends on their seemingly
| not open source backend. Nix needs tools like this, no question,
| but I'd hate to see it get adoption through a way that isn't
| reproducible by anyone. It looks like it could be a repeat of the
| Snap store problem - interesting tech, made unusable through de-
| facto dependencies on proprietary services.
| mongol wrote:
| Is it really a moat though? The same flake can still be
| evaluated without this service, it just takes longer.
| asmor wrote:
| And I can still install a snap on an Ubuntu Core machine by
| building my own distribution method, but it will be a
| downgrade from the experience of running "snap install
| something" that introduced me to it. So nobody does.
|
| Lix (yes, I will mention it again) essentially exists by a
| large degree because the overlap between the people that
| control Nix and DetSys is big, even very big if you look at
| CppNix.
| mongol wrote:
| I don't follow this argument. Unless you chose to step in
| to DetSys ecosystem with this fh tool or whatever, this has
| no impact on you at all.
| larusso wrote:
| I didn't look too deep into Nix the last couple of months (>
| 12) and was wondering while reading: what the hell is fh now.
| Another abstraction? I share your views here!
| biggestlou wrote:
| fh is the CLI tool for FlakeHub. It's been around almost as
| long as FlakeHub itself. It's pretty standard fare, not
| really an abstraction per se.
| rounce wrote:
| 'Standard' if you live in the flakehub ecosystem, which the
| vast majority of Nix users don't.
| biggestlou wrote:
| Standard in the sense of "a non-magical CLI tool that
| wraps a platform's HTTP API," not standard in the sense
| of "used by a majority of users."
| biggestlou wrote:
| Out of curiosity, do you object to the existence of services
| like Cachix, Nixbuild, and Garnix, which also do not have open
| source backends?
| asmor wrote:
| If their founder was the person with principal control over
| what goes into CppNix, yes.
| biggestlou wrote:
| Is there a Nix-based company co-founded by Eelco Dolstra to
| which you wouldn't object?
| asmor wrote:
| Sure, but you seem to sidestep the primary accusation
| that Nix itself got neglected in favor of DetSys. Which
| is most certainly true for at least the installer.
| biggestlou wrote:
| The expectation seems to be that Eelco work on Nix full
| time for free lest he be accused of neglect. Is there a
| way for Eelco to make money that you would approve of?
| asmor wrote:
| I would've replied to you but HN's intransparent timeout
| mechanism hit me.
|
| I'd invite everyone to wonder why Lix exists (and is
| better enough to be used in some parts of Nix's CI now),
| why CppNix stable was stuck 6 versions behind on nixpkgs
| until half a year ago, and what kept Lix contributors
| from working on CppNix instead. Tip: "I want free labor
| from Eelco" is not the answer.
| justinrubek wrote:
| Nobody said this. I suggest that you not escalate the
| discussion in this direction. You're obviously biased
| given involvement with detsys, nobody expects you to
| think differently. This crass dismissal off the issue
| with a strawman isn't right.
| kokada wrote:
| I don't get this argument. It is clear that if it wasn't
| for DetSys making a better installer, we wouldn't have
| one, period.
|
| And now there is some movement from NixOS to bring the
| new installer as a replacement of the old one:
| https://github.com/NixOS/experimental-nix-installer. It
| doesn't seem that Eelco or anyone in CppNix is actively
| trying to sabotage the improvement in the installer, but
| they simple don't care enough to improve the situation
| (that is fair enough, I assume most in the core team use
| NixOS).
| nothrabannosir wrote:
| Is that a fair comparison?
|
| FTA:
|
| > Even if you can (ideally) pull a path from a cache instead
| of building it, you still have to pay Nix's evaluation tax in
| determining the derivation's store path in the first place.
| [...] Fortunately, FlakeHub now offers an elegant solution to
| this problem: resolved store paths.
|
| If I may paraphrase:
|
| > [fundamental problem with Nix] -> [proprietary solution].
|
| Cachix and nixbuild feel like they're solving something at a
| different layer. They fit into the existing Nix ecosystem
| (binary cache, builders) and just provide a hosted version of
| an existing concept. They don't change anything fundamental
| about Nix.
|
| Maybe a better comparison would be devenv?
| mongol wrote:
| This will make discourse.nixos.org explode again. It happens
| every time Determinate System publishes something.
| _huayra_ wrote:
| The unusually high amount of drama in NixOS is one of the
| reasons I have been hesitant to start using it. Jon Ringer (a
| former contributor) posted a giant video [0] outlining his
| history with the project and how he left / was forced out
| (amongst several other key folks). For the record, I don't know
| how to characterize his exit, only that there was an absolute
| ton of drama around it that wasn't really related to even any
| technical aspect.
|
| [0] https://www.youtube.com/watch?v=gp0FI8Gw1iA
| mongol wrote:
| Listened to this video yesterday and it just makes me sad. In
| my opinion, Jon has been let down by the community to which
| he belonged. It is clear there were differences of opinion
| and so on, but I did not see anything that motivated his
| expulsion permanently. It left an especially bad taste that
| it was on recommendation by the Constitutional Assembly.
| Their mission was to propose a constitution, not to act as
| arbiter in moderation matters.
| asmor wrote:
| I don't agree with Jon on a personal level, but who was
| technically in the right aside, he also made some choices
| that'd accelerate your exit from any organization or
| community during the events.
|
| Let's just say his approach wasn't one of reconciliation and
| de-escalation.
| RGamma wrote:
| What a collective waste of energy. Had no idea any of this
| was/is going on. For the sake of my personal systems I hope
| NixOS makes it; wouldn't be the first community that fails
| because of such kindergarten nonsense...
| Smithalicious wrote:
| So you don't use it because of the drama, you only repost the
| drama?
| DannyBee wrote:
| Nah - they look like they are too busy having the n billionth
| argument over telemetry in packages, and the n billionth
| argument over monorepos.
|
| So this will probably get placed into the drama backlog for a
| month or so
| knoopx wrote:
| what a coincidence, my eval times have worsened over the past
| months too!
| jljljl wrote:
| This is cool! We do something similar in Devbox where we pre-
| evaluate store paths from the Nix repository, and then fetch the
| packages directly from the official Nix cache. This greatly
| reduces the evaluation time.
|
| Here's a blog post we wrote on the topic:
|
| https://www.jetify.com/blog/how-we-sped-up-nix-package-insta...
| jkarni wrote:
| We also do something similar at garnix, but when enabling
| incremental builds. Instead of just skipping the build stage,
| we also "normalize" the eval into just the store path, and
| skipping it the second time around.
|
| Mentioned in passing in https://garnix.io/blog/incremental-
| builds. This is even more significant because in this case you
| might otherwise be eval-ing several layers of flakes.
| pinetroey wrote:
| Is this similar to, using nixos-rebuild and target a second
| machine?
|
| I use my laptop's nixos-rebuild and push a new generation to
| another device. The 2nd device doesn't have the config, so it
| won't be able to build it. For these devices, I don't want the
| config to be on the 2nd device.
___________________________________________________________________
(page generated 2025-02-15 23:01 UTC)