[HN Gopher] Nix solves the package manager ejection problem
___________________________________________________________________
Nix solves the package manager ejection problem
Author : rraval
Score : 36 points
Date : 2021-05-31 14:56 UTC (8 hours ago)
(HTM) web link (zeroindexed.com)
(TXT) w3m dump (zeroindexed.com)
| ris wrote:
| Author doesn't even mention the ability to git bisect your
| _entire_ system setup and the power if gives you to track down
| the exact commit that broke something.
| programmer_dude wrote:
| Read the NPM doc but I still don't understand what "ejection" is.
| What are we ejecting? Can someone explain this please?
| dragonwriter wrote:
| It's not a common package manager or NPM thing, but a create-
| react-app thing that a few other things have copied. A detailed
| description of what it involves in CRA is here: https://create-
| react-app.dev/docs/available-scripts/
| tacon wrote:
| My general understanding is that ejection means you are
| abandoning a state that can safely accept updates from the
| original source.
|
| "You're on your own, buddy. Good luck!"
| geofft wrote:
| I _believe_ the confusing bit is that it 's not an NPM thing,
| it's a create-react-app thing. create-react-app is a helper
| tool for, as the name says, creating React applications and
| doing a bunch of things out of the box. Occasionally you reach
| the point where you need to do something more complex than
| create-react-app can handle for you. In that case, you run the
| "eject" script in the generated app (using "npm run" as the
| runner), which removes create-react-app's automatic build
| dependency from your project and sets it up as if you had put
| together all the pieces by hand. But create-react-app still
| works, for the most part.
|
| (In this context, I think what is being "ejected" is the build
| dependency automatically added by create-react-app.)
|
| I don't think it's a perfect analogy, but I see what the author
| is getting at - you need to break the abstraction of some
| packaging tool, but you want the functionality it provided to
| still work as well as possible.
| rraval wrote:
| > I don't think it's a perfect analogy, but I see what the
| author is getting at - you need to break the abstraction of
| some packaging tool, but you want the functionality it
| provided to still work as well as possible.
|
| You got it. I don't think it's the best analogy either but it
| is pithy and works if you squint.
|
| The essence is:
|
| 1. create-react-app is a monolithic transformation of a bunch
| of disparate tools into a managed workflow.
|
| 2. You can update create-react-app like any other dependency
| and get updates to the workflow.
|
| 3. At any point, you can break the abstraction via `npm run
| eject`, which drops you down a level into the raw
| configuration for the "disparate tools" that create-react-app
| was acting as a veneer over. This ejection is a point in time
| transformation, you no longer get the managed workflow
| updates from (2).
|
| The analogy was:
|
| 1. The Linux kernel package on $DISTRO is a monolithic
| transformation of Linux kernel source code to Linux kernel
| binaries.
|
| 2. You can get updates from the package manager.
|
| 3. At any point, you can break the abstraction by dropping
| down a level and forking the current packaging script to
| adjust the transformation (like applying a patch). This fork
| is a point in time transformation, you no longer get the
| package updates from (2).
| kohlerm wrote:
| I think you're correct that the eject mechanism was
| popularized with react. It very much sucks IMHO. At least
| yarn has a mechanism to override a dependency with your own
| version of a component.Therefore I am not sure the article is
| totally correct on this
| yewenjie wrote:
| So, how does Guix compare to Nix in 2021 for the average user
| (who does not want to spend all their time on package
| management)?
| opan wrote:
| Basic commands in guix are simpler, last I compared them. "guix
| install", "guix upgrade", etc. These are equivalent to longer
| guix package commands like "guix package -i" and "guix package
| -u". I recall nix's package install command being odd,
| something like "nix-env -ia".
|
| Package names in guix are very consistent, lowercase and
| hyphen-separated. I recall some nix packages having capital
| letters in them. It could certainly be confusing.
|
| For declarative user packages, guix has manifests. This is a
| first class feature. You make a list of packages in a .scm file
| and apply it, which adds or removes things as needed. With Nix
| you need to use Home Manager, which isn't part of the main Nix
| project. A bit of a shame, as this declarative package
| management is a big selling point, and separating system and
| user packages lets you have a small system list for fast kernel
| upgrades and such, while you can keep stuff like icecat in user
| profile and know that the user package updates may take longer.
| (especially if there's no substitute and it has to be built)
|
| I think more care is taken to making things simple for the user
| with guix. What you lose out on is number of things packaged.
| However, I find guix packages generally have less problems.
| Nix's mpv package performed worse than guix's with guix being
| on two generations older hardware (ThinkPad X220T w/ i5 being
| better than ThinkPad T440p w/ i7). I also ran into strange bugs
| in some things like pcmanfm. It may be not every package gets
| much attention and use.
|
| I would say I'm an average user as far as just using my system
| but not contributing much aside from bug reports.
|
| Note: In all examples I was using each package manager on its
| home distro, not a foreign distro.
| amelius wrote:
| Unfortunately, Nix breaks my CUDA/NVidia support.
| toomanyducks wrote:
| I think the issues with traditional packages all boil down to the
| fact that _packages are not files_. Packages are some transformed
| instance of some code, operating over a computer 's resources in
| some way. Sure, _eventually_ you get down to the fundamental file
| abstraction, but it can take a bit: all the variations of
| building from a single code base. What I like about Nix is that
| it stops representing packages as files and has its own
| declarative syntax. The OP observed what really is a side effect
| of that change, and it 's great! Sometimes, though, I wish it
| went a bit farther than it does - there's room for a few more
| useful transformations between a package's code base, a package,
| and a file. If the nix program itself took a bit more control
| over the system, I think it would end up in a really cool place.
| The specifics of that, though, I do not know.
| totony wrote:
| To be fair Gentoo has /etc/portage/pqtches so that you can patch
| software from official packages instead of having to create a new
| one
| rraval wrote:
| Thanks, I didn't know about this. It's been about a decade
| since I actually used Gentoo and I was writing from the hip.
|
| I've published a correction: https://zeroindexed.com/nix-
| ejection-problem#on-gentoo
| LanternLight83 wrote:
| Great post! I'm a mildly fanatical Gentoo user myself, but
| NixOS has tempted me into migrating to function package
| manager. I've no hesitation to face fiddily systems and work
| around limitations in the name of freedom and flexibility, so
| I've choosen GNU Guix. It inherits these same ideas,
| ostensibly from NixOS itself, but also from the Lisp-driven
| environment of Emacs with it's advice functions (including
| sophisticated extensions to the advice system itself, such as
| el-patch @ https://github.com/raxod502/el-patch)
| comex wrote:
| That works for the kernel. But if you have to patch, say, glibc,
| under Nix doesn't that mean you can no longer use precompiled
| binaries, and have to recompile every single package that uses
| libc, from source?
|
| Sure, it still works. And in the off-chance that you need to make
| a patch that changes the ABI, recompiling the world is exactly
| what you want. But usually you don't need to change the ABI (at
| least not in a backwards-incompatible way), and recompiling the
| world can take a very long time.
| aszen wrote:
| This may change in the future with Nix moving to a content
| addressable store.
|
| I lack the technical know how behind it, but if the output of a
| package doesn't change on changing one of the inputs then nix
| will not rebuild it's cache.
| geofft wrote:
| If you're patching glibc, most of the time, you're going to
| change its output. Determining whether you need to redo
| downstream builds based on changed outputs is great if the
| thing you're changing is docs or optional libraries (such
| that most packages don't have a dependency on the thing being
| changed), but it doesn't help with patching the core.
|
| The interesting case would be if you build separate build-
| time and runtime interfaces - like a libc.so that just has
| dummy symbols for everything that it defines but no actual
| implementations, and a libc.so.6 with the actual
| implementations that can change.
|
| While most Linux distributions have the split of filenames,
| it's not actually done in this way - libc.so is a symlink to
| libc.so.6 (or in the specific case of glibc, a linker
| script), so it requires the actual libc.so.6 to be around and
| used during the build.
|
| It would also be a bit of a semantic change in how Nix
| operates, as I understand it: currently you can run
| ./configure scripts that detect the behavior of libraries by
| actually running code, and if that behavior changes, Nix
| guarantees a rebuild. If you remove runtime libraries from
| what's visible during the build, you can't run such scripts
| anymore (or you're running them against a version of the
| library that doesn't trigger rebuilds when it changes).
| nfoz wrote:
| Off-topic question about Nix. I understand it works by
| redirecting symlinks from, say, one version of a package's files
| to another.
|
| Isn't there a race-condition here -- like if I invoke a program
| at the wrong time while it's in the process of changing symlinks,
| could it pick up the wrong libraries or something? Does Linux OS
| allow a "changeset" of files to be locked and altered together in
| a batch, or anything like that?
| wizeman wrote:
| Nix doesn't work by redirecting symlinks, much less between
| different versions of the same package. So there is no race
| condition as you describe.
|
| Simply, if package A depends on package B then A's files will
| end up mentioning the absolute path to B directly (say,
| /nix/store/abcdef-b-1.0/bin/some-bin-file, where "abcdef" is a
| hash).
|
| If package C depends on a different version of package B, then
| C's files will contain the path to a version of B with another
| hash and possibly a different version number (say,
| /nix/store/zywxabc-b-1.1/bin/some-bin-file).
| nfoz wrote:
| Thanks, let me clarify my question with a proper example.
| Let's say I install A-1.0 and A-1.1. Both versions will
| depend on say a resource file they expect is at
| /usr/share/A.res
|
| I was thinking that nix would symlink /bin/A ->
| /nix/store/abcdef-A-1.0/bin/A, and also /usr/share/A.res ->
| /nix/store/abcdef-A-1.0/usr/share/A.res
|
| So when I'm upgrading to A-1.1, maybe the /bin/A symlink
| would update a moment before the /usr/share/A.res symlink,
| which means invoking A at around the same time as the upgrade
| could pick up the wrong resource.
|
| Do we just try to make sure that binaries know to look for
| resources relative to their binary-path? Or do we use
| chroot/containers? Sorry if this is a dumb question :)
| geofft wrote:
| I don't think "redirecting symlinks" is a totally accurate way
| of describing how Nix works.
|
| Each Nix package, as packaged, has a hard-coded dependency on
| the specific hashed versions of all its dependencies. For
| instance, if you're building NPM and its build scripts hash to
| abcd1234, and it depends on Node.js whose build scripts hash to
| dcba4321, then /nix/store/abcd1234-npm-1.0/bin/npm has a hard-
| coded reference to /nix/store/dbcd4321-nodejs-1.0/bin/node.
|
| Symlinks come into play with user profiles - you don't want to
| type in that full path to npm every time, so you put ~/.nix-
| profile/bin on PATH and you tell Nix to make ~/.nix-
| profile/bin/npm a symlink to
| /nix/store/abcd1234-npm-1.0/bin/npm.
|
| But there isn't a race condition in the underlying packages,
| which are all immutable and more importantly co-installable. If
| you want to upgrade to Node.js 1.1 and its hash is fedc9876,
| then it gets installed to /nix/store/fedc9876-nodejs-1.1. And
| if you rebuild npm against that (even without changing its
| version), the hash of npm's build scripts changes as a result,
| and it ends up in, say, /nix/store/aaaa1111-npm-1.0.
|
| If you upgrade your personal profile to the latest version of
| everything, then Nix will install those two new directories
| into /nix/store, repoint the symlink in ~/.nix-profile/bin/npm,
| and then (eventually) garbage-collect the two old directories.
|
| But at no point does the execution of code _within_ Nix rely on
| mutable symlinks. (As far as I know.)
| taktoa wrote:
| The reason that there's no race condition is that you're not
| "repointing ~/.nix-profile/bin/npm", instead, .nix-profile
| itself is a symlink, so the entire PATH is changed
| atomically.
| kosinus wrote:
| Nix uses symlinks to create user environments, mostly. The rest
| all points to unique paths in `/nix/store`. Sometimes,
| environments are also used for applications, mostly modular
| stuff.
|
| Take a Python service started by systemd. The systemd ExecStart
| points directly to the immutable Nix path of the script, and
| the script also has a shebang that points directly to the
| immutable Nix path of the Python interpreter. (The Python
| interp in turn also links to libraries via direct Nix paths,
| etc.)
| neolog wrote:
| After the profile is built, there is only one link that needs
| to be changed to activate it, so it's atomic.
| n42 wrote:
| not sure if I completely understand the race you describe, but
| I don't think there's any race. the paths and links are created
| at build/install time. nothing is switching around after the
| expression has been applied.
|
| edit: maybe you're describing what happens if a program is run
| while a Nix expression is being applied; I believe what happens
| in that case is that the program that was being run will work
| fine since its environment is pointing to the previous
| generation, and applying the new expression creates a new
| generation
| ubercow13 wrote:
| Isn't that a possibility with any package manager?
| ridiculous_fish wrote:
| How do NixOS users typically manage software that is not a Nix
| package, like a source code tarball where you would traditionally
| run configure && make && make install?
| rraval wrote:
| > How do NixOS users typically manage software that is not a
| Nix package
|
| By writing a Nix package for it (I don't mean for this to sound
| flippant, tone is a bit hard to convey over text).
|
| For example I have this alpha quality rust binary that I'm
| developing but I also want a stable version installed at the OS
| level. I write a Nix package and simply compose it into my
| overall NixOS configuration alongside the more official
| Nixpkgs: https://github.com/rraval/nix/blob/master/git-
| nomad.nix
|
| > like a source code tarball where you would traditionally run
| configure && make && make install?
|
| Nix has a bunch of defaults that make a conventional package
| like this straightforward.
|
| Here's a package for a vanilla C binary + library that does the
| `autoreconf && ./configure && make && make install` dance:
| https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/secu...
|
| It's almost a little misleading because the actual steps are
| largely inherited from the defaults, you can read more about
| `stdenv.mkDerivation` here: https://nixos.org/guides/nix-
| pills/fundamentals-of-stdenv.ht...
| ticketsplease wrote:
| As a NixOS user of 3 years, this rarely happens.
|
| When it does, I either:
|
| * build it via Nix, 10-15 lines, equivalent to packaging it.
|
| * build it in a nix-shell that contains its dependencies
___________________________________________________________________
(page generated 2021-05-31 23:01 UTC)