[HN Gopher] Nix: Taming Unix with Functional Programming
___________________________________________________________________
Nix: Taming Unix with Functional Programming
Author : ingve
Score : 216 points
Date : 2022-08-05 12:49 UTC (10 hours ago)
(HTM) web link (www.tweag.io)
(TXT) w3m dump (www.tweag.io)
| phaer wrote:
| Great introduction and overview on the theoretic foundations of
| the nix ecosystem!
|
| For people new to it, I am trying to provide a quick glossary of
| terms here, as I understand them after about 2 years of using
| nix.
|
| * nix: a _language_ to create derivations _and_ the interpreter
| /package-manager which provides the implementation of said
| language. It currently offers two command-line interfaces, the
| stable on with hyphenated commands like "nix-build", "nix-shell",
| etc. And the newer, "experimental" one which includes support for
| nix flakes and so on, without hyphens: nix build, nix shell, nix
| run, etc. repo: https://github.com/nixos/nix
| docs: https://nixos.org/manual/nix/stable/
|
| * nixpkgs & nixos is a _huge_ mono-repo containing instructions
| how to fetch the source of tenthousands of software packages and
| how to build them on supported platforms. It also contains the
| whole nixos operating system and tooling to support all of that.
|
| repo: https://github.com/NixOS/nixpkgs docs:
| https://nixos.org/manual/nixpkgs/stable/ docs nixos:
| https://nixos.org/manual/nixos/stable/
|
| This tooling includes higher-level helpers for
| language-/environment-specific packaging, like "buildGoModule",
| "buildRustPackage" and so on, as well as e.g. tooling to run
| integration tests in a whole cluster of inter-connected linux
| VMs!
|
| Packages which are submitted to nixpkgs must fulfill certain
| criteria, such as not using "IFD" (input-from-derivation, to
| simplify: "letting nix evaluate nix-code which was generated by
| another deriviation/"nix package".
|
| nixpkgs is alive and well with _lots_ of daily contribution and
| an everlasting effort to keep Hydra, the nix-specific CI /CD
| system and public binary caches up to date and responsive. Thanks
| to all maintainers & contributors!
|
| * flakes are an approach to standardize a way to package nix code
| outside of nixpkgs but to still keep it re-usable. They are still
| "experimental" as the details are figured out, but nevertheless
| used in production. There are some frame-works to keep
| boilerplate low, like "flake-utils", "flake-parts" and others, as
| well as e.g. deployment tools like "colmena" and "deploy-rs" and
| re-usable helpers for system-configuration like e.g.
| https://github.com/nix-community/impermanence
|
| There's lots of other stuff in the community, things like home-
| manager, direnv + flakes and devshells changed my workflow
| fundamentally to the better since I've switched. If you got the
| time and are still interested, join us on matrix or elsewhere :)
| https://github.com/nix-community/awesome-nix
| eulenteufel wrote:
| I'll consider flakes usable for packaging software when they
| support passing options. The respective issue [0] has been
| closed unfortunately. Perhaps I am misunderstanding what flakes
| are meant to be (a more formalized standard way to define nix
| packages and apps), but a lot packages in nixpkgs have a
| plethora of parameters that as of now can not really be mapped
| to a functionality in nix flakes.
|
| [0] https://github.com/NixOS/nix/issues/2861
| pxc wrote:
| Depending on your specific needs, the channelsConfig (modify
| arguments passed to Nixpkgs) and overlay tooling in flake-
| utils-plus may be adequate: https://github.com/gytis-
| ivaskevicius/flake-utils-plus/tree/...
|
| You can take a package from a flake input and call it with
| other args in your overlays, for example
| eulenteufel wrote:
| It's nice that there is a workaround, but passing build
| options is not something that should require a library.
| There should be a well documented standard way to do it.
| pxc wrote:
| Agreed. For _most_ early adopters of flakes, flake-utils-
| plus is the de facto standard solution.
|
| It would be good to see something like it built into Nix
| or Nixpkgs and blessed in the official docs, as the
| flakes feature approaches completion.
| agentultra wrote:
| Nix seems great for build servers. This is a great introduction
| to the motivations behind it.
|
| I'm not sold on using it for managing developer environments
| (another use case it is often used for). It "solves" the problem
| that developers might be using different versions of libraries or
| compilers on their machines... but it comes at the cost of having
| to learn a whole new programming language, a configuration
| language, a whole new jargon, and workflow. It's a bit like using
| Docker as a development environment. It introduces a non-trivial
| amount of friction.
|
| Some folks get excited about package management and
| configuration. Personally I don't care for it enough to over-come
| such a high learning curve. And I don't particularly like the
| workflow it enforces.
|
| However it is pretty great for reproducible CI/CD systems like
| Hydra: https://github.com/NixOS/hydra
| KingMachiavelli wrote:
| In my experience Nix works pretty great for developer
| environments when you don't previously have anything more than
| manually configured dev servers. (In general Nix is much easier
| to buy into if you don't already extensively use Docker, k8s,
| etc.)
|
| For smaller teams with less experienced developers any tool is
| going to have a fairly high learning curve. Instead of having
| everyone learn Nix or Docker, it's very easier to have a few,
| more Linux experienced devs configure servers and devshells
| using Nix while providing simple TUI for developers to access
| various tools.
|
| There are some headaches, if you want to use a full NixOS
| environment then there are some complications with tools like
| VSCode and NodeJS that download dynamically linked binaries but
| its terrible difficult to workaround.
| xyzzy_plugh wrote:
| Hydra is arguably one of the scariest parts of Nix: A giant,
| bespoke CI system written in C++.
|
| Arguably it shouldn't exist. Nix can and should easily slot
| into any CI tool.
| pxc wrote:
| There are couple of CI/CD systems being produced by startups
| that aim to fill that role, based on BuildKit/Docker. For
| now, Nix is still way better than they are in many ways, with
| the notable exception of Windows support.
|
| Maybe visible interest in them can push forward Nix community
| developer interest in polishing Nix for the same use cases.
| octoberfranklin wrote:
| And, unlike nixpkgs, Hydra is pretty unfriendly to
| contributions.
|
| There is _one_ master trusted public key for nix:
| 6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
|
| It is hardwired into the nix source code and every
| (unpatched) build of nix from the last decade or so. There is
| no revocation system. There is no public key infrastructure.
| If that key gets compromised, there is no backup plan. I love
| Nix, but this is batshit crazy.
|
| The Hydra instance has access to the corresponding private
| key. So the people who merge changes to Hydra are
| understandably paranoid. Unfortunately this has turned the
| codebase into a mess.
| infogulch wrote:
| > There is one master trusted public key .. It is hardwired
| into the nix source code .. There is no revocation system
| .. The Hydra instance has access to the private key
|
| > it is built in C++
| jnxx wrote:
| The article talks about the file system abstraction and that
| NixOS is parting ways with it:
|
| > It is not even a new idea for Nix to propose parting ways with
| one of the most pervasive skeuomorphisms in computing, the file
| system, which naturally followed from an era where everything was
| a piece of paper.
|
| What I am wondering is if this is not extremely similar to the
| way that plan9 handles files? As far as I understand, in plan9,
| there is still a file system - but there is no common root, every
| process can have an own view what is, for example, in /bin.
| r3trohack3r wrote:
| If you haven't read the original LISA paper on nix, it's a great
| read. So far it's the best resource I've found that documents
| what nix is, why it exists, and what it's solving:
| https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf
| fesc wrote:
| Man, there's a lot of negative comments here. Just to add a
| different experience: my company loves Nix, it makes it really
| easy to integrate new tools into the dev/build environment
| without needing to document which packages, configuration, ... a
| developer needs to apply to their machine manually.
|
| There's nothing like it, really.
| hamandcheese wrote:
| Are you hiring?
| alch- wrote:
| I'm not GP, but my company's experience[0] is the same, and
| we're definitely hiring. Seeing Nix in the tech stack was one
| of my reasons for applying :)
|
| [0] https://www.channable.com/tech/nix-is-the-ultimate-
| devops-to...
| bogota wrote:
| I dont think anyone is being negative at least not the top
| level comments i read. I like nix but i gave it a test run at
| my company and no one could figure out how to make any change
| that wasn't a copy and paste of what i had done.
|
| The docs don't help much unless you really go diving into them
| and most people who just want the software to run don't want to
| spend the time learning it. I don't blame them.
|
| This is a very valid criticism of any software. Its why things
| like docker (containerization) win out even when it was
| technically around for years before. Someone made it easy to
| use so people used it.
| seqizz wrote:
| Same here. Using Nix and Nixos for years, I have prepared
| nix-shell and packaging files for our internal tools and
| committed to internal gitlab. No one ever touched it even
| after multiple presentations from me. No way to find your way
| out, especially if you don't have functional programming
| background.
| octoberfranklin wrote:
| > most people who just want the software to run don't want to
| spend the time learning it
|
| these people are not nix's target audience
| bogota wrote:
| I personally think that is a bit of a cop out. Most
| developers should be able to do basic tasks in a system
| that has been setup by an expert user.
|
| I think bazel is an ok example of this. Its a pretty
| complex build system but expert users can build macros and
| rules that the average developer can consume without having
| to know a ton about everything that is happening.
|
| IMO the ability to do the above at some level is the sign
| of well crafted software.
| qudat wrote:
| Relegated to a niche subgroup, I see. There's a significant
| difference between spending time learning a tool and
| sinking dozens and dozens of hours into a tool in order to
| perform tasks that are so basic for other package managers.
| whateveracct wrote:
| Nix attracts whiners - 90% people happy with it are happy using
| it and not bothering to waste energy arguing with bad attitudes
| online.
| oneplane wrote:
| There are many things like it. For example, managed operating
| environments where the user doesn't need to do anything (and
| actually can't do anything). Or disposable environments like
| VMs and containers.
|
| Sure, it's not the same as massaging a special pet operating
| system over and over, but most people that need to produce
| software hopped off of that bandwagon years ago.
|
| I get that companies that do functional programming and linux
| and linux on the desktop exist, but I have yet to find any
| company that does that at scale, at a good profit, versus
| competition. That's not to say that "therefore, Nix is bad",
| it's just that the problem isn't a technical one that nix
| suddenly fixes. It seems to be only a problem if you're stuck
| in yum/apt all day and need to get a fix to get out of that.
| otabdeveloper4 wrote:
| Nix is not an operating system. It can be used to build
| operating systems easily, though.
|
| Also, there isn't anything "functional" about Nix. It's a
| nice sales pitch, but underneath it's just a thin layer over
| bash scripts and environment variables.
| octoberfranklin wrote:
| Nix is a metaprogramming language for bash.
|
| The metalanguage is indeed purely functional. The object
| language (bash) isn't.
| otabdeveloper4 wrote:
| Right. You still end up mostly writing bash scripts when
| wrangling Nix. It's not some sort of ivory tower
| Haskelish hermetic ecosystem, it's just a very nice way
| to make bash scripting sane.
| idealmedtech wrote:
| Back in college, the engineering school had a farm of Linux
| servers from which you could access all your documents, as well
| as log in to your account from any number of computer labs around
| campus. Back when I was first learning Linux (not that long ago,
| 2008), I fell in love with how many things you could customize,
| and really create your own experience. When I got to college, I
| wanted to replicate this ability (install ANY program! use ANY
| desktop environment!), but predictably, their machines were
| pretty locked down.
|
| I went snooping around the internet, and found that there was
| this magical software called Nix, which would let you have a
| package manager without root! The Linux computers in the lab
| didn't have root, but they did have GCC. I started learning all
| about build systems (mainly that you could specify a custom
| --prefix and essentially create your own filesystem within your
| filesystem), and got to work compiling Nix (or, i think, GUIX)
| from source. It's actually a fantastic amount of work to go from
| GCC all the way to a functional GUIX, even with access to every
| source tarball on the Internet
|
| Eventually some admin emailed me about this project and I stopped
| working on making it happen shortly after, but it was such a
| formative experience in my tech life that I always think back
| fondly on it when Nix pops up.
| pxc wrote:
| This is also how I got into Nix! I just used a 'normal' Nix
| install with a fake chroot (pivot_root/proot) environment,
| though, so I didn't have to build from source and I got to
| leverage the binary caching. There were some quirks, but
| overall it worked well.
| atemerev wrote:
| Nix is React for system configuration.
| revscat wrote:
| So... over-engineered and expensive to maintain?
| atemerev wrote:
| I don't know, 32kB minified is quite good. And everything is
| React now anyway.
| pxc wrote:
| Overengineered? Maybe, depending on your use case. NixOS is
| pretty popular as a desktop OS within the community, for
| example, and I could see a case that its guarantees and
| strictures are overkill there.
|
| But maintenance is really easy. You're basically never forced
| to rewrite or throw away tons of config. Doing literally
| years worth of updates at once is typically pretty painless.
| (Adding new packages to Nixpkgs or new features to NixOS can
| range from trivial to very hard, just depends on the
| details.)
| atemerev wrote:
| I tried using NixOS a few years ago. Everything went fine,
| except Python packages. There has been quite a lot of
| incompatibilities, sadly.
| pxc wrote:
| There are some different/new tools for creating your own
| Python packages these days. It's still not truly solved
| in the sense of having a single clear winner , but one of
| these new package generation tools might serve you
| better:
|
| poetry2nix: https://github.com/nix-community/poetry2nix
|
| mach-nix: https://github.com/DavHau/mach-nix
|
| dream2nix: https://nix-
| community.github.io/dream2nix/guides/getting-sta...
|
| pynixify: https://github.com/cript0nauta/pynixify
|
| pip2nix: https://github.com/nix-community/pip2nix
|
| The tools available to you at the time (pypi2nix and
| maybe python2nix, if it was a long time ago) have been
| abandoned in favor of the newer tools, I think chiefly
| poetry2nix but I'm not sure.
|
| There's still the Nixpkgs buildPythonPackage stuff, I
| think, if your goal is to upstream a lib into Nixpkgs.
| But if you just want to build your own Python
| applications and vendorize the deps (e.g., for work), you
| might try one of the tools above, which weren't available
| 3+ years ago.
|
| dream2nix is by the author of mach-nix IIRC and has the
| goal of establishing a unified standard and codebase for
| ${proglang}2nix type package generators. But mach-nix is
| still maintained and might be the more feature-complete
| choice between them.
|
| Maybe Nix-y Python users and developers can reply with
| some of their experiences using those tools for real
| projects :)
| mistahchris wrote:
| I have used plain nixpkgs, poetry2nix and mach-nix for
| packaging "real" projects. My biggest take away is that
| python packaging is a hot mess. This isn't exactly news
| that no one has heard before though.
|
| The initial work for packaging a complicated python app
| is dominated by sorting through a lot of confusing
| errors, no matter what tool you use. poetry2nix and plain
| nix has been my best experience so far in python
| packaging though.
|
| For my simple python packages, I'm using plain nix and
| flit, which has been the simplest. It's not feasible for
| python applications that need complicated dependency
| version resolving due to python dependency pinning
| though.
|
| Here's the plain nix + flit example that I really like: h
| ttps://git.sr.ht/~averagechris/deduper/tree/main/item/fla
| ke...
|
| The poetry2nix projects I have worked with are closed
| source sadly so I can't link them.
| nixpulvis wrote:
| Nix needs a new porcelain interface for it's CLIs.
|
| Moreover, this post sums up one of my biggest issues stopping me
| from trying Nix for real again:
| https://ianthehenry.com/posts/how-to-learn-nix/ambiguous-pac...
|
| Finally, needing to rewrite everything in Nix is nice for poorly
| written configurations or undocumented packages in general, but
| seems redundant for well maintained software. Has anyone else
| come up with a sane Nix strategy to avoid the overhead?
| pca006132 wrote:
| I think nix probably needs a more beginner friendly
| documentation as well, by beginner friendly I mean Linux
| beginners. Beginners will not read through nix pills (assuming
| they can understand it) before trying nix, they will just give
| up. And there are a lot of Linux users who don't have much
| technical knowledge about how binaries are being linked, etc.
|
| I think the ecosystem is now mature enough for beginner users
| to ignore the detailed packaging issues, just rely on home
| manager and nixos options for most of the setup. And I think it
| is should be possible to create something like a GUI for home
| manager, to lower the entry barrier. If people are looking for
| a distribution rather than a build system, we shouldn't be
| teaching them how to use nix as a build system, we should show
| them some config that just works.
| SuperSandro2000 wrote:
| Nix is a programming language and there is no visual editor
| for it yet.
| alberth wrote:
| > Nix needs a new porcelain interface for it's CLIs.
|
| The analogy I'd use is that Nix needs what "Github did for
| Git". Meaning, git is actually unnecessarily complex. But
| Github made git easy and accessible.
| nixpulvis wrote:
| No doubt that is needed too! The "human" interface, if you
| will (that which sits on top of the porcelain).
|
| Still, I think Nix could make a better interface for us power
| users.
| SuperSandro2000 wrote:
| That blog post is using nix-env which you shouldn't do if you
| are trying to achieve more than the bare minimum.
| rowanG077 wrote:
| For well maintained software it's generally a few lines of a
| nix definition. It precisely gets hard when it's not properly
| maintained. Patches, insane build configuration (I'm looking at
| you Intel...), binary dependencies etc.
| kaba0 wrote:
| > Has anyone else come up with a sane Nix strategy to avoid the
| overhead?
|
| Didn't need to use it for a long time now, because nowadays
| almost everything is packaged, but there is steam-run CLI app
| that will run the specified executable in a "standard-like"
| environment. It still doesn't support everything, but could
| come in handy.
|
| In the very early days I also just had a debian file system
| laying around and I machinectl-d into it (very lightweight
| virtual machines)
| jnxx wrote:
| I have tried Guix (as a package manager), and it seems much
| better documented.
|
| I also really like the fact that Guix uses a well-established,
| minimalistic, well-implemented, functional-preferred
| configuration language, which is Guile, the GNU implementation
| of Scheme, which is very much tailored to be extended with and
| embedded in other software, for example written in C. In part,
| my love comes from having had to use the alternatives: Huge
| configuration files written in YAML, for example, with no real
| documentation what all the keywords really mean, or things such
| as Conan, which appear declarative and are.... whatever.
| ryukafalz wrote:
| I much prefer Guix UI-wise but it has some downsides:
|
| -- I've had more jank on Guix System as a desktop OS than on
| NixOS. Specifically some dbus-related stuff like
| notifications and appindicators (when running sway + waybar)
| has been very unreliable for me under Guix in ways that it
| hasn't been on any other distro I've tried including NixOS.
| Still haven't figured out why.
|
| - Guix is _slow_ compared to Nix. This is especially
| noticeable on older /weaker hardware.
|
| - Nix home-manager has a lot more options than Guix's
| equivalent - it's really nice being able to rely on it for
| things like sway configuration.
|
| That said, between the two I do lean towards Guix because I
| do gravitate towards Scheme more than the Nix DSL. I just
| wish it were a bit more polished.
| SuperSandro2000 wrote:
| Guix is also very verbose compared to nix.
| rahen wrote:
| GNU Guix, much like Emacs, is a set of LISP (guile) scripts
| running on top of a small C runtime. Nix is mostly C++ in
| comparison.
|
| https://git.savannah.gnu.org/cgit/guix.git/tree/guix
| jnxx wrote:
| > - Guix is slow compared to Nix. This is especially
| noticeable on older/weaker hardware.
|
| Yeah, I have noted it is not the fastest snail on the lawn.
| On the other hand, I have seen so much time wasted on
| integration and reproducibility issues, that I'd be happy
| to run one day a week nothing else than a Guix install and
| not have any of these issues.
| mattarm wrote:
| Nix also runs on macOS. I think it gets a lot more traction
| among devs than Guix does for this reason alone.
| pxc wrote:
| I'm still running Nix and only ever mess around with Guix,
| but I agree: the Guix CLI is beautiful and the documentation
| for it is quite good.
| grumbel wrote:
| > Nix needs a new porcelain interface for it's CLIs.
|
| It already has one with the 'nix' command, it just needs to be
| manually enabled under 'experimental-features', but once done
| there is basically no reason to ever touch any of the old
| commands.
| mikepurvis wrote:
| Yup. I started with Nix a little over a year ago on Nix 2.3,
| and have only ever used flakes and the new CLI. It's complete
| from my point of view-- the main issue is just that the pills
| and all the official documentation still refer to the old
| commands.
|
| I wish they'd flip it over to being the default and update
| the docs. I know that's not trivial, and it's hard when all
| the long-time community members have the legacy commands in
| their muscle memory, but IMO the current state of affairs is
| actively hurting the onboard experience.
| mattarm wrote:
| > it just needs to be manually enabled under 'experimental-
| features'
|
| Same with flakes. My impression is that Nix is either on the
| cusp of a major paradigm and usability change, or the status
| quo will be forever in a state of having "wrong defaults."
| pxc wrote:
| > My impression is that Nix is either on the cusp of a
| major paradigm and usability change, or the status quo will
| be forever in a state of having "wrong defaults."
|
| I think probably both. The Nix community is host to very
| diverse and partially overlapping experimentation, and
| features like rollbacks and version pinning make the
| bleeding edge feel relatively safe, further fostering such
| experimentation.
|
| The new Nix CLI will be huge for new users and for adoption
| once it's finalized. But there will probably always be a
| bunch of Nix users doing weird, cool shit that everyone
| kinda wishes was 'here already' for mainstream use.
| easygenes wrote:
| FWIW, I'd say the new Nix CLI should see broader adoption
| soon. It was only just around the beginning of 2022 that
| they started signaling it was mature and there's been
| good progress to improve things like Home Manager
| compatibility with the new session format brought with
| the new CLI since.
|
| I think mostly now a few people just need to step up to
| overhaul the docs.
| pca006132 wrote:
| I think the really crazy default is that flakes by default
| will not include git submodules, you need to pass a special
| flag to enable it, which is undocumented and have to dig
| through github issues:
| https://github.com/NixOS/nix/issues/4423
| euroclydon wrote:
| I built my router and NAS from NixOS. It was a mostly pleasant
| experience. Being able to sit in an IDE on my laptop, and build
| up a server, incrementally pushing changes to it, with rollback
| if necessary, was great, and I wouldn't want to go back to
| anything else.
|
| I wrote about the router here. It's pretty heavy on router stuff,
| and my own thoughts though...
|
| https://www.jjpdev.com/posts/home-router-nixos/
| mistahchris wrote:
| Wow that is cool. I have been wanting to build both a router
| and a NAS and run nixos on them since I run nixos for
| everything else I do. Thanks for writing and sharing about your
| experience!
| mr337 wrote:
| This is fantastic reading and never really through to use nix
| for a router.
| cerved wrote:
| disappointing that there was no example :/
| hoosieree wrote:
| A few years ago I heard about Nix and NixOS.
|
| I loved the idea, but I did not enjoy the experience. Maybe the
| problem was I was trying to make it work on a less well-supported
| platform (I think it was ARM32). But the packages I wanted to
| install either weren't available, or I kept getting
| incompatibility errors.
|
| I still love the idea, but these days I feel like environment
| managers like Anaconda make (mutable) Python development a little
| more manageable, and things like Docker make (mutable) Linux
| development a little more manageable. Basically these both make
| it less painful to start over with a fresh "thing" when the
| system starts to get crufty.
|
| In my view, there's a spectrum from "immutable and annoyingly
| rigid" to "mutable by default and annoyingly unpredictable". And
| the sweet spot is not at either end, but something like
| "immutable by default, but mutation is possible".
| pca006132 wrote:
| I don't think the problem you encountered is due to
| immutability, the problem is due to the implementation, which
| is now improved due to better documentation and more package
| support.
|
| The possibility of mutation alone will break a lot of
| assumptions and make program analysis a lot harder. I
| personally prefer no mutation at all or only when wrapped
| inside a cell (UnsafeCell), similar to rust. For the latter
| kind, we can treat the states as immutable if we don't have a
| cell, which can help analysis.
| hoosieree wrote:
| That's fair. I blamed immutability when I should have blamed
| the implementation. Thanks for helping me realize my mistake.
| :)
|
| Availability of packages is what makes or breaks a
| distribution, though. If I can't (easily) install the
| software I need to do my job, I choose a distribution that
| can. My home Ubuntu server isn't bringing me joy, so maybe
| now's a good time to give Nix another shot.
|
| Fingers crossed for Nvidia driver support...
| pca006132 wrote:
| I think if you don't use some relatively niche packages,
| nix is fine. However there are some quirks such as setting
| LD_LIBRARY_PATH for CUDA that may need some tinkering.
|
| If you want to give nix another try, I strongly recommend
| you to use nix flakes and home manager. Nix flakes allows
| you to pin dependency versions, and home manager provides a
| lot of configurations for commonly used packages.
| [deleted]
| dpatterbee wrote:
| In my experience the Nvidia driver support has been pretty
| painless (once you figure out the correct settings in
| configuration.nix of course), and the availability of
| packages is by far the best of any linux distro I've used.
| goosedragons wrote:
| I recently tried Guix which is somewhat similar to Nix in
| theory. While I loved the idea I did not love dealing with
| cryptic problems resulting from it. For example I ran into an
| issue where one R package would only work properly if I
| installed other packages in a specific order, doing an update
| though could mess up that order breaking the package. Later
| updates to R broke support for certain things entirely that I
| don't see on non-Guix setups either. Plus getting things that
| required binaries I can't (or easily) compile myself to work is
| a pain.
|
| I just can't justify fiddling with my package manager so much.
| jnxx wrote:
| I am not sure, but it is possible that R want to be its own
| package manager which interferes with Guix? A similar problem
| seems to appear in Python, where there on Distributions like
| Debian there is kind of a struggle which package manager -
| the system-defined packages, or the user-defined ones like
| pip - have the last word.
|
| I have used Guix a bit with Common Lisp libraries, and that
| works like a charm. I also found it useful to be able to use
| new Emacs packages like the newest version of Magit, without
| having to install or supersede the OS installation.
| goosedragons wrote:
| R does have it's own package manager but I was explicitly
| not using it because I read there could be issues, instead
| only installing packages through Guix. It was mainly a
| problem compiling things within R as if the order of how
| gcc-toolchain and the R package were setup it straight up
| wouldn't work. With later versions it just broke entirely
| and I never managed to fix it.
| wdkrnls wrote:
| Despite R being the most commonly packaged software on
| Guix and it being a GNU project and a "common lisp for
| dummies" lispy language which is used by millions of
| students, researchers and data scientists, the developers
| seem to have a noticeable animosity towards it (although
| not as much as the "snake people"). This makes me sad, as
| R users would be the first to sign up for what Guix
| offers if it worked well and the RcppGuile R package
| could be used as a template for bridging the gap between
| the spartan scheme infrastructure and the extremely rich
| and beginner friendly R infrastructure. I feel like this
| could also help motivate the Guix developers to improve
| the documentation and guile porcelain interfaces so they
| could be used directly instead of the copious bash code
| they all seem to relish writing.
| wdkrnls wrote:
| I have not experienced that for what it's worth...
| atleast not after I rebuilt the whole profile.
| rowanG077 wrote:
| I mean if you don't want to fiddle with it I really don't get
| why you would choose Guix to test drive this way of managing
| your packages. It's the way less mature and less supported
| Nix.
| goosedragons wrote:
| I did try Nix briefly. It was even more confusing to
| configure for use with R IMO.
| mattarm wrote:
| This is my experience with both Nix and Guix. They have
| advantages but they are _different_ to the point of being
| unusual.
|
| I very much want the benefits of these kinds of systems, but
| they both produce a run time system (shell environment,
| whatever) that is not typical of how most people use
| software. So, while most other people are helping each other
| out with the "usual" problems, Nix/Guix users have a
| different set of problems. Sure, they are _reproducible_
| problems often shared by all other Nix /Guix users, but those
| communities are niche compared to what is typical.
| smoldesu wrote:
| I use Nix on an array of devices (any combo of x86/aarch64 and
| Linux/Darwin machines you can imagine), and while I really do
| love the experience, you're right that ARM is a sticking point.
| It's been getting better over the past few months, but still
| not close to x86 packaging parity.
|
| On the other side of that coin, I tried switching back to Arch
| a few weeks ago after ~4 months of NixOS. Maybe it's the
| sunken-cost fallacy, but Arch didn't make me feel all starry-
| eyed anymore. Nix feels like a really dependable piece of my
| workflow now, and it's difficult to imagine myself going back
| to Homebrew/pacman.
|
| > the sweet spot is not at either end, but something like
| "immutable by default, but mutation is possible".
|
| Flatpak tried that, you're welcome to draw your own conclusions
| on how that turned out. The problem is that your modifications
| now require build hooks for every update, and you're no longer
| guaranteed a comprehensive runtime. With Nix, these hooks get
| re-written into derivations, which (in my experience) provides
| a more stable, sane alternative to Docker images and Flatpaks.
| It's also not packaged hermetically, which means that not all
| Flatpaks will behave the same on all machines. Something as
| subtle as different environment variables or display server
| implementations can cause your application not to launch.
| mmcdermott wrote:
| Is there a way to get it so that UI applications installed
| with Nix show up in Spotlight Search? I remember that being
| my big annoyance when I tried using Nix instead of Homebrew
| on MacOS.
| smoldesu wrote:
| Oh, I wouldn't know (my Mac is a dumb terminal for testing
| and nothing else). They should have appropriate .desktop
| files for their Linux counterparts, though.
| easygenes wrote:
| For x86, the package availability is almost unparalleled.
|
| NixOS also has tools and options which are not all pure and
| declarative, they're just not considered, "the way."
|
| I find NixOS to have a bit of a high learning curve, but worth
| it for the power and reproducibility.
| rgoulter wrote:
| > For x86, the package availability is almost unparalleled.
|
| https://repology.org/repositories/statistics/total In terms
| of total number of packages, nixpkgs unstable is at 72k,
| while AUR is at 68k.
|
| I'd bet there are many caveats, though.
| soraminazuki wrote:
| A direct comparison with AUR is meaningless though. AUR is
| very unusual because it allows anyone to freely upload
| packages without going through review. So there are many
| duplicates and packaging quality varies wildly.
| hailwren wrote:
| An unfortunate huge one is any way to interact w/ Intel
| Optimus. I can't be the only dev that wants to eat my gpu
| cake and have my battery life optimization cake too.
|
| Bumblebee and optimus-manager both solve this in the aur.
| pxc wrote:
| Bumblebee is well-supported in NixOS. My last mobile
| NVIDIA hardware has died, but I used it Bumblebee on
| NixOS for years.
|
| https://search.nixos.org/options?channel=22.05&from=0&siz
| e=5...
|
| https://search.nixos.org/options?channel=22.05&show=hardw
| are...
| hailwren wrote:
| Oh, this is great info. Thank you. I use optimus-manager
| and it's 100% not, so I thought I was just SoL.
| pxc wrote:
| It'll probably still be a bit of a pain to work out. I
| hate getting NVIDIA Optimus stuff working on a laptop for
| the first time, lol
|
| You might want to consult the unofficial wiki:
| https://nixos.wiki/wiki/Nvidia
|
| (You probably want PRIME in offload mode rather than
| Bumblebee, which wasn't available when I set up my old
| NVIDIA system.)
| whateveracct wrote:
| PRIME offload + the nvidia-offload script works great for
| me.
|
| sync mode actually stopped working for me, but that's not
| a big deal since the offloading works so well. I always
| use my GPU when I need it.
| enoyls wrote:
| I'm not sure what point you're trying to make.
|
| AUR is a supplementary set of packages, and it looks like
| you're comparing it to the total number that Nix supports.
|
| Wouldn't a more fair comparison would be official Arch
| packages + AUR to nixpkgs?
| soraminazuki wrote:
| No, because that would be comparing the number of
| official packages + third party packages with duplicates
| in Arch against the number of official packages in
| Nixpkgs. That's as fair as comparing the number of
| official Nixpkgs packages + every unofficial Nix package
| on GitHub against the number of official Arch packages.
| tjoff wrote:
| > * For x86, the package availability is almost unparalleled
| *
|
| No it is not. Please don't spread that nonsense.
|
| Tons of packages don't work. Many are not maintained.
| Debugging packages are a huge pain.
|
| I've never had so much problems on any distro related to
| packages as on nixos.
|
| Which is expected! Nixos is novel and a niche, few develop
| for it and tons of stuff break because assumptions that work
| on all other distros don't.
|
| But please don't try to give people the idea that nixos
| package situation is great or even good. It is only hurting
| the cause.
| jnxx wrote:
| > Which other results from programming language theory and
| mathematics will we be able to leverage to make software build
| quickly, work reliably, and further tame Unix?
|
| I think Apenwarr's redo (https://redo.readthedocs.io/en/latest/),
| based on an idea from D.J. Bernstein, is a very interesting
| development, because it also has the "purely functional"
| principle at its core - and this allows for much faster parallel
| builds.
| jnxx wrote:
| To add, there is another angle on immutability / purely
| functional definitions, which is compatibility of APIs. Rich
| Hickey (the creator of Clojure) has made a talk titled "Spec-
| ulations", in which he pointed out that there are certain
| operations on an API, like adding functions, adding symbols
| (like enumeration values), loosening preconditions (like, for
| example, adding keyword arguments to a function), tightening
| post-conditions (like, for example, removing possible error
| exit codes), and so on, that keep an API compatible, while
| others, such as removing functions, tightening preconditions,
| widening post-conditions (such as adding error codes or
| exception types), break compatibility.
|
| And then he points out that the API itself can be seen like a
| persistent data structure, like a dictionary where you can add
| new things but not remove old things, because that would break
| client code. And I think this is a very important idea.
| GNOMES wrote:
| Good video about NixOS
|
| https://www.youtube.com/watch?v=ck4J2Faa7Fc
| jrmurray wrote:
| While nix can be very intimidating to get going, I think for just
| getting developer environments spun up it can be somewhat simple.
| I highly reccomend trying to add a `flake.nix` to your projects.
| It makes on boaring new devs a breeze https://medium.com/immuta-
| engineering/nix-and-skaffold-for-p...
| candiddevmike wrote:
| Does Nix still build a lot of things from source? When I tried it
| a while ago everything took forever to install because it was
| compiling locally. Do they have the concept of repos and repo
| mirroring?
| pxc wrote:
| > When I tried it a while ago everything took forever to
| install because it was compiling locally
|
| tl;dr: This is probably due to incompleteness of the binary
| cache. This is pretty rare in general, but it used to be
| relatively easy to hit on macOS on Nixpkgs unstable before the
| community added some channels for use on macOS. Check out the
| darwin stable release channels of Nixpkgs to avoid this issue
| if the current defaults don't show enough improvement for you,
| and see below for a more complete explanation
|
| > Do they have the concept of repos and repo mirroring?
|
| Nix is fundamentally a source-based package manager. This means
| it does not use binary artifacts enriched witb metadata to
| perform dependency calculations at install time. _This_ , in
| turn, means that it doesn't have a use for binary artifact
| repos of the same kind as you see for DEB or RPM.
|
| However, Nix does support caching and distributing binary
| artifacts in a _different_ way. Since all Nix builds are
| deterministic modulo (hopefully inconsequential) indeterminism
| in upstream build processes, once Nix is _right about to_ build
| a source package-- it has figured out all of the build
| parameters and source tarballs to use and so on, for that
| package and recursively for all dependencies-- it can just ask
| a remote server 'Hey, do you have anything for these?'. And
| the remote server can answer without storing or understanding
| any metadata about dependencies, or statefully storing a
| collection of packages at a particular collective repo version,
| or anything like that. If the remote server answers 'no', then
| instead of just choking, like a binary packages manager must
| when a repository is missing a package, Nix just chugs along
| like 'ok, I'll build it myself, then!'.
|
| So with Nix, there _are_ hosted collections of binary
| artifacts, but the metadata associated with them is more
| minimal, and they play a much less crucial role in the install
| process.
|
| The 'repo mirroring' thing likewise has an equivalent: Nixpkgs'
| build artifacts are uploaded to S3 and then distributed via
| CDN. There's no syncing mirrors because there's no state to
| sync (multiple copies of different versions are hosted in the
| same place at once, since they're quasi-content addressed). And
| the CDN hopefully takes care of the local mirror issue for you,
| but you can set up your own Nix build cache as well, or add
| custom binary caches. If the CI/CD system you use to do this
| has 'substituters' (binary caching) enabled, then it will just
| download packages from the main CDN instead of building them,
| just like your local machine would! So aside from _serving_ the
| binary cache publicly, 'building' Nixpkgs is the same as
| mirroring it.
|
| For third-party efforts outside Nixpkgs, it's common to use the
| 'free tier' offered by Cachix, a proprietary, freemium SaaS
| binary cache for Nix builds which is free for open-source
| projects.
|
| Overall, I think this is better than the old-school setup with
| binary package managers and their repos. But one thing that is
| possible here is binary cache misses, where your collection of
| package recipes includes some recipes that have never been
| publically built and cached.
|
| Nix uses the notion of release channels to deal with this: a
| Nixpkgs channel is a snapshot of Nixpkgs which only advances to
| a new version when every recipe in some collection has been
| successfully built (and cached!) by CI/CD. This lets you get
| the best of both worlds: binary caching for everything you
| could want by default, and totally transparent integration when
| you want to install a specific package with your own patches,
| customized build parameters, etc.
|
| Generally speaking, the 'default' channels for Nixpkgs are
| configured based on collections succeeding on Linux/NixOS
| builds, so the recipes on them may not always be 'in sync' with
| the macOS binary caches. If you use one of the channels tested
| against macOS, you avoid this possible mismatch. Nowadays this
| is the default, and there's even a stable release channel for
| macOS. But this was not always so, and consequently you used to
| get kind of a lot of cache misses on macOS.
| breakds wrote:
| Yes it by default build everything from source. However, most
| of the packages will have binary cache on
| https://cache.nixos.org/ so that your installation or update
| will download them instead of building them. Also you can setup
| your own binary cache (https://nixos.wiki/wiki/Binary_Cache)
| and building machine to make your own projects build faster.
| infogulch wrote:
| Nix is running face-first into complexities of build and package
| managers. As an observer it looks like Python's package ecosystem
| in particular is _a giant mess_. This affects Nix
| disproportionately because Nix actually integrates all package
| updates into one channel which nobody else anywhere does (and I
| guess package authors often don 't care to fix), and it's even
| worse because a lot of projects use python as a _build
| dependency_ (?!) which then cascades these issues even farther.
|
| I've been following this thread [1] about the issue which has
| valid interested parties including users, package authors, and
| nix package maintainers, and contains various proposals to solve
| or alleviate the problems.
|
| [1]: https://discourse.nixos.org/t/nixpkgss-current-
| development-w...
| jnxx wrote:
| > As an observer it looks like Python's package ecosystem in
| particular is a giant mess.
|
| Most people would say it is. The better question is how to
| avoid such a mess.
|
| My take is that maintaining backward compatibility is a core
| principle which needs to be strictly observed to solve that
| problem. And yeah, that has also become a cultural issue with
| Python, as the Python2/3 breakage shows.
|
| So, could one sum it up in that Nix magnifies unsolved
| compatibility issues in packaging systems? Because if there
| were a single core Python distribution, like say, Anaconda, but
| nothing else, these issues would not exist. Of course, people
| can avoid the issues if they only use a handful of packages.
| But putting all into a single channel makes the problem much
| more acute.
| infogulch wrote:
| > Nix magnifies unsolved compatibility issues in packaging
| systems ... putting all into a single channel makes the
| problem much more acute
|
| Well said.
| lamontcg wrote:
| > My take is that maintaining backward compatibility is a
| core principle which needs to be strictly observed to solve
| that problem.
|
| Given that software developers never guess the correct design
| up front this means that you always have architecturally
| buggy software, and a bunch of complaining about why buggy-
| looking edge conditions are never fixed. There has to be some
| kind of release valve for software to evolve and break
| backwards compatibility.
| infogulch wrote:
| Yes they've invented that: It's called bumping the major
| version.
| lamontcg wrote:
| The parent comment is complaining about Python 2/3
| breakage which was a major version.
| infogulch wrote:
| Not exactly, they're saying that the Python 2/3 breakage
| debacle became a model for how packages are maintained in
| general.
| lamontcg wrote:
| Well they should clarify because they cite the example of
| the 2/3 breakage which is a major version break, and
| they're not complaining about packages just violating
| SemVer so I don't think your interpretation makes sense.
|
| It also certainly wasn't what I was responding to, and
| responding with "hurr durr major versions" like I've
| never heard of them before is just mildly insulting (and
| kind of insulting to the parent comment by proxy)
| btschaegg wrote:
| I don't disagree, but it doesn't only have to go in this
| one direction. One of the most interesting things about
| Rust for example is how it tackles experimental
| implementations and has concepts in the compiler etc. that
| make unstable language features "first class". I'd say this
| will definitely yield better results than "well a couple of
| guys hacked around on some prototype forks of a compiler,
| and now we're stuck with the result".
|
| Of course, they also make very impressive backwards
| compatibility guarantees for stable stuff (cf. Rust's
| "editions").
| pxc wrote:
| Yeah. This is covering stuff you know if you've read that
| thread, but:
|
| Putting all Python libs into a single channel is something
| Nixpkgs does because Python can't handle different versions of
| the same library in a single process. The Python libs that are
| used in actual applications in NixOS, then, need to be
| compatible or they _might_ cause weird issues when a new or
| existing package tries to leverage them at the same time. Other
| distros do run into this but it might be worse in Nix. (Some
| (all?) C libs have this same issue, but they don 't have
| rampant integration problems from version to version like the
| Python ecosystem does, so having just one copy of them is
| fine.)
|
| And yeah, Python developers seem wary of the kind of
| vendorization that would fix this, and some Python package
| authors are very hostile about integration issues that Linux
| distros are more likely to hit than developers of individual
| downstream applications are. (I guess they don't feel like
| distro integrators are truly 'using their code' in the way that
| application or library developers would be, and those
| integration issues can be a lot of work to figure out for what
| feels like some 'fake user's' configuration problem.)
|
| Python packaging has been an incredible mess for many, many
| years. I don't think the community's processes and institutions
| have the means to meaningfully fix it so that downstream
| consumers of Python packaging infrastructure don't have to use
| a big pile of hacks to successfully package Python
| applications.
|
| Distros will probably see another language replace Python for
| internal tooling and sysadmin applications before they see
| Python packaging unified around something that behaves
| deterministically, works offline, lets programs reason about
| the dependencies of packages that are not
| present/installed/built, pins versions with cryptographic
| hashes by default, disallows arbitrary scripts at install time,
| sanely describes dependencies on native libraries, etc.
|
| PS: This is not a knock on Python. The Python community faces
| some really tough institutional/governance/adoption problems
| because: - the language is mature and the
| ecosystem has a lot of valuable code in it, already packaged in
| various ways - the language was born without a modern
| package management story, because it's very old! - big
| community-wide changes are democratically governed and there
| are a lot of stakeholders who are bound to have opinions
|
| Consensus will be really hard to build and legacy packaging
| processes will stick around for a long time. And some changes
| that could really help, like in-process changes to library
| loading behavior, are likely to be seen (perhaps correctly!) as
| too radical/disruptive to be in the best interest of the
| majority of the existing community.
| SuperSandro2000 wrote:
| Applications can overwrite dependency versions they want to
| use. It's just not possible in the python package set because
| if ever two versions of the same package are in one
| environment very strange things happen and everything falls
| apart.
| pxc wrote:
| Yep, Nixpkgs freely relies on that kind of vendorization to
| solve compatibility problems in other programming
| languages' ecosystems when it has to.
|
| And Nix users can totally override specific Python library
| versions as one-offs for their own Python packages or
| environments.
|
| It's just really unfortunate that some of Python's
| internals make that unfeasible for Nixpkgs' collection of
| Python libraries as a whole. :(
| jnxx wrote:
| > The Python community faces some really tough
| institutional/governance/adoption problems
|
| The Linux kernel people could have the same problems... it is
| genius how they have solved them, by developing git but also
| by establishing processes that scale.
| qbasic_forever wrote:
| In the Linux kernel world Linus is the final arbiter and
| decision-maker--he takes input from people he trusts but
| ultimately he and only he has the final commit. Python used
| to have this model too with Guido at the top as the "BDFL"
| (benevolent dictator for life), however when he stepped
| down they replaced it with a consensus of a steering
| committee: https://peps.python.org/pep-0013/ Both projects
| have very different governance as a result.
| JonChesterfield wrote:
| Python is _really_ tempting as a build (or test) dependency.
| It's an easy way to throw together code generators or similar
| glue that is likely to run on whatever platform is building
| your software.
| jnxx wrote:
| > This affects Nix disproportionately because Nix actually
| integrates all package updates into one channel which nobody
| else anywhere does
|
| Does Arch not do that?
| infogulch wrote:
| Maybe it would be more correct to say that "Nix integrates
| all package updates into one channel which is unique to
| system integrators (like Nix and Arch) but an uncommon
| workflow for regular application developers".
| pxc wrote:
| Yep, I think that revision is perfect.
| SuperSandro2000 wrote:
| The python ecosystem is not a giant mess, its just dependency
| hell. You need to work with what you get from upstream.
|
| > Nix actually integrates all package updates into one channel
| which nobody else anywhere does
|
| If you are using the python system packages then Arch, Debian
| and probably more are doing the same.
|
| > it's even worse because a lot of projects use python as a
| build dependency (?!) which then cascades these issues even
| farther.
|
| In practice this is not a problem at all. Build systems that
| use python underneath have usually very little dependencies.
| The bigger problems we are facing are big python (web)
| applications and anything that moves (very) slowly upstream
| like (sadly) many AI/ML projects.
|
| FYI: I am very active NixOS maintainer and one of my foci is
| the python packages in NixOS.
| infogulch wrote:
| Thank you for your input!
|
| What do you think about the linked thread?
|
| > The python ecosystem is not a giant mess, its just
| dependency hell.
|
| There are dozens (?) of actively used package and environment
| management systems, with no consistency and no lockfiles,
| many packages ignore semver, and having multiple versions of
| a package installed causes weird issues. I'm not sure why "a
| giant mess" is an invalid descriptor, I guess it's just
| arguing semantics.
| rgoulter wrote:
| Neat article.
|
| For me, the key takeaways are:
|
| 1. 'Nix is to `tar -xf && make && make install` as C/C++ is to
| assembly'. In many ways, Nix applies the same kinds of
| improvements that other technologies have.
|
| 2. Nix does try and create an elegant programming model of Unix
| systems.. while the Nix programming language is pure, it
| interfaces with the Unix system by reading files and outputting
| files.
|
| I'm mixed on to what extent articles like this get to the goal of
| make Nix more accessible, though. It seems like preaching to the
| choir to me: if you like the idea of making analogies between
| "software is files, is like dealing with raw pointers", you'll
| prob'ly love diving into Nix as is anyway.
| revscat wrote:
| Same. I read almost every nix-related post that I see pop up on
| HN. I want to be convinced. I have not yet been. It seems like
| `brew install` with (many) extra steps, with little meaningful
| gain.
| rgoulter wrote:
| I'd describe many of the benefits for developers as like
| "docker, without containers".
|
| e.g. if you want to try out helix, you could run `nix run
| nixpkgs#helix`, and it would download + run helix without
| installing it. (Or you could run `nix shell nixpkgs#helix` to
| add helix to the PATH in the current shell, without
| installing helix, etc.).
|
| One use case I'm excited about for developers is the ability
| to declare the dependencies needed to build the project. --
| So rather than copy-pasting `apt-get install` commands, you'd
| rely on nix to fetch the installed dependencies. (e.g. I love
| that I don't have to worry about what packages to install to
| work on qmk_firmware, or repos which provide a nix shell).
|
| VSCode's Remote Containers supports a similar workflow to the
| latter.. but, it relies on containers.
| revscat wrote:
| We use Brewfiles to install binary dependencies needed by
| various projects. This is only for developer's machines,
| but it's lightweight and fast: `brew bundle` and you're
| done.
| noahmasur wrote:
| What happens when you have two projects that use two
| different versions of the same dependency?
|
| With Nix, you can "install" many different versions of
| the same program side by side in the store, and then
| "activate" the one you need at runtime (or with direnv).
| revscat wrote:
| Has never happened. I know this is something that is
| given as a benefit of Nix, but I have personally never
| encountered the situation. For every project I have
| worked in professionally their tool chain was
| standardized enough that the situation never arose.
|
| If parallel installations like you describe is a
| requirement -- and I'm sure that it is -- then Nix looks
| like it could help. That's just not something I have ever
| found myself needing.
| Macha wrote:
| With node this happens everywhere all the time hence the
| popularity of tools like nvm or fnm. At my current
| company we have projects that absolutely require java 8,
| or 11, and I'm sure we'll soon have Java 17 only
| projects, sometimes with corresponding needs regarding
| tomcat or maven etc. versions. It's also a common
| complaint with python and the most common solution seems
| to be a bunch of python3.x packages from your package
| repository, though there have been tools like tox or
| pyenv for this and others that try to combine solving
| this problem and virtualenv management.
|
| That said, if you just want language generic toolchain
| management, asdf seems to have a much lower barrier to
| entry.
|
| Personally I have been using nix as a homebrew
| replacement, because it allows me to sync my packages and
| versions between my personal Arch setup and my day job
| Mac OS setup with a single configuration
| yakshaving_jgt wrote:
| Important difference: Docker is repeatable, but not
| reproducible. Nix is reproducible.
| [deleted]
___________________________________________________________________
(page generated 2022-08-05 23:00 UTC)