[HN Gopher] Nix-Powered Development with OCaml
___________________________________________________________________
Nix-Powered Development with OCaml
Author : dimitrijer
Score : 107 points
Date : 2023-03-06 09:56 UTC (13 hours ago)
(HTM) web link (dimitrije.website)
(TXT) w3m dump (dimitrije.website)
| toastal wrote:
| In 2023, you shouldn't be using nix-env or niv or shell.nix but
| rather just use Nix Flakes. It's listed in the notes but this has
| long been where everyone has moved for a couple of years now (why
| it's still "experimental" is a different question). If this is
| "Nix-powered", why is everything done in the mutable shell
| instead of actually setting up the pure derivation so others can
| consume the build? Looking at opam-nix from Tweag might be a
| better route here. Also, why would anyone put a text editor in
| their devShell?
|
| Question to others: as someone with Nix experience and interested
| in learning OCaml but hasn't quite bit the bullet, what does dune
| actually offer me if I'm using Nix to manage dependencies
| already? It feels like adding an unnecessary layer of LISP when
| Nix and opam should already cover everything.
| momentoftop wrote:
| Nix can replace opam, which is the dependency manager
| supporting isolated ocaml environments. I generally try to use
| Ocaml now without opam, and just using Nix.
|
| You'll still want dune, though I could see an argument for
| generating the dune files from nix expressions.
| toastal wrote:
| Isn't opam read upstream for the public package registry? It
| seems like opam is the one you need.
| baby wrote:
| Dune doesn't handle dependencies for you, you just use it to
| build your ocaml projects with a list of dependencies (often
| without specifying dependency versions, which is why dune
| doesn't really work without nix)
| ghostwriter wrote:
| Why would I give up on shell.nix if flakes API isn't stable
| yet? I've got my own patterns around default.nix and shell.nix
| that provide better ergonomics than flakes.
| ParetoOptimal wrote:
| For your personal use, it sounds like it won't make sense.
|
| In a team setting, enforcing that everyone has to follow
| purity best practices because nothing else is allowed is
| great.
| oever wrote:
| Flakes have a lock file (flake.lock) which means you get to
| decide explicitly when to update the software. nixpkgs does
| not have precise rules on what updates are allowed, so nix-
| shell can give surprising results unless you make the effort
| to stick to one version of the whole nixpkgs explicitly. With
| flakes you can combine packages and versions explicitly.
| ghostwriter wrote:
| > you make the effort to stick to one version of the whole
| nixpkgs explicitly.
|
| the "effort" is just pointing to a particular checkout from
| nixpkgs repo
|
| > With flakes you can combine packages and versions
| explicitly.
|
| it's the same thing with default.nix/shell.nix, just having
| a separate lockfile changes nothing in the underlying
| `builtins.fetchGit` / `builtins.fetchTarball`
| chrissoundz wrote:
| Where does it say you shouldn't use nix-env, niv or shell.nix
| in 2023? I would argue that there is extensive documentation
| and functionality for that already and it's not like it's
| getting deprecated. I've used Nix/Nixos for the past 4 years
| and still haven't looked into flakes - all the Nix stuff I use
| works great so I just can't really be bothered to spend time
| learning more, though I don't doubt it has some advantages.
| chriswarbo wrote:
| I've been using Nix for about a decade, and not bothered with
| flakes yet. However, I also avoid nix-env and channels, as
| the parent suggests:
|
| - Instead of channels, I use fetchGit with pinned revisions
|
| - Instead of managing installed software via nix-env, I use
| buildEnv to collect everything I need in a single derivation.
| If I'm on NixOS I'll put that in systemPackages; or otherwise
| (e.g. on macOS) I'll use nix-env to only install that one
| package.
|
| These have the advantage of being declarative files, which
| can be tracked in git; rather than imperative commands.
| toastal wrote:
| Meanwhile, much of the Nix world is using the composable and
| standardized system of Flakes so they can easily share their
| derivations. Flakes are behind a flag, but are a part of
| `nix` and all of these other tools are adding dependencies
| and complexity to your system. The Nix Flake shells also
| start up much quicker. It's really worth a look because it
| can simplify your setup and make it easier for others to read
| your Nix code and not run into something unexpected or
| nonstandard.
| ParetoOptimal wrote:
| > Where does it say you shouldn't use nix-env, niv or
| shell.nix in 2023?
|
| It's not in official documentation but the biggest reason is
| it's not reproducible by default and a new user could easily
| not pin nixpkgs, then think "wait... Nix isn't reproducible".
|
| nix-shell and niv or otherwise pinning nixpkgs is pretty
| good, though there are some reproducibility advantages flakes
| has over nix-shell + niv such as:
|
| - not allowing access to files not tracked in the git repo -
| forcing the user to lock their inputs with flake.lock whereas
| pinning/niv are optional
|
| Also see:
| https://nixos.wiki/wiki/Flakes#Making_your_evaluations_pure
| AJRF wrote:
| Do you know about nix-shell --pure?
| ParetoOptimal wrote:
| It's a good improvement, but still not as pure:
| https://nixos.wiki/wiki/Flakes#Making_your_evaluations_pure
| omniscient_oce wrote:
| I'm not very familiar with Nix but Dune is a build system not a
| dependency manager. You would need it regardless.
| toastal wrote:
| Nix is a build system + dependency manager. So wouldn't that
| cover the use case? I know some packages use a Makefile with
| ocamlbuild or the like. I want less moving parts and less
| things to learn if I can.
| baby wrote:
| I see nix as a partial build system. It's a more evolved
| makefile that will call bash scripts or other build systems
| to really build things in different languages. For OCaml
| you'll need dune to build your projects, and that's what
| nix will end up calling after setting the dependencies for
| you.
| ghostwriter wrote:
| How would you provide incremental per-module compilation
| for your Nix derivation of the app?
| toastal wrote:
| I don't know; that's why I'm asking. Maybe
| buildOpamPackage already does it?
| https://github.com/tweag/opam-nix#buildOpamProject
| There's a buildDunePackage in the same region of the docs
| that runs `dune build` but that to me implies that dune
| is not needed(?). I haven't read too much into the
| project yet. The example (https://github.com/tweag/opam-
| nix/blob/4bb1a4c372f639f44fc67...) would indicate that
| everything is configured from just the ${PROJECT}.opam
| file, but I don't have enough info on the toolchain to
| know if it's incremental per module or what... or if
| ocamlfind more complicated to work with than dune.
| ghostwriter wrote:
| It does run `dune build`, but the unit of persistence
| within Nix is defined as `buildDunePackage`, i.e. the Nix
| store will only know about the package rather than
| individual modules and a change in a single module will
| cause a cascading rebuild of the entire package to update
| the nix store.
| toastal wrote:
| There are tools out there for other languages that _can_
| handle incremental & per-modules builds, such as purs-nix
| for PureScript, so if opam-nix doesn't now, I'm sure it
| will in the future as a lot of projects are asking for
| this feature.
| ghostwriter wrote:
| > what does dune actually offer me if I'm using Nix to
| manage dependencies already?
|
| purs-nix is a good example that even if per-module
| compilation is made into derivations, the rest of the
| remaining heavy-lifting of constraints resolution is done
| by the external build tool. You can think of Nix as a
| dependency manager, but only with a huge caveat that it
| doesn't actually manage constraints that come with those
| dependencies, especially transitive ones.
| toastal wrote:
| That's a good read but I'm still unclear on dune vs.
| ocamlfind/ocamlopt. Is dune not using those under the
| covers like Nix could?
| ReleaseCandidat wrote:
| While you can _build_ OCaml projects without Dune (although
| I wouldn't want to do that), _editing_ OCaml code I
| wouldn't recommend, as the OCaml-LSP depends on dune. If
| you're an Emacs user: Merlin AFAIK doesn't need dune.
| johnvaluk wrote:
| For a feature that isn't enabled by default, "just use Nix
| Flakes" hardly qualifies as best practice or even commonplace.
|
| I've used Nix/NixOS for years and still don't have a use case
| for flakes. I only enabled it this week to restore some
| functionality to the nix command, which is still a work in
| progress.
|
| Getting there was painful and required a deep dive into the
| documentation. In the end, it was as simple as adding this line
| to my configuration.nix:
| nix.settings.experimental-features = [ "nix-command" "flakes"
| ];
|
| The flexibility of Nix is one of its main selling points, so
| it's impossible to say that "everyone" uses it in a similar
| way. Flakes are a welcome, but disruptive, enhancement.
| Hopefully, the experimental phase will end soon.
| ingenieroariel wrote:
| Most hardcore Nix users/developers I have met have been
| suspicious for Flakes for several years, so your point rings
| true.
|
| That said, it feels like they are slowly coming to terms with
| it and just accepting it as default. Here are two examples of
| maintainers eventually accepting flake support on their repos
| after initial hesitation [1][2].
|
| [1] https://github.com/tpwrules/nixos-apple-silicon/pull/47
| [2] https://github.com/NixOS/mobile-nixos/pull/404
| baby wrote:
| It sounds like a python 2 vs python 3 kind of deal, which
| intuitively makes me think that you should just use flakes
| toastal wrote:
| Counter anecdote: all of the hardcore Nix users/developers
| convinced me it was worth it to switch to Flakes now. They
| even helped me move my NixOS config as well.
| mikepurvis wrote:
| I'm a relative newcomer too-- started about two years ago
| with a Flake-only workflow on Nix 2.3. My sense in
| observing the debate over this time is that about 20% of
| the objections are technical in nature, while 80% are
| around governance-- feeling that the feature was smuggled
| into the main repo as a prototype, despite there being
| existing external projects (niv) that were trying to
| accomplish a lot of the same things.
| williamvds wrote:
| I've got a simple use case: reproducibly building a random
| derivation from someone's Git repo.
|
| If they haven't pinned nixpkgs, I have no idea at what point
| it _was_ building successfully. It's just dumb luck whether
| my system's channel is a compatible nixpkgs.
|
| If it's pinned, great, I can use that pin until I'm bothered
| to bring it back to mainline. At that point I'll override the
| nixpkgs input and bear the cost of updating and maintaining
| it.
|
| Flakes make managing pins those easier, through a nice(r)
| CLI, and a standard format for them.
| giraj wrote:
| I was hoping this post would mention using OCaml with flambda[1]
| enabled. At least for my work, flambda seems to yield a ~10%
| speed up when compiling things. Can you get OCaml binaries with
| flambda enabled through Nix? With opam, I currently have to
| compile OCaml myself to enable it, as I am not aware of any
| binaries being distributed.
|
| [1] https://v2.ocaml.org/manual/flambda.html
| baby wrote:
| I actively work on an OCaml project and couldn't understand
| anything about what flambda does from its readme
| giraj wrote:
| I updated the link to point to the docs, which are more
| informative. As far as I understand, flambda is an
| intermediate representation of OCaml which allows for a
| number of optimisations and better inlining. You can check if
| you already have flambda enabled by runnning "ocamlopt
| -config | grep flambda".
|
| If you're using opam then you can test flambda out by
| creating a new switch opam switch create
| ocaml-flambda ocaml-variants.4.14.1+options ocaml-option-
| flambda
|
| and then running "opam switch set ocaml-flambda". (You can
| replace 4.14.1 with your preferred version above.)
| momentoftop wrote:
| It's a bunch of optimisation passes, I believe mostly around
| inlining. Ocaml is historically quite passive on that.
| ParetoOptimal wrote:
| The nix option for it was added in 2017:
|
| https://github.com/NixOS/nixpkgs/pull/32946
|
| If you find it important, I'd encourage opening a PR to ask for
| flambda to be enabled by default at
| https://github.com/NixOS/nixpkgs/issues/new/choose.
|
| Especially if the ocaml community typically enables flambda by
| default these days.
| ingenieroariel wrote:
| Seems you would need to first override ocaml and then override
| the ocaml used in mkOcamlPackages. [1] let o =
| ocaml.override { flambdaSupport = true; }; in let oP =
| ocaml-ng.mkOcamlPackages o (self: super: {}); in oP.apron
|
| [1] https://discourse.nixos.org/t/install-ocaml-flambda-
| compiler...
| giraj wrote:
| Okay, thanks! But doesn't this still mean you have to compile
| ocaml (with flambda) yourself?
| mindslight wrote:
| If the derivation has already been written (as GP implies),
| then compiling something yourself with Nix is essentially
| just a longer installation process. No downloading
| tarballs, no keeping track of things in /usr/local, no stow
| or manually adding things to your path. Nix is essentially
| a source distribution that pulls common binaries from
| caches.
| baby wrote:
| Why not use the new nix develop, etc. commands?
| ingenieroariel wrote:
| If you are mainly using OCaml and do not mind flakes, this was a
| very good starting point for me:
| https://github.com/brendanzab/ocaml-flake-example/blob/main/...
|
| Using Flakes you can have a devShell, a check target and a build
| target neatly organized.
| weitzj wrote:
| I really would like to use it in the enterprise. But secure boot
| was not easy to do the last time I used it. And i still don't
| have a clue how to package things which are delivered as Deb or
| rpm packages (I.e. a vpn client)
|
| And getting management approval to build a packaging team so that
| you are not the only person In the company to go to, is a hard
| sell
| kaba0 wrote:
| Well if there is no other source/binary available then you have
| to uncompress the deb/rpm file (both have some utility for
| that), look up the dependencies from somewhere, copy the
| extracted files over to the nix output dir, and patch-elf all
| the binaries to link to the nix-version of the dynamic lib.
|
| This sounds more difficult than it is, the result will be a
| copied over binary file that has its "libc.so" and other
| dynamic libs replaced in the ELF-header with
| "/nix/store/hdjdewuieu737-libc/libc.so". I recommend looking up
| a package in nixpkgs which has a similar install story, that's
| the easiest way to write a new package.
|
| In case you only want to run it locally
| https://github.com/thiagokokada/nix-alien and similar programs
| work fine with the binary.
| phaer wrote:
| Afaik it's not finished yet, but there recently has been quite
| a lot of activity regarding secureboot for nixos
| https://github.com/nix-community/lanzaboote/
| kirkfletcher wrote:
| Been a while since I used nix packages or nix-shell more
| specifically, my issue is that the packages are "tampered" with
| and not as distributed, making them prone to issues. I
| experienced quite a few problems previously
| 0x69420 wrote:
| out of curiosity, what sort of tampering/issues?
|
| in my experience such changes are largely to accommodate nix's
| admittedly alien expectations (runtime dependencies living in
| the store, etc etc) and beyond that things behave more vanilla
| than almost any linux distro's repos
| pveierland wrote:
| For those new to Nix and flakes, I created a small OCaml
| development shell demo based on the blog post. MyNixOS lets you
| create flakes using a graphical editor and generates a
| reproducible flake with corresponding Nix code and lockfile.
|
| https://mynixos.com/pveierland/demo-ocaml/outputs
|
| Run using: nix develop
| https://api.mynixos.com/pveierland/demo-
| ocaml/archive/latest.tar.gz
| [deleted]
| schemescape wrote:
| Does anyone know why Nix takes up roughly 500 MB with no packages
| installed?
|
| I want to try NixOS, but the "minimal" image is over twice as big
| as my full (terminal only) development environment using Alpine
| Linux. I was kind of expecting NixOS to be closer in size to a
| glibc-based minimal distribution (after initial install)...
| [deleted]
| l_theanine wrote:
| Ocaml seems somewhat hampered by this weird disconnect between
| upstream maintainers and Jane Street and other third party major
| users. Popular media for ocaml learners describes using Base,
| while documentation focuses on the builtins, which clearly aren't
| used that often in practice.
|
| Do I have this wrong? Why doesn't Ocaml just merge Base (with
| permission) and put their best foot forward for the ecosystem?
| baby wrote:
| It's complicated. From my perspective the core team was too
| slow to support a strong stdlib and so entreprises took over by
| replacing it. Now you have this ugly world where you're split
| between these two things. And since everything jane street is
| poorly documented you are sort of forced into a poor ecosystem
| if you want to do serious OCaml projects.
|
| The same thing is true with the build system dune and opam.
| Half-assed build system and package managers that we're kind of
| stuck with.
|
| I dream of a new cargo-like build+package manager system for
| OCaml that would just take over everything we currently have.
| frou_dh wrote:
| You have it wrong. The official standard library is under
| constant development and in practice many people use it and not
| the Jane Street stuff.
| anentropic wrote:
| I don't think they're wrong
|
| the Jane Street side are quite prolific with blog posts etc
|
| as a newcomer to OCaml one of the first, and nicer-looking,
| intro resources you'll likely encounter is the Real World
| OCaml book https://dev.realworldocaml.org/ which
| unfortunately does everything using Base instead of the
| stdlib
|
| Personally that didn't sit right to me and I prefer to use
| the stdlib by default (which seems fine and not in need of a
| wholesale replacement)
| frou_dh wrote:
| There definitely is indeed a problem, namely a false
| perception that the stdlib is a moribund relic with one
| foot in the grave. What I'm saying is that it's fairly
| healthy and gets used plenty by general OCaml users.
| anentropic wrote:
| I totally agree
| l_theanine wrote:
| Are there some large users or projects that are committed
| to the stdlib and not Base that I could look into or read
| about?
|
| Thanks in advance.
| debugnik wrote:
| Aside from the obvious fact that replacing the Stdlib with Base
| would break existing code, Base has an unstable and quite
| opinionated API and isn't tested on all targets OCaml supports
| (Jane Street is looking to drop support for 32-bit targets
| altogether). There would be a lot of tension to adopt it.
|
| Meanwhile, the built-in Stdlib finally started to acquire much
| needed functions and modules during the late 4.x releases. My
| biggest worry is whether they'll keep up the pace, it doesn't
| even have a resizable array or, alternatively, a Clojure-like
| vector (but neither does Base).
| l_theanine wrote:
| So, you'd say there's a healthy contingent of upstream devs
| dedicated to the health and diversity of the built in stdlib?
|
| May I also ask, have you worked with Ocaml for any medium to
| large projects?
|
| I'd be curious to hear from major users that are NOT using
| Base, and hear how they're getting on. The 5.0 release
| especially has catalyzed my interest in Ocaml, and I've a few
| years left in me still before retirement, I'd like to pick up
| one last niche 'fore I quit.
| domenkozar wrote:
| If you decide to try https://devenv.sh, it has OCaml support
| built-in: $ cat devenv.nix { config, ...}:
| let ocamlPackages = config.languages.ocaml.packages;
| in { languages.ocaml.enable = true;
| packages = [ ocamlPackages.janeStreet.async ]; }
| $ devenv shell ...
|
| Recently also shipped 0.6 with container generation support:
| https://devenv.sh/containers/
|
| Let me know if you give it a try :)
| ocimbote wrote:
| After a lot of hesitation and energized by a work incentive, I
| tried Nix last week.
|
| I've followed the official docs, had to resort on Google et al.
| quite quickly to solve mundane problems (installing a channel is
| not on the default path of the docs for the new learner), and
| after much wandering and trial and error, I evenrially went to
| the discourse and asked what I should use between flakes, nix
| shell or develop... Every single answer was 1/extremely
| informative and nice 2/ crystal clear that Nix was in a
| confusing, in-between stage in its life cycle.
|
| I very much appreciate the community and efforts but I work for a
| company that cannot afford the commitment that Nix requires right
| now. I will happily come back in 2 years and reevaluate.
| Buttons840 wrote:
| What exactly are they in-between? What is changing and how will
| things be better in 2 years?
| ocimbote wrote:
| My most personal example of the CLI being in-between state
| and the community agreeing to it is in this thread:
| https://discourse.nixos.org/t/nix-shell-nix-shell-and-nix-
| de...
| rgoulter wrote:
| The Nix Flakes feature.
|
| Previously, Nix installed its packages using 'channels'. You
| would add and update a channel, and then you could install
| packages from a channel. (This is roughly analogous to apt
| repositories). -- The CLI user experience for this channel-
| based approach is pretty terrible.
|
| Nix flakes are a new feature where a project has a
| `flake.nix` file, which follows a standard format for
| declaring what packages a project provides. -- The CLI
| experience is much nicer, and has several other benefits over
| pre-flake Nix.
|
| Flakes are still (for various reasons) labelled as
| "experimental". Much of the documentation has not been
| updated for flakes.
|
| In 2 years, you'd hope that: the documentation improves, and
| there's less friction for adopting the technology.
| colordrops wrote:
| The people you are talking to are either being too modest or
| are overly picky about what "ready" means. Flakes are 100% the
| way to go for new users and are production ready. They work
| well.
| jon-wood wrote:
| As someone who recently picked up Nix and isn't using flakes
| the reason for that is that none of the official
| documentation seemed to be saying it was a good idea, and all
| I could find to the contrary was random blog posts, none of
| which actually explained what on Earth a flake is. I think I
| vaguely understand them now, but I'm hesitant to rip out the
| setup I've eventually got to and replace it with the new
| hotness just for the sake of it.
| colordrops wrote:
| It is pretty old for being the "new hotness". It allows me
| to idiomatically and completely encapsulate my system
| declaratively in config. Nix isn't complete without this
| IMO.
| shepardrtc wrote:
| > Nix was in a confusing, in-between stage in its life cycle
|
| That's what everyone says about software that is just too
| complex.
|
| I tried Nix for the first time yesterday and had a similar
| experience - it was just too buggy, too brittle, and too
| difficult. It's absolutely not worth it. The software was built
| to solve a difficult problem, and maybe it does that, but it
| became a difficult problem in itself.
|
| For example, if you Ctrl-C while its downloading an update, you
| get in a weird state where nothing works. Ok, so just uninstall
| and reinstall right? Well, there's no uninstall. You have to
| manually delete everything and remove the /nix mount. Its 2023,
| why am I having to hunt all this down? Sure, if you reinstall
| it it tries to do that, but it fails every time, or it just
| misses things. I just don't have time for this. Nix is not even
| close to ready for production.
| colordrops wrote:
| > if you Ctrl-C while its downloading an update, you get in a
| weird state where nothing works
|
| Something else is wrong here. I've aborted updates hundreds
| of times without problems. Uodates are atomic and this
| shouldn't happen.
| ocimbote wrote:
| > That's what everyone says about software that is just too
| complex.
|
| I'm not sure you're saying this to point fingers at the
| software or at the people, that is very confusing :D
|
| That being said, Nix being in-between has been discussed in
| the Discourse of the project here about t supposedly simple:
| https://discourse.nixos.org/t/nix-shell-nix-shell-and-nix-
| de...
|
| It was to me very interesting to see how insiders could be
| diverging and potentially not being fully aware of it.
| mikepurvis wrote:
| My problem space (building ROS packages for robots) is
| difficult enough that it was worth it getting over that hump
| for me, but I totally understand this perspective.
|
| I'm hopeful that in two years, the community will be much
| more aligned around flakes, and the docs (at least the
| onboarding parts) will have a much clearer "happy path"
| oriented around accepted workflows like direnv and home-
| manager.
| one-punch wrote:
| > For example, if you Ctrl-C while its downloading an update,
| you get in a weird state where nothing works. ...
|
| There is a new Nix installer from Determinate Systems, a
| company heavily invested in Nix, which should make installing
| Nix more atomic, reversible, and thus easier:
|
| The Determinate Nix Installer
| https://news.ycombinator.com/item?id=34957953
|
| And after getting pass the installation, Nix is great for
| production.
| lostmsu wrote:
| The parent talks about an interrupted update process.
| Filligree wrote:
| Interrupted updates don't do that. In single-user mode
| you could _maybe_ ctrl-c it during the final activation
| phase, but that 's a small fraction of a second -- it's
| just replacing a few symlinks.
|
| In the default multi-user mode, you can't even do that.
| [deleted]
| kaba0 wrote:
| Yes. This is like a major _benefit_ of nix(os), updates
| and everything is basically atomic. If you interrupt an
| update you will just have a few more downloaded folders
| in your nix store folder, which are trivially GC-d on the
| next run. This is unlike pretty much anything else.
| kaba0 wrote:
| > For example, if you Ctrl-C while its downloading an update,
| you get in a weird state where nothing works
|
| Heh? Oh you mean like during installing it for the very first
| time? Like, what do you expect from a bash script? Is bash
| not ready for production? That's quite a strong statement
| when you didn't even manage to install the thing... like, at
| least give it an honest try before you reach such a
| conclusion.
| pdimitar wrote:
| To be fair, I don't think it's such a huge sacrifice from
| the maintainers to put a check for leftover artifacts at
| the start of the installation script and nuke those from
| orbit before proceeding.
|
| It can be put behind a confirmation Y/N terminal line as
| well.
|
| But yeah, usually bash install scripts are almost never
| idempotent. I feel the maintainers of those scripts just
| give up and IMO they really shouldn't.
| thfuran wrote:
| >Is bash not ready for production?
|
| Almost all bash scripts aren't
| kaba0 wrote:
| (Actually I agree with you, but I used it more as a
| rhetoric :D)
| mikepurvis wrote:
| I don't know if this is what the GP experienced, but you
| can definitely get some inconsistency if you ctrl-C out of
| Nix while it's in single-user/daemonless mode. My sense is
| that single-user is really only meant as a toy and for use
| in unprivileged containers-- in basically every other
| situation you want to be talking to the nix-daemon, which
| maintains proper state and won't get interrupted.
| tomberek wrote:
| I am working on the Nix project; resolving this "in-between
| stage" is our highest priority at the moment. I'd like to
| better understand two independent things:
|
| 1. What were the drivers and motivators for you to try Nix?
| What did you hope to achieve?
|
| 2. Was the current experimental status of the new nix CLI and
| flakes the primary cause of the confusion?
| ocimbote wrote:
| Thanks for your interest, it really confirms the interest
| I've received from the community in general.
|
| 1. I'm leading some Platform Engineering projects at my
| company and one is focused on Developer Experience. We have a
| wide variety of configurations on the engineer's laptops and
| the development environments are not normalized. That causes
| a lot of noise and pain, as one knows. Nix is, or seems to
| be, an opportunity to take back control on how engineers run
| the applications locally.
|
| 2. I'd say it has many factors. The official documentation
| and guides are not yet suitable for a zero-knowledge
| newcomer. As an example, since channels are still recommended
| in the docs, the command to add the default channel is not in
| the "Install" page. Things go from `curl <(...) | bash` to
| `nix-shell`, and it fails (on a M2 Mac) unless I navigate
| somewhere else on the docs to find that I should add the
| default channel first.
|
| Second point is about the terminology. Derivation, Flakes...
| are quite arcane terms. At least to the layman (me).
|
| Third, community blogs etc are covering outdated versions,
| some rely on flakes, others not, it's understandle but also
| understandably disorienting, especially when you're after
| some guidance that the official docs could not provide.
|
| Fourth, the API. `nix-shell` is not a shortcut for `nix
| shell`. Wait what?
|
| ---
|
| When I'm saying I'll come back in 2 years, I mean that it is
| clear that the situation is changing rapidly and dust needs
| to settle. Maybe the direction is not clear yet, and that
| would be fine. My one and only advice would be to only ship a
| feature that is properly documented.
|
| Reading something like "We recommend to use flakes, they're
| the future, but the APIs are in flux, but if you're wary and
| want stability, here are our stable APIs" would have saved me
| a ton of time.
| jen20 wrote:
| > take back control on how engineers run the applications
| locally.
|
| How to piss off engineers in one statement. Developer
| machines are not something for you to "have control" over.
| ocimbote wrote:
| Maybe I phrased poorly, but you didn't try either.
|
| Believe it or not, but the project emerges from the devs
| themselves, it was not even an idea before they spoke up
| and we had to make it a project, to support them.
| citruscomputing wrote:
| You're right -- but it's even worse DX to have to check a
| ton of different files, parse the outdated english
| documentation, guess, ask people, and hope that deps were
| frozen. Especially for old, mostly abandoned projects.
|
| I've started using direnv to switch node versions for
| projects, based on already-existing CI configuration
| files, and it's fantastic. Having a full, working, stable
| development environment that can be shared among the team
| (amd future teams doing software archaeology when
| something breaks!) is my holy grail. I'm really excited
| about the future of nix.
|
| But you're right that at the end of the day, devs should
| retain control. I think having clear, machine readable
| instructions for how to set up the environment enables
| that control. Forcing them to use the blessed config is
| not necessary. HAVING the config lets you know what you
| need to set up, your own way.
|
| The only place such strict control needs to happen is in
| CI/CD, which just falls out of having the dev setup done.
| shakow wrote:
| It's not about ``controlling the machines'', it's about
| running `nix-shell -p our-project` and having the same
| setup on my OpenSuse than Jack on his Arch, Herman on his
| mac, Johann on her Debian, and our prod environment on
| RHEL.
|
| That's a force multiplier in dev & prod, not a hindrance.
| OhSoHumble wrote:
| I'm hesitant to write this as I don't know if you'll see it
| but I feel like what I have to say is different to the
| opinions that have been voiced so far. I will only answer for
| point number one.
|
| I personally feel that Nix documentation and community blog
| posts sits on a gradient of theological meanderings about
| functional purity to actual praxis.
|
| For one, I'm a systems engineer. What drew me to Nix was that
| I wanted a next-gen Ansible. A piece of software that can be
| used to reproduce a hosts OS state in an immutable way. For
| now, my homelab and laptop is all NixOS driven. I use deploy-
| rs and flakes to configure hosts. I can code but am not
| really a software engineer by trade. This is where I'm coming
| from.
|
| Within the context of my established skills, Nix's
| documentation is awful. There were/are actions I want to
| accomplish that sit outside of the paradigm of Nix and when I
| try to accomplish those actions I feel lost.
|
| For example, my main homelab machine has a Nomad server
| installed. The default Nomad package from Nix packages just
| installs the binary. So I needed to handle creating its data
| directory, its configuration, and laying down a systemd unit
| file. Where I got stuck was creating the data directory. I
| could accomplish the systemd configuration and the
| configuration file easily enough but I don't know how to
| create a directory outside of a derivation. I also don't know
| if a derivation is even needed.
|
| But I also don't really know how to fit a derivation in my
| current flake setup so that it's processed when I change host
| state. I ended up solving the problem by having systemd
| manage a StateDirectory and creating the unit file/Nomad conf
| in configuration.nix for the machine. I also don't known if a
| derivation would even be what's needed. Do I use an overlay
| to extend the Nomad package? Can I ad-hoc create directories
| and then shove that into a module?
|
| My point being is that system administrators, network
| engineers, release engineers, etc are all different types of
| engineers that would be interested in Nix... but they are NOT
| functional programming enthusiasts. I think Nix gets
| attention from a lot of people who are comfortable with
| Linux, package management, build processes, and configuration
| management systems like Ansible. But there is no clear runway
| for those types of engineers. Hell, NixOps seems to be an
| abandoned repo that has no documentation whatsoever for it's
| 2.0 release and there are dozen different projects for remote
| deployment to hosts.
|
| If I took a sysadmin and told them "please codify OS state
| using Ansible" then they're likely to come back with a
| solution that could satisfy a technical need within an
| organization. If I took that same sysadmin and told them to
| use Nix to solve the same problem then they're likely to get
| lost for weeks.
| pdimitar wrote:
| Disclosure, I tried it in passing about 2 years ago.
|
| > _1. What were the drivers and motivators for you to try
| Nix? What did you hope to achieve?_
|
| (a) Reproducible dev environments. I worked in a company back
| then had a big mix of different projects: Elixir, PHP, Java,
| JS, C#. We wanted everyone to just `cd` into a directory and
| have identical tools, versions of languages / DBs,
| everything.
|
| (b) Ability to do transactional-like upgrades or downgrades
| of various pieces of software.
|
| (c) Isolate software that usually pokes deep in the guts of
| the system or the user's home directory. PHP and JS were the
| main offenders here, especially with PHP on Linux (and
| Apache2 integration) you had to modify a good amount of files
| in various non-home-dir places. And JS is, well, JS, NPM's
| global `node_modules` directory is a horror to behold as we
| all know.
|
| The experience was mixed. While we had one guy who was super
| invested in Nix and made sure to help us all install it and
| set it up, the day-to-day experience was lacking and people
| had to find ways to "repair" their setup after a few small
| upgrades of software. It felt like a chore. Sorry I can't
| give you more details but for us back then the main goal was
| to reduce the chores, not invent a different kind of them.
|
| All in all, I'd say points (a) and (b) worked somewhat okay,
| while (c) we definitely didn't feel it did so we just
| migrated to Docker for some of our workflows.
|
| > _2. Was the current experimental status of the new nix CLI
| and flakes the primary cause of the confusion?_
|
| Again, my take is outdated, but as a burned out senior dev my
| criticism boils down to the following:
|
| I don't want to care about "flakes" at all. I want all tools
| to dispense with the cutesy pun-y terminology already and
| realize that people use them for work.
|
| Give me one `nix` command with subcommands and good help
| outputs (and good centralized docs website). Reference: the
| `borg` backup program, also `git`. Before Nix gets into that
| state I'll not recommend it anyone.
|
| ---
|
| I am in no way attacking you or disparaging yours and others'
| work but to me and my team back then it felt like Nix simply
| replaced one complexity with another. What drew us in was the
| promise of less complexity and that did not materialize.
|
| I feel the general user experience is often lost on
| maintainers because they live and breathe their tool and to
| them the workflow is second nature. That's why I wrote this
| reply. Hopefully it brings some perspective.
|
| Keep up the good work.
| amardeep wrote:
| > tomberek
|
| 1. Two independent use cases.
|
| a) First was a personal use to carry my dotfiles/home config
| everywhere (office mac, home mac, virtual machines). Ended up
| configuring a flakes based home-manager setup.
|
| b) Second is trying to get good devshell experience for the
| whole team. Especially in a monorepo setting with multiple
| languages and tools. Right now there is tons of setup
| required including installing right versions of java, python,
| nodejs, lots of instructions to get right python version,
| poetry install, setup aliases etc.
|
| 2. No, it is not. It didn't take much of research to conclude
| flakes are the way to go. Honestly, I am confused by so many
| responses about confusion between flakes and non-flakes as I
| don't think it take much effort to realize flakes are the way
| to go even if it is experimental.
|
| Though, to be fair, it does hurt that two of the recommended
| resources - nix.dev and nix pills - do not cover flakes.
___________________________________________________________________
(page generated 2023-03-06 23:02 UTC)