[HN Gopher] The FreeBSD/Firecracker Platform
       ___________________________________________________________________
        
       The FreeBSD/Firecracker Platform
        
       Author : cperciva
       Score  : 211 points
       Date   : 2022-10-18 06:06 UTC (16 hours ago)
        
 (HTM) web link (www.daemonology.net)
 (TXT) w3m dump (www.daemonology.net)
        
       | ComputerGuru wrote:
       | @cperciva
       | 
       | > The FreeBSD kernel now handles such duplicate environment
       | variables by appending suffixes, so that we end up with
       | virtio_mmio.device, virtio_mmio.device_1, virtio_mmio.device_2,
       | et cetera,
       | 
       | Was there any discussion of a cleaner alternative to this,
       | perhaps extending the existing APIs with the ability to retrieve
       | multiple values for one key (without breaking backwards
       | compatibility, obviously) or concatenating the values with some
       | sort of delimiter (colon, literal new line, ascii RS, or even
       | null)?
        
         | cperciva wrote:
         | Problem is that in general we want foo=bar to replace any
         | previous value of foo.
        
           | ComputerGuru wrote:
           | I understand, that's why I was asking about an additional api
           | (get_xx_multi, etc) that would return an array or composite
           | string or whatever, while get_xx would continue to return
           | just the single/latest value.
           | 
           | Not being able to tell whether or not foo was specified twice
           | or foo and foo_1 were specified separately just gives me
           | flashbacks to a bygone era of osdev and gives me security
           | creepy-crawlies.
           | 
           | (also how do you handle `device=xxx device=yyy
           | device_1=zzz`?)
        
             | cperciva wrote:
             | If there's other users for this I'll write a getenv_multi
             | which takes a callback function. Didn't seem like much
             | point doing that until we want to use this somewhere else
             | though.
             | 
             | We end up with device=xxx and device_1=yyy but if you mean
             | that someone specifies device_1=zzz after that, it would
             | show up as device_1_1=zzz. I'm inclined to file that one
             | under "if a user is deliberately obtuse, they get what they
             | deserve".
        
       | plainOldText wrote:
       | Could OpenBSD run Firecracker? I think it would be a nice
       | development. Security focused OS meets security focused microVM.
        
         | 0x457 wrote:
         | No, Firecracker built on top of KVM. IIRC KVM only available on
         | Linux and Illumos. There used to be FreeBSD port, but it's
         | dead.
        
       | jpeeler wrote:
       | From the article: "but Firecracker's implementation had two bugs:
       | It placed the MPTable in the wrong place (above the advertised
       | top of system memory rather than in the last kB) and it set a
       | field containing the number of table entries to zero rather than
       | the appropriate count. In both cases, Linux accepts the broken
       | behaviour; so I added a "bug for bug compatibility" option to the
       | FreeBSD MPTable code."
       | 
       | I'm pretty sure that discoveries like these is why it's good to
       | have operating system diversity. Conversely, sometimes I do
       | wonder if having everyone on the same OS would allow us to have
       | way more cool things and still have less bugs. But it doesn't
       | matter too much because it'll never happen...
        
         | capableweb wrote:
         | > But it doesn't matter too much because it'll never happen...
         | 
         | I'm not saying you're wrong or right, but it's worth
         | remembering that it's hard to guess how it'll be in the future.
         | Maybe Apple acquires Microsoft at one point (or vice-versa) and
         | eventually ends up buying Linux (somehow, don't ask me how),
         | and we'll end up with one OS for absolutely everything. Weirder
         | stuff have happened.
         | 
         | There used to be multiple contenders for what would end up as
         | "the internet", with many countries having their own versions.
         | I'm sure at that point some people had similar sentiments
         | running through their heads, something like "wow, wouldn't it
         | be cool if all these various networks was just one instead? But
         | that'll never happen..."
         | 
         | Similarly how we today look at messaging services not being
         | interoperable with each other, but seems this is slowly (veeery
         | slowly) changing.
        
       | tiffanyh wrote:
       | I don't know if I should be amazed at how 1-person can contribute
       | to an OS ... or if I should be saddened at the current state of
       | OS's given the 10,000s of developers working OS's.
       | 
       | (Another person who comes to mind is Matt Dillon of DragonflyBSD
       | and his huge contributions)
        
         | cperciva wrote:
         | I had a lot of help. I was driving this, but it was absolutely
         | not a one-person effort.
        
       | qalmakka wrote:
       | FreeBSD really deserves all the love it can get. It is an amazing
       | system to work with and develop on, and I'll never stop
       | evangelizing everyone about it.
        
         | CoolCold wrote:
         | If you need fresh ears/someone to practice on - I'm all yours.
         | I'm not ready to discuss usage on desktop - in my opinion it
         | even more useless than Linux, but on server side, for web
         | services related projects I'm fine.
         | 
         | My upper limit would be ~ 5000 servers in use, 3-500 web
         | related developers for that system/servers.
        
         | mistrial9 wrote:
         | Is it really a good idea to make your graphical logo as "a
         | Devil smiling" ?
        
           | xxpor wrote:
           | This is one of those cases of if you're going to raise a
           | stink about it, the community probably doesn't want to deal
           | with you regardless.
        
           | ComputerGuru wrote:
           | That ship sailed a long time ago, friend.
        
         | nurettin wrote:
         | In your opinion, what is the best thing that FreeBSD offers?
        
           | nix23 wrote:
           | https://klarasystems.com/articles/deep-diving-into-the-
           | stren...
           | 
           | https://vermaden.wordpress.com/2020/09/07/quare-freebsd/
        
             | flanked-evergl wrote:
             | I'm not sure anything of the first article can really be
             | said to be unique value propositions, though to be fair
             | this is not what you are claiming either. But just to go
             | into some specifics.
             | 
             | > One of the best features of FreeBSD is that it can be
             | used as a general-purpose operating system. That means that
             | FreeBSD can be used as a server, as an embedded system, or
             | in networking.
             | 
             | Same for Linux.
             | 
             | > FreeBSD provides simplified software management with the
             | help of its packaging system and the ports collection.
             | 
             | Not a ports experts but the difference between ports and
             | Gentoo portage does not really seem that big, certainly
             | nothing of FreeBSD itself makes it more suitable to working
             | with ports than a Linux kernel. And there is guix and nix
             | also which really should cover most software management
             | needs and flexibility anyone has.
             | 
             | > One of the many great advantages of FreeBSD is that it
             | allows you to customize the operating system, according to
             | your needs.
             | 
             | Again Gentoo, nix and guix covers this quite well.
             | 
             | > You don't need lots of cores - but you can definitely run
             | it on huge multicore systems, can easily run a highly
             | graphical interface, or choose the simpler way; FreeBSD
             | will support you either way.
             | 
             | Linux definitely does scale up and down, maybe FreeBSD
             | scales down a bit smaller than Linux, but I would have to
             | see numbers to back this up, and I don't think it is that
             | significant in difference.
             | 
             | > FreeBSD has been ported to a variety of instruction set
             | architectures.
             | 
             | Again this is not something unique to FreeBSD and Linux has
             | a proven track record of portability.
             | 
             | > FreeBSD includes two native file systems, the Unix File
             | System (UFS) and the Z File System (ZFS).
             | 
             | I guess "native" ZFS is the closest it comes to a UVP, but
             | given Linux support for ZFS is quite good to the point
             | where ZFS on root is possible I'm nto sure it qualifies.
             | 
             | Not saying FreeBSD is not great, but not sure it really has
             | any unique value propositions.
        
               | dangerface wrote:
               | > Not a ports experts but the difference between ports
               | and Gentoo portage does not really seem that big,
               | certainly nothing of FreeBSD itself makes it more
               | suitable to working with ports than a Linux kernel.
               | 
               | As far as I know gentoo portage is based on freebsd's
               | ports. Where Gentoo has different kernels (linux or
               | freebsd) and an ever changing userland, FreeBSD has
               | consistency they make a complete operating system they
               | don't bolt stuff together like Linux distros do. The
               | holistic approach of freebsd results in a more cohesive
               | and considered environment linux doesn't have this.
               | 
               | The difference is not that one has the command pkg and
               | the other portage but how the entire environment has been
               | considered to work with ports.
               | 
               | > Linux definitely does scale up and down, maybe FreeBSD
               | scales down a bit smaller than Linux
               | 
               | In my experience they both scale the same, it makes sense
               | that they would both watch each other and implement any
               | improvements they seen since they are both open source.
               | 
               | > I guess "native" ZFS is the closest it comes to a UVP
               | 
               | If you are looking for a killer feature that destroys
               | linux you aren't going to find it they are both very
               | similar, its like looking for a UVP for Fedora vs Gentoo
               | vs Debian they all do pretty much the same thing but in
               | different ways. I use Fedora on desktop and Debian on
               | servers as they both seem to be well suited to those
               | environments, I could just change one to work like the
               | other but why bother? Same is true of FreeBSD.
        
               | philjohn wrote:
               | The killer for me is that it's a full-fledged operating
               | system, not just a kernel.
               | 
               | Kernel and Userspace are developed, and packaged
               | together, in the same repo.
        
               | CoolCold wrote:
               | I have internal (happens just for me, in my head) bet
               | that recent Postgres 15 won't be even benchmarked on
               | FreeBSD by any meaningful vendor. By vendor here I refer
               | to companies like Percona, 2ndQuadrant, EnterpriseDB,
               | Greenplum and/or hosting vendors - like AWS/AZURE/GCP
               | (like they may find out that on ARM servers FreeBSD +
               | Postgres15 are a sweet couple) or some Postgres devs -
               | not a random person on internet.
               | 
               | Lack of such benchmarking, will be indicator of "no one
               | cares on that platform anymore".
               | 
               | Nginx can be other one "indicator" - Nginx started as
               | FreeBSD focused product in company heavily focused on
               | FreeBSD (I know couple of persons from there by that
               | times), nowdays not even mentioning ARM64 in list of
               | supported product
               | 
               | > FreeBSD 3 -- 12 / i386; FreeBSD 5 -- 12 / amd64;
               | FreeBSD 11 / ppc; FreeBSD 12 / ppc64;
               | 
               | from [1] and [2]. Not a big deal for _now_ but, but -
               | more and more ARM servers in major hostings, even Hetzner
               | has 2 of them. Seems this train have been missed too.
               | 
               | Let's see will I win my bet or not.
               | 
               | [1] https://nginx.org/en/#tested_os_and_platforms [2]
               | https://docs.nginx.com/nginx/technical-
               | specs/?_ga=2.23463900...
        
               | 1500100900 wrote:
               | https://buildfarm.postgresql.org/cgi-
               | bin/show_status.pl?memb...
        
               | nix23 wrote:
               | Oracle Cloud supports FreeBSD on Arm64 and Amd64, but
               | overall i have no idea what you try to say...in your
               | head...
        
           | dangerface wrote:
           | Consistency, I have been using linux for 20+ years but if its
           | not redhat or debian based I couldn't tell you how to start
           | or stop services etc as its always different from the unix
           | standard and every other linux. FreeBSD is unix it doesn't
           | try to be that and something else at the same time.
        
             | anthk wrote:
             | This, this. On Debian I had to run rcconf since Debian
             | Sarge. On OpenBSD there's rcctl and /etc/rc.conf.local.
             | MUCH saner. If I had internet at home before 2010, a lot of
             | things would be far easier.
             | 
             | My fist Linux was from 2003-4, Debian Woody. Why I learned
             | was until 2007-8 mainly the bunch of tools from Debconf
             | (the installer, update-alternatives, apt, dpkg-reconfigure,
             | make-kpkg (to compile a newer kernel), debhelper and so
             | on), not the Unix way to do things. In 2006-2009 I knew how
             | to compile Mplayer from source (tgz) and a bunch of other
             | tools (Cedega from CVS) with apt-get build-dep to fetch the
             | deps and a bit more, that's it.
             | 
             | From 2009-2010, I've got a brief match with Slackware and
             | Arch, and from that, OpenBSD. OpenBSD was dumb simple in
             | design, seriously. With Linux you had to do voodoo with
             | Artsd/ESD _disabling_ them and then more fuckery with UCM
             | files and DMIX. OSS under BSD 's (I liked FreesBIE, a
             | lovely live CD with FreeBSD and XFCE) just _worked_. And it
             | ran faster than Linux.
             | 
             | Also, the Bash guides from the Howto's were utterly
             | difficult, much more than the Ksh ones from Orelly
             | (learning Perl for scripting was far easier, trust me).
        
             | nix23 wrote:
             | >but if its not redhat or debian based I couldn't tell you
             | how to start or stop services
             | 
             | The same way as in freebsd "service blabla start" it's a
             | compatibility hook.
        
               | dangerface wrote:
               | on fedora its systemctl instead of service because they
               | are all systemd up, other distros might use something
               | other than systemd and a compatibility hook is entirely
               | dependant on the distro deciding it cares about backwards
               | compatibility which isn't the case for fedora or debian
               | at least in comparison to freebsd you can still build
               | work the old school way.
        
               | cesarb wrote:
               | > on fedora its systemctl instead of service because they
               | are all systemd up
               | 
               | Fedora (and its derivatives like RHEL/CentOS/etc) still
               | has the "service" command, it forwards to the
               | corresponding systemctl command. The same for Debian (the
               | comments in Debian's "service" command even mention it's
               | a modified version of the RHEL/Fedora one).
        
           | pjmlp wrote:
           | Being a proper UNIX descendant, but it hardly matters to most
           | folks.
        
             | DominoTree wrote:
             | Funny enough, it's not quite the same, but Huawei has a
             | Linux distribution that is certified to conform to the Unix
             | 03 standard
             | 
             | https://en.wikipedia.org/wiki/EulerOS
        
           | adrian_b wrote:
           | I have been using both FreeBSD and Linux for many decades.
           | 
           | Until around 2003 to 2005, FreeBSD had much better
           | performance and reliability than Linux.
           | 
           | That changed after Intel introduced Pentium 4 with multiple
           | threads in 2003 and AMD introduced CPUs with multiple cores
           | in 2005. Before that, multiple hardware threads were
           | available only in very expensive computers, which were
           | inaccessible for most people.
           | 
           | Linux was adapted very quickly to work on multi-core or
           | multi-thread CPUs, while FreeBSD required many years for that
           | transition.
           | 
           | At that time, Linux got a great advance in performance over
           | FreeBSD, which was difficult to recover later.
           | 
           | Nowadays, FreeBSD has device driver support for a much less
           | number of devices than Linux, so you must be more careful
           | when choosing hardware for it.
           | 
           | Nevertheless, even if I use Linux in laptops, desktops and
           | computational servers, I have continued to use FreeBSD in
           | networking servers until today.
           | 
           | The main reason is that the administration of a FreeBSD
           | server requires much less time than the administration of a
           | Linux server (which requires much less time than the
           | administration of a Windows server).
           | 
           | With FreeBSD, I may not do anything manually for years, if I
           | have configured it to make certain automatic updates.
           | 
           | With Linux, at least with the distributions that I like for
           | other reasons, I have to always supervise the updates,
           | because the updates frequently break things. (I assume that
           | this might not happen with something like Debian stable, but
           | whenever I have looked to a Linux distribution like that, it
           | was too antiquated for my needs.)
           | 
           | One of the main culprits is always Python, because there are
           | a huge number of packages that insist that they are
           | compatible only with a certain Python version, and they
           | conflict between themselves even when I have all the possible
           | Python versions installed. I could understand that passing
           | from Python 2 to 3 has broken things, but why some packages
           | like Python 3.9 and dislike 3.10 is beyond me. Also the
           | compatibility with various Qt versions frequently causes
           | update problems.
           | 
           | On FreeBSD all the system is more consistent, so I need to
           | spend no more than a couple of hours for a server after
           | several years of 24/7 functioning, when I make some hardware
           | update and when I might replace the major FreeBSD version at
           | the same time.
        
             | capableweb wrote:
             | I have just about zero experience running any kind of BSD
             | systems, but I just wanted to know a bit more about this:
             | 
             | > With Linux, at least with the distributions that I like
             | for other reasons, I have to always supervise the updates,
             | because the updates frequently break things
             | 
             | What distributions have you run into these issues with and
             | how often is "frequently" for you?
             | 
             | I'm running Arch Linux on my desktop and bunch of servers,
             | for more than five years each one and can count the number
             | of breakages on one hand. On the other hand, I run Ubuntu
             | on my laptop and stopped counting the amount of issues I've
             | had with upgrades.
             | 
             | But I'd consider that amount of breakage from my Arch
             | installs very infrequent, but maybe I'm comparing it to the
             | wrong thing, as as I said before, I don't have any
             | experience with BSD systems, maybe breakage there is even
             | more uncommon?
        
               | CoolCold wrote:
               | the secret sauce here is that "ports collection/packages
               | collection" aka anything useful beyond service NAT
               | gateway/network router is not a system!
               | 
               | from the docs:
               | 
               | > FreeBSD is bundled with a rich collection of system
               | tools as part of the base system. In addition, FreeBSD
               | provides two complementary technologies for installing
               | third-party software: the FreeBSD Ports Collection, for
               | installing from source, and packages, for installing from
               | pre-built binaries.
               | 
               | Thus, when FreeBSD guys say "my system is solid" - means
               | he can use ping/top/csh. But if update breaks say Firefox
               | - it's not part of the system and haha, system is stable!
        
               | nortonham wrote:
               | I take your point, but I've used freebsd on a laptop
               | before and never had 3rd party software (like firefox)
               | break after an update. FreeBSD really is stable. For
               | desktops/laptops I would probably lean towards a linux
               | distro first due to hw support, but FreeBSD being
               | stale/solid is no joke.
        
               | adrian_b wrote:
               | I never use csh, the first step when I install FreeBSD is
               | to install bash.
               | 
               | Most of the programs that I use on FreeBSD come from
               | ports, not from the base system (they are various
               | networking applications like squid or samba or nginx),
               | even if I use some functions from the base system, e.g.
               | firewall/NAT/routing, some times the NFS server.
               | 
               | Nevertheless, I did not have problems with
               | inconsistencies in the ports collection, and that
               | includes Firefox (which I use some times together with
               | vncserver, so I can use it remotely even if I do not
               | normally install the X11 server on FreeBSD).
               | 
               | It is true however that on FreeBSD I have used only
               | seldom programs with GUI interfaces, Firefox being an
               | exception to that, and those are usually among the
               | programs more likely to be broken by updates on Linux
               | systems.
        
               | 0x457 wrote:
               | nah, I used FreeBSD on desktop, and I was tracking
               | CURRENT with weekly rebuilds.
               | 
               | The only time things "broke" for me is when X11 on
               | FreeBSD switched to libinput by default.
               | 
               | Never had issues with anything else:
               | 
               | - nvidia drivers always worked, unlike ubuntu that tried
               | to install opensource-not-working-driver every time it
               | gets
               | 
               | - My keyboard remapping worked every upgrade. Ubuntu
               | broke it major release, Arch broke it from time to time.
               | Fedora I didn't even bother.
               | 
               | - Most of my software was always on bleeding edge release
               | when possible
               | 
               | - Docker in bhyve worked better than on linux because I
               | could just restart VM
               | 
               | - Not once, my bootloader went foobar in 20 years (with
               | GRUB I live in constant fear)
               | 
               | Now, if FreeBSD folks stopped with perfectionism and
               | finally landed GPU and WiFi drivers...I'd be still using
               | it. Now I won't switch back though, I'm far too friendly
               | with Nix.
        
               | throw0101a wrote:
               | > _- nvidia drivers always worked, unlike ubuntu that
               | tried to install opensource-not-working-driver every time
               | it gets_
               | 
               | Many, many moons ago a video game called _Return to
               | Castle Wolfenstein_ was released, and in addition to a
               | Linux server where you could host multi-player games, it
               | actually had a _Linux client_.
               | 
               | I played on the Linux client for a while with an NVidia
               | GPU. But my main desktop was FreeBSD at the time, and so
               | instead of dual-booting I used FreeBSD's Linux API/ABI
               | emulation to play RtCW with FreeBSD NVidia drivers. I
               | even got a few more FPS under FreeBSD.
               | 
               | * https://en.wikipedia.org/wiki/Return_to_Castle_Wolfenst
               | ein
        
               | anthk wrote:
               | FreeBSD users with issues will just recompile Firefox
               | from the ports.
               | 
               | Also, I am an OpenBSD user (not FreeBSD), and I can say
               | packages are much better maintained than in the 99% of
               | the distros.
        
               | CoolCold wrote:
               | I was trying to produce good example on differences in
               | reading "stable system" on FreeBSD vs Linux worlds, when
               | definition on what is "system" differs.
               | 
               | No anything specific I have to put against Firefox or any
               | other application here.
        
               | anthk wrote:
               | Being a Linux user since Debian Woody and others until
               | 2011, I can say most of them.
               | 
               | Debian and Ubuntu breackage was guaranteed. On RH/RHEL,
               | it was not easy somtimes.
               | 
               | Arch had two major switchs (SystemD and anothers).
               | 
               | On OpenBSD, you just read the upgrade guide, download the
               | bsd.rd, boot it and upgrade. Or with sysupgrade from a
               | few releases. Much reliable than Linux.
               | 
               | https://www.openbsd.org/faq/upgrade71.html
        
               | nortonham wrote:
               | what breakage do you remember from Debian? I remember
               | using debian on the same machine and going through three
               | major releases without issue. ubuntu, not so much.
               | 
               | agreed about the ease of openBSD upgrades from one
               | release to another
        
               | anthk wrote:
               | I think Woody->Sarge, some ALSA settings. I used bf24
               | (2.4) kernel in Woody.
        
               | adrian_b wrote:
               | I almost always use only programs that I compile from
               | sources (with few exceptions, like the NVIDIA drivers and
               | libraries or some commercial EDA/CAD programs). I also
               | strongly dislike systemd (not based on hearsay, but after
               | testing it for a couple of months a few years ago, while
               | using Arch Linux). These 2 conditions disqualify many
               | Linux distributions for me.
               | 
               | So I most frequently use Gentoo or some distribution
               | derived from it, e.g. Funtoo.
               | 
               | I have switched to Gentoo around 2002/2003, after using a
               | few older distributions, including Slackware, Redhat and
               | SUSE.
               | 
               | During the first decade of using Gentoo, the Portage
               | collection was better maintained and I would have never
               | had problems with upgrades, except that when I have
               | tested KDE 4 I was astonished by how the team which
               | hijacked its development had been able to remove all the
               | features that I liked in KDE 3.5, so I have wiped KDE 4
               | and I have reverted to KDE 3.5.
               | 
               | Then, for a couple of years I had to fight more and more
               | at each upgrade to avoid the breakage of the preserved
               | KDE 3.5, until that became so difficult that I gave up
               | and I replaced KDE with XFCE.
               | 
               | Then for a few years there have been no problems with
               | upgrades, but during the last decade they have begun to
               | appear sporadically. In many cases the upgrades still
               | work without surprises, but every few months there is one
               | that fails because some package does not compile for
               | various reasons, e.g. because some library is not listed
               | as a dependence so it has not been updated, or because
               | there is a circular package dependence that appears only
               | with the specific combination of USE flags that are set
               | on my system, or because there is some conflict between
               | the dependencies of some packages that prevent them to be
               | installed together without certain workarounds.
               | 
               | What I like at Gentoo is that it permits an extreme
               | customization of the system, but that is what makes very
               | difficult for the package maintainers to test their
               | package build instructions, because it is hard to
               | generate all the combinations of options that might have
               | been chosen by some user.
               | 
               | The breakages are not frequent, but because they happen
               | sometimes I am not confident to allow them to be
               | performed automatically, without supervision, like I
               | allow them to be done on FreeBSD.
               | 
               | I would prefer a package system like that of Nix, but I
               | have not found yet enough time to play with Nix, or maybe
               | with Guix, to evaluate which will be the consequences of
               | converting to it.
        
           | efortis wrote:
           | An alternative OS for an heterogeneous infrastructure.
           | 
           | For example, when there is a security bug in Linux you can
           | shutdown those servers and spin up FreeBSD ones while the
           | patch is being made.
        
             | hobo_mark wrote:
             | I guess that makes some sense if you were already running
             | both operating systems in production. Maintaining an
             | alternative stack would be almost twice the work otherwise.
        
               | efortis wrote:
               | It depends on the risk. Sometimes shutting down
               | vulnerable servers is not worth it.
        
           | qalmakka wrote:
           | There are several, to be honest, but my favourite by far is
           | that it cleanly separates, like all BSDs, the base system and
           | the external packages. You can have a stable or bleeding edge
           | base system depending on what suits your tastes, and on it
           | install the lastest extra packages you need. They completely
           | reside under /usr/local and are installed through different
           | channels (the ports), cleanly separating the system from the
           | user software, like it happens on Windows or macOS but
           | arguably less clunky.
           | 
           | This is something is sincerely miss in Linux, where more
           | often than not you have to either decide between having
           | bleeding edge packages on a bleeding edge system (like Linux,
           | or Debian Sid) or having to deal with old packages and third
           | party repositories (like on Debian or Red Hat).
           | 
           | Also, the fact the whole system is shipped, tested and built
           | by a closely knit group of developers is a huge bonus to me -
           | the overall experience feels polished and well integrated,
           | something Linux distros often seriously lack. At times on
           | Linux it just feels as if some packages have just been
           | bundled haphazardly together with zero care about documenting
           | their interactions or how they are supposed to work, or that
           | the packagers do not agree with upstream and do random
           | modifications that just complicate things (like on Debian or
           | Red Hat). Lots of stuff in FreeBSD has no upstream - FreeBSD
           | and the other BSDs write and maintain their own kernel, libc,
           | core utils, init system, bootloader, ...
        
             | anthk wrote:
             | >You can have a stable or bleeding edge base system
             | depending on what suits your tastes,
             | 
             | Not with OpenBSD. If you run -current, please, do not mix
             | -release and -current pkgs/ports.
             | 
             | NetBSD it's bound to major releases:
             | 
             | https://www.netbsd.org/docs/misc/index.html#package-
             | manageme...
        
               | chalst wrote:
               | NetBSD's pkgsrc makes it easy to run several different
               | package repositories, mixing releases.
        
             | CoolCold wrote:
             | Q1: what's your impression/general thoughts on immutable
             | OSes? To name a few - Bottlerocket, Flatcar and Talos
             | Linux, Fedora Silverblue and Android?
             | 
             | Q2: a bit different, but similar as the end goal -
             | immutable infrastructure where you say update your AMIs and
             | migrating services there, shutting down old "unapgraded"
             | servers at all?
        
           | jlouis wrote:
           | Documentation.
           | 
           | Everything is in one place, and it is maintained. In
           | contrast, Linux requires you to go look up documentation in a
           | lot of different places, and it is highly dependent on
           | distribution what to do in a given situation.
           | 
           | One reason for this is that FreeBSD maintains not only a
           | kernel, but also a userland. This means a change in the
           | kernel can immediately be followed by appropriate changes in
           | the userland and vice versa.
        
             | zeendo wrote:
             | Yeah. And the exception (as a desktop user) - ArchLinux -
             | really proves the rule here. I like Arch for a handful of
             | reasons but the documentation is really the standout one.
             | 
             | But my experience with FreeBSD's docs are even better - and
             | most of that experience is quite old at this point. I can't
             | imagine how much better the situation has gotten since
             | FreeBSD only seems to mature and not degrade (based on
             | comments like yours which are very common).
        
               | xxpor wrote:
               | I use Arch docs all the time, even for non-Arch distros.
               | What really makes the wiki stand out is its focus on
               | practical problems with examples, more than man-pages
               | style docs. Is there anything similar for FreeBSD? I know
               | the man pages are supposed to be amazing, but it's just a
               | different problem to solve IMO.
               | 
               | I also seriously dislike the BSD's tendency towards
               | making --help useless, forcing you to bring up the man
               | page. This is just for practical reasons. It's easier to
               | hit up arrow, dash dash help pipe less than up arrow,
               | ctrl+a, man, alt+f, ctrl+k to kill the arguments you've
               | already written :)
        
               | anthk wrote:
               | > forcing you to bring up the man page
               | 
               | Man pages in OpenBSD for instance are a "must". In the
               | GNU world, often man pages are really badly maintained
               | pointing to GNU info. Also, the so called "Linux Howtos"
               | were a disaster full of obsolete points.
               | 
               | Just compare the OpenBSDxy.html upgrading page (where m
               | and n are release numbers) for OpenBSD releases (and the
               | FAQ) to the Arch Wiki.
        
               | rabf wrote:
               | https://docs.freebsd.org/en/books/handbook/
        
               | xxpor wrote:
               | Oh this looks really nice!
               | 
               | /me adds a new project to the backlog...
        
             | anthk wrote:
             | That was my first problem with Linux, getting docs to
             | configure the system was horrible. From Debian Woody to
             | SuSE and then Debian Sarge, every distro was chaotic and
             | Yast and Debconf overwrote everything.
             | 
             | With NetBSD and OpenBSD everything looked _sane_. They
             | explained you everything, from  /etc/profile, managing
             | services without making you mad with runlevels, to
             | launching X and so on.
        
               | CoolCold wrote:
               | Yeah, I double this
        
       | agapon wrote:
       | Nice development. Thank you, Colin!
        
       | anotherhue wrote:
       | Incredible. Maybe we can use this to shoe horn in docker support
       | without the long lived VM
        
         | nix23 wrote:
         | https://bastillebsd.org/
        
       | dkjaudyeqooe wrote:
       | A welcome development. I wonder how quickly it boots.
        
         | kevans91 wrote:
         | Colin's done a lot of work in improving boot time on EC2 and in
         | general, so I'm a little surprised he didn't actually mention
         | the boot time in this article. In 2022Q1 he had boot time down
         | to 8s (in EC2, IIRC); based on how cut down the FIRECRACKER
         | config he added is and that there's no loader involvement, I
         | don't think it'd be too surprising if he hits ~5-6s or less.
        
           | cperciva wrote:
           | It took me four days to get the blog post written and I only
           | finished it last night because the toddler went to sleep
           | early. I didn't want to drag it out any longer!
           | 
           | But to answer the question: On small VMs, we can reach
           | /sbin/init in under 50 ms and /etc/rc finishes running in
           | under 500 ms. (Larger VMs take longer -- past a few GB of
           | RAM, the time for initializing paging tables and launching
           | APs starts to dominate.)
        
       | rwmj wrote:
       | Is it better / worse (in terms of say performance or security)
       | than Bhyve?
        
         | eyberg wrote:
         | Firecracker trades run-time performance in favor of faster
         | boot-time latency. It is something that is being addressed but
         | afaik still exists. This of course doesn't mean anything if you
         | want firecracker to boot a multi-gig JVM installation (the
         | larger the filesystem generally, expect a longer boot time).
         | 
         | Security wise there are some minor differences. For instance
         | bhyve supports virtio-rng but firecracker doesn't want to:
         | https://github.com/firecracker-microvm/firecracker/issues/16...
         | .
         | 
         | I think if your app requires fast boot time and your app
         | supports that than it's fine (so services that spin up and down
         | on demand) but apps that daemonize for extended periods of time
         | or take forever to initialize probably not a great fit.
        
       | tinco wrote:
       | Nice! If I'd be looking into running FreeBSD it would be because
       | I heard about its supposed security and I/O performance
       | characteristics. Do those qualities hold up when run on
       | firecracker this way? What applications would you choose FreeBSD
       | for?
        
         | LunaSea wrote:
         | Is I/O performance still better on FreeBSD after the last round
         | of Linux improvements like io_uring and eBPF?
        
         | dangerface wrote:
         | In my experience FreeBSD performance and security are great but
         | not significantly better than linux. ZFS support on freebsd
         | seems better than linux certainly its been supported a lot
         | longer, bhyve works really well for virtualising freebsd and
         | linux never tried it with windows.
         | 
         | The biggest strength is the ports / package manager works
         | better than anything I have tried on linux and it sticks to
         | unix principles closer than linux.
         | 
         | The biggest weakness for freebsd is docker and mono support is
         | very poor. There is a desktop distro but in general any gui
         | stuff linux beats freebsd hands down I only use freebsd for
         | server stuff.
         | 
         | I use it for production web servers, backup / storage servers,
         | routers, virtualisation servers. I find its rock solid and more
         | consistent than linux.
        
           | exogenousdata wrote:
           | Agreed that the lack of docket support is frustrating given
           | how pervasive docker has become. However I've been using
           | jails[0] in my home lab for 12 years and it's a delight to
           | use.
           | 
           | [0] - https://docs.freebsd.org/en/books/handbook/jails/
        
             | soupbowl wrote:
             | I love jails, bastille as a jail manager/builder is really
             | great.
             | 
             | https://github.com/BastilleBSD/bastille
        
       | foomer wrote:
       | That's a neat technical challenge solved, but I'm surprised by
       | how much effort is being put in for these operating systems that
       | basically no-one uses.
        
         | dangerface wrote:
         | Lots of people and places use freebsd like netflix. I only see
         | the popularity of freebsd increasing as linux moves more and
         | more away from unix.
        
           | yamtaddle wrote:
           | I'm slowly switching over for server-side, and next time I
           | try a free OS desktop it'll probably be freebsd. Only thing I
           | miss is docker, which I mostly just use as a package manager
           | and service manager so I don't have to screw with distro-
           | specific trivia for those things, and that's less of a
           | problem on FreeBSD anyway.
           | 
           | I've long tolerated Linux but never _loved_ it, and it 's
           | moving away from anything I'm interested in with each year,
           | it seems (the competing mostly-bad image-based app
           | distribution systems and the way Wayland's going are...
           | yikes). Glad FreeBSD's there as an alternative.
        
             | tcmart14 wrote:
             | On FreeBSD, I would recommend looking at jails and
             | something like CBSD as a way to manage jails.
             | 
             | https://github.com/cbsd/cbsd
        
               | yamtaddle wrote:
               | Right, I'm familiar with them, but a huge part of the
               | benefit of docker is that it's basically a cross-distro
               | (and, to some degree, cross-platform) very up-to-date
               | server-oriented package manager. There are some efforts
               | to create something like that for Jails but AFAIK none of
               | them have gotten very far. Docker's benefit is that
               | ~everyone uses it so there's usually an official docker
               | image for anything one might want to run, kept in step
               | with releases, plus a substantial history of images for
               | rollbacks and testing and such.
               | 
               | It turns "how do I install and run this?" from a bunch of
               | platform- and distro-specific instructions into `docker-
               | compose up` for everything, with the bonus that the
               | compose file also tells you exactly where data & config
               | for the service live, so it's _also_ concise, guaranteed-
               | accurate, and extremely useful documentation. Old LTS
               | Ubuntu? Can run the same version of a service, with the
               | same command, as on bleeding-edge Arch. On Arch, exact
               | same command. Debian, Red Hat? Same command. MacOS, if
               | you 've got docker set up, same command. Windows, ditto.
               | The latter two, non-natively, sure, but it works. Learn
               | docker, and you can use it everywhere, unlike becoming
               | proficient with, say, dpkg/apt or portage or pacman. It
               | can even _kinda_ save you from having to worry about how
               | a given platform 's init system and process management
               | works, at least for simple use cases.
               | 
               | Again, if you're doing most or everything on FreeBSD,
               | development and hosting both, those benefits mostly
               | vanish. But the reality is many workplaces or even
               | personal dev+hosting environments are mixed, and docker
               | suits those better than anything else I'm aware of--but
               | if FreeBSD is in the mix, it's the odd-man-out. It's good
               | not (only) because of its kinda-Jails-like isolation, but
               | because of the excellent coverage of the image
               | repository.
        
         | capableweb wrote:
         | Basically no one uses FreeBSD? Last time I checked, companies
         | like Netflix, Sony, Yandex and more uses FreeBSD for most of
         | their services, not sure I'd put those in the bracket of "no
         | one".
         | 
         | Also, Hacker News runs FreeBSD, for what it's worth :)
        
           | nix23 wrote:
           | Distrowatch too...it's kind of hilarious ;)
        
             | sigjuice wrote:
             | Why is it hilarious that a website covering Linux
             | Distributions and Berkeley Software Distributions is
             | running on FreeBSD?
        
               | nix23 wrote:
               | Because they changed to linux and then back to freebsd,
               | bit like sdf.org.
        
         | stragies wrote:
         | I thought also Apple regularly cherry-picks interesting stuff
         | to integrate into macOS/iOS
        
           | tcmart14 wrote:
           | I believe when Apple used to make the Airports, it was pretty
           | much NetBSD on those.
        
         | comprev wrote:
         | You might not hear about them publicly but *BSD boxes usually
         | run the backbone of network infrastructures - DNS servers,
         | firewalls, file servers, etc.
         | 
         | A predictable release schedule and praised upgrade paths mean
         | companies can plan product updates long into the future.
        
         | gruturo wrote:
         | FreeBSD's license causes it to be used in lots (lots!) of
         | places where GPL isn't suitable - most are probably not super
         | famous, but still ubiquitous. But as others have mentioned in
         | their replies to you, there are many rather high profile use
         | cases as well.
         | 
         | So - your premise is wrong - but even if it wasn't, why be
         | surprised? Some stuff is just fun to hack on, people have
         | passions and interests too.
        
         | nix23 wrote:
         | It's funny to see peoples in such a deep bubble. For example in
         | japan you hear *bsd as much if not more than linux.
         | 
         | Also many firewalls use freebsd/openbsd, from pfSense to
         | opnsense to genugate.
        
           | nortonham wrote:
           | there's japanese ISP's that use NetBSD in their hardware
           | correct?
        
             | nix23 wrote:
             | Yes i heard something too.
             | 
             | And Apple used it for their Apple AirPort:
             | 
             | https://www.netbsd.org/gallery/products.html
        
       ___________________________________________________________________
       (page generated 2022-10-18 23:03 UTC)