[HN Gopher] Jails on FreeBSD
___________________________________________________________________
Jails on FreeBSD
Author : rodrigo975
Score : 124 points
Date : 2023-05-25 09:37 UTC (1 days ago)
(HTM) web link (ogris.de)
(TXT) w3m dump (ogris.de)
| codetrotter wrote:
| Let me chime in and say, Michael W Lucas has an awesome book
| about FreeBSD Jails. I bought it recently and I read the whole
| thing and it helped me a lot. I still had to figure some things
| out by myself because the book is for a slightly older version of
| FreeBSD. But it is an awesome book.
|
| I use only the tools included in base system for setting up my
| jails. No "ezjail" or anything.
|
| If you read his whole book you will see how it might be the
| correct choice to just do it yourself. Depending on what you want
| to do etc.
|
| For me I am definitely much better off having set it up myself
| with the help of mwl's book.
|
| https://www.freebsdmall.com/cgi-bin/fm/bsdmjails
|
| Buy the physical copy of the book.
|
| PS: Use vnet interfaces for most of your jails.
| chunk_waffle wrote:
| All of Michael W Lucas' books are great, they're so easy to
| digest.
| jedberg wrote:
| Just one nit, FreeBSD had jails before March of 2000. I know this
| because I was using them in 1999. And in fact they were merged
| into CURRENT in early 1999 and STABLE in mid/late 1999. It was
| merged into RELEASE in March 2000.
| erk__ wrote:
| They were actually developed a year before, but the person who
| paid for development had a year of exclusive use.
|
| http://freebsd.dk/sagas/jails/
| thehigherlife wrote:
| Since this is gaining some traction. I'm curious if you are using
| jails and a jail manager, what are you're using now? I'm still
| using IOCAGE (Version 1.2) on freeBSD 13, but it seems like its
| not the best way to do it anymore? Documentation is all over the
| place on what is the best way to manage jails now.
| kevans91 wrote:
| iocage as well... I've been trying to sucker someone into
| writing a new jail manager in flua, since we have that in base.
| It hasn't worked yet.
| soupbowl wrote:
| I use BastilleBSD, it has a simple template system which I
| really like and decent docs.
| https://github.com/BastilleBSD/bastille
|
| Bastille also has a sister project 'rocinante' which allows you
| to use Bastille templates on the host. I converted my ansible
| scripts to bastille templates and it works a lot better for
| *ME*. I found I spend more time updating ansible scripts
| whenever I needed to use them, it costed more time then just
| using a setup.sh script, which rocianate basically is.
| https://github.com/BastilleBSD/rocinante
|
| Another new kid on the block for jails is AppJail, it has some
| interesting features. I have not played with it enough to say
| how stable it is. https://github.com/DtxdF/AppJail
| UI_at_80x24 wrote:
| I'm in the same boat.
|
| Started with ezjail, switched to iocage, now thinking about
| bastille or roll-my-own.
| kitsunesoba wrote:
| Still on ezjail myself. Set my NAS/home server up several years
| ago with a blog post tutorial centered around ezjail and have
| just carried that forward. Eventually will need to do a fresh
| reinstall though and would be interested to know what's
| considered best practice these days.
| soupbowl wrote:
| There are a lot of jail tools these days, it would be nice if we
| posted more modern FreeBSD guides. Myself I use BastilleBSD
| heavily for personal and small business use.
|
| https://github.com/BastilleBSD/bastille
| mikece wrote:
| Is there a "best article" out there comparing _BSD Jails to
| Docker and the strengths and weaknesses of choosing one or the
| other?
|
| (I'm also curious if _BSD Jails are the same thing as Solaris
| Zones but with a different name or if there is significant nuance
| making them different).
| paulddraper wrote:
| For starters, Docker runs as a daemon.
| __turbobrew__ wrote:
| For starters starters, Docker is a few layers up in the stack
| from jails. You would be better off comparing runc to jails
| or just cgroups+namespaces to jails.
| paulddraper wrote:
| No.
|
| Docker and runc are very similar.
|
| Docker run Docker containers. runc runs OCI containers.
|
| And jails operate at a similar level, though without an
| image format.
| __turbobrew__ wrote:
| Right, so jails do not have images or any of the tooling
| around building and distributing images which is the vast
| majority of what makes up docker. This is why
| cgroups+namespaces are much closer to jails than docker
| is.
|
| There are systems which build upon jails such as iocage
| and ezjail which are more similar to docker.
| tptacek wrote:
| Use whichever one you're more comfortable with and that suits
| the rest of your stack best. There's nothing Jails are going to
| allow you to do that Linux containers can't do. All the shared-
| kernel workload isolation schemes are comparably safe.
| whalesalad wrote:
| https://blog.jessfraz.com/post/containers-zones-jails-vms/
| gumballindie wrote:
| Every time I think of freebsd and how cool it was and is I
| wondering what caused linux to overtake it. I like linux as well,
| but I just don't understand the mechanics behind freebsd not
| gaining traction.
| jbverschoor wrote:
| For me it was Java. I think there was something with mmap().
|
| I ran FreeBSD servers with jails around '00. They worked fine.
|
| Not for me, but in hindsight, maybe the FreeBSD daemon mascotte
| had something to do with it.
|
| But around that time bea weblogic and ibm websphere also became
| popular, and I don't think Java was officially supported. It
| was also a period where people tended to run oracle, but I'm
| not sure if that tab on Linux or something else. I think it was
| some custom Linux "unbreakable oracle"
| alberth wrote:
| Off topic: I miss the days when web pages looked like this link.
|
| High info density, consistent look, "responsive" without being
| responsive.
| jmclnx wrote:
| Very nice
|
| I used jails on FreeBSD and nothing in Linux comes close. Yes, it
| is not a pointy-clicky setup like Linux likes to do. But IMHO
| Jails are far more secure, in a way: you get what you 'pay' for.
| irusensei wrote:
| I don't know about the low level stuff but LXC is pretty much
| the same end result as FreeBSD jails. At least for my very
| basic needs that is.
| johnisgood wrote:
| What about `firejail` on Linux? Is it not much more secure than
| FreeBSD Jails which is just based on chroot?
| bell-cot wrote:
| FreeBSD Jails are _conceptually_ just based on chroot. Their
| actual design and implementation are a few orders of
| magnitude more secure.
| johnisgood wrote:
| Thanks, I'll check out the specifics.
| mardifoufs wrote:
| A few order of magnitudes? Can you be more specific? I'm
| genuinely wondering what you could be referring to that
| I've missed.
| bell-cot wrote:
| https://en.wikipedia.org/wiki/Operating_system-
| level_virtual...
|
| or
|
| https://en.wikipedia.org/wiki/Freebsd_jail
| NexRebular wrote:
| Also the SANE paper:
|
| http://therbelot.free.fr/Install_FreeBSD/jail/jail.pdf
| tptacek wrote:
| chroot more or less isn't secure at all. All they're
| really saying is that Jails aren't built on top of
| chroot, but rather are a ground-up reimplementation
| motivated by improving chroot.
| mardifoufs wrote:
| Ah! For some reason I thought jails were built on
| something similar to chroot at the kernel level, meaning
| that it also came with the same security caveats.
|
| (to be more accurate, I knew that it was not "on top" in
| the same way as Linux containers are built on top of
| chroot. )
| tptacek wrote:
| Jails are not, in fact, far more secure than Linux containers.
| Arguably they're a better, more coherent design, but both are
| shared-kernel isolation schemes, and the high-order bit of
| their security is exactly the same. The best argument you'll
| come up with is that FreeBSD kernel LPEs are rarer than those
| of Linux, but now we're just back to the standard Linux vs.
| FreeBSD security debate, which never goes anywhere.
| zie wrote:
| I agree with all of this, with perhaps one tiny addendum.
| Defaults matter, and last I checked Docker/container defaults
| are miserable for security. I'm guessing if you randomly
| inspected production container vs jails out in the world, way
| more container solutions would have giant holes in them
| compared to jails.
| cpuguy83 wrote:
| Defaults on Docker specifically are pretty locked down.
| Even if if the container process has the root uid it is not
| able to do most things normal root can do, and is pretty
| much limited to being able to access root owned files and
| binding to privileged ports (which in a container doesn't
| tend to matter).
| yjftsjthsd-h wrote:
| > Defaults on Docker specifically are pretty locked down.
|
| That is _strongly_ dependent on your threat model. The
| default docker configuration completely bypasses the
| firewall, making it trivial for containers to be exposed
| to the open internet with no way for admins to prevent
| it[0]. Likewise, I hesitate to call docker 's default of
| running as root safe since it means anyone with access to
| the docker socket immediately has root on the host.
|
| [0] It is quite easy for someone even slightly
| inexperienced to accidentally write, say `-p 1234:1234`
| instead of `-p 127.0.0.1:1234:1234` and thereby cause a
| security incident or near-miss; _ask me how I know._
| cpuguy83 wrote:
| Docker _has_ to run as root, or use otherwise insecure
| methods ( "rootless" is a sham, it requires suid binaries
| and CVE ridden unprivileged user namespaces).
|
| I agree with ports, working[0][1][2] on it.
|
| [0] https://github.com/moby/moby/discussions/45524
|
| [1] https://github.com/moby/moby/issues/45532
|
| [2] https://github.com/moby/moby/pull/45076
| cpuguy83 wrote:
| That said, docker does have support for running
| "rootless"
| betaby wrote:
| Please provide technical details of `Jails are far more
| secure`.
| throw0101c wrote:
| Multiple CVEs for escaping? E.g.:
|
| * https://blog.aquasec.com/cve-2022-0185-linux-kernel-
| containe...
|
| * https://securitylabs.datadoghq.com/articles/dirty-pipe-
| conta...
|
| * https://snyk.io/learn/docker-
| security/top-5-vulnerabilities/
|
| * https://www.container-
| security.site/attackers/container_brea...
|
| The only escaping of jails that I've heard of in the last ~20
| years is one not in the jails code, but tunnelling out
| through devfs:
|
| * https://www.freebsd.org/security/advisories/FreeBSD-
| SA-14:07...
| tptacek wrote:
| Misleading: the most important vulnerabilities that break
| shared-kernel isolation systems like Jails, Zones, and
| containers are kernel LPEs, and they aren't reported as
| "escapes".
| betaby wrote:
| No? Because many of them Docker specific. Neither I
| mentioned Docker anywhere, not Docker is a good example of
| software with a good security record. Another link lumps
| together k8s, runc and so on. I specifically mentioned
| systemd-nspaw in the beginning as contender to jails, not
| other solutions.
| gtirloni wrote:
| What makes Jails so far ahead of containers in your opinion?
| doublerabbit wrote:
| Not op, but for me:
|
| You can setup Linux in a jail.
|
| You can nest bhyve virtual machines in a jail.
|
| You can assign individual NICs to a jail.
|
| You can encrypt with ZFS a jail.
|
| You can run browsers in a jail with full set of features.
| jmillikin wrote:
| Most of those are supported by "containers", which are
| after all just a chroot and a process tree with restricted
| visibility into certain kernel objects. Assigning
| individual NICs or running a browser with extra isolation
| is one of the classic Linux namespace tutorials.
| > You can setup Linux in a jail.
|
| I don't see how this can be possible. Could you explain
| more how to boot a Linux kernel within a FreeBSD jail?
|
| edit: I'm not talking about running binaries compiled for
| Linux under FreeBSD. The parent said it's possible to set
| up Linux within a jail, so I want to see instructions to
| boot an actual Linux kernel as a FreeBSD process.
| throw0101c wrote:
| > _Assigning individual NICs or running a browser with
| extra isolation is one of the classic Linux namespace
| tutorials._
|
| With FreeBSD you can set up a standalone virtualized
| network stack:
|
| * https://klarasystems.com/articles/virtualize-your-
| network-on...
|
| * https://man.freebsd.org/cgi/man.cgi?vnet
|
| * https://wiki.freebsd.org/Jails/VNET
| the8472 wrote:
| I read the quote you're replying to as "you can _also_ do
| this under linux ", i.e. it didn't dispute that freebsd
| jails can do it. But it's not a distinguishing feature of
| jails.
| [deleted]
| rwaksmunski wrote:
| You use FreeBSD kernel and Linux userland. FreeBSD can
| emulate Linux system calls.
| NexRebular wrote:
| Using the linuxulator.
|
| https://wiki.freebsd.org/LinuxJails
| betaby wrote:
| All of those available on Linux and even more, see
| https://www.man7.org/linux/man-pages/man1/systemd-
| nspawn.1.h...
| doublerabbit wrote:
| I don't see the guide to run FreeBSD in a container from
| that man page.
|
| And what about if my Linux Distro doesn't use system-md,
| like Slackware.
| betaby wrote:
| > And what about if my Linux Distro doesn't use system-
| md, like Slackware.
|
| It's a choice of Slackware not use systemd infra. However
| on Linuxes you have more that one choice to run
| containers, your next option would be LXC/LDX
| https://docs.slackware.com/howtos:misc:lxc
| doublerabbit wrote:
| Sure and my choice not to use it too. Well aware of LXC
| too.
|
| I'm not claiming Linux can't. I was stating why. I know
| Linux can do what FreeBSD can do, but BSD does it better.
|
| However both are pretty much mimicked concept from
| Solaris Zones
| sillystuff wrote:
| > However both are pretty much mimicked concept from
| Solaris Zones
|
| Your timeline is off.
|
| FreeBSD and Linux were within a year of each other (not
| in-tree on linux, but Debian packaged VServer kernels and
| VServer + GRSec kernels; we used vserver+grsec debian
| packaged kernels at work in the mid through late '00s).
|
| Solaris containers came years later.
|
| 2000 - FreeBSD jails
|
| 2001 - Linux VServer
|
| 2004 - Solaris Containers
|
| https://blog.aquasec.com/a-brief-history-of-containers-
| from-...
| 5e92cb50239222b wrote:
| I wonder why would anyone spend time on developing and
| maintaining that. FreeBSD needs Linux userland and
| syscall support to get roughly the same level of software
| compatibility. The opposite isn't true (maybe it is for
| your special snowlake internal application developed
| specifically for FreeBSD, but it isn't so in general
| case).
|
| All other things you have mentioned are possible with
| pretty much any of the Linux container runtimes. systemd-
| nspawn is just one of them, and I don't think it is
| actually used very much compared to the alternatives.
| 1over137 wrote:
| Instruction are for FreeBSD 12, I wonder if it's still current
| for 13 and 14...?
| BanazirGalbasi wrote:
| In my limited experience with BSD over the years (I tinker with
| it every year or so), a lot of the core experience remains the
| same. Additions to rc.conf should keep working, as well as
| configuration files for things like jails.
| UI_at_80x24 wrote:
| Not really.
|
| Specifically the suggestion to use lo# instead of vnet.
| layer8 wrote:
| In the aughts (2000s) I rented a "managed" VPS that was based on
| FreeBSD jails and an overlay filesystem. It was great in that you
| didn't have to maintain most of the software yourself (e.g.
| security updates) because it was maintained by the hoster on the
| base filesystem layer, while at the same time you were still root
| on your own filesystem overlay, where you could add or modify any
| file from the base filesystem. If you messed up somehow, you
| could simply revert to the base filesystem version of the
| respective files. Furthermore only the files on your overlay
| counted against your filesystem quota. Unfortunately the hoster
| discontinued that offering after a couple of years.
| codetrotter wrote:
| > Unfortunately the hoster discontinued that offering after a
| couple of years.
|
| I'm planning on starting a FreeBSD based hosting service, but I
| am not sure if anyone actually wants what I have in mind.
|
| The thing is that I have some couple of very specific ideas in
| mind.
|
| I will offer a small amount of storage, and not general
| internet connectivity.
|
| Users will have 25GB space and the idea is that they will be
| able to connect over Wireguard, but they cannot make outbound
| connections from the host.
|
| It'll be like a /home/user in the cloud.
|
| So you can keep some files there and connect from wherever in
| the world and use the cli tools we all know and love. But idk
| if there is an actual market for that or not.
|
| The selling point is that I will be focusing on the storage of
| that data. ZFS with redundancy and offsite backups - the works.
| That's why the amount of data should be low. Only keep the most
| important files there.
|
| Also I'm gonna accept payments in Bitcoin only, and people will
| have to sign up for many years upfront.
|
| I think maybe the number of people that want this can be
| counted on one hand heh
| layer8 wrote:
| No outbound connections would likely be a deal-breaker for
| many.
| gtirloni wrote:
| One question you may want to answer is why anyone would use
| your solution vs something like rsync.net or a small VM in
| DigitalOcean, etc. You said the selling point would be
| storage. What would be the differentiator in that area? Would
| price be it?
|
| Just trying to help with question to validate the idea. Best
| of luck.
| codetrotter wrote:
| > something like rsync.net
|
| My impression of rsync.net is that it is for backup.
| Whereas my service would be a live service you ssh into.
| Basically, connect to Wireguard VPN and then ssh into your
| cloud home.
|
| The idea being that this is where you keep, and interact
| with, your files that are important.
|
| > DigitalOcean
|
| My service would have less system administration for the
| user, and high level of storage redundancy and offsite
| backups
|
| > Just trying to help with question to validate the idea.
| Best of luck.
|
| Thank you, I appreciate it :)
| gtirloni wrote:
| Got it! This: less system administration, that's a good
| selling point!
| porker wrote:
| Verio VPS FTW! That was the first proper hosting I had.
| st3fan wrote:
| I upvoted this because I have a lot of love for FreeBSD and I
| think Jails are great.
|
| However, I really wish the FreeBSD folks would educate themselves
| a bit more about what is actually available on Linux and how
| those options compare to FreeBSD Jails.
| doublerabbit wrote:
| And vice versa. I really wish the Linux folk would educate
| themselves a bit more about what is actually available on
| FreeBSD.
| tptacek wrote:
| Why would they? What difference would it make to them?
| mnd999 wrote:
| Well, they might realise their use case it's better suited
| to FreeBSD. But probably not better-enough to switch OS.
|
| But mainly they might make make more educated comments.
| Sometimes, and maybe this is more a historic thing, the
| Linux advocates can get a bit carried away.
| adr1an wrote:
| Who are you referring to? Linux or FreeBsd people? I guess
| both should look to each others'. But it's common to see
| this in FLOSS, right? Goes beyond namespaces/ isolation.
| tptacek wrote:
| I don't mean to sound snide, I'm just saying: if your
| stack is Linux and the container ecosystem (for instance,
| if your stack is K8s) what would be in it for you to
| investigate Jails?
| mnd999 wrote:
| If your stack is K8s and you actually need K8s then none.
| Most people using K8s don't actually need it and are
| practicing resume driven development.
| tptacek wrote:
| You get that resume-driven development is a real thing,
| right? It's not irrational. Improving your future
| prospects is a very legitimate reason to invest in K8s
| development.
|
| I'm speaking against my own interests here; I don't like
| K8s.
| bojo wrote:
| I think the point was aimed at people that want to debate
| about jails vs. linux namespaces. Not much of a debate if
| you don't understand the strengths and weaknesses of both
| sides.
|
| If they're invested in an existing linux container
| ecosystem and don't intend to change then yeah probably
| not much gained.
| [deleted]
| jmillikin wrote:
| Whenever an article about FreeBSD's jails gets posted here, the
| comments tend to drift toward an argument about jails vs Docker
| containers. People call them "Linux containers", but really
| they're talking about Docker (or its clones like Podman).
|
| This may be me shouting into the void, but I wish there were an
| article directly comparing jails with _namespaces_ , which is the
| Linux functionality that Docker uses. I can totally believe that
| FreeBSD jails provide a better / more unified / more secure
| experience than Docker, but to extend that into saying "FreeBSD
| jails are better than Linux namespaces" feels like a category
| error.
|
| Questions I would like to see answered in that article:
|
| * Can jails be used to run subprocesses in the normal filesystem,
| but with a different network environment (for example making a
| given command run its net traffic through TAP)?
|
| * Can jails be used to limit memory/cpu/IO/network for
| subprocesses? For threads within a process?
|
| * Can live processes be moved into or out of a jail?
|
| * Can jails be used to make a process think it's running as a
| different user?
|
| I feel like the answer to these questions is generally "no,
| that's not what jails are for", which is (1) a fine answer given
| the apparent goal of being a better chroot(), and (2) reinforces
| that jails and namespaces are addressing different problem
| domains.
| throw0101c wrote:
| > _Can jails be used to run subprocesses in the normal
| filesystem, but with a different network environment (for
| example making a given command run its net traffic through
| TAP)?_
|
| Jails are basically like a VM-like/light: it appears like jail
| is its own system. Not quite as heavy as an actual VM under
| (e.g.) Qemu with virtualized hardware. A jail can have its own
| network stack:
|
| * https://klarasystems.com/articles/virtualize-your-network-
| on...
|
| You can then create a 'virtual patch cable' between the host
| and the jail and send one side of the 'cable' to the jail and
| do routing and stuff on the host:
|
| * https://man.freebsd.org/cgi/man.cgi?epair
|
| But a (sub)process cannot be 'sent' to a jail: jails 'boot up'
| like a normal system does, and so you'd have your PID 1 run
| your regular daemon startup.
|
| > _Can jails be used to limit memory /cpu/IO/network for
| subprocesses? For threads within a process?_
|
| Yes:
|
| * https://wiki.freebsd.org/JailResourceLimits
|
| > _Can live processes be moved into or out of a jail?_
|
| No: jails are VM-like in functionality.
|
| > _Can jails be used to make a process think it 's running as a
| different user?_
|
| The jail, being VM-like, would have its own passwd.
| toast0 wrote:
| > But a (sub)process cannot be 'sent' to a jail: jails 'boot
| up' like a normal system does, and so you'd have your PID 1
| run your regular daemon startup.
|
| It's an option to run jails with a whole init and everything;
| then it's like a separate host. But you can also just run
| stuff in the jail, without a whole everything. I currently
| run two daemons in a vnet jail to get a separate network for
| those daemons, but chrooted to /, because they don't need a
| separate filesystem. At my last job, we ran a TLS termination
| proxy chrooted to a very limited directory, because we didn't
| trust OpenSSL after Heartbleed; the chroot had just the
| executable, ld-elf and the libraries it loaded, unix sockets
| to communicate with the origin server, and logfiles, most of
| which was chflags schg.
| heavyset_go wrote:
| > _I currently run two daemons in a vnet jail to get a
| separate network for those daemons, but chrooted to /_
|
| When you do this, do the jails/chroots act like separate
| overlays on top of / to the daemons?
|
| For example, if you use systemd-nspawn or systemd-run, you
| can run daemons in containers that get their own view of /,
| but writes take place in separate overlay file systems.
|
| I'm curious how that compares with what you're doing with
| jails.
| toast0 wrote:
| No, my jailed deamons don't have an overlay, writes
| manifest to the filesystem the same as if they weren't
| jailed (which is good for me). Someone elsewhere
| described a vps they user which did have an overlay
| system, but I haven't used that.
| tptacek wrote:
| You can run Linux containers with their own systemd's
| (obviously, they have their own PID 1). And, of course,
| that doesn't make them VMs.
|
| If you don't trust a component running inside a Jail or a
| Linux container, you shouldn't be running it there; the
| kernel attack surface is big. At that point you've adopted
| the same security model as a phone jailbreaker has (except
| you don't get to dedicate hardware to the anti-jailbreaking
| problem the way the phone vendors do).
| toast0 wrote:
| Well what are you going to do when you have to run
| something you don't trust?
|
| Can't run TLS termination on an isolated host, because
| then the traffic to the origin goes over the network,
| which you also don't trust.
|
| Don't want to run OpenSSL integrated into the daemon,
| because OpenSSL is garbage.
|
| Couldn't run anything else in the immediate aftermath of
| Heartbleed, because GnuTLS was worse, and LibreSSL and
| BoringSSL hadn't been released yet.
| tptacek wrote:
| If you really don't trust it, you run it inside a virtual
| machine.
|
| The truth is, you really do trust OpenSSL. You just trust
| it less than other things. That's fine! Layer controls on
| top of it. In Linux-land, this is the point where you'd
| start thinking about things like seccomp-bpf.
|
| A truly untrusted workload is, like, a compute job you've
| accepted from a SAAS customer; it's arms-length
| multitenancy. You can't share kernels in that situation.
| sophacles wrote:
| Worth pointing out that VM escape is a thing too:
| https://en.wikipedia.org/wiki/Virtual_machine_escape
| tptacek wrote:
| It is. It's a _much_ smaller attack surface.
| tptacek wrote:
| Jails are _not_ VM-like. The defining attribute of a VM is
| that it runs its own kernel. FreeBSD has a virtualization
| system: it 's called bhyve.
| fragmede wrote:
| > The defining attribute of a VM is that it runs its own
| kernel.
|
| _of a VM_.
|
| The difference between "VM" and "VM-like" is the trailing
| modifier "-like". This means something is similar, but not
| the same thing. If it looks like a duck, but is made of
| rubber, we call it a "rubber ducky." If it looks like a
| duck, is wearing a blue shirt, has a speech impediment, and
| is not wearing any pants, we call it Donald Duck.
|
| Thus the fact that jails, and docker _look like_ VMs
| because they have their own PID 1, their own file system,
| their own slice of memory /cpu/IO/network; that is why
| people are, accurately, using the descriptor "VM-like" to
| describe them. People understand that docker is not a true
| virtual machine because it's not running its own kernel. In
| cases where it matters, pedantry between true VM vs fake
| VM, or VM-like (for eg security isolation between VM and
| docker container) is crucial, but most discussions where
| "VM-like" is used to help people understand OpenVZ, LXC,
| jails, docker, cgroups, etc aren't focused on the
| possibility of an RCE in the container escaping the
| container, but helping people understand what a container
| even is in the first place.
|
| Or to put it another way, strcmp("VM", "VM-like") != 0.
| heavyset_go wrote:
| I believe I recall there being a period quite a while ago
| when things like LXC and OpenVZ were described/marketed as
| being "VM-like" around the same time that jails were
| popularized.
|
| It feels like jails are still stuck with being described
| using older terms even after Linux containers and the
| distinction between them and VMs were mainstreamed.
| tptacek wrote:
| I don't know about any of that. LXC containers are also
| not VM-like.
| the8472 wrote:
| Calling "VM-like" is not helpful because containers have also
| been called that, it also doesn't explain anything. Is it
| running a separate kernal image? Does it do hypercalls to the
| host kernel? If not then I wouldn't call it VM-like.
|
| The low-level details and fiddly knobs matter, not how it
| looks like from userspace when all is done.
|
| Based on the jail(8) manpage I get the impression that jails
| are more like all the namespace and cgroup things rolled into
| a single entity. Which I guess makes it more difficult to use
| them incorrectly. But it also prevents other uses such as the
| one that jmillikin has in mind.
| 0x457 wrote:
| It's called VM-like because:
|
| - Often boots user land like a VM (i.e. PID 1 is not the
| process you want to run, but whatever metal or VM freebsd
| runs as PID 1)
|
| - Normally has its own network stack (VIMAGE)
|
| - Normally runs a bunch of background services like a
| regular VM or on-metal would.
|
| - Normally has the entire copy of user-land
|
| - I've never seen short-lived jails either - you make a
| base dataset, clone it, but after that, you just run
| `freebsd-update` like you would in VM.
|
| The majority of jail users treat them like lightweight VMs,
| only difference between VM and Jail from consumer
| standpoint: shared kernel, access to a subset of host's FS
| (sharing host FS to VMs in freebsd is not as easy as it is
| on linux).
|
| To be clear, it's possible to use Jails like Docker, there
| is just no good tooling to do it. People would yell that
| whatever we have for jails is all we need with foam from
| their mouths every time you mention it.
| the8472 wrote:
| That's just saying how people tend to use it, not what
| the technology does. Namespaces/Cgroups can be used both
| ways, to isolate a single process (or even partially-
| isolate individual threads) or entire userland stacks.
|
| The question that was asked what the technology can do,
| not how it's being used. What are the primitives, the
| atoms its made of.
| fragmede wrote:
| > That's just saying how people tend to use it
|
| People tend to use words that describe how they use
| things to describe what they are.
| righttoolforjob wrote:
| Well, take a car as an example.
|
| The question is not what you use a car for, but rather
| how the car is built.
| 0x457 wrote:
| I think you need to look into usage of "-like" suffix.
| 0x457 wrote:
| > But a (sub)process cannot be 'sent' to a jail: jails 'boot
| up' like a normal system does, and so you'd have your PID 1
| run your regular daemon startup.
|
| Technically, you don't have to. It's just because jails don't
| have convenient tooling around them (like docker or podman)
| it's easier to just boot it up like a normal system.
| heavyset_go wrote:
| Hate to nitpick, but a comparison to bare namespaces wouldn't
| be apt, either, as Docker et al use more features than just
| namespaces in their container implementations, like control
| groups, union/overlay/snapshot file systems, nftables, etc.
|
| For several of those questions about jails, the answer would be
| the same if you were just evaluating the questions against
| namespaces by itself.
| vermaden wrote:
| First of all - the Namespaces and Cgroups on Linux are not
| about security. They are about features.
|
| FreeBSD Jails provide both secure isolation from the host and
| also provide the separated network namespace and resources with
| rctl(8) when needed.
|
| To have Linux containers separated and secured you need
| additional layer for that - like SELinux or AppArmor. Only then
| Docker/Podman/other/... Linux containers are isolated and
| secure.
|
| Regards.
| icedchai wrote:
| You're telling those namespace / cgroup features have nothing
| to do with security? Not in any way, shape, or form? I'm a
| little skeptical.
| craftkiller wrote:
| > Can live processes be moved into or out of a jail?
|
| That might be possible on DragonflyBSD. They have support for
| freezing a running process to disk and restoring the process
| again (potentially on another machine) [1], and they have
| support for jails, so I don't see why you wouldn't be able to
| freeze a running process on the host and restoring it in the
| jail or vice versa.
|
| [1] https://www.dragonflybsd.org/features/#index9h2
| toast0 wrote:
| > * Can jails be used to run subprocesses in the normal
| filesystem, but with a different network environment (for
| example making a given command run its net traffic through
| TAP)?
|
| Yes, you can run a vnet jail chrooted to /; same filesystem as
| the host, but a separate network system (you have to setup the
| network for the jail at this point; but I imagine the jail
| tools help with that). This is a fine use for jails. I
| currently run a vnet jail chrooted to / to do some crazy
| network stuff, but I just need a separate network, no other
| separation.
|
| > * Can jails be used to limit memory/cpu/IO/network for
| subprocesses? For threads within a process?
|
| I haven't used it, but it looks like yes/maybe with the rctl
| subsystem. This allows limits on lots of things, include memory
| of several types, cpu of a few types, filesystem io in bytes
| per second and operations per second, number of threads. Valid
| subjects for resource limits are process, user, loginclass and
| jail. Based on the manual, this doesn't let you limit network
| as you wanted, or place limits on threads within a process like
| you wanted. If you run a vnet jail, you can potentially set
| networking limits in other ways, but only if you pass virtual
| interfaces (such as epair, or taptun) to the jail rather than
| physical interfaces. This is a reasonable use for jails, but
| you might not need a jail for this?
|
| > * Can live processes be moved into or out of a jail?
|
| A process can move itself into a jail, but only if it's
| superuser. This is generally used administratively. Jailed
| processes can not be moved out of the jail, afaik; when the
| jail is destroyed, all processes within are killed. This is not
| within the design scope of jails.
|
| > * Can jails be used to make a process think it's running as a
| different user?
|
| I don't think so, but I'd use something with LD_PRELOAD to
| override getuid/geteuid for this; but I think I must not
| understand the question. You can certainly have entirely
| different users inside jail vs on the host?
| assimpleaspossi wrote:
| The article is about how to set up jails on FreeBSD. The
| article has nothing to do with anything you are talking or
| asking about.
___________________________________________________________________
(page generated 2023-05-26 23:01 UTC)