[HN Gopher] Podman: A Daemonless Container Engine
___________________________________________________________________
Podman: A Daemonless Container Engine
Author : lobo_tuerto
Score : 244 points
Date : 2021-02-11 14:06 UTC (8 hours ago)
(HTM) web link (podman.io)
(TXT) w3m dump (podman.io)
| yobert wrote:
| I've been trying to use podman in production and it is not
| working very well. I'm excited for this technology but it is not
| ready.
|
| - podman layer caching during builds do not work. When we
| switched back to docker-ce, builds went from 45 minutes to 3
| minutes with no changes to our Dockerfile
|
| - fuse-overlayfs sits at 100% CPU all day, every day on our
| production servers
|
| - podman loses track of containers. Sometimes they are running
| but don't show in podman ps
|
| - sometimes podman just gets stuck and can't prune or stop a
| container
|
| (edit: formatting)
| windexh8er wrote:
| I could very well be wrong but Podman seems to have missed the
| time-frame of opportunity. It was always a knife fight between
| Red Hat and Docker with regard to tooling. Red Hat wanting to
| own the toolchain for containers so they didn't have to deal
| with, and so they could box out, competitors like (now
| basically defunct) Docker Enterprise.
|
| I've taken a look at podman from time to time over the years
| but it seems like it's just never formalized, never been
| polished and almost always has been sub-par in execution. On
| this list the builds and container control are things that I've
| run across. I guess - what's the point? The rootless argument
| leaned on so heavily is pretty much gone, the quality of Podman
| hasn't (seemingly) improved and now IBM owns Red Hat
| (subjective, but a viable concern/consideration given what's
| recently happened with CentOS).
|
| You're more than safe leveraging Docker and buildkit (when and
| where needed). Quite honestly, given the relatively poor
| execution of Red Hat with these tools over the years, I don't
| see the point. I'm sure there are some niche use cases for
| Podman/Buildah, but overall it just seems to come up as an
| agenda more than an exponentially better product at this point.
| Red Hat could have made things better, instead they just
| created a distraction and worked against the broader effort in
| the container ecosystem.
| yobert wrote:
| Reading through the comments though it looks like I should try
| making a BTRFS mount for the container storage and that might
| help the fuse-overlayfs problems though.
| znpy wrote:
| It feels weird that a containerisation technology imposes
| requirements on the underlying storage technology to work
| well, tho.
|
| Now I understand that btrfs is not really a requirement.
|
| But I've been thinking about using podman to replace docker-
| in-docker on our fleet of gitlab runners where we build our
| images, and running btrfs is a deal breaker. We really don't
| want to add complexity to the mix. Fuse-overlay burning
| through the whole CPU is not really good looking either. Mh.
|
| I still have to properly dig deep into this, but I'm keeping
| my hopes high.
| tw04 wrote:
| I know it tends to be a controversial technology around here,
| but I've had great luck with OpenZFS for container storage.
| dastx wrote:
| I've been running multiple services through podman on a pi
| without any issues. Certainly not the same as production
| though.
|
| Layer caching seems to work for me. Note that rootless stores
| images (and everything else), in different place from rootfull.
| It may be that you're caching the wrong directory.
| nednar wrote:
| What are the Github issue IDs for these 4 problems?
| bartvk wrote:
| What are you trying to say here? Your comment feels a bit
| passive-aggressive.
| nednar wrote:
| > What are you trying to say here?
|
| I want to look at the problems. But I don't want to do a
| guessing game with the 159 open issues, which one may
| relate to one of these points or not.
| qbasic_forever wrote:
| Where are the four unit tests in podman's codebase that
| should be catching these issues?
|
| I find it very strange that your reaction is, "the software
| is infallible, it is _YOU_ who have failed the software by
| not logging bugs!" It is perfectly reasonable to have a
| conversation about the quality or issues with a piece of
| software SEPARATE TO the logging or finding of bugs in said
| software.
| corty wrote:
| Even less IPv6 support than docker. With docker you can at least
| get it to work somehow, even if it is totally different from
| IPv4, weirdly. Podman just has no IPv6 support to speak of.
| 3np wrote:
| For airgapped networks, this is great.
| flowless wrote:
| Not entirely true.
|
| I was researching this few hours ago and according to
| https://github.com/containers/podman/issues/6114#issuecommen...
| it just works when you add another network.
|
| Docker registry having no IPv6 is another fun story tho.
| corty wrote:
| Well, from that issue from 2020-12-24: "--ipv6 has not landed
| yet", so you cannot assign static external v6 adresses to a
| container. No activity on that bug since. There is tons of
| stuff in docker that works automatically, where in podman it
| is like "Please be aware you have to configure ip6tables (or
| whatever your OS firewall is) for forwarding.". Yeah, well,
| if i have to do everything by hand or program a ton of
| scripts around it, it is quite useless, isn't it?
|
| IPv6 support is always like this: half-baked at best, someone
| got it to work somehow, developers declare it done since it
| worked for someone. Then crickets...
|
| IPv6 support isn't done until you can replace IPv4 with it,
| with no changes to your config except addresses. Even docker
| isn't there yet. And podman's is still in a larval state.
| Datenstrom wrote:
| Glad to see Podman here, worked with it at a past job where
| services were deployed on RedHat. Converting all existing
| deployment scripts to podman was as easy as do a recursive search
| and replace `s/docker/podman/` and there was no learning curve as
| all the commands were the same. By default root is not required
| which is great also.
| fillest wrote:
| Well, "daemonless" is kind of marketing - there is still this
| daemon-per-container 'conmon' thing
| https://github.com/containers/conmon and I don't get why it is
| needed because 1) who actually needs to re-attach anyway? 2)
| container's streams are already properly handled by whatever
| supervisor (e.g. systemd). You can't disable conmon and I'm not
| sure if its usage is not hardcoded throughout the codebase.
|
| I would very much like to use Podman as a finally proper
| container launcher in production (non-FAANG scale - at which you
| maybe start to need k8s), but having an unnecessary daemon moving
| part in thousands lines of C makes me frown so far.
| ddtaylor wrote:
| I liked podman for what I was working on with it. The only hangup
| I found was that it couldn't do networking related routing? Does
| anyone know more?
| dathinab wrote:
| As far as I know it can do so but not when running rootless, I
| also don't thing it's currently possible to be done rootless.
|
| You still can run podman as root without damon.
|
| Or you can run a podman deamon (through systemd) which is even
| compatible with docker-compose but has most of the drawbacks of
| running docker.
| Xophmeister wrote:
| Isn't Singularity[1] also daemonless?
|
| [1] https://sylabs.io/
| SEJeff wrote:
| Yes, but singularity is a gigantic suid root binary where as
| podman uses user namespaces for unprivileged stuff.
| geerlingguy wrote:
| What's the status on ease of running on Mac? I know last time I
| seriously considered testing it out in my workflow, it was kind
| of crazy that I had to have a separate machine (VM or real) just
| to run container images...
|
| I see in the docs [1]:
|
| "Podman is a tool for running Linux containers. You can do this
| from a MacOS desktop as long as you have access to a linux box
| either running inside of a VM on the host, or available via the
| network. You need to install the remote client and then setup ssh
| connection information."
|
| And yes, I know that Docker (and any other Linux container-based
| tool) also runs a VM--but it sets it up almost 100% transparently
| for me. I install docker, I run `docker run` and I have a
| container running.
|
| [1] https://podman.io/getting-started/installation#macos
| quadrifoliate wrote:
| Yeah without a transparent workflow like that for Mac, there is
| no reason for me to switch to this project.
|
| I wish they had tried to collaborate with Docker and contribute
| upstream instead of this project.
| eadmund wrote:
| > I know last time I seriously considered testing it out in my
| workflow, it was kind of crazy that I had to have a separate
| machine (VM or real) just to run container images...
|
| Why does that seem 'kind of crazy' to you? A container is
| really just a namespaced Linux process, so ... you either need
| a running Linux kernel or a good-enough emulation thereof.
|
| What seems kind of crazy to me is that so many folks who are
| deploying on Linux develop on macOS. That's not a dig against
| macOS, but one against the inevitable pains which arise when
| developing in one environment and deploying in another. Even
| though I _much_ prefer a Linux environment, it would seem
| similarly crazy to develop on Linux and deploy on Windows or
| macOS. Heck, I think it is kind of crazy to develop on Ubuntu
| and deploy on Debian, and they are generally very very close!
| oauea wrote:
| > And yes, I know that Docker (and any other Linux container-
| based tool) also runs a VM--but it sets it up almost 100%
| transparently for me. I install docker, I run `docker run` and
| I have a container running.
|
| Comments like this help reinforce the stereotype of lazy macos
| developers ;)
| geerlingguy wrote:
| It doesn't matter what platform I'm on, I like tools that let
| me be lazy ;)
| throwaway894345 wrote:
| The docker machine or whatever it's called these days is pretty
| impressive but when we've tried to use it in developer
| workflows on mac it would eat all of the CPU allotted to it in
| order to marshal filesystem events over the host/guest VM
| boundary. We tried a number of workarounds but none solved the
| problem. I'm sure it's possible, but we ultimately gave up and
| moved back to developing against native processes on MacOS.
|
| For other use cases I imagine it's just fine.
| dzolvd wrote:
| this is an issue I have battled as well. Docker is great for
| normalizing the execution environment; which should be a huge
| boon for developer tooling but on MacOS having your laptop
| sound like a hovercraft just to have some files watched for
| hot rebuilds is no bueno.
| zerkten wrote:
| It seems like they haven't gotten to this, don't understand the
| needs and the potential, or aren't doing this intentionally. My
| understanding is that Docker maintains what amounts to a
| wrapper VM with nice integration to enable the workflow. This
| doesn't exist for Podman as far as I've seen.
|
| On Windows, the WSL2 feature gives you that easy wrapper in a
| different way. It's sets up and manages the underlying VM
| although you have to choose your distro etc. Once this is
| running after a simple setup you are just using Linux from that
| point on. It's less specialized than how Docker on macOS seems
| to work.
|
| If someone knows of something that follows the WSL2 approach
| without VirtualBox or VMware Fusion I'd be all ears. That would
| be more versatile than how Docker seems to work right now.
| Docker's business interests probably aligned well with getting
| a workflow running, so unless someone is motivated to do
| similar for Podman, you are going to be out of luck. At least
| recognizing this deficiency would be a start though.
| jjtheblunt wrote:
| > And yes, I know that Docker (and any other Linux container-
| based tool) also runs a VM--but it sets it up almost 100%
| transparently for me. I install docker, I run `docker run` and
| I have a container running.
|
| That's exactly not true, considering you said Linux.
|
| The utility of Linux containers is that they share the same OS
| instance, but have an embellished notion of Unix process group
| where, within a container i.e. embellished process group, they
| see their own filesystem and numbering for OS resources, AS IF
| they were on individual VMs, but they're not.
| daitangio wrote:
| Happy to see docker alternative, but the road is still long for
| it to replace docker
| long_warmup wrote:
| I use podman in my projects, no problems. And since cli is
| compatible with docker all my work I have done using docker
| simply runs with podman.
| dathinab wrote:
| I don't think so.
|
| It's doing a better job for most of the use cases I had
| recently.
|
| You can run it rootless which makes it a far better use cases
| for a lot of dev use cases (where you don't need to emulate a
| lot of network aspects or heavy filesystem usage).
|
| Also you can run it as root without deamon which is much better
| for a bunch of use cases and eliminates any problems with fuse-
| overlayfs. This is the best way to e.g. containerize systemd
| services and similar (and security wise still better as e.g. it
| still works well with the audit subsystem).
|
| If worse comes to worse you can run it as a deamon in which
| case it's compatible with docker compose, and has more or less
| the same problems.
|
| In my experience besides some integration testing use-cases is
| either already a good docker replacement without running it as
| a deamon, or the use case shouldn't be handled by docker either
| way (sure there are always some exceptions).
|
| Lastly I had far less problems with podman and firewalls then
| with docker but I haven't looked into why that's the case. (But
| it seems to be related to me using nftables instead of
| iptables).
| curt15 wrote:
| The ability to start containers as a normal user ("rootless
| mode") is quite interesting and imo the most significant feature
| missing from LXD, which requires you to either use sudo or be a
| part of the "lxc" group (equivalent to having sudo privileges).
| hda111 wrote:
| You can afair configure LXD to disallow escalating privileges
| via group membership.
| mixmastamyk wrote:
| Wouldn't the lxc group have much more focused permissions than
| sudo?
| symlinkk wrote:
| Who cares? It's one command to start the daemon.
| williesleg wrote:
| Yay IBM!
| op00to wrote:
| Every complaint here boils down to: "it's not docker, it doesn't
| work exactly like docker, therefore it's bad".
| jeffbee wrote:
| Which is a remarkable position considering how bad docker is.
| You'd think that not being docker would be a feature by itself.
| shyim wrote:
| I still miss an easy way to setup multiple containers in a single
| network like with docker-compose. podman-compose is not really
| useable.
| dathinab wrote:
| You can use docker-compose with podman.
|
| Through it has similar drawbacks as using docker-compose with
| docker.
|
| As far as I know there is currently no rootless way on Linux to
| setup custom internal networks like docker compose does (hence
| why podman-compose doesn't support such things).
| chippiewill wrote:
| There definitely is a way as creating custom networks works
| perfectly fine with rootless docker. I'm surprised if podman
| doesn't support that since it uses rootlesskit (same as
| rootless docker) which does the heavy lifting for rootless
| networking.
| pydry wrote:
| What's wrong with it?
| dathinab wrote:
| With docker all container have their own ip address and
| (potential) internal DNS name.
|
| But this can't be done rootless.
|
| So with rootless podman all container map to the same ip
| address but different ports.
|
| This is for some use cases (e.g. spinning up a DB for
| integration testing) not a problem at all. For others it is.
|
| More over you can run multiple groups of docker containers in
| separate networks, you can't do so with rootless podman.
|
| Through you can manage networks with rootfull podman (which
| still has no deamon and as such works better with e.g.
| capabilities and the audit sub system then docker does).
|
| Through to get the full docker-compose experience you need to
| run it as a deamon (through systemd) in which case you can
| use docker-compose with podman but it has most of the
| problems docker has.
| Spivak wrote:
| An Ansible playbook with the podman_container module is my go-
| to. It's not as easy because it's less magic but it's also less
| magic which I count as a win.
| SEJeff wrote:
| podman 3.x natively supports docker-compose, but it requires
| running a podman daemon and has the same drawbacks as docker:
|
| https://www.redhat.com/sysadmin/podman-docker-compose
|
| That says, you literally can use the same docker-compose app
| shipped from docker and it works using podman.
| akvadrako wrote:
| You can also use pod files for this, which is nice in that it's
| easy to migrate to k8s.
| Cu3PO42 wrote:
| I have recently found out you can actually run Docker containers
| with LXC as well. I've yet to get deep into it, but it seems to
| work fine*.
|
| * Manual config tweaking may be required.
| linkdd wrote:
| IIRC, Docker's first versions were only a high-level API to
| LXC. containerd/runc/... came after.
| maweki wrote:
| One very interesting piece of tech coming from it, is toolbox
| (https://github.com/containers/toolbox). Basically throwaway (or
| keeparound) rootless containers with their own root directory but
| shared HOME. Install hundreds of dev-dependencies to build this
| one piece of software? Yeah, not gonna install those packages
| permanently. Spin up a toolbox, build it, install it in my
| home/.local.
|
| You have root in the container without having root in the host
| system. That takes care of a lot of issues as well.
| johnchristopher wrote:
| I usually share a volume between containers (eg:: a volume for
| wp-cli cache, another for -g npm_modules).
|
| What benefits would toolbox add ?
| westurner wrote:
| Is the title of this page out of date?
|
| AFAIU, Podman v3 has a docker-compose compatible socket and
| there's a daemon; so "Daemonless Container Engine" is no longer
| accurate.
|
| "Using Podman and Docker Compose"
| https://podman.io/blogs/2021/01/11/podman-compose.html
| lifeisstillgood wrote:
| I redid some docker builds yesterday with podman, and filled up
| my laptops 99GB entirely with layers. Docker was handling it
| somehow better.
|
| I used --squash, cleaned up the various apt-gets but boy it made
| me realise that for every abstraction we get leakage - sometimes
| serious leakage
| yamrzou wrote:
| I you'd like to know what it is like to use Podman, I've found
| those terminal recordings by Matthew Heon (Podman contributor)
| quite helpful: https://asciinema.org/~mheon
| SEJeff wrote:
| Under the covers, both podman AND docker use runc. Redhat is
| writing a new version named "crun" which is lower overhead and
| faster:
|
| https://github.com/containers/crun
| Ciantic wrote:
| The podman targets the crun too:
|
| > Runtime: We use the OCI runtime tools to generate OCI runtime
| configurations that can be used with any OCI-compliant runtime,
| like crun and runc.
| lrossi wrote:
| Upside: At least they used a different name.
|
| On other occasions, they just hijack the name. Like they did
| with dstat.
| dralley wrote:
| That was a misunderstanding where they saw that there had
| been no commit activity for 3 years, and no responses to
| issues/PRs filed in the past 18 months, and assumed the
| upstream was dead.
|
| Making that assumption was reasonable - moving forwards
| without at least making an attempt to contact the maintainer
| "just in case" was not. It would have been courteous to at
| least try.
|
| But it's not exactly the deliberate hostile takeover that you
| make it out to be.
|
| https://bugzilla.redhat.com/show_bug.cgi?id=1614277#c9
|
| >> To my knowledge, there is no need (legally) to obtain
| consent to use the name 'dstat' for a replacement command
| providing the same functionality. It might be a nice thing to
| do from a community perspective, however - if there was
| someone to discuss with upstream.
|
| >> However, dstat is dead upstream. There have been no
| updates for years, no responses at all to any bug reports in
| the months I've been following the github repo now, and
| certainly no attempt to begin undertaking a python3 port.
|
| >> Since there is nobody maintaining the original dstat code
| anymore, it seemed a futile exercise to me so I've not
| attempted to contact the original author. And as pcp-dstat is
| now well advanced beyond the original dstat - implementing
| features listed in dstat's roadmap for many years, and with
| multiple active contributors - I think moving on with the
| backward-compatible name symlink is the best we can do.
| lima wrote:
| Great idea - let's rewrite the most security-critical piece of
| container tech in plain C!
|
| Red Hat literally took a memory safe Go program and rewrote it
| in C for performance, in 2020.
| coder543 wrote:
| Yeah. I just don't think "runc" has ever been a bottleneck in
| any systems I've worked with.
|
| It's been years since I've been able to get excited about
| rewriting systems from a memory safe language into a memory
| unsafe language for "performance reasons". As an industry, we
| have too much evidence that even the best humans make
| mistakes. Quality C codebases still consistently run into
| CVEs that a memory safe language would have prevented.
|
| Rust exists, if performance is so critical here and if Go
| were somehow at fault... but it sounds like most of the
| performance difference is due to a different architecture, so
| rewriting runc in Go probably could have had the same
| outcome.
|
| I wish I could be excited about crun... I'm sure the author
| has put a lot of effort into it. Instead, I'm more excited
| about things like Firecracker or gVisor that enable people to
| further isolate containers, even though such a thing
| naturally comes with some performance impact.
| SEJeff wrote:
| "Any of the systems you've worked with"
|
| I suspect it has been a bottleneck in some of the systems
| redhat has worked with, or they would have no benefit in
| writing it. As the authors of openshift I'm sure they've
| seen some pretty crazy use cases. Just because it isn't
| your use case doesn't make it invalid.
| [deleted]
| lima wrote:
| > _I wish I could be excited about crun... I'm sure the
| author has put a lot of effort into it. Instead, I'm more
| excited about things like Firecracker that attempt to
| enable people to further isolate containers, even though
| such a thing naturally comes with some performance impact._
|
| gVisor is also very exciting due to how it handles memory
| allocation and scheduling. Syscalls and I/O are more
| expensive, though.
| coder543 wrote:
| Heh, I actually edited gVisor in right before your reply.
|
| I agree completely!
| zapita wrote:
| It's not for performance. It's to settle an old grudge with
| Docker. All code coming from Docker must be erased.
|
| See also: podman; cri-o; buildah.
| SEJeff wrote:
| Grudge? You mean when Jessie Frazelle walked around
| dockercon with a badge saying, "I will not merge patches
| for systemd" just out of principal? In hindsight it is
| fairly obvious picking a fight at redhat summit was a very
| poor long term strategy from docker. It forced them to
| ultimately sell the orchestration parts of their business
| and massively limit the free tier of docker hub.
| zapita wrote:
| Thank you for illustrating my point perfectly.
|
| "Grudge?" _(proceeds with details of grudge)._
| kbenson wrote:
| What was described could be the basis of a grudge, or it
| could be that Red Hat were convinced that the current
| project would not accept changes Red Hat would benefit by
| even if it helped many other people. At that point, and
| at the size of Red Hat and how important this technology
| is becoming for them, it only makes sense for them to
| secure some of the technological foundations they rely
| on. It costs them very little in resources in relation to
| the business they are basing on that technology.
|
| I don't think it's a stretch to look at the events as
| described and see them as Red Hat doing what they need to
| work around someone else's grudge. It could be any number
| of things, but I don't think the information presented so
| far lends itself towards Red Hat having a grudge.
| zapita wrote:
| runc, containerd and the docker client are critical
| infrastructure to a whole lot of people, and they seem to
| be doing just fine running it and, when needed,
| contributing back. Only Red Hat seems to have a problem
| with using those projects without forking or re-writing
| them. Could it be because Red Hat considers it of
| strategic importance for their business, not just for
| those projects to be reliable (they are), but to be
| _controlled_ by them to a sufficient degree that they can
| act as gatekeepers to upstream, thus preserving their
| business model?
| kbenson wrote:
| That could be, but if Frazelle did actually walk around
| with a button saying "I will not merge patches for
| systemd" I think it's simpler and safer to assume Red Hat
| decided that relying on Docker was risky given how
| important it was to their future products and the
| apparent animus on display from the Docker maintainer.
|
| If I carpool with someone and they like to go around and
| loudly proclaim "I refuse to listen to anything kbenson
| has to say", maybe I find someone else to carpool with or
| drive myself? That only seems prudent. That doesn't mean
| I have a grudge against that person, it just means I
| don't want to trust them with getting me to and from work
| anymore.
| zapita wrote:
| It's no secret that there were tensions betwen Docker and
| Red Hat that culminated around 2016, when Red Hat's
| desire to capitalize on the success of containers ran
| into Docker's own ambitions to compete with Red Hat.
| Those tensions were eventually resolved (or at least
| diminished) by Red Hat shifting gears to Kubernetes as
| their "next Linux".
|
| The drama you're talking about is from that period -
| 2015-16. But "crun" was launched in 2020, a full four
| years later. Frazelle left Docker years ago, as well as
| most of the people involved in those feuds. Runc,
| containerd and docker are chugging along drama-free, and
| are maintained by multi-vendor teams. There is no
| interest from anyone outside of Red Hat in forking or re-
| writing those tools. It's just a tremendous waste of
| energy and in the end will be problematic for Red Hat,
| because they will be building their platform on less used
| and therefore less reliable code.
|
| The world's container infrastructure runs on containerd,
| runc and docker. Not crio, crun and podman. Someone at
| Red Hat clearly is having trouble accepting that reality,
| in my opinion because they're having trouble moving on
| from old grudges. I really wish they did, because all of
| that wasted engineering effort could be deployed to solve
| more pressing problems.
| kbenson wrote:
| > It's just a tremendous waste of energy
|
| > The world's container infrastructure runs on
| containerd, runc and docker
|
| I don't consider alternate implementations of widely used
| software to be wasted evergy almost ever. Was Clang a
| waste of energy because everyone was using GCC? There are
| many reasons why multiple implementations may be useful,
| competition and being able to cater to slightly different
| common use cases obvious ones.
|
| I'm not sure why any end user would wish for there not to
| be multiple open source projects looking to satisfy a
| similar technical need.
|
| > in the end will be problematic for Red Hat
|
| That may be, but I don't really care about whether it's
| good for Red Hat. I care that it increased user choice
| and maybe at some point features and capabilities offered
| to users (whether through podman or pressure on docker).
| SEJeff wrote:
| runc and containerd wouldn't exist if Redhat hadn't
| created the OCI specification and work around docker to
| make things interoperable. Docker, Inc was not playing
| ball and then redhat had to work around them. It wasn't a
| grudge it was just them keeping their business safe. The
| Open Containers Initiative was essentially designed to
| force Docker Inc to try to play better with others. They
| donated containerd to the OCI due to the reality that
| docker was about to be left in the dust.
|
| For reference, this was the shirt that Docker Inc was
| giving away at Redhat summit years ago. I was there. I
| saw the shirt, and thought it was in very poor form.
| https://twitter.com/SEJeff/status/1125871424126767104
|
| As Rambo said, "But Colonel, they drew first blood."
| zapita wrote:
| Your history is wrong. Docker created the OCI, not Red
| Hat. Docker donated both the spec and the runc
| implementation.
|
| Containerd was donated (also by Docker) to CNCF, a
| different organization, and I believe a few years later.
|
| You are correct that Docker did those things because of
| pressure to be more interoperable.
|
| Reading your linked tweet ("don't mess with an
| engineering behemoth") you seem to agree that Red Hat is
| rewriting away Docker code because of a past grudge?
| hda111 wrote:
| So why does podman include code from docker?
| zapita wrote:
| Excellent question, especially since the code is copy-
| pasted without crediting the original authors. Could it
| be that they're trying to minimize the role of Docker as
| much as possible even when re-using their code?
| tpush wrote:
| > Excellent question, especially since the code is copy-
| pasted without crediting the original authors.
|
| That seems like a serious allegation. Can you back that
| up?
| zapita wrote:
| It's right there in the repo history. Start from the
| first commits.
| 8fingerlouie wrote:
| Contrary to popular belief, memory safe C is possible to
| write, and with modern linters it's even easier.
|
| Yes it's easier to shoot yourself in the foot, but with
| practice you can learn to miss every time.
| tshtf wrote:
| It probably isn't worth trying.
|
| djb wrote remotely exploitable C for qmail, applications
| written by the OpenBSD team have had exploitable memory
| issues, Microsoft still doesn't get it right, neither do
| Linux kernel devs.
| CivBase wrote:
| You say that as if the likes of Microsoft or Linux kernel
| devs consist exclusively of elite C devs.
|
| I'm very glad languages like Go and Rust exist, but
| saying you shouldn't write C because you might create
| memory leaks is kind of like saying you shouldn't write
| multi-threaded code because you might create race
| conditions. Yeah, it adds complexity to your code, but
| it's sometimes worth the overhead it saves. Whether that
| trade-off is worth it is always up for debate.
| tshtf wrote:
| Oh, I agree it is a tradeoff. But the parent said "you
| can learn to miss every time". Can anyone point me to a C
| language project where the developers consistently miss
| every time?
| 8fingerlouie wrote:
| It's possible with rules, experience and guidance.
|
| Granted, I learned to write C 25+ years ago, and have
| worked as a C programmer for 15 of them, writing mostly
| embedded software for mobile phones (pre smartphone),
| airport sorters, but have also written financial software
| (mostly for parsing NASDAQ feeds), but the point is that
| most of the software I've written has had close to a
| decade of "runtime", and while I started out making the
| same mistakes as everybody else, you learn to put ranges
| on your heap pointers like strncpy instead of just
| blindly doing strcpy.
|
| Checking the size of your input vs your allocated memory
| takes care of a lot of it. As for memory leaks, it's not
| exactly hard to free a pointer for every time you malloc
| one.
|
| People are terrified of C, and yes, Go, Rust, Java, C#
| makes it harder to make _those_ mistakes, but that
| doesn't mean it's impossible to write good C code.
|
| And it's not like projects written in Go or Rust are
| error free. They just struggle with different errors.
|
| As for good C projects, check stuff like postfix, nginx,
| and yes Linux or FreeBSD.
| CivBase wrote:
| Are you saying that every project written in C has
| suffered from a memory leak issue at some point? Most
| C/C++ code I've personally written doesn't even use the
| heap.
|
| I work on avionics and we use C/C++ now and then. We have
| a ton of rules regarding memory management (pretty much
| everything stays on the stack) and I can't recall
| anything I've ever been involved with suffering from a
| memory leak.
| tshtf wrote:
| All I'm saying is that it's a trivial oversimplification
| to say you can learn to miss every time.
| legulere wrote:
| It's not just the developers. The code gets reviewed and
| merged as well. I don't know which projects but those are
| the highest quality C code based.
|
| Writing in C doesn't create complexity but lots of traps
| to fall in. Writing in C doesn't save overhead over Rust.
| Your rust code can be very low level and safe and C isn't
| that close to hardware as it used to be anyway.
| SEJeff wrote:
| It is almost 50% faster for launching containers. That's
| considerable, and at scale could be an actual cost savings.
| If nothing else, efficiency is good.
|
| Also, small c projects can be written properly with
| discipline. This is a very talented team on a very small and
| well scoped project. It can be written properly.
| jnwatson wrote:
| There's also the copious amount of static analysis and
| warnings built into gcc and clang these days, not to
| mention safer build options. Writing C today is quite a bit
| better than even 10 years ago.
| eeZah7Ux wrote:
| External static analyzers also improved a lot.
|
| You can ensure memory-safety in C by enforcing strict
| rules since the first commit.
| tene wrote:
| Can you share any good examples of C codebases that have
| enforced these rules and successfully avoided having
| memory safety problems? Can you share any details on what
| rules a team could use to reliably avoid having memory
| safety problems when writing C?
| SEJeff wrote:
| Indeed. Pretending it isn't is equally silly.
| lima wrote:
| Better, yes. Safe, no. Even with ASAN and modern C++,
| vulnerabilities happen all the time.
| SEJeff wrote:
| This is C, not C++. Modern C++ is kind of a mess with how
| much stuff they've added to the language. This is not
| relevant.
| adev_ wrote:
| > Modern C++ is kind of a mess with how much stuff
| they've added to the language. This is not relevant.
|
| Yeah and C++ would be a way better language to write
| critical system daemon in 2020 than C. Both safer and
| more productive while keeping the exact same portability
| and performance as C when necessary.
|
| Most safety issues of C (buffer overflows, use-after-
| free, stack smash) are not a problem anymore in modern
| C++.
|
| Yes, writing new userland software in C in 2020(1) is
| non-sense.
|
| - Use at least C++ if you are conservative.
|
| - Use Rust if you aim for absolute safety.
|
| - Use Go if you accept the performance hit and do not
| need libraries.
|
| There is zero excuses to C in userland in 2020.
|
| The only excuse is some Red Hat folks seems to practice
| C++ hating as some kind of religion.
|
| That exactly what give us "beautiful" monstrosities like
| systemd or pulseaudio with their associated shopping list
| of CVEs [^1]
|
| Even GCC maintainers switched to C++, by the sake of god,
| do the same.
|
| ----
|
| [^1]:
| https://www.cvedetails.com/product/38088/Freedesktop-
| Systemd...
| MaxBarraclough wrote:
| The Linux kernel still has memory safety issues to this
| day.
|
| The C++ approach is to offer a feature-rich language so
| the programmer doesn't have to reinvent common
| abstractions. The C approach is to offer a minimal and
| stable language and let the programmer take it from
| there. It's not obvious _a priori_ which approach should
| result in fewer memory safety issues. If I had to guess
| my money would be on C++ being the better choice, as it
| has smart pointers.
| coder543 wrote:
| Saying C is easier to secure than Modern C++ is just
| confusing. The point of a lot of recent C++ features is
| to make memory safety easier to achieve. I personally
| don't think it gets anywhere close to safe enough for me
| to recommend, when Rust is such a competent option that
| people can choose instead. I would still rather people
| have the tools of C++ at their disposal instead of having
| to do everything themselves in C.
|
| Please find me a well-known C code base that doesn't
| suffer from memory safety CVEs. You're the one making the
| claim that they exist... I can't prove that such a thing
| doesn't exist, but I can point to how curl[0][1],
| sqlite[2][3], the linux kernel[4][5], and any other
| popular, respected C code base that I can think of
| suffers from numerous memory safety issues that memory
| safe languages are built to prevent.
|
| Writing secure software is hard enough without picking a
| memory unsafe language as the foundation of that
| software.
|
| "Just find/be a better programmer" isn't the solution.
| We've tried that for decades with little success. The
| continued prevalence of memory safety vulnerability in
| these C code bases shows that existing static analysis
| tools are insufficient for handling the many ways that
| things can go wrong in C.
|
| I think you mentioned railcar elsewhere in this thread,
| and I would find it easier to be interested in a runc
| alternative that is written in a memory safe language...
| which railcar is.
|
| [0]: https://curl.se/docs/CVE-2019-5482.html
|
| [1]: https://curl.se/docs/CVE-2019-3823.html
|
| (among numerous others)
|
| [2]: https://www.darkreading.com/attacks-
| breaches/researchers-sho...
|
| [3]: http://cve.mitre.org/cgi-
| bin/cvekey.cgi?keyword=SQLite
|
| (the second link has quite a few memory safety issues
| that are readily apparent, including several from SQLite
| that affected Chrome)
|
| [4]: https://www.cvedetails.com/cve/CVE-2019-11599/
|
| [5]: https://www.cvedetails.com/cve/CVE-2018-15471/
| coldtea wrote:
| And yet NASA uses C, not Rust.
| speedgoose wrote:
| I don't think you want to develop your software project
| like the NASA develops its critical software projects.
| daenney wrote:
| > This is a very talented team on a very small and well
| scoped project. It can be written properly.
|
| Talent does not preclude you from making mistakes
|
| Most projects start out small and well scoped. That doesn't
| mean it's going to last.
| kragniz wrote:
| Remember that runc contains quite a few lines of C:
|
| https://github.com/opencontainers/runc/blob/4d4d19ce528ac40c.
| ..
|
| https://github.com/opencontainers/runc/blob/4d4d19ce528ac40c.
| ..
| vishvananda wrote:
| Go is actually a really poor choice for the container runtime
| because much of the container setup cannot be done from
| multithreaded code[0], so it has to be done in C before the
| go runtime initializes. I do think rust is a better choice
| for this layer than C because there are still security risks,
| but getting rid of Go for this layer is a win. I'm not sure
| why RH chose to rewrite it in C rather than using rust[1].
|
| [0]: https://www.weave.works/blog/linux-namespaces-and-go-
| don-t-m... [1]: https://github.com/drahnr/railcar
| xyzzy_plugh wrote:
| > Go is actually a really poor choice for the container
| runtime because much of the container setup cannot be done
| from multithreaded code[0]
|
| This was addressed in 2017/2018 [0], it's no longer a poor
| choice.
|
| [0]: https://github.com/golang/go/commit/2595fe7fb6f272f920
| 4ca3ef...
| vishvananda wrote:
| While the particular issue of network namespaces and
| locking the os thread was fixed, there is still c code
| that must run before the go runtime starts to work around
| the issue that you cannot do some of the necessary
| nsenter calls once you have started additional threads.
| The c code to make runc work is encapsulated in
| libcontainer/nsenter[0]
|
| [0]: https://github.com/opencontainers/runc/tree/master/l
| ibcontai...
| [deleted]
| SEJeff wrote:
| This I entirely agree with you on but I'd expect it due to
| the resources available at the time were more fluent with C
| than rust. A shame really.
| Brian_K_White wrote:
| Or they want to be more platform-agnostic than rust and
| llvm.
| coder543 wrote:
| Which platforms does Red Hat support Linux containers on
| that Rust doesn't support? Which platforms is anyone else
| frequently using Linux containers on that Rust doesn't
| support? I'm pretty sure the venn diagram here would be a
| single circle. Rust supports the major architectures.
|
| Portability seems like an unlikely reason here, but I
| would pick memory safety over portability every single
| day of the week. Rust is far from the only language that
| offers memory safety, but it is certainly one of the most
| compelling ones for projects that would otherwise be
| written in C.
|
| "runc" is battle tested, written almost entirely in Go,
| which offers memory safety, and the performance delta
| under discussion is only about 2x for a part of the
| system that seemingly only has a measurable performance
| impact if you're starting containers as quickly as
| possible.
|
| This just isn't a very compelling use case for C.
| Brian_K_White wrote:
| Who knows? Who cares?
|
| If I worked for RedHat I would still seek to write as
| portable as possible just on general principle or for the
| sake of my own future self.
|
| But I agree it's merely a possible reason and may not be
| a major one.
| [deleted]
| [deleted]
| hda111 wrote:
| Most security software is written in C. WireGuard, OpenSSH,
| OpenSSL, OpenVPN, all the Software from OpenBSD folks. Crun
| is being used without root privilege in most cases so it's
| not a huge problem if the runtime is ever escaped.
| xerxes901 wrote:
| I did some hacking on crun this week to fix a bug, so maybe I
| can offer a perspective about why it's written in C.
|
| A huge part of a container runtime is pretty much just
| issuing Linux syscalls - and often the new ones too, that
| don't have any glibc bindings yet and need to be called by
| number. Plus, very careful control of threading is needed,
| and it's not just a case of calling runtime.LockOSThread() in
| go - some of these namespace related syscalls can only be
| called if there is a single thread in the process.
|
| It's _possible_ to do this in go, of course; runc exists
| after all. But it's certainly very convenient to be able to
| flip back and forth between kernel source and the crun code
| base, to use exactly the actual structs that are defined in
| the kernel headers, and to generally be using the lingua
| franca of interfacing with the Linux kernel. It lowers
| cognitive overhead (eg what is the runtime going to do here?)
| and lets you focus on making exactly the calls to the kernel
| that you want.
| gobblegobble2 wrote:
| The most security-critical part of container tech lives in
| Linux kernel and has always been in plain C.
| cashewchoo wrote:
| Adduser vs useradd is older than I am and I still can't keep
| them straight. Just put this with the rest of the fire, I
| guess, heh!
| lrossi wrote:
| Indeed. After 30 years of Linux, we should have got better at
| naming things.
| SEJeff wrote:
| I think it is sort of a play on words. Since runc is written
| in go and is slower, crun is written in c and is faster.
| Stupid nerd joke I know.
| osipov wrote:
| Any plans for Docker to offer a daemonless version?
| acec wrote:
| It is almost 100% compatible with docker. I only had to do
| minimal changes in my Dockerfiles to use it.
| lima wrote:
| But not Buildkit and docker-compose.
| lmeyerov wrote:
| ... but not docker-compose
| rq1 wrote:
| What's the difference with systemd-nspawn? (with or without -U)
| eeZah7Ux wrote:
| ...that nspawn is already available on 90% of Linux systems.
| Klasiaster wrote:
| Podman can run under your user and use your UID (e.g., 1000) as
| UID 0 for the root user in the container. Any other user in the
| container gets mapped to a temporary unprivileged UID on your
| system (the range to pick from is specified in /etc/subuid).
| Brian_K_White wrote:
| 100 years ago I wrote a simple sysv init script that
| started/stopped/queried/enabled/disabled lxc containers, graceful
| host shutdown (all containers shut down gracefully before the
| host shuts down), you could start/stop/query/enable/disable
| individual containers as well as the the meta-service of all
| containers, all from a few k of bash, and no daemon, not even in
| the form of a shell loop that stayed running.
|
| Right about then systemd came along (as in, my distro at the time
| adopted it) and essentially required a service to monitor (some
| process, tcp port, or file to monitor), but my efficient little
| script had no service by design, because none was needed and I
| hate making and running anything that isn't needed to get a given
| job done. (Plus about that time my company went whole-hog into
| vmware and a managed service provider and so my infant home-grown
| lxc containers on colo hardware system never grew past that
| initial tiny poc)
|
| I laugh so many years later reading "Hey! Check it out! You don't
| need no steeenkeeeng daemon to run some containers!"
| superbcarrot wrote:
| Can anyone explain what daemonless means in this case and what
| the advantages are? (I don't work in this space and my knowledge
| ends with knowing what a container is.)
| _joel wrote:
| You don't need root access to run the containers
| 8fingerlouie wrote:
| IIRC, docker by default runs a daemon as root, which spawns all
| your containers. It is possible to run it rootless as well
| (https://docs.docker.com/engine/security/rootless/), though
| you'd need a separate daemon for each user you'd want to run
| containers as.
|
| Podman doesn't have that. It spawns containers without the help
| of a controlling daemon, and can spawn containers both as root
| and rootless.
|
| Rootless is of course a fairly big deal considering if you run
| docker containers as root, and runc has a vulnerability, you
| could potentially escape the container and become root, where a
| rootless installation would just let you escape to whatever
| user is running the container.
| minieggs wrote:
| I have a container I stick in Syncthing that I do 100% of my
| development within across all my machines. One day I decided to
| switch to podman on a whim and forgot I made the switch. That was
| about nine-ish months ago.
| runeks wrote:
| Sincere question: why would I want to use Podman over Docker?
|
| What does being "daemonless" actually buy me in practice?
| otabdeveloper4 wrote:
| podman runs out of the box in normal user mode and doesn't
| require sudo.
| archi42 wrote:
| This. SO MUCH THIS. We use docker for our CI, which is mostly
| okay. But we devs don't have root on our workstations, so we
| can't just load the docker image and test stuff in the CI
| env. Enter podman: I've written a small perl wrapper around
| it and now everyone can just call `ci-chroot` to drop to a
| CI-equivalent shell. They can get virtual root (default) or
| normal user (--user), and the script takes care of everything
| (binding some useful folders, fetching/updating our internal
| image, starting/stopping the actual container and cleaning
| up,...). Only change necessary to our [Manjaro] environment
| was adding podman to the package list and a small bash script
| that generates the `/etc/sub{u,g}id` from our LDAP.
| eeZah7Ux wrote:
| > we devs don't have root on our workstations
|
| What's the justification for this... while at the same time
| allowing the use of containers?
| archi42 wrote:
| With root it's easy to mess up the system badly ("I'll
| just fix that small error") and/or let all systems slowly
| diverge. And if something fails on my workstation, it
| will fail on many others as well and needs a coordinated
| solution. Also, giving every dev root is a security
| liability, especially when there is absolutely no work
| reason that requires us to become root. So only a few
| "core IT" people have root access.
|
| I don't see how that contradicts the use of containers?
| When we only used docker for the CI we didn't have access
| to them, because escalating to root from within docker is
| still pretty easy. But with podman they're running with
| our privileges (and due to gid/uid mapping, I can run our
| installer "as root" in the container and see if it sets
| up everthing correctly - while in reality the process
| runs under my uid).
|
| Disclaimer: This is in a professional enviroment. At home
| I despise containers and prefer to have all my service
| installed and easily updated by the system's package
| manager.
| AlphaSite wrote:
| Why does it matter what happens to your machine?
|
| The big corp I work for has no issue with any of this so
| I genuinely don't understand the motivation for this
| restriction.
|
| Your machine shouldn't have anything important on it and
| if it does I'm not sure that root/non-root will protect
| you from that.
| doliveira wrote:
| Developing anything without root in your own machine
| these days must be an absolute nightmare. If I may ask,
| what stack are you using? I can imagine Java, but that's
| about it. Go, Python, Node, etc, all of them I've needed
| root access to test some packages, for instance.
|
| I would guess developers must be using a lot of
| "workarounds" without management's knowledge.
| kohlerm wrote:
| In big companies there isn't always a good justification
| for a restriction ;-)
| lmeyerov wrote:
| $ for your favorite infra oss startup or volunteer
|
| rhel ~forces you to do podman in rhel 8+. They make it hard
| enough to do a docker install that corp policies will reject
| docker (non-standard centos 7 repos etc), while allowing ibm's
| thing. Ex: Locks podman over docker for a lot of US gov where
| rhel 8 is preferred and with only standard repos, and anything
| else drowns in special approvals.
|
| subtle and orthogonal to podman's technical innovations.. but
| as a lot of infra oss funding ultimately comes from what
| enterprise policies allow, this a significant anti-competitive
| adoption driver for podman by rhel/ibm.
| op00to wrote:
| Because people can't install non-Red Hat software due to
| internal policy, that's somehow anti-competitive?
| lmeyerov wrote:
| The main repos have plenty of non-rhel sw, and people can
| expend social capital working around IT policies, but that
| doesn't change it being against the grain.
|
| We are fine because we use Ubuntu for the generally better
| GPU support, but I don't have control over the reality of
| others. I repeatedly get on calls with teams where this
| comes up, and I feel sorry for the individuals who are
| stuck paying the time etc. cost for this. Docker for RHEL 8
| is literally installing Docker for Centos 7, so this isn't
| a technology thing, but political: A big corporate abusing
| their trusted neutral oss infra position to push non-
| neutral decisions in order to prop up another part of its
| business. This is all bizarre to operators who barely know
| Docker to beginwith, and care more about say keeping the
| energy grid running. ("The only authorized thing will
| likely not work b/c incompatibilities, and I have to spend
| 2w getting the old working thing approved?")
|
| And again, none of this is technical. Docker is great but
| should keep improving, so competitive pressure and
| alternatives are clearly good... but anti-competitive stuff
| isn't.
| mbreese wrote:
| But RHEL repositories are always out of date, sorry, I
| mean "stable". The disconnect is when you try to `yum
| install docker` on a RHEL system, you get an old version.
| If you want a newer version of Docker, you need to use
| the Docker provided repositories.
|
| But this is the case for all software on RHEL. It's all
| older and more stable. Try installing any programming
| language, database, etc... it's always a version or two
| older in the official RHEL repositories. RedHat written
| stuff is always going to be more "current" because they
| control the repositories, but this isn't any different
| from how any other package is treated. RHEL has always
| been slow to update. (Which is why Docker created their
| own repositories in the first place).
|
| It makes sense though... because RHEL has to check all
| packages that are part of their repository. Specifically
| because things in their repository are safe and stable.
| There is only so much time to go around testing updates
| to packages.
| lmeyerov wrote:
| Oof I wish more enterprises/govs could do that. In RHEL
| 8, a team would need to modify their trust chain to
| include centos 7 / epel to get at the old and still
| working `yum install` binaries, and that gets into IT
| signoffs before we even get into discussing updated
| versions.
|
| I'd totally expect a trade-off for old & stable over new
| and shiny for RHEL: They're getting paid for that kind of
| neutral & reliable infra behavior. That'd mean RHEL is
| packaging docker / docker-compose, and maybe podman as an
| optional alternative. Instead IBM/RHEL is burning IT dept
| $ and stability by not allowing docker and only
| sanctioning + actively promoting IT teams use their
| alternative in unreliable ways.
| greggyb wrote:
| I can't answer the question well, but I think the opposite is
| just as valid a question: What does a daemon buy you in
| practice? Why do you need to run the Docker daemon if it is
| possible (as appears so from podman's existence) to do the same
| thing without a daemon?
| CraigJPerry wrote:
| It removes a privilege escalation scenario. With docker i can
| easily gain root, with podman i can't.
|
| Might not be relevant if it's your own machine though but for
| $BIG_CORP desktop that might be useful.
| zeroxfe wrote:
| Anyone have experience with this? I love the idea of it being
| daemonless.
| shaicoleman wrote:
| I tried it, and went back to Docker with BuildKit.
|
| * The newer version of Docker+BulidKit supports bind-
| mounts/caching/secrets/memory filesystem etc. we used it to
| significantly speed up builds. You could probably find a way in
| buildah to achieve the same things, but it's not standard.
|
| * Parallel builds - Docker+BulidKit builds Dockerfile in
| parallel, in podman things run serially. The combination of
| caching and parallel builds and Docker being faster even for
| single-threaded builds, Docker builds ended up an order of
| magnitude faster than the Podman ones.
|
| * Buggy caching - There were a lot of caching bugs (randomly
| rebuilding when nothing changed, and reusing cached layers when
| files have changed). These issues are supposedly fixed, but
| I've lost trust in the Dockerfile builds.
|
| * Various bugs when used as a drop-in replacement for Docker.
|
| * Recurring issues on Ubuntu. It seemed all the developers were
| on Fedora/RHEL, and there were recurring issues with the Ubuntu
| builds. Things might be better now.
|
| * non-root containers require editing /etc/subuid, which you
| can't do unless you have root access.
|
| More information about the new BuildKit features in Docker:
|
| https://pythonspeed.com/articles/docker-buildkit/
| feanaro wrote:
| FWIW, I'm using Podman and I'm happy with it given that it's
| fast enough for my purposes and I haven't encountered any
| caching bugs.
| lima wrote:
| Buildkit's caches are a killer feature. They go a long way
| towards making Docker builds usable for iterative
| development.
|
| We also migrated _back_ to Docker from Podman because of
| Buildkit and the general bugginess of Podman.
| jka wrote:
| From limited experience: podman and the associated toolset
| (buildah, skopeo) show a lot of promise. That said, they are
| evolving and may currently require administrative care and
| attention when used in production.
|
| If your environment would benefit from smaller, isolated
| containerization tools then my two cents would be that it's
| worth keeping an eye on these as they mature, and perhaps
| perform early evaluation (bearing in mind that you may not be
| able to migrate completely, yet).
|
| The good:
|
| - Separation of concerns; individual binaries and tools that do
| not all have to be deployed in production
|
| - A straightforward migration path to build containers from
| existing Dockerfiles (via "buildah bud")
|
| - Progressing-and-planned support for a range of important
| container technology, including rootless and multi-architecture
| builds
| edwintorok wrote:
| If you run it as non-root it is significantly slower than
| docker as root. Docker can us the overlay2 kernel driver,
| whereas podkan would use fuse-overlayfs in userspace. This has
| a high CPU overhead (e.g. don't try to run AFL inside podman),
| and a 1024 FD limit for the entire container (so a 'make -j40'
| usually dies).
|
| There are ways around it: raise the ulimit for your user and
| run new enough podman to raise limit for fuse-overlayfs, use
| the 'vfs' driver (it has other perf issues). I heard (but
| haven't tested yet) that the 'btrfs' driver avoids all these
| problems and works from userspace. Obviously requires an FS
| formatted as btrfs...
|
| There are also compatibility issues with Docker. E.g. one
| container was running sshfs inside Docker just fine, but fails
| with a permission error on /dev/fuse with podman.
| stuaxo wrote:
| Are these fuse limitations or fuse-overlayfs limitations ?
| edwintorok wrote:
| I think FUSE limitations: fuse is served by a single
| userspace process, which is limited the same way as any
| other userspace process by ulimit. It is not a fundamental
| limitation of podman, just of podman's default rootless
| behaviour.
| SEJeff wrote:
| You can run podman as root, but it doesn't default to it, for
| generally sensible security reasons.
|
| Also, docker runs as root, so it won't have permissions
| problems. You can change the permissions of /dev/fuse if you
| want to allow podman containers to access it or update the
| group of the user launching podman.
| eriksjolund wrote:
| If I understand correctly support for native rootless mounts
| is currently under development:
| https://github.com/containers/storage/pull/816 The
| functionality requires Linux kernel 5.11 (soon to be
| released)
| snickms wrote:
| I've used podman-compose for a few web projects. Rootless means
| you can't bind to privileged ports etc. but after a bit of
| fiddling I can now spin up my choice of DB/server/mock pretty
| quickly - just like I could with docker.
|
| The lack of 'cannot connect to dockerd' mysteries makes for a
| much-improved developer experience if you ask me.
| sevagh wrote:
| I have a Makefile for a Rust project which binds the local
| repository to a Docker volume, builds it in the container using
| muslrust, and then does a chown to change the target directory
| back from root ownership to my own user.
|
| All I had to do was 's/docker/podman/g' and remove the chown
| hack and it works fine:
| https://github.com/sevagh/pq/commit/6acf6d05a094ac2959567a9a...
|
| It understands Dockerfiles and can pull images from Dockerhub.
| thechao wrote:
| I'm a GPU driver dev and you guys are talking ~alien~, right
| now. What does "daemonless" mean? Why is that good?
| [deleted]
| shaded-enmity wrote:
| In Docker each container process is a child of the Docker
| daemon process. If you need to apply security patches to the
| Docker daemon it kills all your running containers.
| raesene9 wrote:
| Technically the parent process for a contained process with
| Docker is containerd-shim.
|
| Also, Docker does support live restore if you want to keep
| containers running over daemon restarts
| https://docs.docker.com/config/containers/live-restore/
| dnautics wrote:
| I've been using my company's docker compose script
| transparently with podman-compose for three months now and I
| have basically forgotten that I've switched. I actually didn't
| use docker before this, so I can't comment on what the
| difference is.
|
| I feel like the podman experience is binary, either it works
| perfectly or it sucks balls. My suggestion is to give it a try,
| if it fails, then maybe file a bug report, fail fast and fall
| back on docker.
| raesene9 wrote:
| Out of curiosity, why do you prefer daemonless setups?
| stuaxo wrote:
| It's a pain having to setup root access or a user for Docker.
| At a financial institution I worked at we had to waste about
| half a day to get this setup (and that's once we worked out
| who we had to speak to).
| raesene9 wrote:
| FWIW you can run Docker rootless (as an ordinary user) now.
| georgyo wrote:
| If I have 30 containers running, why should a single daemon
| being restarted cause all 30 to shutdown as well?
|
| Similarly, the docker security model is that there isn't a
| security model. If you can talk to the docker socket you have
| what ever privileges the daemon is running as.
| raesene9 wrote:
| So Docker supports live restore
| https://docs.docker.com/config/containers/live-restore/
| which addresses the first point.
|
| Second point, yep if you run Docker as root and someone can
| access the socket file they get root.
|
| If that's a concern, you can run Docker rootless.
|
| And as we're talking file permissions on a local host to
| allow that access, the same applies to podman containers
| does it not? If there are permission issues allowing
| another user to use access the container filesystems, you
| have the same problem.
| zaarn wrote:
| Rootless Docker is basically a joke, I've tried to run
| production workloads on it for about a year before I gave
| up. Numerous docker images will refuse to work, a bunch
| of things will subtly fill your error logs with warnings
| and it doesn't mesh well with running docker swarm at
| all.
| raesene9 wrote:
| Rootless docker only left experimental status with 20.10
| which came out in December 2020, so maybe they would have
| addressed some of those issues...
|
| As to swarm, I was comparing Docker rootless to podman,
| which is more a developer use case than prod. container
| clusters.
| mbreese wrote:
| When you use a system-level daemon[0], the daemon has to have
| privileges to start a container as anyone (that is... root).
| In a daemon-less environment, you only need the privileges of
| the user who is starting the container.
|
| [0] I suppose you could have a user-level daemon that runs
| for each user that needs to run containers, but that's even
| more overhead.
| raesene9 wrote:
| Docker does allow for daemonless execution, but as you say
| one daemon per user will add a bit of overhead.
|
| There's some tradeoff I guess though, between rootful setup
| and per user, as images duplication per user could add up.
| igneo676 wrote:
| I had pretty negative experiences with it. I'm not sure how
| much of my experience is _actually_ the fault of Podman and how
| much of it was around using it as a drop-in replacement for
| Docker. It baffled me though and I'm a long time Docker and
| Linux user.
|
| I used Arch Linux and followed their docs[1], mainly because I
| wanted it to also run without root. Big mistake. Attempting to
| run a simple postgres database w/ and w/o root permissions for
| Podman resulted in my system getting into an inconsistent
| state. I was unable to relaunch the container due to the state
| of my system.
|
| I mucked around with runc/crun and I ended up literally nuking
| both the root and non-root directories that stored them on my
| computer. I reset my computer back to Podman for root only,
| hoping that I had just chosen a niche path. It still would
| leave my system broken, requiring drastic measures to recover.
| No thanks.
|
| After much debugging, finally switching back to Docker, I
| realized my mistake: I had forgotten a required environment
| variable. Silly mistake.
|
| Docker surfaced the problem immediately. Podman did not.
|
| Docker recovered gracefully from the error. Podman left my
| system in an inconsistent state, unable to relaunch the
| container and unable to remove it
|
| Again, I'm not sure how much of my experience was just trying
| to force Podman into a square hole? I'm sure there are other
| people that make it work just fine for their use cases
|
| Edit: I should note that I used it as a docker-compose
| replacement, which is probably another off-the-beaten path
| usage that made this more dramatic than it should have been
|
| 1. https://wiki.archlinux.org/index.php/Podman
| dnautics wrote:
| I don't know when you tried this but I recall it having
| sucked in the summer when I tried it and I gave up. New job
| in the fall, decided to give podman a shot "if it doesn't
| work on the first try I'm going to use docker", then podman-
| compose 3rd party instructions (I can't remember which) just
| works out of the box. I'm running a fleet that includes
| postgres and redis.
| perlgeek wrote:
| One huge advantage over docker is that if you mount a directory
| into the container, and the container writes files there, on
| the host system they always have the owner of the user that
| started the container.
|
| That makes it much more convenient for build environments,
| without having to hard-code user IDs both in the container and
| on the host.
| Spivak wrote:
| If you're on a Linux system you can actually make this work
| better with sssd. So sssd's architecture is actually client-
| server over a unix socket.
|
| So all you need to do is create a very simple base container
| layer that just installs sssd-client, and wires up
| /etc/nsswitch.conf to use it (your package manager will
| almost surely do this automatically). Then just bind mount
| the sssd socket into your container and boom, all your host
| users are in the container.
|
| If you already log in with sssd you're done. But if you only
| use local users then you'll need to configure the proxy
| provider so that sssd reads from your passwd on your host. In
| this case the host system doesn't actually have to use sssd
| for anything.
| perlgeek wrote:
| Sounds interesting, do you have a link to an example where
| this is done/demonstrated?
|
| (Also I'm not sure how that's better (and not just
| different), except maybe it allows more than one host user
| in the container, but I haven't had a use case for that).
___________________________________________________________________
(page generated 2021-02-11 23:01 UTC)