Is Docker Swarm really dead … reddit / selfhosted ---------------------------------------------------------------------- Toorms · 2023-10-08 12:46 … or still useable for docker cluster - and make this sense in 2023 for a small docker cluster? ---------------------------------------------------------------------- GlumKnowledge5933 · 2023-10-08 12:57 Portainer, no? ---------------------------------------------------------------------- Toorms · 2023-10-08 13:02 Portainer is cluster compatible in community edition? ---------------------------------------------------------------------- KoenigPhil · 2023-10-08 13:10 My 12 production clusters are working very well. Onprem and on cloud. ---------------------------------------------------------------------- neroita · 2023-10-08 13:21 I have 4 swarm on production that works really well. ---------------------------------------------------------------------- GhoulishPaladin · 2023-10-08 13:39 Yes. It works well. ---------------------------------------------------------------------- danial23 · 2023-10-08 13:55 I love how lightweight Swarm is ---------------------------------------------------------------------- sv-giampa · 2023-10-08 14:04 We have a 12 nodes cluster at university and our Docker- Swarm/Portainer/NFS is working very well for Big Data analysis experiments. It is very useful for quickly deploying new containers for performance tests. ---------------------------------------------------------------------- ElevenNotes · 2023-10-08 14:14 No, who told you swarm is dead? K8s fanboys? ---------------------------------------------------------------------- kiwimonk · 2023-10-08 14:31 I've always avoided it due to it being not recommended for new deployments. I guess I should try it, it always sounded like exactly what I was looking for. ---------------------------------------------------------------------- psicodelico6 · 2023-10-08 14:41 Does portainer need clúster filesystem? Can it migrate container when its down? ---------------------------------------------------------------------- GhoulishPaladin · 2023-10-08 14:42 No, I think setting up docker swarm beforehand handles that. The container will stop and restart on another node when the original node goes down. ---------------------------------------------------------------------- Biog0d · 2023-10-08 14:45 Swarm orchestration takes care of migrating containers. Portainer is best deployed via their official yaml which takes care of portainer agent as well. It is best ran as a single instance, while the agent runs as global instance. If you want portainer instance to be able to restart / migrate to any manager node, you’ll need shared storage. I use glusterfs runs great ---------------------------------------------------------------------- Biog0d · 2023-10-08 14:49 The business ed has a 5 node free license (3 node now sadly) I have the 5 node free license but I have 6 nodes so it makes the over the limit grace period last 10 days - I just remove the stack and reinstall it again as needed ---------------------------------------------------------------------- probablyjustpaul · 2023-10-08 14:51 Swarm is not dead. Mirantis (Docker's parent company) uses the [internals of swarm in their Kubernetes offering](https://www.mirantis.com/software/swarm/). And the OCI maintains (Swarmkit)[https://github.com/moby/swarmkit] as part if the Moby project (Docker's upstream). For what it's worth anecdotally, I just rebuilt my entire system from the ground up. New hardware, new network back bone, new storage, all of it. New setup is three distributed storage nodes and five compute nodes and the orchestrator for all of it is Swarm. ---------------------------------------------------------------------- bityard · 2023-10-08 14:57 There are two versions of swarm, an old one that is deprecated and not maintained, and a newer one that is maintained and works quite well. Unfortunately, the docker company named them both the same thing and the official docs still refer to both without clarifying which one they mean. It's entirely stupid. ---------------------------------------------------------------------- chkpwd · 2023-10-08 14:58 I’m fairly sure it’s not “k8s fanboys”. Rather, those who regurgitate misleading information. Docker Swarm is deprecated. Whereas, Docker SwarmKit or Docker Swarm Mode (the latter being tightly integrated into Docker), is the new implementation that replaces Docker Swarm. ---------------------------------------------------------------------- KoenigPhil · 2023-10-08 15:33 I use portainer -CE on 10x 6 nodes without problem. ---------------------------------------------------------------------- nerdyviking88 · 2023-10-08 15:44 'new' is not a word I'd put attached to Swarmkit ---------------------------------------------------------------------- kayson · 2023-10-08 15:50 Not dead! https://dockerswarm.rocks/ ---------------------------------------------------------------------- the_cainmp · 2023-10-08 15:54 I’m curious how you’re handling clustered storage in your set up ---------------------------------------------------------------------- probablyjustpaul · 2023-10-08 15:58 Glusterfs. Though speaking of dead software, there's a rumor that I haven't been able to find concrete news about that RH is going to be killing it soon. ---------------------------------------------------------------------- mb2m · 2023-10-08 16:12 Will be dead by end of the year… For six years now. ---------------------------------------------------------------------- rocket1420 · 2023-10-08 16:33 No. Classic Swarm is deprecated. ---------------------------------------------------------------------- ninadpchaudhari · 2023-10-08 16:41 I swear your Information Technology office must be nice …. Many places they don’t like the idea of giving container access to researchers. To adhere to their so called “security” objectives 😖. It’s a great place to research at, but ITS at University at Albany, has fallen into this trap.. they want you to use either Jupyter notebooks or SLURM… Which university do you attend ? ---------------------------------------------------------------------- the_cainmp · 2023-10-08 17:00 Yeah, I have heard that too. And the lack of a good darker swarm aware, clustered storage solution really is what has prevented me from deploying it. ---------------------------------------------------------------------- dRaidon · 2023-10-08 18:06 Swarm just does it's thing with no complications. ---------------------------------------------------------------------- SIN3R6Y · 2023-10-08 18:48 Gluster isn’t dying, but the RHGS enterprise release of gluster has already been killed more or less. ---------------------------------------------------------------------- seizeheures · 2023-10-08 19:41 Yeah one of them seems to be a plugin and the other, currently supported one is "swarm mode" and comes built-in ---------------------------------------------------------------------- tomdaley92 · 2023-10-08 20:31 As others here have said. The old Docket Swarm was an enterprise offering and that has long since been deprecated. Docker "Swarm Mode" is built into Docker Engine and is still maintained. I've been using it production at work for a multi billion dollar company as well as in my homelab for just about everything including GPU passthrough for Plex. Works well. Just doesn't get as much out-of-the-box love as the Kubernetes ecosystem. Over the years I've ended up with some pretty mature IaC to setup swarm nodes with monitoring, centralized logging, reusable IaC to be used within any CICD pipelines for deploying just about anything on Docker Swarm using Ansible. Feel free to check out any of my open repos [here](https://github.com/Diesel-Net) ---------------------------------------------------------------------- tomdaley92 · 2023-10-08 20:38 The docs have had a comment about it in the bottom of the intro for quite awhile now https://docs.docker.com/engine/swarm/ ---------------------------------------------------------------------- adamshand · 2023-10-08 21:08 Does glusterfs require low latency between storage nodes? ---------------------------------------------------------------------- desqviewX · 2023-10-09 02:03 No, there are TONS of shops that still run it, despite it being declared "dead" years ago. It's a really solid middle-ground between no-orchestration and Kubernetes (k8s). Despite what you see in here and in places like /r/homelab, k8s is not as popular as they'd have you believe and is a splickin' nightmare to manage. ---------------------------------------------------------------------- youngsecurity · 2023-10-09 03:28 You have it good, but you don't realize how good. I have seen data scientists and researchers install a database service in a container with default credentials. That container was compromised and turned into a crypto miner almost immediately. Unless you're researching how to orchestrate containers, you should not want the responsibility of managing your containers. Leave the architecture and cybersecurity to the architect and focus on doing research. There is a lot more risk when you're given the privilege to manage the container. It could be that your organization does not have the resources to allow that privilege securely. There are layers to this. ---------------------------------------------------------------------- ninadpchaudhari · 2023-10-09 03:42 Building on this idea, it’s certainly not a good idea to not expose things to internet ! And many issues can arise due to say mis-configuration of files. Eg. Database ports being open on internet is a very common issue. But just blocking access doesn’t remove the threat. Researchers just resolve to AWS/cloud providers to do the same thing… ultimately the loss remains same. Or they just buy new machines and set things up in their lab… and face same issues. Also, specially in academic a case can be made is to educate the researchers for potential issues… it’s an academic institute. Let people learn! There are other ways to mitigate these issues, for eg: only allow these things on a local intranet. Setup monitoring on research project files that have the potential of facing the internet. Reduce the surface area and impact in case of an attack…. Maintain backups… And, yes there are cases when this is just not feasible eg.when working with sensitive data. But that stage, the researchers are aware of the sensitivity and there are already rules on how this data has to be handled. Anyways, Having said that, We do have it good…. The computer cluster we have is just amazing! And sure things work and they are great. But there are always steps to make things better :) ITS should work with the researchers to make things happen! Not just act as a policing force. ---------------------------------------------------------------------- PaulEngineer-89 · 2023-10-09 04:19 With Docker the standard setup is that you run a stock database like Postgres in its own container shared over a private network to your “web interface” and maybe a third “experimental” container. All three can see each other but only the “web interface” offers any sort of direct connection. ---------------------------------------------------------------------- youngsecurity · 2023-10-09 04:51 I had a feeling you would go that route. That's why I did say there are layers to this. It sounds like you're more focused on cybersecurity than you are doing research. It is the job of information security or cybersecurity to be the police, not IT. If you can't get your research done, there should be channels to address the issue, and I'm sure there are. I don't fully understand the first paragraph. Specifically, "it’s certainly not a good idea to not expose things to internet !" Look into OpenZiti. Blocking access to something does mitigate the threat to the organization's systems. That's precisely what blocking, policies, and controls are for. All the solutions you mention are not needed when you have been provided the service you have. You have to make a valid case. You can't just try to solve problems that don't exist with solutions that cost money to implement. You won't get far with your IT and InfoSec dept. What you describe is similar to shadow IT but outside the responsibility model because you're saying the person goes to a CSP or builds a home lab if they don't get access to the container directly from the organization. There is no threat to the origination if a person acquires services from a CSP and builds a home lab. The organization does not manage that infrastructure. You do not have a shared responsibility model with your organization when you go outside of the organization's policies. I agree that education helps, but you are not researching cybersecurity unless you are. If a system like the one you have been provided is in place, there is no value to that level of training, and it will be seen as a waste of the researcher's time. If no system like that existed, then yes, it should have gone without me saying that you have to spend more money and time training non- cybersecurity people on how to be better at cybersecurity. Many doubt the overall effectiveness of security awareness training as it is. It is better than most if the organization's policies and procedures keep you from worrying about how your tools are deployed. It is a sign of a mature operation that wants researchers to focus on doing research and not on how to build cloud infrastructures. ---------------------------------------------------------------------- baux80 · 2023-10-09 05:35 It's not deprecated nor dead. It just do the work without continuos changes and idiosyncronized ever changing abstraction ---------------------------------------------------------------------- AllanMarsh · 2023-10-09 07:42 Shades of Outlook Express vs Outlook vs outlook.com. Do these companies never learn, or are they just completely unoriginal? ---------------------------------------------------------------------- fat_keepsake · 2023-10-09 07:52 Not dead but I no longer trust it for critical systems. I used to run it in production across 20+ nodes and it would always end up in some unrecoverable state after awhile. Since then I've migrated to HashiCorp Nomad and never looked back. ---------------------------------------------------------------------- MightyDillah · 2023-10-09 09:33 And people say docker swarm is complicated … ---------------------------------------------------------------------- boli99 · 2023-10-09 10:58 they learned that confusion often causes folk to buy stuff they dont really need. ---------------------------------------------------------------------- socheat78 · 2023-10-09 12:48 I have just made a collection of Docker Swarm stacks I used at the current company I worked at. Just refactoring the deployment to make it easy to understand and re- write the docs. Hope you find it useful https://github.com/YouMightNotNeedKubernetes ---------------------------------------------------------------------- sv-giampa · 2023-10-09 16:47 As already suggested by someone in here, we won't have many services exposed to the internet, we have just a portainer instance with SSL. I know that some security problems could happen, but we just use the cluster for performance analysis and we deploy compose stacks with images pre-built on our development machines and shared among Docker Hub. Normally, after tests the unused containers/stacks are stopped. Moreover, we won't have important data stored on the cluster. By the way, there is always someone that is using it (we have set some schedules) and we do a large use of htop and some other CPU-monitoring tools, so a potential miner cannot easily escape our eyes 😉 We had to study all the Docker ecosystem because it is the kernel of our research project, so the time dedicated to deploying a Docker Swarm cluster was practically zero. I found this was a fortune and probably a very good study case. ---------------------------------------------------------------------- sv-giampa · 2023-10-09 16:57 I attend University of Calabria, in Italy. We have not many security policies about the cluster. But I think we should define some policies, beacuse I think it's not a good idea to expose every test container to the internet. A researcher that is not so expert about security could expose unsecured services on the internet, focusing only about its research work. On the other side, we already are used to largely use strong ssh public key authentication for accessing everything. ---------------------------------------------------------------------- 4x10m2 · 2023-10-09 22:22 Mint y, mint l, mint x ? ---------------------------------------------------------------------- protecz · 2023-10-10 02:18 Not updated since 2020 though. Is this guide reliable? ---------------------------------------------------------------------- kayson · 2023-10-10 02:52 In my experience, yes. And part of that is probably because there haven't been any significant swarm updates since then. I don't really see that as a problem because it's really well suited for small clusters of self hosted docker containers ---------------------------------------------------------------------- protecz · 2023-10-10 03:13 Thanks. I've been burned too much lately by using "managed" services such as lightsail containers etc. Definitely going to try this from now on.. ---------------------------------------------------------------------- kayson · 2023-10-10 03:32 Here's another great site to check out for swarm stuff - https://geek- cookbook.funkypenguin.co.nz/ ---------------------------------------------------------------------- greenblock123 · 2023-10-19 14:50 cool stuff. How is the templating and reusability in your opinion? I kept having issues between envs, so I built a lightweight templating engine - https://github.com/neuroforgede/nothelm.py Samples here: [https://github.com/neuroforgede/nothelm- charts](https://github.com/neuroforgede/nothelm-charts) Would be great to hear feedback from someone that is maintaining templates as well. ---------------------------------------------------------------------- socheat78 · 2023-10-20 12:23 Thank you, and also for showing your project. It a great project. For my personal use, I found that I often reach to gomplate for both templating stack deployment and sometime also use in the container for generating application configuration. https://docs.gomplate.ca/ ---------------------------------------------------------------------- AntonOfTheWoods · 2023-10-21 07:44 Um, yeah, pretty sure there are lots of k8s variants that are actually pretty easy to both set up and manage, and take you from the dev desktop to smallish clusters. For example, k3d + k3s is an absolute pleasure to set up and manage. You get all the power of kubernetes and an absolutely effortless scale-up if you ever need to. Updating to new versions of kubernetes is as simple as rerunning the original install script and you magically get updated. I've been updating every few months for around 3 years now and have never had an issue on a 4-node multi-master cluster. ---------------------------------------------------------------------- bluepuma77 · 2023-11-15 07:25 The old question. Everyone is doing k8s, should you do it, too. About every quarter I look into k8s, but every time I find it to be way more complicated than Swarm, even when using "easy" distros like k3s. If you are by yourself or a (very) small team of developers and you just need 3-15 servers, then I think Swarm is working just fine. Everyone tells me that you need 2 FTE to run a k8s in production, and they probably need a month-long training to do it. Unless you want to get into k8s for learning, I would not necessarily run a cluster with k8s. Swarm development has been realllly slooow for the last years, some ready-made pull requests stuck for months and years, no one with access rights cares to merge, but it still serves its purpose very well. PS: There's even a Reddit for that ;-) https://www.reddit.com/r/DockerSwarm/ PPS: The people at Docker forum are also very helpful https://forums.docker.com/c/open-source-projects/swarm/17 ---------------------------------------------------------------------- theAddGardener · 2023-12-01 12:21 r/DockerSwarm has like 4 posts in the last 9 months. 🙈 ---------------------------------------------------------------------- bluepuma77 · 2023-12-01 13:56 Yeah, I know, it's a very sad story. We are still happy with it and not ready to invest to move to k8s. ---------------------------------------------------------------------- theAddGardener · 2023-12-01 17:09 Same here. Just one part-time admin in an eight person company, running 3x3 nodes with three LAMP stacks on them. ---------------------------------------------------------------------- Shaamaan · 2023-12-07 13:34 We're using Swarm for some internal DevOps agents (an agent, after a job, is refreshed from the base image). That said, the swarm itself isn't always... entirely stable, I feel. But this has little to do with the Swarm itself and everything to do with the fact that we need to have both Linux AND Windows nodes. Or, to put it in another way, Windows containers are "meh", but I'm yet to find a better way... ---------------------------------------------------------------------- iodineman999 · 2023-12-10 21:49 it just updated today. it’s deprecated