[HN Gopher] PM2: Production Process Manager with a Built-In Load...
___________________________________________________________________
PM2: Production Process Manager with a Built-In Load Balancer
Author : thunderbong
Score : 76 points
Date : 2024-04-07 03:42 UTC (19 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| arcza wrote:
| I like PM2 and it's successfully been keeping my APIs alive quite
| well, but how does the future look given the popularity of
| Kubernetes? What is the overlap like and what's not overlapping?
| 4pkjai wrote:
| I think Kubernetes is unnecessary for the vast majority of
| deployments. Simple apps run perfectly fine with things like
| PM2, Monit and init.d
| occz wrote:
| Be that as it may - if you already have Kubernetes up and
| running, what would be the additional value of having both
| PM2 running alongside Kubernetes?
|
| Fwiw, last time I asked myself this question my conclusion
| was that there was no additional value, and we opted to not
| use PM2. This was a long time ago however, so take it with a
| grain of salt.
| 4pkjai wrote:
| I agree with you on that
| sublinear wrote:
| PM2 is great if you have an app composed of multiple services
| that need to share an environment. You may want to reduce
| communication overhead between these node processes or avoid
| involving kubernetes to share resources. Implementation details
| would drive this.
|
| Also, kubernetes cluster workers can fail. How do you want your
| workload to behave in that case? When your container is killed,
| should these services fail together? Do you want a better
| separation of concerns between your app dev team and your
| devops/middleware team (think disaster recovery and cluster
| maintenance)? Where does the app dev end and the devops begin?
| For other languages such as Java or C++, it would be out of the
| question to let these details matter to anyone but app devs.
| Their entire apps are in one docker image. Kubernetes should
| only manage instances of the app and the obvious dependencies
| that should be externalized (redis, memcached, postgres, mysql,
| etc.)
|
| In my experience it's really a question of failure modes in
| your architecture and squeezing out small gains in performance
| that matter at scale.
| keepamovin wrote:
| I use PM2 with BrowserBox and it's been great. Reliable,
| scalable, performant. PM2 has been with us since the early
| days. We've never looked for an alternative.
| pkstn wrote:
| PM2 was really nice until I moved to using Docker Swarm Mode
| which is even better in so many ways!
| cocochanel wrote:
| For some reason I thought Docker Swarm was abandoned. From the
| Docker docs,
|
| > Docker Swarm mode is built into the Docker Engine. Do not
| confuse Docker Swarm mode with Docker Classic Swarm which is no
| longer actively developed.
| azophy_2 wrote:
| the change from docker classic swarm into current docker
| swarm happened years ago. from what I can tell, usage-wise
| there are not much difference between the two.
|
| however the sad thing is currently swarm is held by other
| company (mirantis) after being sold by docker inc. and the
| new company doesn't seem that invested in improving swarm.
| you could say its in life support now save for a few
| community contribution
|
| more explanation by dockerswarm.rocks website:
| https://dockerswarm.rocks/swarm-or-kubernetes/
| victorbjorklund wrote:
| Just used PM2 on a recent project for deploying a nodeapp on a
| single server. Very easy to use and just works.
| SSchick wrote:
| Tangentially:
|
| Writing this at 1:30am, might not be very insightful.
|
| We actually are phasing out the use of pm2 in our org since we
| are already running within kubernetes. And running pm2 within k8s
| seemed redundant (and is likely a relict of previous deployment
| strategies).
|
| It also made certain debugging tasks trickier as we very rarely
| like to attach debuggers to our staging environment and the
| cluster/fork behavior of pm2 made it hard to effectively target
| indivual executions/processes.
|
| Further we had some issues running nextjs after upgrading to the
| latest version within pm2 when using custom configuration files
| (we did not end up finding the root cause but decided to drop
| pm2, least path of resistance etc).
|
| One other thing that also gave us headaches was the fact that pm2
| allows specifying different env variable values for different
| environments but no other configuration (such as cluster count
| etc.)
|
| We were successful entirely replacing it with docker-compose
| configs locally that are ultimately closer to how our services
| are deployed in production.
|
| Overall I think pm2 is suitable for local dev and prototyping
| small to medium complexity services.
| ilaksh wrote:
| It seems strange that you would even consider using pm2 still
| if you have all of that other stuff.
|
| I use it to avoid the complexity of those things since my
| operational needs are usually very modest.
| hosh wrote:
| At work, we have kubernetes and pm2 running on a pet server
| outside of kubernetes. I cannot wait to get rid of pm2 and the
| pet server. For smaller projects, I'd rather just work directly
| with systemd or better yet, use the podman-systemd
| integrations.
| Aldipower wrote:
| I am running a 18 node SOA since 4 years with pm2 and I am very
| happy. Very reliable. No need for the complexity of Kubernetes or
| even Mesosphere DCOS back in the day. I do not even consider to
| change anything.
| 0x6c6f6c wrote:
| All on a single host, or is this spread across multiple
| servers? Curious about how people run it with the latter
| approach if they do
| Aldipower wrote:
| Actually on 3 hosts, but only the primary is at the edge, one
| other is a hot-standby at another geographic location and the
| third is for incremental backup purposes via ZFS. The
| database sync connection are tunneled via Wireguard. Failover
| to the hot-standby is done via DNS (ClouDNS). Also I run the
| setup in a Linux Container (LXC). Deployment is done via SCP
| and Tar. :-D This setup is very reliable to me and at the
| other hand cheap!
| contextnavidad wrote:
| Had good experiences with PM2, however be cognizant of it's copy
| left license. We have found it redundant with the likes of
| Kubernetes.
| byyll wrote:
| > be cognizant of it's copy left license
|
| why?
| contextnavidad wrote:
| Some organisations forbid copy left licenses
| andrewstuart wrote:
| Is systemd capable of doing this?
|
| I thought process managers like supervisor and pm and the others
| had all been superseded by systemd, unless you're in a non
| systemd distro of course.
| Kharacternyk wrote:
| I don't think systemd has a built-in load balancer. Also,
| running multiple copies of the same service isn't typical PID-1
| stuff, so maybe doing that with systemd involves some friction.
| stephenr wrote:
| Running multiple "copies" of a service is very much in
| systemd's wheelhouse; it's called templated units.
| Kharacternyk wrote:
| I know about templates, but I don't believe they are
| designed for fungible copies. You can run a bunch of my-
| app@a-random-port.service, but it's not the same UX as `pm2
| start my-app -i number-of-copies`.
| roryrjb wrote:
| I created a Node.js binding for sd-notify to cover at least
| some of the use case here: https://github.com/systemd/node-sd-
| notify
| e1g wrote:
| Systemd also gives you cpu/memory limiting, file system
| isolation, locked down capabilities for the process, CPU
| affinity, can use different Node versions for different apps,
| etc. It removes the need for Docker for the vast majority of
| cases.
|
| Systemd won't do load balancing, but you should have
| nginx/caddy in front of Node anyway, and either will handle
| load balancing just fine.
| ametrau wrote:
| What about creating new processes?
| e1g wrote:
| Yep, systemd is an excellent and stable process runner - it
| can start your processes whenever, with dependencies (eg
| redis/pg), or on a flexible schedule etc.
| aorth wrote:
| systemd can run Node.js apps easily, but unless your
| applocation is doing something special for multithreading that
| is single threaded. PM2 makes it trivial to run an application
| in cluster mode.
| BearScale wrote:
| I set up my first nodejs backend project on an ec2 instance with
| pm2 almost 9 years ago. I've also been toying with an instant
| deploy pipeline that takes advantage of its watch and restart.
| edwinjm wrote:
| PM2 is very good and I really wonder why it's still somewhat
| unknown.
|
| People complaining that Node.js only use one CPU core, or
| benchmarks with low scores for Node.js. There are solutions for
| this! PM2 is one of them and is very good. Use it.
| nesarkvechnep wrote:
| Just spawn more processes. Great solution! /s
| sublinear wrote:
| Would you rather scale with more containers? That's heavier.
|
| Properly written node apps are bound by I/O throughput. CPU
| and memory cost for additional processes is minimal, but
| latency is significantly decreased with a worker pool.
| jitl wrote:
| > Properly written node apps are bound by I/O throughput
|
| Does that mean that node apps shouldn't do any computation
| on data, just move bytes between a file and a network
| socket? If I need my program to think, I wrote it wrong or
| in the wrong language?
|
| I would rather have shared memory multithreading be a
| usable feature in my program's runtime. SharedArrayBuffer
| exists but 98% of existing code uses objects and arrays so
| can't be easily shared between processes without copying,
| and the cost of copying objects with structuredClone puts a
| huge optimization barrier in between most code and parallel
| processing.
| sublinear wrote:
| No, just write your javascript to do more I/O than
| compute. Apps should do app things not low level tasks
| that slow things down.
|
| If you need to compute something complicated, write it in
| a compiled language and make it a node module. Many
| popular npm libraries are exactly this.
| nesarkvechnep wrote:
| Or... don't write Node at all? I'd choose Elixir over
| Node any day.
| throwitaway1123 wrote:
| > Does that mean that node apps shouldn't do any
| computation on data, just move bytes between a file and a
| network socket? If I need my program to think, I wrote it
| wrong or in the wrong language?
|
| This is a very uncharitable interpretation of Node's
| primary use case. I think it would be more accurate to
| say don't use Node if all of the following apply:
|
| - Your task is CPU bound rather than IO bound (e.g.
| you're not waiting to insert or retrieve data from the
| database).
|
| - Your task is highly parallel (some computations are
| sequential and linear).
|
| You can't use worker threads because:
|
| - You have too much data to bear the cost of message
| passing via the structured clone algorithm.
|
| - You can't share memory and avoid structured cloning
| because your data can't be represented using an array
| buffer.
|
| This is a far cry from the claim that Node can't handle
| problems that require "thinking". When you're on the JIT
| compiler's happy path Node can think pretty quickly
| actually. In practice, many high volume web servers are
| handling enough requests that there's more than enough
| jobs for each Node process to have exclusivity over its
| jobs, and you don't need to share work between processes.
| Each process or worker thread can be largely independent.
| Not every task is going to fall within those boundaries,
| but that's ok, Node doesn't have to fulfill every use
| case.
| mort96 wrote:
| I mean... yes
| bionhoward wrote:
| PM2 is a life saver for sidestepping Kubernetes imho
| chamomeal wrote:
| How do you feel about using pm2 inside of a container?
|
| At work, most of our app is docker-composed in one VPS (early
| B2B product with few customers). I would like to have the
| multiple-process benefits of PM2, but I don't want to spin up a
| whole load balancer for it, or split the node service from the
| other services. Everything is so simple right now and I'd like
| to keep it that way lol.
|
| Last time I googled this I found conflicting opinions, but if I
| can just stuff PM2 into my node container, I'd be a happy dev!
| ilaksh wrote:
| Node.js cluster https://nodejs.org/api/cluster.html or
| worker_threads
| dalore wrote:
| We use PM2 in a docker container since we got a memory leak
| and PM2 has a memory limiter option that allows us to kill
| and restart a process when it uses too much memory. If we let
| it grow the response time starts to drop due to GC taking
| longer and longer.
| hosh wrote:
| You have to do so many things just to use more than a single
| CPU core. There are other language platforms and runtimes where
| using up the entire available CPU cores is the default option,
| and taking advantage of multiple cores within the app is easy
| by design.
|
| I can see why people like pm2, similarly to why I can see why
| people like Nodejs. But I think pm2 is half-baked, and Nodejs's
| greatest design feature (single-threaded async execution) is
| flawed by design.
| nsonha wrote:
| Could some one please explain to me if this is still gonna be
| useful if we already have load balancing on the API gateway?
| lioeters wrote:
| PM2 has worked very well in my experience. I have about a dozen
| services (APIs and full-stack applications) across a number of
| servers, remote and local. Mostly Node, these days trying with
| Bun. Some of them have been running for years without a hitch.
| It's been totally reliable.
|
| I'm sure there are alternatives, lighter-weight and closer to the
| metal, but I haven't had the need. PM2 does the job.
| mynegation wrote:
| It may well be that I did not learn to use it properly, but - to
| provide a counter opinion - pm2 left a sour taste with me. Tried
| to use it few years back and by default it does not rotate the
| logs, did not handle machine reboots well, and at times I was
| confused what is running and what is not. I settled on docker for
| small projects and kubernetes for larger ones and these things
| made more sense to me. I admit it could be a matter of habit.
| sublinear wrote:
| My experience starting out with PM2 was the same. You need a
| plugin for the log rotation, saving the default environment
| variables is confusing, and stale processes cause weird
| behavior.
|
| You may already have your own in-app log rotation before adding
| PM2 so may want to avoid printing to stdout/stderr. For usage
| with docker you want pm2-runtime.
| zarzavat wrote:
| I came to the opposite conclusion for the same reasons.
|
| Docker/kubernetes is fine for things that make money, but for
| small personal projects it is far too much complexity and I
| dumped it. pm2 gets the job done with less effort overall.
| dartos wrote:
| I've found pm2 more complicated than docker-compose for
| local/self hosted small projects.
|
| Why did you prefer pm2?
| __s wrote:
| pm2 gets past involving containers
|
| I get to rsync my code up to a server, & pm2 restart it.
| The server process runs like any other, no container issues
| to consider, just a json file saying how I'd run it myself
| if I wanted to
|
| tbf I'd rather replace pm2 with systemd, since I'm using it
| to run a Rust program, but if I was using node their
| cluster mode looks neat
| zarzavat wrote:
| Honestly, I hate using docker. It's a pain to deal with in
| your dev environment, especially on a Mac. It's not
| lightweight for your dev machine. It requires a good
| internet connection and it uses a lot of data. It's all
| very "enterprise".
|
| If you're sat in an office, you're on a work computer, you
| need reproducible dev environments, probably none of that
| bothers you.
|
| However, if I'm working by myself, I don't want to deal
| with any of that Rube Goldberg shit. I use shell scripts,
| ssh, rsync and pretend it's the 00s again, and oh my god
| I'm so much happier in that universe.
| dartos wrote:
| Maybe it's different on Mac, maybe I'm biased, but I've
| been using docker since it was a small project and I've
| never felt that enterprise cruft.
|
| It saved me from the Rube Goldberg esque pile of bash
| scripts and puppeteer configs of the 00s.
|
| Funny how perception changes.
| petre wrote:
| It uses a VM on Macs, it's usually slow and requires too
| much thinkering to get it to work decently. I gave up
| development with Linux tools on Macs, very annoying and
| not worth the hassle. I think it even works better in
| Windows since WSL.
|
| I'm using runit instead of pm2. Systemd should also do
| fine. It's over engineered.
| noizz wrote:
| As others mentioned - logrotate is available as an option,
| since the system one is more preferable for many. As per reboot
| handing - it's available via "pm2 startup" script.
| hosh wrote:
| I'm with you. I think PM2 is half-baked software. It tries to
| do too much and none of them well. It has two source-of-truths,
| the dump file and the ecosystem files. It doesn't have a way to
| stagger restarts.
|
| For small projects, there is now a way to run docker containers
| or pods (through podman) via systemd units, so if you are using
| a systemd enabled distro, then that may be a better alternative
| than pm2, especially if you already have proficiency in docker
| and kubernetes.
| teaearlgraycold wrote:
| Be warned. It's AGPL. I've never been able to reach the
| developers to get commercial licensing.
| diamondfist25 wrote:
| I used pm2 to run my adhoc Python jobs/bot. Tried with Django and
| celery and docker, ended up being a pain in the ass since I have
| to modify on the fly.
|
| Pm2 is neat, but the monitor solution is pricey.
| TZubiri wrote:
| It's just something that's not needed.
|
| You can come up with a Unix solution by using bash, piping to
| logs, daemonizing with &.
|
| And as a bonus, it works for any program, not just with node
| stuff.
|
| Remember, use the tools available in your foundation before
| adding more blocks to the pyramid
| iTokio wrote:
| Is there a robust generic solution already existing? To launch
| multiples processes, load balance and automatically restart if
| one crash..
| erichocean wrote:
| Does anyone have an alternative to pm2 that's reliable? That's my
| biggest issue with it: it gets slower and slower to do commands
| like `pm2 list` over time, and eventually, pm2 itself crashes--
| taking everything it runs with it.
|
| Also, `pm2 flush` doesn't actually truncate all logs, just the
| logs of whatever processes pm2 happens to be running at that
| moment.
|
| I workaround this by killing pm2 entirely and `rm -Rf .pm2` once
| a month or so.
|
| (This is all on Linux BTW.)
| koshergweilo wrote:
| I've had a lot of luck with dokku. It uses docker so it's more
| heavyweight, but it's super simple and scales really well
| lmeyerov wrote:
| It's a fascinating project, and esp for from before docker+k8s
| normalized and nodejs was reinventing the universe
|
| Nowadays, docker compose or k8s ecosystems do a lot of this stuff
| in a more team-friendly & ecosystem-supported manner. I always
| liked pm2 bc the dynamic API layers... except docker & k8s also
| have APIs, and you get all sorts of extras with them by being a
| layer below
|
| I wonder if there is a name for this waller garden phenomena. The
| pm2 team built a wonderful universe and it keeps improving
| afaict, but tech ecosystems also have been improving around them
| and at faster & bigger levels...
| mvkel wrote:
| I've experienced some strange behavior with pm2 on my projects
| where a reboot would completely wipe the configs and I'd have to
| rebuild the processes (even after a pm2 save)
|
| If it happens again I'll likely look into doing it the old
| fashioned way with systemd.
|
| Skill issue? Most certainly. It still sucks to think I've figured
| it out (to the point of testing with manual reboots) only to one
| day have nothing running and have no clue why.
___________________________________________________________________
(page generated 2024-04-07 23:02 UTC)