[HN Gopher] Postgres: A better message queue than Kafka?
___________________________________________________________________
Postgres: A better message queue than Kafka?
Author : tylermcginnis
Score : 24 points
Date : 2022-10-04 17:27 UTC (5 hours ago)
(HTM) web link (dagster.io)
(TXT) w3m dump (dagster.io)
| benadam11 wrote:
| "Many haters want you to buy their message queue product" lol
| mannyv wrote:
| You can spew messages into sqs then have a lambda on sqs sending
| to pg.
|
| Funny, we were going to use kafka too, but just sending to mysql
| worked just fine <shrug>. One day that will change, obviously.
| eurasiantiger wrote:
| It works fine on the happy path, but if mysql goes down, you
| lose things. Having a distributed, resilient queue has
| availability benefits.
| pritambaral wrote:
| > It works fine on the happy path, but if mysql goes down,
| you lose things.
|
| Why do you have to lose things if the DB goes down? Agreed,
| untuned & unconfigured MySQL (and MongoDB) out-of-the-box can
| lose things due to bugs and design issues, but that is the
| case even when they are running. However, DBs, in general are
| made precisely for the purpose of safely storing things and
| not losing them.
|
| OTOH, the number of Kafka setups I have seen that'd lose
| things when something goes down ... maybe this is not a
| guaranteed win for the Kafka side of arguments.
|
| > Having a distributed, resilient queue has availability
| benefits.
|
| High availability is not a function exclusive to Kafka. On
| the other hand, there's some functions that may come in handy
| to use in a queue that Kafka simply cannot provide, but DBs
| can. Off the top of my head: ACID, instant scalability (both
| up and down) of consumer groups, and the sheer flexibility
| (and power) that comes with a DB in general.
|
| ----
|
| Overall, there's some merits to using a distributed log as a
| message queue, sure, but there are also merits to using a DB
| for that.
| hbrn wrote:
| A controversial, but a very pragmatic take.
|
| Queues are great for semi-infinite scalability, but you rarely
| need it.
|
| There's numerous subtle benefits to using db compared to regular
| message queues that are often overlooked.
|
| Being able to delete, reorder, or edit specific messages can be a
| lifesaver when things go wrong.
|
| Most common issue with queue-based systems is getting overwhelmed
| with messages. Either your consumers went down. Or your producers
| had a bug / executed too frequently.
|
| Being able to recover with a simple SQL query is a blessing.
| hbrn wrote:
| Here's just a few examples from my experience:
|
| 1. Huge number of messages from test system were accidentally
| inserted into production.
|
| Queue solution: disable consumers, move messages to a temporary
| queue while filtering them, move messages back to the old
| queue, enable consumers
|
| DB solution: just delete rogue messages
|
| 2. We want to store some of the messages, but we're not ready
| to process them yet
|
| Queue solution: create a separate queue for each message type,
| insert messages to different queues, manually move them when
| you're ready to process them (and keep in mind that not every
| queue can persist messages forever, SQS for example can't hold
| messages longer than 14 days)
|
| DB solution: just skip those messages while processing
|
| 3. Consumers went down and the queue now contains a big number
| of duplicate messages. While it was fine to just wait a couple
| hours to let it stabilize, a whale customer started complaining
|
| Queue solution: none (any hacky solution would take longer than
| it takes for the system to naturally stabilize)
|
| DB solution: move whale customer messages to the front of the
| queue
| lightbendover wrote:
| File this under "If a headline is asking a question, then the
| answer is NO." Honestly, I'm not even sure what point the author
| is trying to make besides "anecdotally and at low scale, unusual-
| for-the-purpose technology X solved problem Y." A near-infinite
| number of bad patterns can solve problems along happy paths and
| resolve plenty of edge cases to boot. 99.9% availability was a
| goal post here? There are systems where 7 9s is unacceptable. The
| author didn't even provide data backing the measurement goals.
| hbrn wrote:
| What are those mythical systems where seven nines is
| unacceptable? What are the chances you're going to work on one
| of those?
|
| Google Spanner for example is _up to_ five nines. Are there a
| lot of systems that need to be three orders of magnitude more
| reliable than Google Ads?
| lightbendover wrote:
| You're right, nothing is more important than serving ads.
| hbrn wrote:
| Nothing is more _profitable_ than serving ads.
|
| I'm still waiting for an answer.
| lightbendover wrote:
| While I have no idea what kind of distributed system
| would _need_ 7+ 9s, there are plenty of solution
| providers who at least promise it as part of their
| marketing. I truly disbelieve that these companies don 't
| have any point of failure within their system that would
| take them under that threshold, even if it hasn't been
| challenged to date, but that level of availability is
| still the published goal. Hopefully needless to say, but
| 99.9% is not acceptable for most "important" (and yes,
| advertising is important) applications outside of US-
| East-1 somehow. Quick examples, no experience with any of
| these: * https://www.infinidat.com/en/news/press-
| releases/infinidat-h... *
| https://vindicia.com/blog/9999999-global-system-uptime/ *
| https://www.ibm.com/downloads/cas/A856LOWK
|
| Bad examples? Certainly yes. Are there any good examples?
| Maybe, doubtful. Does any system actually need 7 9s?
| Maybe, doubtful.
| bsaul wrote:
| "The write path. We built a daemon that would select log entries
| that were older than two weeks, copy them into a file in S3, and
| delete the rows from the database"
|
| Seriously... How can you ever consider saying "a rdbms is just
| fine as a kafka alternative" under those conditions ?
| peterhunt wrote:
| Why not?
| andrewstuart wrote:
| Ideology.
|
| There's alot of people who are ideologically opposed to
| database backed message queues. They're usually reluctant to
| give detailed explanations why, because it's an emotional
| thing.
| srajabi wrote:
| I don't think this article made a compelling reason not to use
| Kafka. In fact it may have made the opposite point.
|
| Wouldn't it have been easier to just use Kafka?
| greymalik wrote:
| From TFA Postgres was an established tool in their org they
| already had expertise with. The author argues that adopting a
| new tool would have been risky and costly.
| andrewstuart wrote:
| Not just Postgres.
|
| You can do exactly this with MySQL and SQL server too because
| they both support SKIP LOCKED.
|
| Interestingly, the plain old file system on Linux also makes the
| basis of a perfectly acceptable message queue for many use cases
| - the thing that makes it work is that the file move operation is
| atomic. Atomic moves are what make queuing systems possible.
|
| You could write a file system based message queue in 100 lines of
| async python, which I did here:
|
| https://github.com/bootrino/arniesmtpbufferserver
|
| File system based message queues can be written in any language,
| extremely simple and, most importantly - zero configuration. One
| of the most frustrating things about queuing systems is
| configuration - that includes database backed queuing systems.
| They can also be fast - I wrote one in Rust which maxed out the
| hard disk's random write capability well before maxing out the
| CPU - from memory it beat most of the common queuing systems in
| terms of messages per second.
|
| Not all use cases for queues need to be able to globally
| distributed messages queues with the sort of guarantees needed
| for financial transaction processing. I would suggest to you that
| in fact most queues out there are used as outbound SMTP queues,
| which are then over engineered to use something like Celery,
| which is a nightmare to configure and debug.
| hbrn wrote:
| What makes it atomic is running publishers and consumers on the
| same box (since you're sharing filesystem between those).
|
| Also listdir is a big bottleneck here: while
| True: # get files in outbox
| files_in_outbox = [f'{PREFIX}/outbox/{x}' for x in
| os.listdir(f'{PREFIX}/outbox')]
| steeve wrote:
| Honestly that's comparing apples and oranges. Also, that title is
| clickbait. It feels like a PR article disguised as a tech
| article.
___________________________________________________________________
(page generated 2022-10-04 23:02 UTC)