[HN Gopher] We "solved" C10K years ago yet we keep reinventing i...
___________________________________________________________________
We "solved" C10K years ago yet we keep reinventing it (2003)
Author : birdculture
Score : 96 points
Date : 2025-12-25 22:35 UTC (3 days ago)
(HTM) web link (www.kegel.com)
(TXT) w3m dump (www.kegel.com)
| gnabgib wrote:
| (2011 / 2003)
|
| Title: _The C10K problem_
|
| Popular in:
|
| 2014 (112 points, 55 comments)
| https://news.ycombinator.com/item?id=7250432
|
| 2007 (13 points, 3 comments)
| https://news.ycombinator.com/item?id=45603
| trueismywork wrote:
| With nginx and 256 core Epycs, most single servers can easily do
| 200k requests per sec. Very few companies have more needs
| intothemild wrote:
| I can't tell if this is sarcasm or not.
|
| They didn't have this kind of compute back when the article was
| written. Which is the point in the article.
| trueismywork wrote:
| Half serious. I guess what Iwas saying is that it is that
| kind of science which is still very useful but more to nginx
| developers themselves. And most users now dont have to worry
| about this anymore.
|
| Should have prefixed my comment wirh "nowadays"
| hinkley wrote:
| In spring 2005 Azul introduced a 24 core machine tuned for
| Java. A couple years later they were at 48 and then jumped to
| an obscene 768 cores which seemed like such an imaginary
| number at the time that small companies didn't really poke
| them to see what the prices were like. Like it was a typo.
| fweimer wrote:
| Before clusters with fast interconnects were a thing, there
| were quite a few systems that had more than a thousand
| hardware threads: https://linuxdevices.org/worlds-largest-
| single-kernel-linux-...
|
| We're slowly getting back to similarly-sized systems. IBM
| now has POWER systems with more than 1,500 threads
| (although I assume those are SMT8 configurations). This is
| a bit annoying because too many programs assume that the
| CPU mask fits into 128 bytes, which limits the CPU
| (hardware thread) count to 1,024. We fixed a few of these
| bugs twenty years ago, but as these systems fell out of
| use, similar problems are back.
| alexjplant wrote:
| > Driven by 1,024 Dual-Core Intel Itanium 2 processors,
| the new system will generate 13.1 TFLOPs (Teraflops, or
| trillions of calculations per second) of compute power.
|
| This is equal to the combined single precision GPU and
| CPU horsepower of a modern MacBook [1]. Really makes you
| think about how resource-intensive even the simplest of
| modern software is...
|
| [1] https://www.cpu-monkey.com/en/igpu-apple_m4_10_core
| fweimer wrote:
| Note that those 13.1 TFLOPs are FP64, which isn't
| supported natively on the MacBook GPU. On the other hand,
| local/per-node memory bandwidth is significantly higher
| on the MacBook. (Apparently, SGI Altix only had 8.5 to
| 12.8 GB/s.) Total memory bandwidth on larger Altix
| systems was of course much higher due to the ridiculous
| node count. Access to remote memory on other nodes could
| be quite slow because it had to go through multiple
| router hops.
| hinkley wrote:
| My Apple Watch can blow the doors off a Cray 1. It's
| crazy.
| marcosdumay wrote:
| The article was written exactly because they had machines
| capable enough at the time. But the software worked against
| it on every level.
| Maxatar wrote:
| I don't see how you could have read the article and come to
| this conclusion. The first few sentences of the article even
| go into detail about how a cheap $1200 consumer grade
| computer should be able to handle 10,000 concurrent
| connections with ease. It's literally the entire focus of the
| second paragraph.
|
| 2003 might seem like ancient history, but computers back then
| absolutely could handle 10,000 concurrent connections.
| api wrote:
| I'm shocked that a 256 core Epyc can't do millions of requests
| per second at a minimum. Is it limited by the net connection or
| is there still this much inefficiency?
| otterdude wrote:
| 256 Processes x 10k clients (per the article) = 256K RPS
| mrweasel wrote:
| Aren't you of by a zero? 10K requests per core / per
| second, time 256 cores is 2.560.000 RPS.
|
| There's probably going to be some overhead, but it seems
| like you could do 1M, if you have the bandwidth.
| tempest_ wrote:
| Like anything it really depends on what they are doing, if
| you wanted to just open and close a connection you might run
| into bottle necks in other parts of the stack before the CPU
| tops out but the real point is that yea, a single machine is
| going to be enough.
| zipy124 wrote:
| It almost certainly can, even old intel systems with dual CPU
| 16 core systems could do 4 and a half million a second [1].
| At a certain point network/kernel bottlenecks become apparent
| though, rather than being compute limited.
|
| [1]: https://www.amd.com/content/dam/amd/en/documents/product
| s/et...
| tempest_ wrote:
| This is how I feel about this industries fetishization of
| "scalability".
|
| A lot of software time is spent making something scalable when
| in 2025 I can probably run any site the bottom 99% of most
| visited sites on the internet on a couple machines and < 40k
| capital.
| tbrownaw wrote:
| > _any site the bottom 99% of most visited sites on the
| internet_
|
| What % is the AWS console, and what counts as "running" it?
| tempest_ wrote:
| > What % is the AWS console
|
| 0%
|
| Prior to the recent RAM insanity(a big caveat I know) a 1u
| supermicro machine with 768GB some NVME storage and twin 32
| core Epyc 9004s was ~12K USD. You can get 3 of those and
| and some redundant 10G network infra(people are literally
| throwing this out) for < 40k. Then you just have to find a
| rack/internet connection to put them in which would be a
| few hundred a month.
|
| The reality is most sites don't need multi region setups,
| they have very predicable load and 3 of those machines
| would be massive overkill for many. A lot of people like to
| think they will lose millions per second of down time, and
| some sites certainly do but most wont.
|
| All of this of course would be using new stuff. If you
| wanted to use used stuff the most cost effective are the 5
| year old second gen xeon scalables that are being dumped by
| cloud providers. Those are more than enough compute for
| most they are just really thirsty so you will pay with the
| power bill.
|
| This of course is predicated on assumption you have the
| skill set to support these machines and that is
| increasingly becoming less common though as successful
| companies that started in the last 10 years are starting to
| do more "hybrid cloud" it is starting to come back around.
| oblio wrote:
| Raw technical excellence doesn't rake in billions, despite
| what IT people keep saying.
|
| Otherwise Viaweb would be the shining star of 2025. Instead
| it's a forgotten footnote on a path to programming with money
| (VC).
| Animats wrote:
| The sites that think they need huge numbers of small network
| interactions are probably collecting too much detailed data
| about user interaction. Like capturing cursor movement. That
| might be worth doing for 1% of users to find hot spots, but
| capturing it for all of them is wasteful.
|
| A lot of analytic data is like that. If you captured it for
| 1% of users you'd find out what you needed to know at 1% of
| the cost.
| otterdude wrote:
| When people talk about a single server they're not talking
| about one hunk of metal, they're talking about 1 server
| process.
|
| This article describes the 10k client connection problem, you
| should be handling 256K clients :)
| marcosdumay wrote:
| When people talk about a single server they are pretty much
| talking about either a single physical box with a CPU inside
| or a VPS using a few processor threads.
|
| When they say "most companies can run in a single server, but
| do backups" they usually mean the physical kind.
| Maxatar wrote:
| The term is absolutely ambiguous and I know I've run into
| confusion in my own work due to the ambiguity. For the
| purpose of the C10K, server is intended to mean server
| process rather than hardware.
| marcosdumay wrote:
| > You can buy a 1000MHz machine with 2 gigabytes of RAM
| and an 1000Mbit/sec Ethernet card for $1200 or so. Let's
| see - at 20000 clients, that's 50KHz, 100Kbytes, and
| 50Kbits/sec per client. It shouldn't take any more
| horsepower than that to take four kilobytes from the disk
| and send them to the network once a second for each of
| twenty thousand clients.
|
| It was about physical servers.
| Aloisius wrote:
| This is mostly talking about running more than 10K
| connections on a single daemon. Machines were generally
| dedicated and you were typically running a single daemon
| per machine.
|
| We pushed past 10K connections on a single machine in
| 1999 (we didn't need to listen on any particular port
| which made things easier), but much of the stuff talked
| about this page (polling mechanisms, threading/forking
| strategies, etc) is stuff I went through when scaling
| past 10K connections on a single daemon.
| dilyevsky wrote:
| At the time this was written powerful backend server only had
| like 4 cores. Linux only started adopting SMP like that same
| year. Also CPU caches were tiny
|
| Serving less than 1k qps per core is pretty underwhelming
| today, at such a high core count you'd likely hit OS
| limitations way before you're bound by hardware
| Grosvenor wrote:
| Linux had been doing SMP for about 5 years by that point.
|
| But you're right OS resource limitations (file handles, PIDs,
| etc) would be the real pain for you. One problem after
| another.
|
| Now, the real question is do you want to spend your
| engineering time on that? A small cluster running erlang is
| probably better than a tiny number of finely tuned race-car
| boxen.
| dilyevsky wrote:
| My recollection is fuzzy but i remember having to recompile
| 2.4-ish kernels to enable SMP back in the day which took
| hours... And I think it was buggy too.
|
| Totally agree on many smaller boxes vs bigger box
| especially for proxying usecase.
| hinkley wrote:
| I don't think I even heard of C10K until around 2003.
| hoppp wrote:
| We solved it 2 decades ago but then decided to use javascript on
| the server ...
| wmf wrote:
| Node.js is actually pretty good at C10K but it failed at
| multicore and C10M.
| mifreewil wrote:
| Node.js uses libuv, which implements strategy 2. mentioned on
| the linked webpage.
|
| "libuv is a multi-platform C library that provides support for
| asynchronous I/O based on event loops. It supports epoll(4),
| kqueue(2)"
| marcosdumay wrote:
| Except that it wastes 2 or 3 orders of magnitude in
| performance and polls all the connections from a single OS
| thread, locking everything if it has to do extra work on any
| of them.
|
| Picking the correct theoretical architecture can't save you
| if you bog down on every practical decision.
| mifreewil wrote:
| I'm sure there is plenty of data/benchmarks out there and
| I'll let that speak for itself, but I'll just point out
| that there are 2 built-in core modules in Node.js,
| worker_threads (threads) and cluster (processes) which are
| very easy to bolt on to an existing plain http app.
| IgorPartola wrote:
| So think of it this way: you want to avoid calling
| malloc() to increase performance. JavaScript does not
| have the semantics to avoid this. You also want to avoid
| looping. JavaScript does not have the semantics to avoid
| it.
|
| If you haven't had experience with actual performant code
| JS can seem fast. But it's is a Huffy bike compared to a
| Kawasaki H2. Sure it is better than a kid's trike but it
| is not a performance system by any stretch of the
| imagination. You use JS for convenience, not performance.
| jauntywundrkind wrote:
| What a sad miserable cult it is, to be obsessed with
| performance, and to so desperately need to beat up on a
| language. I don't know if people spewing this shit are
| aware of how dull their radical extremism is?
|
| You really don't have to go far down the techempower
| benchmarks to get to JS. Losing let's just say 33%
| performance versus an extremely tuned much more
| minimalist framework on what is practically a micro
| benchmark is far from the savage fall from civilization &
| decadence of man, deserving far less scorn and shame than
| what the cult of js hate fuels their fires on.
|
| I could go on about how JS has benefitted from having
| incredibly work poured into it, because it is the most
| popular runtime on the planet, because it's everywhere,
| because there was a hot war for companies to try to beat
| each other on making _their_ js runtime good (one of the
| only langs with many runtimes is _interesting_ ). It's a
| bit excessive & maybe it should have been a more
| deserving language perhaps (we can debate endlessly), but
| man... It just doesn't matter. Stressing out, being so
| mean and nasty (huffy vs Kawasaki, @hoppp's even worse
| top-voted half sentence snark takedown), trying to
| impress upon people this image that it's all just so bad:
| I think there's such a ridiculous off kilter tilting way
| too hard, and far less of it is about good reasons and
| valid concerns, and so so so much of it is this bandwagon
| of negative energy, is radical overconcern.
|
| Like most tools & languages, _it 's what you do with it._
| With JS, we have a problem that (mainstream) software
| hadn't faced before, which is client server architecture,
| that the client might be a cruddy cheap phone somewhere
| and/or on a dodgy link. We are trying to build
| experiences that work across systems, with significant
| user perceived latency sometimes. And so data & systems
| _architecture_ matters a lot. Trying to get keep the
| client primed with up to date data that it needs to
| render, doing client work without blocking /while
| maintaining user responsiveness, are hard fun multi-
| threaded (webworker) challenges, for those folks that
| care.
|
| And those challenges aren't unique to js. Other languages
| have similar challenges. Trying to multithread a win32 UI
| to avoid blocking also was a bit of a nightmare, working
| off main thread. Doing data sync is a nightmare. There's
| so many ways to get stuff wrong. And I think a lot of the
| js code out there does get it wrong. But we experience
| hundreds or thousands of websites a week, and crucial
| tools we use that are js client-server are badly
| architected. I _sympathize_ with why js has such a bad
| rap. To me it usually seems like architectural app design
| issues, that companies are too busy building features to
| really consider the core, to establish data architectures
| that don 't block or lag. And that's not a specific js
| problem.
|
| There are faster systems yes, but man, the energy being
| poured into blaming the worst of the world on JS seem
| ridiculous to me, like such a sad drag, that avoids any
| interest or fascination on what is so so interesting & so
| worthy. The language is the least remarkable part of the
| equation, practically doesn't matter, and the marginal
| performance levels are (with some exception for very
| specific cases) almost never a remotely critical factor.
| Just so so so tired, knowing such enormous and pointless
| frivolous scorn and disdain is going to overwhelm all
| conversations, going to take over every thread forever,
| when it matters so so little, is so very rarely a major
| determinant.
|
| JS does not _have_ to be the thought terminating cliche
| to every thread ever (and humbly, I 'd assert it doesn't
| deserve that false conviction either. But either way!).
| mystraline wrote:
| Tl;dr.
|
| Care to summarize?
| jauntywundrkind wrote:
| Architecture is far more important than runtime speed.
| (People are so easily swayed by "JS SUCKS LOL" because of
| experiences with terrible & careless client-server
| architectures, far more than js itself being "slow".)
|
| The people ripping into js suck up the interesting
| energies, and bring nothing of value.
| dilyevsky wrote:
| Nothing about current js ecosystem screams good
| architecture it's hacks on hacks and a culture of totally
| ignoring everything outside of your own little bubble.
| Reminds me of early 2000s javabeans scene
| IgorPartola wrote:
| If we are discussing C10K we are by definition discussing
| performance. JavaScript does not enter this conversation
| any more than BASIC. Yes of course architecture matters.
| Nobody has been arguing otherwise. But the point is that
| if you take the best architecture and implement it in the
| best available JS environment you are still nowhere close
| to the same architecture implemented in a systems
| language in terms of performance. You are welcome to your
| wishful thinking that you do not need to learn anything
| besides JavaScript when it comes to this conversation.
| But no matter how hard you argue it will continue being
| wishful thinking.
|
| We are discussing tech where having a custom userland TCP
| stack is not just a viable option but nearly a
| requirement and you are talking about using a lighter JS
| framework. We are not having the same conversation. I
| highly recommend you get off Dunning-Kruger mountain by
| writing even a basic network server using a systems
| language to learn something new and interesting. It is
| more effort than flaming on a forum but much more
| instructive.
| jauntywundrkind wrote:
| Techempower isn't perfect, but a 33% loss of speed is
| really not bad versus best of the best web stacks. Your
| attempt to belittle is the same tired heaping scorn, when
| the data shows otherwise. But it's so cool to hate!
|
| Libuv isn't the most perfect networking stack in the
| planet, no. It could be improved. We could figure out new
| ways to get io-uring or some eBPF based or even dpdk
| bypass userland stack running. Being JS does not limit
| what you do for networking, in any way. At all. It adds
| some overhead to the techniques you choose, requires some
| glue layer. So yes, some cpu and memory efficiency
| losses. And we can debate whether thats 20% or 50% or
| more. Techempower shows it's half an order of magnitude
| worse (10^0.5, 33%). Techempower is a decent proxy for
| what is available, what you get, without being extremely
| finicky.
|
| Maybe that is the goal, maybe this really is a shoot for
| the moon effort where you abandon all existing work to
| get extreme performance, as c10k was, but that makes the
| entire lesson not actually useful or practical for almost
| everyone. And if you want to argue against techempower
| being a proxy for others, for doing extreme work & what
| is possible at the limit, you have to consider a more
| extreme js networking than what comes out of box in
| node.js or Deno too, into a custom engine there too.
|
| It's serious sad cope to pretend like it's totally
| different domains, that js is inherently never going to
| have good networking, that it can't leverage the same
| networking techniques you are trying to vaunt over js.
| The language just isn't that important, the orders of
| magnitude (<<1 according to the only data/evidence in
| this thread) are not actually super significant.
| tguvot wrote:
| 33% loss of speed is really not bad if you don't care
| about 33% loss of speed.
| IgorPartola wrote:
| Look you seem to have made up your mind and are unwilling
| to listen to knowledge or experience. The tech industry
| does not do well with willful ignorance so I wish you
| luck with all that.
| jauntywundrkind wrote:
| > _unwilling to listen to knowledge and experience_
|
| I'm more than happy to listen! But there's not really
| content or lessons happening. It's just an endless stream
| of useless garbage no effort dunking and "trust me bro."
|
| I would be far less offended by the "js sucks lol"
| thought terminating cliche & dunking if they came at all
| correct. But I'm the only one citing any evidence here
| and your assertions that better networking is impossible
| in js, js could never userland bypass, just don't wash or
| make sense.
|
| _I don 't think js is ideal to spend all your time
| optimizing for._ It's a bit ridiculous to imagine a
| userland kernel bypass like dpdk that's married to v8.
| But it doesn't seem inconceivable at all.
|
| Assume both of us are being willfully to our ends. I'd
| way rather be willfully of possibility, than to be so
| ironclad locked down assured beyond belief willful of
| impossibility. I think that's an insult to the hacker
| spirit. And it's a sad fate that every conversation has
| the same useless loud bellowing that JS is impossibly
| bad; this thought terminating cliche is a wicked evil to
| inflict on the world, adding chiefly to din. _Especially_
| when it never bring any evidence, when it 's so self
| assured.
| hinkley wrote:
| I was a Java programmer out of school. My first
| specialization became performance. It's the same problem
| with JavaScript. There's at least an order of magnitude
| speed difference between using your head and just
| vomiting code, and probably more like a factor of forty.
| Then use some of the time you saved to do deep work to
| hit an even two orders of magnitude and it can be fast.
| If you're not an idiot that is.
|
| bcantril has an anecdote about porting some code to rust
| and it was way faster than he expected. After digging he
| found it was using btrees under the hood, which was a big
| part of the boost. B-trees being a giant pain in the ass
| to write in the donor language but a solved problem in
| Rust. It's a bit like that. You move the effort around
| and it evens out.
| winrid wrote:
| (to be fair the memory manager reuses memory, so it's not
| calling out to malloc all the time, but yes a manually-
| managed impl. will be much more efficient)
| IgorPartola wrote:
| Whichever way you manage memory, it is overhead. But the
| main problem is the language does not have zero copy
| semantics so lots of things trigger a memcpy(). But if
| you also need to call malloc() or even worse if you have
| to make syscalls you are hosed. Syscalls aren't just
| functions, they require a whole lot of orchestration to
| make happen.
|
| JavaScript engines also are also JITted which is better
| than a straight interpreter but except microbenchmarks
| worse than compiled code.
|
| I use it for nearly all my projects. It is fine for most
| UI stuff and is OK for some server stuff (though Python
| is superior in every way). But would never want to
| replace something like nginx with a JavaScript based web
| server.
| gbuk2013 wrote:
| IIRC V8 actually does some tricks under the hood to avoid
| malocs which is why Node.js can be be unexpectedly fast
| (I saw some benchmarks where it was only 4x of equivalent
| C code) - for example it recycles objects of the same
| shape (which is why it is beneficial not to modify object
| structure in hot code paths).
| hinkley wrote:
| Hidden classes is a whoooole thing. I've switched several
| projects to Maps for lookup tables so as not to poke that
| bear. Storage is the unhappy path for this.
| drogus wrote:
| Worker threads can't handle I/O, so a single process
| Node.js app will still have the connection limit much
| lower than languages where you can handle I/O on multiple
| threads. Obviously, the second thing you mention, ie.
| multiple processes, "solves" this problem, but at a cost
| of running more than one process. In case of web apps it
| probably doesn't matter too much (although it can hurt
| performance, especially if you cache stuff in memory),
| but there are things where it just isn't a good trade-
| off.
| hinkley wrote:
| And I have confirmed to my own satisfaction that both PM2
| and worker threads have their own libuv threads. Yes very
| common in node to run around one instance per core, give
| or take.
| hinkley wrote:
| Libuv now supports io_uring but I'm fuzzy on how broadly
| nodejs is applying that fact. It seems to be a function by
| function migration with lots of rollbacks.
| drob518 wrote:
| FFS, yep. Sigh.
| _qua wrote:
| I personally think it's more of a https://c25k.com/ time of year.
| alwa wrote:
| Apparently this refers to making a web server able to serve
| 10,000 clients simultaneously.
| IgorPartola wrote:
| It has been long enough that C10K is not in common software
| engineer vernacular anymore. There was a time when people did
| not trust async anything. This was also a time when PHP was
| much more dominant on the web, async database drivers were rare
| and unreliable, and you had to roll your own thread pools.
| amelius wrote:
| Yes. But it's easy to reinvent it, with modern OSes and tools.
| readthenotes1 wrote:
| The internationally famous Unix Network Programming book. An
| icon, a shibboleth, a cynosure
|
| https://youtu.be/hjjydz40rNI?si=F7aLOSkLqMzgh2-U
|
| (From Wayne's World--how we knew the comedians had smart
| advisors)
| senko wrote:
| The date (2003) is incorrect. The article itself refers to events
| from 2009, is listed at the bottom of the page as having been
| last updated in 2014, with a copyright notice spanning 2018, and
| a minor correction in 2019.
| throwaway29303 wrote:
| The date (2003) is incorrect.
|
| You're right, it's even older than that; it should be (1999).
|
| https://web.archive.org/web/*/https://www.kegel.com/c10k.htm...
| petters wrote:
| Seems to be some kind of living document. It was refered to as
| an "oldie" in 2007: https://news.ycombinator.com/item?id=45603
| zahlman wrote:
| > And computers are big, too. You can buy a 1000MHz machine with
| 2 gigabytes of RAM and an 1000Mbit/sec Ethernet card for $1200 or
| so. Let's see - at 20000 clients, that's 50KHz, 100Kbytes, and
| 50Kbits/sec per client. It shouldn't take any more horsepower
| than that to take four kilobytes from the disk and send them to
| the network once a second for each of twenty thousand clients.
| (That works out to $0.08 per client, by the way. Those
| $100/client licensing fees some operating systems charge are
| starting to look a little heavy!) So hardware is no longer the
| bottleneck.
|
| It seems to me that there are far fewer problems nowadays with
| trying to figure out how to serve a tiny bit of data to many
| people with those kinds of resources, and more problems with
| _understanding how to make a tiny bit of data relevant_.
|
| It still absolutely can be. We've just lost touch.
| nine_k wrote:
| This particular case, with the numbers given, would work as a
| server for profile pictures, for instance. Or for package
| signatures. Or for status pages of a bunch of services
| (generated statically, since the status rarely changes).
|
| Yes, an RPi3 might be adequate to serve 20k of client requests
| in parallel, without crashing or breaking too much sweat. You
| usually want to plan for 5%-10% of this load as a norm if you
| care about tail latency. But a 20K spike should not kill it.
| cxr wrote:
| Stop fucking editorializing the fucking submission titles.
___________________________________________________________________
(page generated 2025-12-28 23:01 UTC)