[HN Gopher] My PS4 a month server can handle 4.2M requests a day
___________________________________________________________________
My PS4 a month server can handle 4.2M requests a day
Author : mark_mcnally_je
Score : 253 points
Date : 2021-09-07 20:01 UTC (2 hours ago)
(HTM) web link (mark.mcnally.je)
(TXT) w3m dump (mark.mcnally.je)
| taf2 wrote:
| I wonder what the 99th - 95th percentile response times look
| like...
| Wronnay wrote:
| If you host it on GitHub Pages, GitLab Pages or Vercel it's even
| PS4 less :o
| reilly3000 wrote:
| That estimate can be verified with load testing systems like
| Artillery. My theory is that things would break far sooner than
| estimated along the following lines:
|
| - Too many WSGI connections if the timeouts aren't tweaked
|
| - Too many database connections, especially without caching and
| tuning
|
| - on the Apache side if MaxRequestWorkers isn't set there will be
| memory issues with 1GB RAM
|
| - the disk could easily hit IOPS limits, especially if there is a
| noisy neighbor
|
| It's not likely all or any of these things will hit IRL, but that
| all depends on traffic and usage patterns. It matters not, if you
| were getting 4.2 M requests each day you'd be in the Alexa Top
| 1000 and could probably shell out for the $8 server :)
| markandrewj wrote:
| Normally benchmarks for things like this are measured in how many
| concurrent requests can be handled, i.e the C10K problem, not by
| how many requests you are able to serve in a day. It's also well
| known that you can serve a large amount of requests on limited
| hardware.
|
| https://en.wikipedia.org/wiki/C10k_problem
|
| "By the early 2010s millions of connections on a single commodity
| 1U rackmount server became possible: over 2 million connections
| (WhatsApp, 24 cores, using Erlang on FreeBSD),[6][7] 10-12
| million connections (MigratoryData, 12 cores, using Java on
| Linux).[5][8]"
|
| Although I do understand the boxes listed above have more
| resources then the VPS you are using. I am also not criticizing
| your write up, or results, bench-marking is in general
| interesting to do. I just wanted to provide some additional
| information.
| habibur wrote:
| Right. I calculated what 5m/day converts into. And it's like 60
| req/sec. Considering non even distribution and spikes, I would
| assume its like 200req/sec.
| adtac wrote:
| unrelated but 4x increase isn't really a spike
| ijidak wrote:
| How does a single server run millions of active connections?
|
| Wouldn't you run out of TCP sockets?
|
| What am I missing?
| dreyfan wrote:
| A connection isn't just a dest_port, it's the unique
| combination of 4 components:
| source_ip:source_port:dest_ip:dest_port
| [deleted]
| arthurcolle wrote:
| socket multiplexing
| jfrunyon wrote:
| I would guess they may be muxed over fewer sockets, by their
| LBs, but that's not strictly necessary.
|
| I'm not sure exactly what you mean by "run out of TCP
| sockets", but theoretically speaking, the only limitation is
| how much memory is available to store the necessary info
| about the socket (like address/protocol info and process
| info).
|
| In practice, OS's do have a "max socket" or "max FD" limit,
| but that's usually configurable and (with enough RAM) could
| easily be set to "millions".
| polote wrote:
| What's the point of this post? OP is serving a file at 50req/sec.
| There is not even mention of a dB query. How is that able to
| relate to any kind of normal app?
|
| I guess that the post was written as an answer to the mangadex
| post [1]. Mangadex was handling 3k req/sec involving dB queries.
| It was not just a cached Html page.
|
| 50req/sec for a Html file is super low which shows that a $4
| month server cant do much actually. So yes this is enough for a
| blog, but a lot of websites are not blogs
|
| [1] https://news.ycombinator.com/item?id=28440742
| great-potential wrote:
| Exactly, I dont see the point bragging about this nevertheless
| posting about it on HN ...
| [deleted]
| TruthWillHurt wrote:
| +1. I remember modest VPS/Parrallels serving PHP at 350r/s
| quickthrower2 wrote:
| Something like: <?php echo("this is a
| benchmark") ?>
| napworth wrote:
| It's called boasting
| gaptoothclan wrote:
| Is this just an apache bench mark
| vymague wrote:
| > What's the point of this post? OP is serving a file at
| 50req/sec.
|
| I'd guess a response to the mangadex thread?
| https://news.ycombinator.com/item?id=28440742
| throwdecro wrote:
| > How is that able to relate to any kind of normal app?
|
| There's too much competition involved in writing normal apps,
| which often attract significant investment that bootstrapped
| startups struggle to compete with.
|
| It's interesting to see what kind of performance is possible
| for next to no money, when you throw out basic assumptions like
| using a database, and then start thinking about what you could
| build out of it.
| quickthrower2 wrote:
| My recent submission of HNBadges was made like this. It's
| just 3 files (html, css, JS) which I hosted for free on
| Netlify, but could have been hosted on a setup like OP. I
| used other services for XHR requests. I imagine it got a
| tonne of traffic from being on the first page, but I wasn't
| taking metrics.
|
| Another example of clever use of resources is the
| https://haveibeenpwned.com/ website. Using a bloom filter (I
| think) to turn what could have been a back-end lookup into a
| "front end lookup" by requesting a small file from the server
| based on the password hash.
|
| The only issue I have with the OP is his assumption that
| you'd get a nice smooth 60 request/second throughout the day!
| Most likely will be lumpy, and in the top of the lumpy
| periods (where most of your visitors visit) performance will
| be bad.
| Arch-TK wrote:
| >There is not even mention of a dB query.
|
| Did you read the post?
| yupper32 wrote:
| Well the front page of HN won't get you 4.2M views today, but
| it's a pretty good real world test!
| mark_mcnally_je wrote:
| It is! It's crashed my analytics but the website seems to be
| doing fine.
| jonplackett wrote:
| True, but it may very well get you a lot more than 50 a minute.
| louwrentius wrote:
| I'm hosting my static blog site on a physical Raspberry Pi 3B+
| powered by Solar [1].
|
| That blog post got hugged by HN but it didn't even raise the CPU
| above 10% on a single core.
|
| And a Raspberry Pi 3B+ is dog slow. And severely limited by
| bandwidth, unlike the Raspberry Pi 4B+. (But it uses less power
| so that's why I use a 3B+).
|
| However I have another point to make. Professional rack-mount
| servers from HP and Dell can be had second hand for dirt cheap
| and you get a ton of CPU (20+ cores) and an ocean of RAM for next
| to nothing.
|
| For many applications, an old Gen8 or similar Dell server will
| perform more than adequately. Even more so if you have a little
| bit more to spend on Gen9.
|
| They are so cheap that you can like buy four to eight, sprinkle
| them across two different datacenters and even if one breaks, you
| won't be in any hurry.
|
| [1]: https://louwrentius.com/this-blog-is-now-running-on-solar-
| po...
| 5faulker wrote:
| That's a good spec for 4 bucks. With cloud hosting you might be
| able to push the cost down a bit with less CPU resource and
| memory.
| mark_mcnally_je wrote:
| I am cloud hosting using OVH cloud, I brought the server a
| couple of years ago now so they probably have some better specs
| for the price I am paying.
| zxcvbn4038 wrote:
| Combined with a CDN and you can do a lot more, I like to think in
| terms of origin requests rather than raw req/s. The problem I
| have is getting people to really understand how caching works at
| the cdn and browser layers and design their frontends and backend
| responses around that. There is also lot you can do with edge
| compute to clean the incoming requests before the CDN evaluates
| them to increase cache hits. Even if you are trying to give a
| "real time" view of data, caching it for even a second and
| allowing stale data to be served while it is updating can reduce
| origin requests significantly. I've seen people hammer sites
| hundreds of times a second looking for changes that only happen
| once every fifteen seconds or once per minute - the best thing
| you can do for yourself is handle all those requests at the CDN
| level (eventually you'll do log analysis and see the activity and
| can take other measures, but in the mean time don't let all of
| those requests go to the origin). Your CDN is probably giving you
| better rates for network egress then Amazon or Google anyway -
| the later are more focused on incentivizing you to use their
| ecosystem exclusively by penalizing you for sending data
| "outside". Cheap VPS hosts discourage you exceeding your
| bandwidth allocation because they are overselling their capacity
| and heavy usage upsets that - so again you want to shift as much
| as you can to your CDN.
| ppeetteerr wrote:
| How did this make it to the number two spot on HackerNews?
| [deleted]
| [deleted]
| ok_coo wrote:
| IMHO, it's better than a deluge of political posts.
|
| I don't mind reading about politics but I come to HN to read
| about tech. We can go elsewhere to get whatever politics we
| desire.
| dreyfan wrote:
| Broadly speaking people on HN have no clue how to setup a
| performant httpd/app server and are impressed by abysmal
| performance/cost metrics like this or the MangaDex post.
| Everything these days is obscured through multiple layers of
| SaaS offerings and unnecessary bloat like kubernetes.
|
| ~10k rps (it was concurrent connections but close enough) was
| state of the art in 1999. Now 22 years later ~50 rps is somehow
| impressive.
| watermelon0 wrote:
| I would guess this is close to a default Apache + mod_wsgi
| setup, and this is one of the easiest way of hosting Python
| web apps, so basically achievable by anyone on HN.
|
| I assume (based on 180 req/s for static page) that he is
| using mpm_prefork, where each Apache child handles a single
| connection. If he switched to mpm_event, which uses event
| loop like nginx, ~10k rps should easily be achievable, but I
| don't think WSGI would work with that.
| mark_mcnally_je wrote:
| Yeah it's all default apache + mod_wsgi. This is also my
| first Django setup and I made it over-complicated as a
| learning exercise. mpm_event is something I have not heard
| of before, thanks for bringing that to my attention.
| ppeetteerr wrote:
| The author provides no information other than to say it's an
| Apache + Django setup. That's pretty bare-bones and is part
| of most tutorials online.
| reilly3000 wrote:
| In my opinion it isn't so much of a lost art or lamentable
| accretion of useless abstractions, but an increase in the
| scope of what web apps do these days. Most of us aren't
| working on static sites or simple CMS publishing- those are a
| solved problem. Instead we're building mobile banks,
| diagnostic systems, software tooling, 3D games, and shopping
| malls. The complexity is inherent in the maturity of the web
| and it's many uses, as well as its global scale. Hugs of
| death are rare these days thanks to better architecture and
| infrastructure, even though the scale of users has grown
| 100X.
|
| Yes there are wildly unnecessary abstractions that are used
| for small sites/apps, but I would contend they are artifacts
| of someone who it's trying to learn something new, and/or get
| promoted. I have no problem with the former.
| Zababa wrote:
| > ~10k rps (it was concurrent connections but close enough)
| was state of the art in 1999. Now 22 years later ~50 rps is
| somehow impressive.
|
| I honestly don't understand how that can be true. I'm not
| suggesting you're lying of course, but when you put it this
| way it's almost like people are actively trying to slow their
| programs down. I have a few ideas on why that might be the
| case (switch to slow interpreted languages, switch to bigger
| web frameworks, bigger payload) but even that wouldn't
| explain all of it. Do you have any idea why things are this
| way?
| crazy_horse wrote:
| This is just basic use the right tools for the right job
| 101. You've got what is basically a static website. You
| want to serve static files. To do that, you use a fast
| language and/or servers written in those languages.
|
| It's something anyone who has done this for any length of
| time knows, that HN is impressed by this is confusing to
| some of us. If you were _trying_ to get as little out of
| your server as possible you 'd serve cached content using
| this framework in this language.
|
| Is this stuff not being learned?
| Zababa wrote:
| I thought you meant from 10k to 50 rps doing the same
| work, not that most of the work could be avoided in the
| first place.
|
| > Is this stuff not being learned?
|
| I don't know if it is. I recently finished my studies,
| and most people had no curiosity at all. As in, they
| learned a framework early, used it everywhere, and got a
| job using it. I do remember reading a few times on
| tutorial that you should put a Nginx as reverse proxy in
| front of your Django/Flask/Express server to server
| static files, so I think most people know/do that but I'm
| not sure.
|
| On the other hand, having the wisdom of knowing what can
| be static in the first place? I don't think that it's
| something teached. In fact this kind of wisdom can be
| hard to find outside of reading lots of sources
| frequently in hope of finding little nuggets like that. I
| don't think I was ever taught explicitely "You should
| first try to find out if the work you're trying to do is
| necessary in the first place". In a way it's encoded in
| YAGNI, but YAGNI isn't universal, and is usually
| understood at the code level and not the tooling level.
| wpietri wrote:
| An awful lot of professional programmers work in such
| heavyweight contexts that they don't have a good idea of how
| fast modern hardware can be.
|
| I was talking with an architect at a bank whose team was having
| trouble getting under a 2-second maximum for page views. They
| blamed it on having to make TCP requests to other services, and
| said something like "at a couple hundred milliseconds per
| request, it adds up quickly!" My head nearly exploded at that.
| I spun up some quick tests in AWS to show exactly how many
| requests one could make in 2000 ms. I don't have the numbers
| handy, but the number is very large.
|
| This junky slice of a server handling full page requests in 20
| ms is a fine example to counter thinking that's endemic in
| enterprise spaces.
| ppeetteerr wrote:
| This is a person running a blog and pulling HTML data from a
| database. Is it that impressive that they are getting 50r/s?
| wpietri wrote:
| It's not.
|
| My point isn't that it's impressive in some ultra-tuned
| performance sense. It's that doing pretty mundane things on
| pretty basic servers is still very fast compare with a) the
| past, or b) what a lot of developers are used to
| professionally. That's why it is interesting to the crowd
| here.
| dragontamer wrote:
| I had a discussion with a coworker about something about slow
| memcpys, that roughly went the same way except... you know,
| DDR4 RAM has a speed of roughly 40GBps.
|
| Also, that "awful" 1MB memcpy is likely all in L3 cache these
| days. But even if it weren't in cache, we're talking about an
| operation that takes 50 microseconds (1MB read + 1MB written
| == 25microseconds + 25 microseconds).
|
| Given that modern CPUs have like 16+ MBs of L3 cache (and
| more), and some mainstream desktop CPUs have 1MB of L2
| cache... its very possible that this memcpy is far faster in
| practice than you can imagine.
|
| 1MB is big, its a million bytes. But CPUs are on the scale of
| billions, so 1MB is actually rather small by modern
| standards. Its surprisingly difficult to get intuition
| correct these days...
| ComputerGuru wrote:
| Absolutely. Even a terrible Wordpress instance can be beautifully
| (and transparently!) cached behind either nginx or varnish with
| ease, in which case you're just serving static html pages and can
| probably handle any traffic you are likely to ever get.
| MangoCoffee wrote:
| this story remind me, the dot com bubble. dotcom companies bought
| servers from Sun Microsystem. they needs to handle the large
| traffics that "PC" server can't handle.
|
| anyone remember Cobalt server?
|
| https://en.wikipedia.org/wiki/Cobalt_Networks#/media/File:Co...
| sgt wrote:
| I remember many stories and discussions on Slashdot about it.
| johnklos wrote:
| This is a good, simple way to show how much can be done with
| modest resources.
|
| Sometimes we see people fetishizing bigger and faster, then
| gatekeeping when people want to do the same work with modest
| means, whether it a four quid a month hosting service or a first
| generation Raspberry Pi. Not everyone has the money or desire for
| bigger & faster, and it's nice to see that here.
| mark_mcnally_je wrote:
| That's why I made this post, was very happy to see how much it
| could theoretically handle.
| jonplackett wrote:
| #1 on HN and still up. That speaks for itself.
| mark_mcnally_je wrote:
| :)
| jonplackett wrote:
| Are your starting to feel the pressure yet?
| mark_mcnally_je wrote:
| My blog is fine, my self hosted analytics (on another
| server) not so much!
| idworks1 wrote:
| One of my proudest moment in my career is when I lowered our app
| processing time from ~8hrs to 17 minutes. When I deployed my
| first update, it reduced it to 2 hours. The sysadmin immediately
| contacted me that there was something unusual. I confirmed the
| results but he was skeptical.
|
| Then with my second update, he told me that the app must be
| broken or that the script must be dying. There is no way it could
| complete this fast.
|
| What was the issue? We processed terabytes of data. Each and
| every single line processed created a new connection to the
| database and left it hanging. A try catch was added when the
| connections failed and restarted the process. Removing the
| connection from the for loop and properly handling it reduced the
| time drastically.
|
| And... why would you loop through millions of records when you
| can use batches? Also this was a phperlbashton* script. I turned
| it into a single PHP script and called it a day.
|
| As a consequence, backup time was reduced to 2 hours as opposed
| to 12 hours (no one was allowed on the website until the back up
| was done).
|
| Modern machines are incredibly fast.
|
| * PHP/Perl/Bash/Python
| great-potential wrote:
| omg what is this, we need some Brendan Gregg posts to elevate
| the level here, this is HN not reddit :D
| jaymzcampbell wrote:
| I'm not sure there was ever an argument saying otherwise. The
| ease of processing X million requests is heavily dependant on
| what those requests actually _do_. Trivial use cases shouldn 't
| be a surprise to have high throughput.
| fbrchps wrote:
| I understand your excitement for being able to handle a decent
| amount of requests on such a small server, but just like many
| other websites that get on the frontpage of HN, your site is
| taking multiple seconds to load for me, depending on when I
| refresh.
|
| As you said in your post, adding caching to your site increased
| your throughput by ~20% (or +10/req/sec). What you and other
| sites seem to lack is a more distributed caching, a la
| CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only
| really work well for a static site, however as mentioned in your
| post that's essentially what you're serving.
|
| While I'm all for having a free-as-in-freedom hosting solution
| and keeping things lean, the internet is a fickle beast, and
| nothing looks worse for a company who posts on HN when their
| technology-oriented site can't handle a few thousand requests per
| minute. (Or in this case, when a blog claims to handle 4.2M
| requests a day -- 2.9k req/min)
| jeroenhd wrote:
| Are you sure that's related to the server itself? The page
| loads instantly for me and the DNS is still resolving to an OVH
| IP address.
|
| Timing info from Firefox: Blocked: 0ms DNS resolution: 8ms
| Connecting: 9ms TLS setup: 12ms Sending: 0ms Waiting: 30ms
|
| The very last resource (favicon.ico) loaded after 466ms and
| that's mostly because of the other files being requested only
| after the CSS has come in (after about 195ms). All in all the
| entire site (without the Matomo tracking JS) loaded in half a
| second.
|
| Maybe the website has switched hosts in the last ten minutes, I
| guess, but I doubt it. I think this is more likely to be a
| problem related to distance to origin and saturation of the
| underlying connection.
| mark_mcnally_je wrote:
| Nope, website has not changed at all - speaking of the Matomo
| tracking; that is running on a separate server and has
| actually crashed!
| _fizz_buzz_ wrote:
| Weird, this site loads really fast for me actually. Much faster
| than most sites that I visit.
| mark_mcnally_je wrote:
| Thanks! Just goes to show that it runs quickly even when it
| hits #1 on HN ;)
| fbrchps wrote:
| Hey OP, in case it wasn't clear from my original comment, I
| am impressed with how stable your site is! "A few seconds"
| is wonderful for a #1 post, and far better than what
| usually happens to lightly-hosted sites that get to this
| point. Those are usually timeouts or outright failures to
| connect.
|
| I just figured I'd start some conversation on the post,
| since there wasn't any comments when I initially looked.
| For better or for worse it seems like I got people talking.
| spicybright wrote:
| Working extremely fast for me right now. Your post was about 20
| minutes ago. I don't know how HN traffic fluctuates, but it
| seems really solid compared to most sites.
| umvi wrote:
| Loaded instantly for me
| crazy_horse wrote:
| 200 rps is not great.
| fridif wrote:
| Wikipedia as a whole has 8k rps, and that's with multiple
| racks in multiple data centers.
|
| I haven't read recently, but they were only doing 200 rps per
| server.
| floren wrote:
| Looks fine over here, and he doesn't have to route through a
| fucking Internet gatekeeper like Cloudflare or Amazon... let's
| enjoy this golden era before Chrome starts flagging any site
| which isn't fronted by a "reputable" cache like Cloudflare,
| Amazon, or whatever Google decides to introduce.
| fbrchps wrote:
| It would also be possible for OP to spin up their own Redis
| cache, and have multiple POPs near their target audience, and
| handle DoS type attacks against their site if need be, and
| easily be able to brush aside bot traffic, and...
|
| Not all the above apply to a hobby-blog style site, but I
| wasn't referring only to OP's site in my original comment. I
| understand that not everyone needs to feed into "fucking
| Internet gatekeeper"s as you described, but the fact that
| they provide valuable services is undeniable. They make a
| complex operation -- one that could mean the difference
| between a company being able to sell their product or not --
| simple.
| antihero wrote:
| Could just install Varnish locally.
| mark_mcnally_je wrote:
| Yeah a varnish install locally is the next step up from
| what I am doing I guess. Spinning up multiple POPs +
| cloudflare is way to much IMO
| fbrchps wrote:
| Haven't used Varnish myself directly, but yeah that would
| also work.
|
| For OP, I'd also be interested to see the benchmarks
| between this PS4 server, and a PS8 or PS10 one, same
| stack.
| ezfe wrote:
| Oh I hate Google as much as the next guy, but that's not
| something they've shown any interest in doing.
| shtps wrote:
| Google AMP comes to mind.
| JohnWhigham wrote:
| People like you also probably thought the "Don't be evil"
| slogan would be around forever too, huh?
| mark_mcnally_je wrote:
| Yeah if I was building a business website I would want
| distributed caching/a CDN, mainly to support spikes, like what
| is happening now!
| fbrchps wrote:
| Working in the space, that's one of the more frustrating
| things to see on HN/Reddit/etc. It's not a complex or niche
| thing, and especially for sites that only make profit when
| people can actually visit them, it's kind of a necessity to
| stay up as much as possible.
|
| (Obviously the sales thing doesn't apply to OP)
| tomrod wrote:
| For those of us that are new to the space, lack background,
| or wear 50 hats in a startup, can you point us to best
| practices here?
| tick_tock_tick wrote:
| Just throw cloudflare in front it's free.
| fbrchps wrote:
| I don't have a "Do all These for a Fast Website" list
| handy, but here are some key points I've found can be
| applied to most sites:
|
| - Make sites that are fast by default: Small bytes sent
| over the wire, beyond just initial page load, too. Yes,
| that does mean that your giant Google Tag
| Manager/Analytics/3rd party script is bloated. Reach out
| to 3rd parties about reducing their payload size, it's
| saved me several MB over the years. Also, not writing
| efficient CSS is a huge killer when it comes to byte
| site. Devs shouldn't "leave it just in case" when it
| comes to code, you have version control for a reason. And
| when a new feature comes out, clear out the old cruft.
|
| - Avoid unnecessary DB calls: Obviously you need to get
| the data onto the page somehow, but if you can server-
| side render, then cache that result, you're reducing the
| overall calls to the DB. Also, optimizing queries to
| return only-what-you-need responses helps reducing total
| bytes over the wire
|
| - Balance between Server and Client side: Not only are
| servers getting more powerful, so are client devices.
| Some logic can be offloaded to clients in most cases, but
| there needs to be a balance. Business-critical logic
| should probably be done server side, but things like
| pagination & sorting -- so long as they client will
| likely see or use all the data -- is fine in my book.
| Having 2000 rows of JSON in memory is totally OK, but
| rendering 2000 at once might cause some issues. Again,
| balance
|
| - Hopping on the latest-and-greatest bandwagon isn't the
| best: Devs hate re-writing the site every 6 months, and
| really the newest framework might not be the best for
| your use case. Keep up to date with new technology, but
| saying "not for me" is fine.
|
| - Don't let (non technical) managers make technology
| decision: See above. More often than not, C-levels want
| to use shiny new things they read an article about on
| LinkedIn once, no matter if it fits the needs of the
| company or not. Thankfully I've only been at one place
| that was like that, but while I was there it was hell.
| Current VP was an original developer on the site back in
| the early 00's, so he knows how to deflect BS for us.
| That VP also knows that he's outdated in his knowledge by
| now, so he trust the Devs to make technical decisions
| that are best for the company.
| blfr wrote:
| _your site is taking multiple seconds to load for me, depending
| on when I refresh._
|
| Barely over a second here. Much better than vast majority of
| "webscale" services.
| fbrchps wrote:
| For sure, OP's site is handling this much better than most.
| And like I said, it's not every time that it takes multiple
| seconds. Some websites featured on HN/Reddit don't load at
| all when under load. However I was able to get it to take
| ~30s to load multiple times, over a period of around 10
| minutes.
| Tenoke wrote:
| In their defence the sites that fail to load or take too
| long are usually full webapps that do a lot of work rather
| than just static sites.
| fabian2k wrote:
| The important message there is that if you can change
| your problem from serving slow dynamic content to serving
| static content you can gain enormous performance
| benefits.
|
| Whether that means actually using static sites for stuff
| that can be static or just properly caching expensive
| things. Even dynamic content doesn't have to be slow, but
| many CMS are seriously inefficient without a cache. I'm
| not really blaming the CMSes entirely here, part of that
| is because they need to be extremely flexible, but once
| you need dozens of DB queries per page it'll fall over
| quickly on small hardware.
| fbrchps wrote:
| We're using Next.js at my current company with a custom
| MongoDB based CMS.
|
| Next has a thing called Incremental Static
| Regeneration[0] which allows us to grab the top ~100
| pages from the CMS at build time, generate the pages,
| then cache them for however long we want. The rest of the
| pages are grabbed when requested, then are cached for the
| same amount of time. After the time, they're re-grabbed
| from the DB, then re-cached. Overall I think we're down
| to around 5-10% of the way things were done before, which
| was -- you guessed it -- hit the DB on every page load
| _just in case_.
|
| Sit the Next.js site behind CloudFlare, and then we also
| don't really pay data transfer costs. Our servers are
| just low-tier GKE nodes, and we run around 3k/visitors at
| any given time, sometimes spiking up to 8k concurrent.
|
| [0] https://nextjs.org/docs/basic-features/data-
| fetching#increme...
| fabian2k wrote:
| Even database queries aren't that slow on reasonable
| hardware, as long as the queries are simple. The problem
| appears once you have dozens of DB queries per page. It's
| really not a fair comparison to the site this topic is
| about, but for trivial queries you can easily get a few
| thousand requests per second out of Postgres on desktop
| hardware without any real tuning as long as the DB fits
| into memory.
|
| But static content is of course still much faster and
| also much simpler.
| Rd6n6 wrote:
| Re: benchmarking, sometimes the bottleneck is the machine or
| server that issues the requests, not the receiver that you are
| testing. To figure out your actual capacity, you sometimes need
| multiple request servers or a more powerful request server. This
| was the case for a project I did a few years ago. Not a critique
| of the blog post, just remembering something out loud
|
| His site, https://peepopoll.com/, took about 10s to load for me.
| It's also good to chart other metrics like response times while
| you benchmark. Requests per second isn't the same as a low
| response time
| hu3 wrote:
| Indeed. Recently a client needed to bench raw req/s processing
| power of their application server and I had to ask for a
| powerful server running on the same DC in order to discard any
| potential routing issues.
| [deleted]
| nicoburns wrote:
| So 50req/sec. I'd hope it could handle a lot more than that!
| nostrademons wrote:
| People tend to severely underestimate how fast modern machines
| are and overestimate how much you need to spend on hardware.
|
| Back in my last startup, I was doing a crypto market intelligence
| website that subscribed to full trade & order book feeds from the
| top 10 exchanges. It handled about 3K incoming messages/second
| (~260M per day), including all of the message parsing, order book
| update, processing, streaming to websocket connections on any
| connected client, and archival to PostGres for historical
| processing. Total hardware required was 1 m4.large + 1 r5.large
| AWS instances, for a bit under $200/month, and the boxes would
| regularly run at about 50% CPU.
| danudey wrote:
| A lot of that is due to absolutely lousy code.
|
| We had a system management backend at my last company. Loading
| the users list was unbearably slow; 10+ seconds on a warm
| cache. Not too terrible, except that most user management tasks
| required a page reload, so it was just wildly infuriating.
|
| Eventually I took a look at the code for the page, which
| queried LDAP for user data and the database for permissions
| data. It did: get list of users
| foreach user: get list of all permissions
| filter down to the ones assigned directly to the user
| foreach user: get list of all groups
| foreach group: get list of all permissions
| filter down to the ones assigned to the group
| filter down to the ones the user has
|
| I'm no algorithm genius, but I'm pretty sure O(n^2+n^3) is not
| an efficient one.
|
| I replaced it with get list of all users
| get list of all groups get list of all permissions
| <filter accordingly>
|
| Suffice to say, it was a lot more responsive.
|
| Also worth noting was that fetching the user list required
| shelling out to a command (a python script) which shelled out
| to a command (ldapsearch), and the whole system was a
| nightmare. There were also dozens of pages where almost no
| processing was done in the view, but a bunch of objects with
| lazy-loaded properties were passed into the template and always
| used, so when benchmarking you'd get 0.01 seconds for the
| entire function and then 233 seconds for "return render(...)'
| because for every single row in the database (dozens or
| hundreds) the template would access a property that would
| trigger another SQL call to the backend, rather than just doing
| one giant "SELECT ALL THE THINGS" and hammering it out that
| way.
|
| Note that we also weren't using Django's foreign keys support,
| so we couldn't even tell Django to "fetch everything non-
| lazily" because it had no idea.
|
| If that app were written right it could have run on a Raspberry
| Pi 2, but instead there was no amount of cores that could have
| sped it up.
| JJMcJ wrote:
| Your pattern is quite powerful: get data from several sources
| and do the rearranging on the client (which might be a web
| server), instead of multiple interactions for each data item.
|
| For SQL you can also do a stored procedure. Sometimes that
| works well if you are good at your DBMS's procedure language
| and the schema is good.
| IfOnlyYouKnew wrote:
| I believe the parent's point was that code tends to be
| _faster_ than what people expect, not slower.
| namenotrequired wrote:
| I think the child's point is that people expect code to be
| slower than it is because they have seen code be slow far
| more than necessary.
| tra3 wrote:
| This is an example of N+1 problem [0]. It should be a
| FizzBuzz for anyone doing any CRUD apps.
|
| [0]: https://stackoverflow.com/questions/97197/what-is-
| the-n1-sel...
| fizwhiz wrote:
| Would you mind describing your stack in more detail? Did you
| use gRPC with Go?
| nostrademons wrote:
| Sure, startup is defunct now and I think arbitrage & data on
| centralized exchanges is a dead market now. Wall Street HFTs
| got into the arbitrage game, and the data sites laypeople
| actually visit are the ones started in 2014.
|
| Codebase was pure server-side Kotlin running on the JVM.
| Jackson for JSON parsing, when the exchange didn't provide
| their own client library (I used the native client libraries
| when they did). Think I used Undertow for exchange
| websockets, and Jetty for webserving & client websockets.
| Postgres for DB.
|
| The threading model was actually the biggest bottleneck, and
| took a few tries to get right. I did JSON parsing and
| conversion to a common representation on the incoming IO
| thread. Then everything would get dumped into a big
| producer/consumer queue, and picked up by a per-CPU
| threadpool. Main thread handled price normalization (many
| crypto assets don't trade in USD, so you have to convert
| through BTC/ETH/USDT to get dollar prices), order book
| update, volume computations, opportunity detection, and other
| business logic. It also compared timestamps on incoming
| messages, and each new second, it'd aggregate the messages
| for that second (I only cared about historical data on a 1s
| basis) and hand them off to a separate DB thread. DB would do
| a big bulk insert every second; this is how I kept database
| writes below Postgres's QPS limit. Client websocket
| connections were handled internally within Jetty, which I
| think uses a threadpool and NIO.
|
| Key architectural principles were 1) do everything in RAM -
| the RDS machine was the only one that touched disk, and
| writes to it were strictly throttled 2) throw away data as
| soon as you're done with it - I had a bunch of OOM issues by
| trying to put unparsed messages in the main producer/consumer
| queue rather than parsing and discarding them 3) aggregate &
| compute early - keep final requirements in mind and don't
| save raw data you don't need 4) separate blocking and non-
| blocking activities on different threads, preferring non-
| blocking whenever possible and 5) limit threads to only those
| activities that are actively doing work.
| ochoseis wrote:
| Would you use Kotlin again for the back end? Having not yet
| used it for that purpose, it seems like you'd get the
| benefit of the JVM ecosystem along with a nice language
| (but perhaps too many power-features).
| dirkg wrote:
| may be OT, but how do you subscribe to these trade feeds, is
| there a unified service or do you need to do it individually
| for each source, and how much does it cost approximately ?
|
| I'm guessing if you put all this data into Kinesis or message
| queues it would end up costing quite a bit more.
| meltedcapacitor wrote:
| Never heard of a crypto exchange that charges for data feeds,
| the norm is free and fast. One of the positive of the
| industry compared to old school finance.
|
| They're rent seeking in other ways though, no worries.
| nostrademons wrote:
| There are probably unified services that let you do it - I
| was kinda competing in this area but didn't want to deal with
| enterprise sales, and it's a bit of a hard sell anyway.
|
| If you do it individually, there are public developer docs
| for each exchange that explain how their API works. It's
| generally free as long as you're not making a large number of
| active trades.
| Gepsens wrote:
| I'm running a crypto trading platform I'm developing on 30$ on
| DigitalOcean. I coded exclusively in Rust and recently added a
| dynamic interface to python. Today during the BTC crash it
| spiked at 20k events/s, and that's only incoming data.
| giancarlostoro wrote:
| is that $30 for a single droplet or is it spread out between
| a few different services? I'm kind of curious since I use DO
| for small projects myself.
| bochoh wrote:
| I'm also curious on this. My stack currently is a SQLite
| file -> ORM -> .net core on Linux on a single box
| danudey wrote:
| > I coded exclusively in Rust
|
| This reminds me of back in 2003, a friend of mine worked for
| an online casino vendor; basically, if you wanted to run an
| online casino, you'd buy the software from a company and
| customize it to fit your theme.
|
| They were often written in Java, ASP.NET, and so on. They
| were extremely heavyweight. They'd need 8-10 servers for 10k
| users. They hogged huge amounts of RAM.
|
| My friend wrote the one this company was selling in C. Not
| even C++, mind you, just C. The game modules were chosen at
| compile time, so unwanted games didn't exist. The entire
| binary (as in, 100% of the code) compiled to just over 3 MB
| when stripped. He could handle 10k concurrent users on one
| single-core server.
|
| I'm never gonna stop writing things in Python, but it still
| amazes me what can happen when you get down close to the
| metal.
| Andrew_nenakhov wrote:
| Probably, Erlang would be a good fit for your task.
| blacklion wrote:
| Crypto markets are very small :-)
|
| I'm working and company which process "real" exchanges, like
| NASDAQ, LSE, and, especially, OPRA feed.
|
| We've added 20+ crypto exchanges in our portfolio this year,
| and all of them are processed on one old server which is unable
| to process NASDAQ Total View in real-time anymore.
|
| On the other hand, whole OPRA feed (more than 5Gbit/s or
| 65B/day, yes, it is billions, messages of very optimized binary
| protocol, not this crappy JSON) is processed by our code on one
| modern server. Nothing special, two sockets of Intel Xeons (not
| even Platinums).
| joering2 wrote:
| Could you share some more about that very optimized binary
| protocol? I know there are ways to be more efficient than
| JSON but since you call it crappy, your solution must be much
| much better. Honestly interested to readup more.
| blacklion wrote:
| It is not "our" protocol, it is protocol designed by
| exchange and we need to support it, as we can not change it
| :). Simple binary messages, with binary encoded numbers,
| etc. No string parsing, no syntax, nothing like this, only
| bytes and offsets. Think about TCP header, for example.
|
| JSON is very inefficient both in bytes (32 bit price is 4
| bytes in binary and could be 7+ bytes as string, think
| "1299.99" for example) and CPU: to parse "1299.99" you need
| burn a lot of cycles, and if it is number of cents stored
| as native 4-byte number you need 3 shifts and 4 binary ors
| at most, if you need to change endianness, and in most
| cases it is simple memory copy of 4 bytes, 1-2 CPU cycle.
|
| When you have binary protocol, you could skip fields which
| you are not interested in as simple as "offset = offset +
| <filed-size>" (where <filed-size> is compile-time
| constant!) and in JSON you need to parse whole thing
| anyway.
|
| Difference between converting binary packet to internal
| data structure and parsing JSON with same data to same
| structure could be ten-fold easily, and you need to be very
| creative to parse JSON without additional memory
| allocations (it is possible, but code becomes very dirty
| and fragile), and memory allocation and/or deallocation
| costs a lot, both in GC languages and languages with manual
| memory management.
| paraph1n wrote:
| There are many binary encoding protocols. A popular one is
| protobufs[1], which is used by gRPC.
|
| [1]: https://developers.google.com/protocol-buffers
| rewq4321 wrote:
| And msgpack if you want an order of magnitude faster
| serialization/deserialisation and can put up with worse
| compression (I think mainly due to schema overhead since
| protobuf files don't store the schema?)
|
| https://msgpack.org/index.html
|
| Good protobuf vs msgpack comparison:
| https://medium.com/@hugovs/the-need-for-speed-
| experimenting-...
| wffurr wrote:
| https://www.opraplan.com/datafeeds
| nostrademons wrote:
| Some Googling turned up this protocol descriptor:
|
| https://uploads-
| ssl.webflow.com/5ba40927ac854d8c97bc92d7/5bf...
|
| If you're optimizing for latency JSON is pretty terrible,
| but most people who use it are optimizing for
| interoperability and ease of development. It works just
| fine for that, and you can recover decent bandwidth just by
| compressing it.
| [deleted]
| fhood wrote:
| People may underestimate how fast modern machines are, but that
| is probably in part because, at least in my fairly relevant
| experience, I have literally never seen a CPU bottleneck under
| normal circumstances. Memory pressure is nearly always the
| driving issue.
| foobarbazetc wrote:
| Yeah. Now that CPUs are insanely powerful and you have NVMe
| SSDs etc the bottleneck is always memory.
| xfitm3 wrote:
| In my experience disk i/o is the biggest bottleneck. It
| used to be sync()ing writes to disk for strict consistency
| but that's been pushed down to the DB now. I just looked at
| my DB systems and CPU is low but disk is nearly pegged.
|
| My data sets are far too big to fit into memory/cache. Disk
| pressure can be alleviated by optimizing queries but it's a
| game of whack-a-mole.
|
| I have exhausted EBS i/o and been forced to resort to dirty
| tricks. With RDS you can just pay more but that only scales
| to a point - normally the budget.
| zozbot234 wrote:
| Sure, but as far as software is concerned, optimizing for
| memory bandwidth (the typical bottleneck in modern systems)
| is not so different from optimizing for CPU.
| nostrademons wrote:
| It's also amazing how much you can fit in RAM if you're
| careful. I remember ~2007 people were aghast at Facebook's
| 4T memcached deployment that stored basically everyone's
| social network posts; now you can get single servers for
| ~$4K with 4T of RAM.
|
| The trick is basically that you have to eschew the last 15
| years of "productivity" enhancements. Pretty much any
| dynamic language is out; if you must use the JVM or .NET,
| store as much as possible in flat buffers of primitive
| types. I ended up converting order books from the obvious
| representation (hashtable mapping prices to a list of Order
| structs) to a pair of SortedMaps from FastUtils, which
| provides an unboxed float representation with no pointers.
| That change ended up reducing memory usage by about 4x.
|
| You can fit _a lot_ of ints and floats in today 's 100G+
| machines, way more than needed to represent the entire
| cryptocurrency market. You just can't do that when you're
| chasing 3 pointers, each with their associated object
| headers, to store 4 bytes.
| NetToolKit wrote:
| > now you can get single servers for ~$4K with 4T of RAM
|
| Does the $4K include the cost of the RAM? Where can I
| find these servers? Thanks!
| nine_k wrote:
| The CPU is rarely used up to 100% because most code fails to
| utilize several cores efficiently.
|
| OTOH a service loading the single core with the main thread
| is a frequent sight :( Interpreted languages like Python can
| easily spend 30% of time just on the deserialization
| overhead, converting the data from a DB into a result set,
| and then into ORM instances.
| OneEyedRobot wrote:
| I've seen exactly the opposite although you certainly can't
| ignore memory speed.
| [deleted]
| bob229 wrote:
| Pepe ftw
| olingern wrote:
| I might get downvoted for not getting on the 2000s style of
| development bandwagon, but do you really need a web server to
| serve static text?
| bellyfullofbac wrote:
| What do you suggest, Gopher?
| crazy_horse wrote:
| Nginx.
| Zababa wrote:
| Nginx is a web server though. You're assuming that by "web
| server" the first comment meant "web framework"?
| CyanLite2 wrote:
| Cloudflare
| sigg3 wrote:
| Put Wordpress on it, and do a new battery of TTFB tests ;)
| celsoazevedo wrote:
| Even WordPress would work fine if we use a plugin like WP Super
| Cache (no idea why they don't cache things by default). It
| wouldn't beat a simple static page, but WordPress + Cache
| plugin + cheap VPS can easily handle #1 on HN.
| jka wrote:
| There are a few comments in here that predictably suggest that
| simple static sites can handle large request rates easily.
|
| Sure, that's true - but to try to progress the conversation: how
| would you measure the complexity of serving web requests, in
| order to perform more advanced cost comparisons?
|
| (bandwidth wouldn't be quite right.. or at least not sufficient -
| maybe something like I/O, memory and compute resource used?)
| throwaway20371 wrote:
| You can also handle 50 requests per second on a 66MHz 486DX2 with
| 16MB of RAM and a 10Mbit/s network card. Not with modern "I have
| infinite resources" software, but we used to handle more than
| that traffic regularly in the early 90s.
| fabian2k wrote:
| 50 rps not that much, though of course easily sufficient for many
| situations. This is also Django which certainly isn't the fastest
| choice. I played around with it a long time ago and liked it
| quite a bit, but you don't choose Django for performance but for
| the other benefits.
|
| I'm really more surprised that static serving is so slow at 180
| rps. This should be able to easily saturate the network,
| statically serving files is very, very fast. From what I see in
| the blog I doubt that the files are very large, so there is
| probably some other bottleneck or I'm missing something here.
| janmo wrote:
| There is a difference between being able to handle 4.2M requests
| a day, and handling 4.2M requests per day.
|
| Visitors don't come neatly one after the other. You might only
| have 1M requests a day but get random spikes with 100 requests at
| the same time.
| bArray wrote:
| Very true. It also makes a difference as to which resource is
| being pulled, whether it is cached, what transport is being
| requested (SSL, compression, etc).
|
| I really suspect the website would fall long before it hits
| anything close to 4.2 million requests (which the author also
| seems to except).
|
| That all said - long live tiny web servers!
| great-potential wrote:
| Dont mean to be the negative Joe but you dont need a webserver if
| you're serving a static-able website.
| nayuki wrote:
| You mean use someone else's web server instead?
| great-potential wrote:
| I mean this is just basic stuff, for me it just sounds like a
| developer putting a site in production, no auto-scaling,
| patching ... might as well outsource this to a CDN since
| there is no database/redis/varnish ...
| sleepyhead wrote:
| So...someone else's web server instead?
| great-potential wrote:
| No offense but I think you've missed the point.
| [deleted]
| pluc wrote:
| > Not taking into account any issues that may occur around
| CPU/RAM/Disk IO due to sustained levels of traffic as well as
| bandwidth issues
|
| congrats?
___________________________________________________________________
(page generated 2021-09-07 23:00 UTC)