[HN Gopher] My PS4 a month server can handle 4.2M requests a day
___________________________________________________________________
My PS4 a month server can handle 4.2M requests a day
Author : mark_mcnally_je
Score : 727 points
Date : 2021-09-07 20:01 UTC (1 days ago)
(HTM) web link (mark.mcnally.je)
(TXT) w3m dump (mark.mcnally.je)
| taf2 wrote:
| I wonder what the 99th - 95th percentile response times look
| like...
| KronisLV wrote:
| It feels to me like most websites out there could run on way less
| hardware, if only people would embrace a few things.
|
| #1 Minimalism. You don't need 400 KB of JS to display some mostly
| text content to your users with some interactivity sprinkled in.
|
| You don't need to reinvent office software, or very rich text
| editors in browsers, stop using the web as a universal delivery
| platform/mechanism, because that's not what it was meant for.
| When browsers will ship integrated dependencies so that even CDNs
| don't need to be hit (like versions of jQuery, Bootstrap and
| numerous JS frameworks as well as WASM code like Blazor which
| contains a .NET runtime), then you'll be able to do that, but
| arguably that will never happen.
|
| Use the web as a platform for displaying primarily text content
| with the occasional images, forms and a little bit of
| interactivity sprinkled in. Most sites out there simply aren't
| and shouldn't be like this (that said, when you have exceptional
| reasons for throwing aside that suggestion, do so):
| https://geargenerator.com
|
| #2 Static content. You don't need to use Wordpress, Drupal,
| Joomla or many of the other CMSes out there, since they can get
| really heavyweight with numerous plugins and are not only a
| security challenge, but are also problematic from a performance
| perspective.
|
| Consider using static site generators instead. When reading an
| article of yours, the DB shouldn't even be hit, since most of the
| article contents are unlikely to change often, so you should be
| able to pre-render each of the article versions as a set of
| static HTML and use the common JS/CSS that you already have for
| the rest of the articles. Furthermore, it's easy to just jump
| into CMSes and introduce ungodly amounts of complexity, all of
| which cause your back end to process bunches of code for each
| request. Static files don't have that drawback.
|
| #3 Caching. Know when and what to cache, and how. Images, JS
| files, CSS files and even entire HTML pages should be cache
| friendly. Know which ones aren't, make exceptions for those and
| cache everything else.
|
| Not only is it not necessary to hit the DB for many of the pages
| in your site at all, but also sometimes you shouldn't even hit
| the back end either. The most popular pages of your site should
| just live in a cache somewhere, be it within your web servers or
| a separate solution, so that they can be returned instantly. HTML
| is good for this, use it.
|
| Furthermore, know what cache policies to use. Sometimes even the
| cache resources shouldn't be redownloaded, if the user already
| has these resources loaded from a different page. Use bundle
| splitting responsibly, extract common functionality in easily
| cacheable bundles and set the appropriate headers.
|
| And yet, i've seen a surprising amount of ignorance in regards to
| caching, static site generation and even how large webpages have
| gotten: https://idlewords.com/talks/website_obesity.htm
|
| I don't claim to know it all, but working towards the goal of
| efficiently using pages should definitely be viewed as an
| important one: be it because you want to pay less for your
| infrastructure, or care about the environment, or even just want
| to manage fewer nodes.
|
| Instead, nowadays far too many orgs just try to be the first to
| market and ignore the engineering based approach to ensuring that
| the solutions are not only functional but also sustainable. That
| saddens me.
| iradik wrote:
| "If we can handle 50 requests a second that means we can handle
| 4.2 million requests a day."
|
| Not really. Real world traffic won't be uniform over one entire
| day. 50 QPS would be more accurate.
| danbrooks wrote:
| I hosted draftsim.com on a $3/month hosting plan for a few years.
|
| We served 500GB of data the first month.
|
| I imagine that the hosting company lost money on us (but they
| never called to complain).
| flobosg wrote:
| Just wanted to thank you for draftsim.com! As someone who got
| into MTG Arena a few months ago it has been very useful to
| learn some basics.
| privacyonsec wrote:
| My 1cpu 2gb ram server: - nginx - slack bot behind Django -
| OpenVPN - Minecraft server (20 players) - tunneling (reverse
| proxy for local dev)
| whatsthatabout wrote:
| You're running an 20 players mc server with that hardware? How?
| In my experience Minecraft-servers are insanely resource
| hungry, especially whilst generating new parts of the map or
| larger red stone contraptions. Back when I played around with
| hosting servers I used the "most optimized" mc-server fork
| called Tuinity - and even with that I had to allocate way more
| cpu and ram to the vms then u use. Would love to hear about
| your setup.
| mktk1001 wrote:
| Press X to doubt.
| Wronnay wrote:
| If you host it on GitHub Pages, GitLab Pages or Vercel it's even
| PS4 less :o
| bullen wrote:
| My Java HTTP app. server manages that on a Raspberry 2 at 2 watts
| serving this reddit clone: http://talk.binarytask.com
|
| Really we need to compare apples to apples (how many watt)!
|
| Most of you have an external IP address, open port 80 and put it
| to good use before they put you behind a shared IP!
| shultays wrote:
| The new metrics for server performance is "unique buzzwords per
| paragraph". If you don't write a 4 page blog post with at least
| 4-5 ubpp then it is shit.
| reilly3000 wrote:
| That estimate can be verified with load testing systems like
| Artillery. My theory is that things would break far sooner than
| estimated along the following lines:
|
| - Too many WSGI connections if the timeouts aren't tweaked
|
| - Too many database connections, especially without caching and
| tuning
|
| - on the Apache side if MaxRequestWorkers isn't set there will be
| memory issues with 1GB RAM
|
| - the disk could easily hit IOPS limits, especially if there is a
| noisy neighbor
|
| It's not likely all or any of these things will hit IRL, but that
| all depends on traffic and usage patterns. It matters not, if you
| were getting 4.2 M requests each day you'd be in the Alexa Top
| 1000 and could probably shell out for the $8 server :)
| markandrewj wrote:
| Normally benchmarks for things like this are measured in how many
| concurrent requests can be handled, i.e the C10K problem, not by
| how many requests you are able to serve in a day. It's also well
| known that you can serve a large amount of requests on limited
| hardware.
|
| https://en.wikipedia.org/wiki/C10k_problem
|
| "By the early 2010s millions of connections on a single commodity
| 1U rackmount server became possible: over 2 million connections
| (WhatsApp, 24 cores, using Erlang on FreeBSD),[6][7] 10-12
| million connections (MigratoryData, 12 cores, using Java on
| Linux).[5][8]"
|
| Although I do understand the boxes listed above have more
| resources then the VPS you are using. I am also not criticizing
| your write up, or results, bench-marking is in general
| interesting to do. I just wanted to provide some additional
| information.
| habibur wrote:
| Right. I calculated what 5m/day converts into. And it's like 60
| req/sec. Considering non even distribution and spikes, I would
| assume its like 200req/sec.
| adtac wrote:
| unrelated but 4x increase isn't really a spike
| [deleted]
| spyder wrote:
| Yes, handling 50 requests per second doesn't mean the server
| can handle 4.2 million a day. That can only happen if they are
| uniformly distributed throughout the day, which isn't the case
| for most website traffic.
| ijidak wrote:
| How does a single server run millions of active connections?
|
| Wouldn't you run out of TCP sockets?
|
| What am I missing?
| dreyfan wrote:
| A connection isn't just a dest_port, it's the unique
| combination of 4 components:
| source_ip:source_port:dest_ip:dest_port
| [deleted]
| arthurcolle wrote:
| socket multiplexing
| jfrunyon wrote:
| I would guess they may be muxed over fewer sockets, by their
| LBs, but that's not strictly necessary.
|
| I'm not sure exactly what you mean by "run out of TCP
| sockets", but theoretically speaking, the only limitation is
| how much memory is available to store the necessary info
| about the socket (like address/protocol info and process
| info).
|
| In practice, OS's do have a "max socket" or "max FD" limit,
| but that's usually configurable and (with enough RAM) could
| easily be set to "millions".
| diath wrote:
| > I'm not sure exactly what you mean by "run out of TCP
| sockets", but theoretically speaking, the only limitation
| is how much memory is available to store the necessary info
| about the socket (like address/protocol info and process
| info).
|
| Probably the 65k port limit since each connection will get
| assigned a remote port, which can be solved by binding to
| multiple local ports and using a load balancer in front or
| using multiple network interfaces.
| jfrunyon wrote:
| The 65k limit is per client address _and port_. Each
| client can have 65,535 connections to a single port on
| your server _from each and every 65,535 of their own
| ports_. (65,536? I dunno what would actually happen if
| you tried to use port 0.)
| winrid wrote:
| You can have more than one connection per port, so you
| don't even need the load balancer or more interfaces! :)
| remram wrote:
| On my default Ubuntu install, the hard limit for open files
| of a process is 1048576 (ulimit -Hn). So you you have to run
| a handful of processes.
| ignoramous wrote:
| I run a FOSS Pi-Hole esque public DoH resolver in the most
| expensive way possible (over Cloudflare Workers) and it costs $2
| for 4M requests. Granted the CPU slice is limited (50ms) but the
| IO slice (30s) is plenty for our workloads (stub dns-resolution).
|
| The reason this is cheaper in a sense is because Workers deploys
| globally and needs zero devops. Per our estimates, this setup
| (for our workload) gets expensive once the request range goes
| beyond 1.5 billion a month after which deploying to colos
| worldwide becomes cheaper even with associated cost of devops.
| polote wrote:
| What's the point of this post? OP is serving a file at 50req/sec.
| There is not even mention of a dB query. How is that able to
| relate to any kind of normal app?
|
| I guess that the post was written as an answer to the mangadex
| post [1]. Mangadex was handling 3k req/sec involving dB queries.
| It was not just a cached Html page.
|
| 50req/sec for a Html file is super low which shows that a $4
| month server cant do much actually. So yes this is enough for a
| blog, but a lot of websites are not blogs
|
| [1] https://news.ycombinator.com/item?id=28440742
| great-potential wrote:
| Exactly, I dont see the point bragging about this nevertheless
| posting about it on HN ...
| [deleted]
| TruthWillHurt wrote:
| +1. I remember modest VPS/Parrallels serving PHP at 350r/s
| quickthrower2 wrote:
| Something like: <?php echo("this is a
| benchmark") ?>
| napworth wrote:
| It's called boasting
| sillycube wrote:
| Boasting of what? Serving static sites doesn't even need a
| server.
|
| Use apache to serve Django + wsgi? Just use Django asgi and
| nginx and you will get a higher number.
| wyager wrote:
| My $5/mo server can handle several thousand requests per
| second. It's mostly a question of what server software you use.
| If you use some node, python, ruby thing, it's going to be slow
| as shit and need a reverse proxy in front of it. If you use a
| fast compiled language with a good framework, you can rip
| through requests no problem.
|
| I tried a bunch of different stuff and ended up using Haskell -
| all of its popular web libraries are fast as hell. Go was fast
| but its standard library leaked sockets or I was not cleaning
| up connections properly or something, and it would tank
| whenever something went viral. All the popular interpreted
| language backend I tried were absurdly slow, like tens of RPS.
|
| Source for my current thing is at http://yager.io/Server.hs. It
| also does all my RSS stuff, image processing for my photo
| gallery, etc.
| pdimitar wrote:
| I'm curious if you evaluated Rust?
| wyager wrote:
| At that point there were not any tried and true Rust web
| frameworks, although a priori I would expect any Rust web
| offerings to be pretty solid.
| [deleted]
| gaptoothclan wrote:
| Is this just an apache bench mark
| vymague wrote:
| > What's the point of this post? OP is serving a file at
| 50req/sec.
|
| I'd guess a response to the mangadex thread?
| https://news.ycombinator.com/item?id=28440742
| SPBS wrote:
| No way, OP is essentially serving a static webpage from a
| database. That's nothing to brag about if comparing with
| mangadex.
| throwdecro wrote:
| > How is that able to relate to any kind of normal app?
|
| There's too much competition involved in writing normal apps,
| which often attract significant investment that bootstrapped
| startups struggle to compete with.
|
| It's interesting to see what kind of performance is possible
| for next to no money, when you throw out basic assumptions like
| using a database, and then start thinking about what you could
| build out of it.
| quickthrower2 wrote:
| My recent submission of HNBadges was made like this. It's
| just 3 files (html, css, JS) which I hosted for free on
| Netlify, but could have been hosted on a setup like OP. I
| used other services for XHR requests. I imagine it got a
| tonne of traffic from being on the first page, but I wasn't
| taking metrics.
|
| Another example of clever use of resources is the
| https://haveibeenpwned.com/ website. Using a bloom filter (I
| think) to turn what could have been a back-end lookup into a
| "front end lookup" by requesting a small file from the server
| based on the password hash.
|
| The only issue I have with the OP is his assumption that
| you'd get a nice smooth 60 request/second throughout the day!
| Most likely will be lumpy, and in the top of the lumpy
| periods (where most of your visitors visit) performance will
| be bad.
| Arch-TK wrote:
| >There is not even mention of a dB query.
|
| Did you read the post?
| latexr wrote:
| From the guidelines[1]:
|
| > Please don't comment on whether someone read an article.
| "Did you even read the article? It mentions that" can be
| shortened to "The article mentions that."
|
| [1]: https://news.ycombinator.com/newsguidelines.html
| wokwokwok wrote:
| > These benchmarks show that a very cheap server can easily
| handle 50 requests a minute to a "full stack" website.
|
| I did, and all I see is someone spinning some numbers idly,
| like, hey, if I can lay 1 brick every second, then with 20000
| people we can build a house in one second! So good!
|
| a) entirely and totally lacking in experience running a heavy
| load website.
|
| b) 50 requests _a minute_ is so atrociously bad, it's not
| even worth talking about.
|
| c) there _isnt_ any db load going on here, this is a full
| page single table query. See
| https://docs.djangoproject.com/en/3.2/ref/contrib/flatpages/
|
| Sure maybe a db exists, but it's not relevant when you
| compare this to the complexity of doing _write operations_.
|
| Ie. this is some hiiiigh level arm chair commentary right
| here.
|
| Sure, they're just talking about their website, but anyone
| going "oh yeah, look at this, those mangadex guys should
| learn a thing or two and run it on django". ...has no idea
| what they're talking about.
| Arch-TK wrote:
| >there isnt any db load going on here
|
| This is shifting the goal posts. Your initial comment
| claimed that there are no database queries being made.
|
| In any case, you've amended your argument, I have no
| further comments.
| ac29 wrote:
| > b) 50 requests a minute is so atrociously bad, it's not
| even worth talking about.
|
| That was a typo, the worst performance they tested was 54
| reqs / second.
| stef25 wrote:
| And here I am building a simple API using Lumen (Laravel's
| stripped down, hopefully faster cousin) getting response times
| that are just abysmal.
|
| The raw queries themselves are fast enough, but for some reason
| running them in a framework, transforming them in to a Resource
| and dumping it as json takes so long that I'm scared to find out
| what this super popular framework is even doing under the hood.
|
| Once I learn enough Python I'd like to compare its performance to
| something like FastAPI. But even that probably won't come near
| what these recent posts are describing.
|
| (Disclaimer - it's just a side project and I haven't really
| looked in to making it faster)
| Kiro wrote:
| That's strange. Laravel should be able to handle thousands of
| requests a second even on the cheapest hardware.
| euoia wrote:
| In my experience with Laravel, it is often the transformation
| to JSON that takes some time for large responses.
| [deleted]
| yupper32 wrote:
| Well the front page of HN won't get you 4.2M views today, but
| it's a pretty good real world test!
| mark_mcnally_je wrote:
| It is! It's crashed my analytics but the website seems to be
| doing fine.
| jonplackett wrote:
| True, but it may very well get you a lot more than 50 a minute.
| louwrentius wrote:
| I'm hosting my static blog site on a physical Raspberry Pi 3B+
| powered by Solar [1].
|
| That blog post got hugged by HN but it didn't even raise the CPU
| above 10% on a single core.
|
| And a Raspberry Pi 3B+ is dog slow. And severely limited by
| bandwidth, unlike the Raspberry Pi 4B+. (But it uses less power
| so that's why I use a 3B+).
|
| However I have another point to make. Professional rack-mount
| servers from HP and Dell can be had second hand for dirt cheap
| and you get a ton of CPU (20+ cores) and an ocean of RAM for next
| to nothing.
|
| For many applications, an old Gen8 or similar Dell server will
| perform more than adequately. Even more so if you have a little
| bit more to spend on Gen9.
|
| They are so cheap that you can like buy four to eight, sprinkle
| them across two different datacenters and even if one breaks, you
| won't be in any hurry.
|
| [1]: https://louwrentius.com/this-blog-is-now-running-on-solar-
| po...
| 5faulker wrote:
| That's a good spec for 4 bucks. With cloud hosting you might be
| able to push the cost down a bit with less CPU resource and
| memory.
| mark_mcnally_je wrote:
| I am cloud hosting using OVH cloud, I brought the server a
| couple of years ago now so they probably have some better specs
| for the price I am paying.
| gruellan wrote:
| Hey! Someone else from Jersey! Nice one
| zxcvbn4038 wrote:
| Combined with a CDN and you can do a lot more, I like to think in
| terms of origin requests rather than raw req/s. The problem I
| have is getting people to really understand how caching works at
| the cdn and browser layers and design their frontends and backend
| responses around that. There is also lot you can do with edge
| compute to clean the incoming requests before the CDN evaluates
| them to increase cache hits. Even if you are trying to give a
| "real time" view of data, caching it for even a second and
| allowing stale data to be served while it is updating can reduce
| origin requests significantly. I've seen people hammer sites
| hundreds of times a second looking for changes that only happen
| once every fifteen seconds or once per minute - the best thing
| you can do for yourself is handle all those requests at the CDN
| level (eventually you'll do log analysis and see the activity and
| can take other measures, but in the mean time don't let all of
| those requests go to the origin). Your CDN is probably giving you
| better rates for network egress then Amazon or Google anyway -
| the later are more focused on incentivizing you to use their
| ecosystem exclusively by penalizing you for sending data
| "outside". Cheap VPS hosts discourage you exceeding your
| bandwidth allocation because they are overselling their capacity
| and heavy usage upsets that - so again you want to shift as much
| as you can to your CDN.
| ppeetteerr wrote:
| How did this make it to the number two spot on HackerNews?
| [deleted]
| [deleted]
| ok_coo wrote:
| IMHO, it's better than a deluge of political posts.
|
| I don't mind reading about politics but I come to HN to read
| about tech. We can go elsewhere to get whatever politics we
| desire.
| dreyfan wrote:
| Broadly speaking people on HN have no clue how to setup a
| performant httpd/app server and are impressed by abysmal
| performance/cost metrics like this or the MangaDex post.
| Everything these days is obscured through multiple layers of
| SaaS offerings and unnecessary bloat like kubernetes.
|
| ~10k rps (it was concurrent connections but close enough) was
| state of the art in 1999. Now 22 years later ~50 rps is somehow
| impressive.
| watermelon0 wrote:
| I would guess this is close to a default Apache + mod_wsgi
| setup, and this is one of the easiest way of hosting Python
| web apps, so basically achievable by anyone on HN.
|
| I assume (based on 180 req/s for static page) that he is
| using mpm_prefork, where each Apache child handles a single
| connection. If he switched to mpm_event, which uses event
| loop like nginx, ~10k rps should easily be achievable, but I
| don't think WSGI would work with that.
| mark_mcnally_je wrote:
| Yeah it's all default apache + mod_wsgi. This is also my
| first Django setup and I made it over-complicated as a
| learning exercise. mpm_event is something I have not heard
| of before, thanks for bringing that to my attention.
| Matthias1 wrote:
| I generally agree with your comment. But the point here is
| not this is cutting edge performance or anything, but rather
| that people on HN know how to set up this type of website.
|
| If you have any tips on how to get ~10k rps (or even a more
| reasonable improvement) on a PS4 a month server, I at least
| would be very interested in hearing about them.
| ppeetteerr wrote:
| The author provides no information other than to say it's an
| Apache + Django setup. That's pretty bare-bones and is part
| of most tutorials online.
| reilly3000 wrote:
| In my opinion it isn't so much of a lost art or lamentable
| accretion of useless abstractions, but an increase in the
| scope of what web apps do these days. Most of us aren't
| working on static sites or simple CMS publishing- those are a
| solved problem. Instead we're building mobile banks,
| diagnostic systems, software tooling, 3D games, and shopping
| malls. The complexity is inherent in the maturity of the web
| and it's many uses, as well as its global scale. Hugs of
| death are rare these days thanks to better architecture and
| infrastructure, even though the scale of users has grown
| 100X.
|
| Yes there are wildly unnecessary abstractions that are used
| for small sites/apps, but I would contend they are artifacts
| of someone who it's trying to learn something new, and/or get
| promoted. I have no problem with the former.
| Zababa wrote:
| > ~10k rps (it was concurrent connections but close enough)
| was state of the art in 1999. Now 22 years later ~50 rps is
| somehow impressive.
|
| I honestly don't understand how that can be true. I'm not
| suggesting you're lying of course, but when you put it this
| way it's almost like people are actively trying to slow their
| programs down. I have a few ideas on why that might be the
| case (switch to slow interpreted languages, switch to bigger
| web frameworks, bigger payload) but even that wouldn't
| explain all of it. Do you have any idea why things are this
| way?
| ratww wrote:
| It's death by a thousand paper cuts. Lots of things that
| aren't really that slow in isolation, but in aggregate (or
| under pressure) they slow down the system and become
| impossible to measure.
|
| Let's do web development. Since you mentioned payloads:
| today they're bigger, and often come with redundant fields,
| or sometimes they're not even paginated! This slows down
| the database I/O, requires more cache space, slows down the
| serialisation, slows down compression, requires more memory
| and bandwidth...
|
| And then you also have the number of requests per page. Ten
| years ago you'd make one request that would serve you all
| the data in one go, but today each page calls a bunch of
| endpoints. Each endpoint has to potentially
| authenticate/authorise, go to the cache, go to the
| database, and each payload is probably wasteful too, as in
| the previous paragraph.
|
| About authentication and authorisation: One specific
| product I worked on had to perform about 20 database
| queries for each request just for checking the permissions
| of the user. We changed the authentication to use a JWT-
| like token and moved the authorisation part to inside each
| query (adding "where creator_id = ?" to objects). We no
| longer needed 20 database queries before the real work.
|
| About 15 years ago I would have done "the optimised way"
| simply because it was _much easier_. I would have used SQL
| Views for complex queries. With ORMs it gets a bit harder,
| and it takes time to convince the team that SQL views are
| not just a stupid relic of the past.
|
| Libraries are often an issue that goes unnoticed too. I
| mentioned serialisation above: this was a bottleneck in a
| Rails app I worked. Some responses were taking 600ms or
| more to serialise. We changed to fast_jsonapi and it went
| to sub-20ms times for the same payload that was 600ms. This
| app already had responses tailored to each request, but
| imagine if we were dumping the entire records in the
| payload...
|
| Another common one is also related to SQL: when I was a
| beginner dev, our on-premises product was very slow in one
| customer: some things on the interface were taking upwards
| of 30 seconds. That wasn't happening in tests or in smaller
| customers. A veteran sat down by my side and explained
| query plans, and we brought that number down to
| milliseconds after improving indexing and removing useless
| joins.
|
| A few weeks ago an intern tried to put a javascript
| _.sort()_ inside a _.filter()_ and I caught it.
| Accidentally quadratic (actually it was more like _O(n^4)_
| ). He tried to defend himself with a "benchmark" and show
| it wasn't a problem. A co-worker then ran anonymised
| production data into it and it choked immediately. Now
| imagine this happening on hundreds of libraries maintained
| by voluntaries on Github:
| https://accidentallyquadratic.tumblr.com
|
| All those things are very simple, and you certainly know
| all of them. They're the bread and butter of our
| profession, but honestly somewhere along the way it became
| difficult to measure and change those things. Why that
| happened is left as an exercise.
| Zababa wrote:
| That's interesting, thanks for sharing.
|
| > All those things are very simple, and you certainly
| know all of them.
|
| I wonder about that. Most of the people I graduated with
| didn't know about complexity. Some had never touched a
| relational database, most probably didn't know about
| views. I doubt most of them knew what serialization mean.
| ratww wrote:
| I wonder if it's a matter of background. I never really
| had tutorials when starting out. I never had good
| documentation, even.
|
| (Sorry in advance for the rant)
|
| I also remember piercing together my first programs from
| other people's code. Whenever I needed an internet forum
| I'd build one. Actually, all my internet friends, even
| the ones who didn't go into programming, were doing web
| forums and blogs from scratch!
|
| Today people consider that a heresy. "How dare you not
| use Wordpress".
|
| My generation just didn't care, we built everything from
| scratch because it was a badge of honor to have something
| made by us. We didn't care about money, but we ended up
| with abilities that pay a lot of cash. People who started
| programming post the 2000s just didn't do it...
|
| I think it is visible that I sorta resent the folks (both
| the younger, and the older who arrived late at the scene)
| constantly telling me I shouldn't bother "re-inventing
| the wheel". Well, guess what: programming is my passion,
| fuck the people telling me to use Unity to make my game,
| or Wordpress to do my blog.
| throwaway20371 wrote:
| Software has become slower and less efficient because we
| have faster hardware to run it on and nobody's asking for
| the software to get more efficient. Back in the day you had
| inefficient, expensive hardware with limited software.
| There was demand to make software as performant as possible
| so it didn't cost you ten grand to serve a popular website.
| But now you can load up on hardware for pennies on what it
| used to cost. 16 cores? Sure! 32 gigs of ram? No sweat.
| Software can now be bloated and slow and it won't break the
| bank. On top of that, we now have free CDNs, free TLS
| certs, even free cloud hosting, and domains are a couple
| bucks. You can serve traffic for free today that would have
| cost $50K 15 years ago.
|
| Sometimes there's also a misunderstanding of metrics that
| leads devs to not think about performance tuning. Like,
| "4.2M requests a day" is clearly incorrect at this 50rps
| benchmark. Traffic is not linear, it's bursty. You will
| never serve 50rps of human traffic steadily for 24 hours.
| If you're serving 4.2M requests per day, 90% of it will be
| in a 12 hour window, peaking at whenever people have lunch
| or get off work, with a short steep climb leading to a
| longer tail. So to not crash your site at peak visitorship,
| you realistically need to handle 300+ rps in order to
| achieve 4.2M requests per day. (But also that's requests
| per second... if under load it takes 5 seconds to load your
| site, you can still serve a larger amount of traffic, it's
| just slower... so a different benchmark is also "how many
| requests before the server literally falls over")
| tedunangst wrote:
| Computers used to be expensive. Now you just click a button
| and hyper scale.
| crazy_horse wrote:
| This is just basic use the right tools for the right job
| 101. You've got what is basically a static website. You
| want to serve static files. To do that, you use a fast
| language and/or servers written in those languages.
|
| It's something anyone who has done this for any length of
| time knows, that HN is impressed by this is confusing to
| some of us. If you were _trying_ to get as little out of
| your server as possible you 'd serve cached content using
| this framework in this language.
|
| Is this stuff not being learned?
| Zababa wrote:
| I thought you meant from 10k to 50 rps doing the same
| work, not that most of the work could be avoided in the
| first place.
|
| > Is this stuff not being learned?
|
| I don't know if it is. I recently finished my studies,
| and most people had no curiosity at all. As in, they
| learned a framework early, used it everywhere, and got a
| job using it. I do remember reading a few times on
| tutorial that you should put a Nginx as reverse proxy in
| front of your Django/Flask/Express server to server
| static files, so I think most people know/do that but I'm
| not sure.
|
| On the other hand, having the wisdom of knowing what can
| be static in the first place? I don't think that it's
| something teached. In fact this kind of wisdom can be
| hard to find outside of reading lots of sources
| frequently in hope of finding little nuggets like that. I
| don't think I was ever taught explicitely "You should
| first try to find out if the work you're trying to do is
| necessary in the first place". In a way it's encoded in
| YAGNI, but YAGNI isn't universal, and is usually
| understood at the code level and not the tooling level.
| ratww wrote:
| _> On the other hand, having the wisdom of knowing what
| can be static in the first place? I don 't think that
| it's something teached._
|
| I think the trick is realising that reaching for a
| "programming language" is just one of the tools we have
| to solve a certain problem, and probably the last one we
| should reach for! For a stable system, you _want_ less
| moving parts. A good programmer fights for it.
|
| Can you solve a problem just by storing a JSON file
| somewhere? Can you solve a problem without a backend? Can
| you solve a frontend problem with just CSS or just HTML?
| Can you solve a problem without Javascript? Can you solve
| a data storage problem with just a database instead of
| database+Redis? Do you really need a full-fledged web
| framework where a micro-framework would suffice? Do you
| need micro services, Kubernetes, containers and whatnot
| for your site before it gets its first visitor?
|
| I find that a lot of people go for the "more powerful"
| tool just to cover their asses. They don't want surprises
| in the future, so they just go for something that will
| cover all bases. But what you actually want is the things
| with the least power [1].
|
| Another issue is that intelligent people have an anti-
| superpower called "rationalisation". They can justify
| _every single decision_ they make, as misguided as it is.
| So it doesn 't matter if a website could be done with a
| single HTML file: it is always possible to find a
| reasonable explanation for why it needed k8s, micro-
| services and four languages.
|
| [1] Using the least powerful tool also has other
| advantages, see Tim Berners-Lee "Principle of Least
| Power" for example - https://blog.codinghorror.com/the-
| principle-of-least-power/
| yc12340 wrote:
| > On the other hand, having the wisdom of knowing what
| can be static in the first place? I don't think that it's
| something teached.
|
| All traffic is static by definition. You are not
| modifying bytes when they are in transit to user. And you
| don't have to serve different bytes each microsecond just
| because users want to be "up-to-date". The network
| latency is usually around 40ms or so. If your website
| serves 1000s of requests per second, you should be able
| to cache each response for 10ms, and no one will ever
| notice (today this is called "micro-caching").
|
| Of course, most webpages can't be cached as whole -- they
| have multiple "dynamic" parts and have to be put together
| before serving to user. But you can cache each of those
| parts! This is even simpler if you do client-side
| rendering (which is why MangaDex abysmal performance is
| pathetic).
|
| Then there are ETags -- arbitrary strings, that can be
| used as keys for HTTP caching. By encoding information
| about each "part" into a substring of ETag you can
| perform server-side rendering and still cache 100% of
| your site within static web-server, such as Nginx. The
| backend can be written in absolute hogwash of language
| such as Js or Python, but the site will run fast because
| most requests will hit Nginx instead of slow backend.
| ETags are very powerful, -- there is literally no
| webpage, that can't be handled by a well-made ETag.
|
| Even pages, that need to be tailored to user's IP can be
| cached. It is tricky, but possible with Nginx alone.
|
| Instead of "static" you are better off thinking in terms
| of "content-addressable".
| Zababa wrote:
| I didn't know about ETag, thank you for that.
| wpietri wrote:
| An awful lot of professional programmers work in such
| heavyweight contexts that they don't have a good idea of how
| fast modern hardware can be.
|
| I was talking with an architect at a bank whose team was having
| trouble getting under a 2-second maximum for page views. They
| blamed it on having to make TCP requests to other services, and
| said something like "at a couple hundred milliseconds per
| request, it adds up quickly!" My head nearly exploded at that.
| I spun up some quick tests in AWS to show exactly how many
| requests one could make in 2000 ms. I don't have the numbers
| handy, but the number is very large.
|
| This junky slice of a server handling full page requests in 20
| ms is a fine example to counter thinking that's endemic in
| enterprise spaces.
| ppeetteerr wrote:
| This is a person running a blog and pulling HTML data from a
| database. Is it that impressive that they are getting 50r/s?
| wpietri wrote:
| It's not.
|
| My point isn't that it's impressive in some ultra-tuned
| performance sense. It's that doing pretty mundane things on
| pretty basic servers is still very fast compare with a) the
| past, or b) what a lot of developers are used to
| professionally. That's why it is interesting to the crowd
| here.
| dragontamer wrote:
| I had a discussion with a coworker about something about slow
| memcpys, that roughly went the same way except... you know,
| DDR4 RAM has a speed of roughly 40GBps.
|
| Also, that "awful" 1MB memcpy is likely all in L3 cache these
| days. But even if it weren't in cache, we're talking about an
| operation that takes 50 microseconds (1MB read + 1MB written
| == 25microseconds + 25 microseconds).
|
| Given that modern CPUs have like 16+ MBs of L3 cache (and
| more), and some mainstream desktop CPUs have 1MB of L2
| cache... its very possible that this memcpy is far faster in
| practice than you can imagine.
|
| 1MB is big, its a million bytes. But CPUs are on the scale of
| billions, so 1MB is actually rather small by modern
| standards. Its surprisingly difficult to get intuition
| correct these days...
| wpietri wrote:
| For sure. I think keeping one's intuition for systems
| updated is one of the biggest quests for anybody trying to
| have a career in tech.
| Ice_cream_suit wrote:
| In contrast, the Australian 2016 National Census crashed and
| burned since the system could not cope with over 250 requests a
| second.
|
| Parliamentary enquiry:(PDF)
| https://www.aph.gov.au/DocumentStore.ashx?id=0a7f6bd5-8716-4...
|
| https://www.zdnet.com/article/census-2016-among-worst-it-deb...
|
| https://www.theguardian.com/australia-news/2016/aug/10/compu...
| erdo wrote:
| Working on other companies' mobile apps, about half the
| performance problems I've discovered have been down to some
| accidental crazy, like initialising something you only need once,
| in a loop, in 3 different places (because the code has become so
| unnecessarily complicated that no one really knows what it's
| doing). The rest are due to some piece of code accidentally
| blocking the UI thread.
|
| A well written mobile app doesn't really have any need to be
| sluggish at all, including smooth animations and fast scrolling
| lists, it was doable 10 years ago, it's doable now. (*I don't
| know about games).
|
| But unlike on the server side, the accepted wisdom in most places
| I've worked at is that the answer to the performance problems is:
| a new framework.
|
| (I feel like this is a lie that developers tell the business
| side, and maybe themselves. It avoids having to explain that
| software is hard, sometimes you don't get it right the first
| time, and if you don't spend time and effort tending to it, it
| can turn into an ungodly and expensive mess - and that's got
| nothing to do with the hardware or the framework)
| bob1029 wrote:
| All these "X requests per unit time" posts are starting to make
| me want to break out some of my experimental code... I have some
| services that can process several million events per second. This
| includes: compressing the event batch, persisting to disk,
| validation of business logic, execution of all view updates
| (state tracked server-side), aggregation and distribution of
| client update events, etc. These implementations are easily
| capable of saturating NVMe flash.
|
| If you want to see where the theoretical limits lie, check out
| some of the fringe work around the LMAX Disruptor and .NET/C#:
|
| https://medium.com/@ocoanet/improving-net-disruptor-performa...
|
| You will find the upper bound of serialized processing to be
| somewhere around 500 million events per second.
|
| Personally, I have not pushed much beyond 7 million per second,
| but I also use reference types, non-ideal allocation strategies,
| etc.
|
| For making this a web-friendly thing: The trick I have found is
| to establish a websocket with your clients, and then pipe all of
| their events down with DOM updates coming up the other way. These
| 2 streams are entirely decoupled by way of the ringbuffer and a
| novel update/event strategy. This is how you can chew through
| insane numbers of events per unit time. All client events get
| thrown into a gigantic bucket which gets dumped into the CPU
| furnace in perfectly-sized chunks. The latency added by this
| approach is measured in hundreds of microseconds to maybe a
| millisecond. The more complex the client interactions (i.e. more
| events per unit time), the better this works. Blazor was the
| original inspiration for this. I may share my implementation at
| some point in the near future.
| mariushn wrote:
| > The trick I have found is to establish a websocket with your
| clients, and then pipe all of their events down with DOM
| updates coming up the other way. These 2 streams are entirely
| decoupled by way of the ringbuffer and a novel update/event
| strategy.
|
| Could you detail this, please? I don't get it. What is the
| flow?
|
| 1. Browser is sending events to web server via web socket,
| instantly as the event is occurring (?)
|
| 2. ? (what exactly does the server do?)
| bob1029 wrote:
| You got 1 correct. Everything that happens gets sent
| immediately as an event to the server (e.g. KeyDownEvent).
| These are pushed without blocking for a response to each -
| The websocket guarantees delivery and ordering.
|
| Upon receiving an event from the client socket, it is
| immediately inserted into the LMAX ring buffer for
| processing.
|
| Updates to the client are triggered by events+state
| determining when a redraw is required and issuing a special
| "ClientRedraw" event into the same queue. These events are
| grouped by client so that we can aggregate multiple potential
| updates in a single actual redraw. These result in view
| updates being pushed back down to the relevant clients. One
| performance trick here is that the client redraw is
| dispatched asynchronously from the server, so there is no
| blocking on processing the subsequent batches each time.
|
| You can think of an E2E client view update as always
| requiring 2 events - the client event that triggered the
| change to domain state, and the actual redraw event(s) that
| result. For applications where the client should update at a
| fixed interval (e.g. game), a high performance timer
| implementation injects periodic redraw events. Because the
| upper bound of the ring buffer latency is around a
| millisecond, this allows for incredibly low jitter on real
| time events. Scheduling client draws as simple domain events
| is feasible.
| mariushn wrote:
| Thank you!
| ComputerGuru wrote:
| Absolutely. Even a terrible Wordpress instance can be beautifully
| (and transparently!) cached behind either nginx or varnish with
| ease, in which case you're just serving static html pages and can
| probably handle any traffic you are likely to ever get.
| MangoCoffee wrote:
| this story remind me, the dot com bubble. dotcom companies bought
| servers from Sun Microsystem. they needs to handle the large
| traffics that "PC" server can't handle.
|
| anyone remember Cobalt server?
|
| https://en.wikipedia.org/wiki/Cobalt_Networks#/media/File:Co...
| sgt wrote:
| I remember many stories and discussions on Slashdot about it.
| [deleted]
| noduerme wrote:
| When I launched my former bitcoin casino in 2011 (it's gone, but
| it was a casino where all games, even roulette tables were
| multiplayer, on a platform built from scratch starting in '08), I
| handled all web requests through a server in Costa Rica that cost
| about $6/mo. Where I had a shell corporation for $250/year. Once
| the front end -- the bullet containing the entire casino code,
| about 250kb -- loaded, from Costa Rica, and once a user logged
| in, they were socketed to a server that handled the gaming action
| in the Isle of Man. Graphics and sounds were still sent from the
| Costa Rica server. I didn't have a gaming license in the IoM,
| though - that was around $400k to acquire legally. So I found a
| former IoM MP who was a lawyer, who wrote a letter to the IoM
| gov't stating that we didn't perform random calculations on their
| server, thus weren't a gambling site under IoM law. Technically
| that meant that no dice roll or card shuffle leading to a
| gambling win or loss took place on that server. So the IoM server
| handled the socketed user interactions, chat, hand rotation and
| tournament stuff. Also the bitcoin daemon and
| deposits/withdrawals. But to avoid casino licensing, I then set
| up a VPS in Switzerland that did only one thing: Return random
| numbers or shuffled decks of cards, with its own RNG. It was a
| quick backend cUrl call that would return a fresh deck or a dice
| roll, for any given random interaction on the casino servers. The
| IoM server would call the Swiss server every time a hand was
| dealt or a wheel was spun; the user was still looking at a site
| served from a cheap web host in Costa Rica. And thus... yeah, I
| guess I handled millions of requests a day over a $6 webserver,
| if you want to count it that way.
| amelius wrote:
| It would be even cooler if you derived the random numbers from
| a lava lamp visible through a window of the Swiss embassy.
| bcrosby95 wrote:
| Why go through IoM at all? Why not Costa Rica -> Switzerland,
| rather than Costa Rica -> IoM -> Switzerland?
| noduerme wrote:
| Good question. The original plan was to take normal payment
| methods (Visa/MasterCard) but it became apparent after Bush
| passed the ban on online poker that Costa Rica was going to
| follow suit (or that visa/mc would soon start holding up
| payments from CR casinos... in which case we might be stuck
| with debt we couldn't use to pay winnings). Setting up a CR
| bank acct as a shell requires you to hand over power of
| attorney to a CR citizen, and also given how shady the entire
| corporate structure there was and the legal outfit we hired
| (who thought we could take $100 deposits as payments through
| their fake real estate portal) I evaluated other routes.
| These included landing funds in Cyprus and processing through
| Israeli banks at a 10% markup, and other shady sounding
| things. I had begun to give up on my little side code project
| when Bitcoin showed up. The benefit of the Isle of Man was
| that all funds could be landed there - in a Bitcoin wallet,
| on a caged dedicated server - without triggering any other
| financial issues. the only trouble was randomness and
| gambling.
| ricardobeat wrote:
| With all this trouble, why not just set up shop in one of
| the many places where online gambling is legal?
| noduerme wrote:
| I spent 3 years writing the software in my spare time,
| and launched it on the $20k I had in savings...
| personally bankrolling the games. When I began the alpha
| phase, you could still get an "information processing"
| license to run a casino in Costa Rica that would allow
| you to host gambling and which the credit card companies
| tacitly accepted and would do business with. (Those
| gambling payments to shady jurisdictions are how Paypal
| got big, and how Elon Musk made his first nine figures).
| But by 2010/11, all payment processors had abandoned
| Costa Rica, and there was nowhere in the world offering a
| license I could come close to affording.
|
| I looked for investment, put together thick books of
| plans every few months, built 24 games... No one would
| get behind it, and I couldn't just shelf it, so this was
| how I launched it without breaking any laws.
|
| A few months in, one existing online casino network
| offered me $100k to just hand it over to them and then
| come work for them, but I considered the offer extremely
| insulting.
| zschuessler wrote:
| This story is fascinating. What ended up happening with
| the project, did you at least break even?
|
| I would read a book about this, fwiw.
| j1elo wrote:
| Focusing on a small but important detail that some have already
| mentioned but with a more aggressive tone... was your
| "loophole" system tested in an actual litigation at any point?
|
| What I mean is that this:
|
| > _The IoM server would call the Swiss server every time a hand
| was dealt_
|
| might seem like a clever loophole around the laws in IoM, but
| in reality it sounds to me like the kind of technicalities that
| wouldn't really pass the reasoning of a human judge, who in
| their duty of _interpreting_ the law and its intended spirit,
| would probably consider this an invalid trick and thus that the
| RNG of the system still resided in IoM, even if technically it
| didn 't.
|
| But of course, none of this matters if the casino never had any
| legal battle to fight where this idea could be tested in court,
| which is the equivalent of not being "caught".
| noduerme wrote:
| It was never legally tested. It was what I felt I had to do
| such that the randomness didn't take place on the island. And
| no randomness ever did. I was in touch with a lot of officers
| of the large casinos operating out of there at the time, who
| were curious but skeptical about Bitcoin. I think by the time
| they realized it was a potentially valuable thing, I had
| already shut down operations, because I wasn't willing to
| chase the market into legally gray areas.
| carlmr wrote:
| If you go by technicalities you could have used the
| argument that you're not generating random numbers, only
| pseudorandom numbers.
| mvc wrote:
| He got an MP on his side which seems to be all it takes these
| days in the UK.
| [deleted]
| madaxe_again wrote:
| You'd be amazed at how cheap it is. You can usually get an
| MP to do your bidding or say your piece in Parliament for a
| donation of PS10k or less. I've never heard of anyone
| paying more.
| [deleted]
| gnfargbl wrote:
| It would be a contempt of Parliament to either request or
| agree to such an arrangement [1]. If you have definitive
| evidence of this having happened in the recent past, you
| really should pass it on to the Parliamentary
| Commissioner for Standards.
|
| [1]
| https://erskinemay.parliament.uk/section/5023/corruption-
| or-...
| madaxe_again wrote:
| This is why it's a donation to the party - not to them or
| their office - although they can want a campaign
| contribution, which again doesn't count as going to them.
|
| I've had an MP literally solicit this from me - he
| emailed a bunch of local businesses basically stating his
| price list. He's no longer an MP, or in politics - but
| you see this stuff everywhere, all the time. Any member's
| bill you see has almost certainly been sponsored.
| Grustaf wrote:
| Does that include the lobbyist's cut, or would you go
| directly to the MP? But how much do you really gain from
| buying a single MP?
| toyg wrote:
| MPs typically move in packs ("areas" or "currents", that
| may or may not be officially organized around a thinktank
| or club), so buying the right MP you can actually move
| several of them. And then your representative goes and
| trades favors with the colleagues in government - "I'll
| vote for your controversial legislation if you pull that
| other lever over there". Representative democracy is a
| big sausage factory: in go blood and guts, out comes
| "edible" rules that society will live on.
| silverscania wrote:
| The IoM isn't in the UK and nor does it have MPs. Probably
| just semantics but I have to point that out.
| MagnumOpus wrote:
| IoM has MPs, just not UK MPs. The OP almost certainly
| meant an MP from the Manx Tynwald parliament.
| silverscania wrote:
| It doesn't though, that's my point. The Isle of Man has
| MHKs and ministers, not "MPs":
| https://en.wikipedia.org/wiki/House_of_Keys.
|
| It's like saying a politician in the UK is a "senator"
| and expecting people think you are an authority on the
| subject.
| amelius wrote:
| > in reality it sounds to me like the kind of technicalities
| that wouldn't really pass the reasoning of a human judge
|
| Yes, only big companies can successfully "hack" the law based
| on its letter, see e.g. tax evasion.
| seanalltogether wrote:
| Yeah, that's like setting up a casino in one location with a
| permanent phone line open to switzerland to ask where the
| ball landed on the roulette wheels. Doesn't seem like it
| would hold up under scrutiny.
| noduerme wrote:
| That's exactly what it was... and it really depends on how
| a locale defines gambling. In Costa Rica, for instance, you
| can run the game of chance _as long as the money isn 't
| landed onshore_, because you're just generating random
| numbers. IoM was slightly different in that they didn't
| mind you landing cash, they just wouldn't allow you to
| generate the numbers. So it seemed natural to co-locate,
| and then Switzerland was a better backstop than either.
| j1elo wrote:
| In any case I must admit that the trick, while we agree
| wouldn't (probably) hold water in court, it might have
| actually helped (we'll never know) to keep _that_ casino
| out of some law enforcement watch list... especially if
| the officers were too overburdened with more important
| issues / lazy enough / not looking into that kind of
| activities at the time.
| wiz21c wrote:
| Does this article count as "Cost Rica leaks ?"
| noduerme wrote:
| yeah, I'm blowing my HN cover.
| [deleted]
| arcticbull wrote:
| Very crafty. Nice. I'm legitimately impressed by every part of
| that.
| ajnin wrote:
| Thanks for sharing that, I hope it does not land you into
| trouble. So, how many billions did you make with that site ?
| [deleted]
| Gepsens wrote:
| You're a fricking genius.
| elcomet wrote:
| He's a frickin genius because he found ways to break the law
| with impunity for his own profit ?
|
| I'm surprised by HN sometimes.
| nix23 wrote:
| >He's a frickin genius because he found ways to break the
| law with impunity for his own profit ?
|
| Obviously he did not break the law, finding holes like that
| is the number one job of a tax-lawyer.
| dna_polymerase wrote:
| The breaking the law consists entirely of not acquiring a
| license. Otherwise everything they did was perfectly legal.
| noduerme wrote:
| finally, someone understands :)
| concordDance wrote:
| " _the_ law "?
|
| There is no "The law", just a bunch of different
| jurisdictions with different laws. He didn't break
| Switzerland law. Does Switzerland's law not matter for some
| reason?
| nickthemagicman wrote:
| You mean like the people who smoked marijuana before it was
| legal?
|
| Or the people who got same sex married before it was legal?
|
| Or the slaves who escaped slavery before it was legal?
|
| Should the slaves have just not try to free themselves
| because it was "illegal"?
|
| The law does not equal morality.
| abstractbarista wrote:
| Yes. Gambling should be legal anyway.
| noir_lord wrote:
| You can respect the ingenunity if not the morality of it.
|
| His solution was a clever hack because he worked around the
| law without apparently breaking it directly - clever.
|
| That said gambling systems are specifically one class of
| software I refuse to work on.
| hnbad wrote:
| I think the lawyer who convinced the gov't that the
| operation was legal being a former MP and this taking
| place in a nation of less than 100,000 people is a
| crucial element to the story.
|
| Incidentally both gambling AND fintech (including crypto)
| are on the list of industries I refuse to do work with.
| So I guess BTC gambling would have been off the table for
| two reasons.
| [deleted]
| noduerme wrote:
| Uh, excuse me, but this whole story demonstrates that I
| went to extreme lengths to _never break any laws_ , no
| matter how large or small the jurisdiction. Had I been
| willing to bend laws, I'd surely be a lot richer today.
| skrebbel wrote:
| Come on, cURL'ing to a foreign server to get a random
| number and not just reading /dev/urandom is logically
| identical. It's a hack, just like calling into GPL'ed
| code over HTTP is a hack to avoid "linking" the GPL'ed
| code. It doesn't _really_ suddenly turn a site from
| gambling site into a non-gambling site.
|
| I mean, I have mad respect for the hustle with the former
| MP etc. I agree with what you say in that you did not
| actually break the law - because you found a loophole
| (made a loophole? hustled it? again, I'm impressed). You
| ran a gambling site from IoM though :-)
| the_absurdist wrote:
| Logically identical is not legally identical.
| noduerme wrote:
| What's so funny is - I was in a peer group of early btc
| founders who were starting random Bitcoin hustles and
| didn't care at all about laws. No one at that time had
| even approached the IoM about whether BTC was currency -
| nor would they have, because trying to find legal haven
| was the furthest thing from their minds. And so not
| surprisingly, the IoM didn't have a ready answer when I
| asked them whether gambling with Bitcoin was actually
| gambling. But the letter of the law was that gambling
| occurs _in the jurisdiction where_ the random chance
| takes place. So it really was different from hitting a
| local RNG, legally.
|
| It added a nice little feature too, which was that every
| spin and deck could be stored on a separate server that
| would show them all at the end of the day. This was a
| little before "proven randomness" took off in btc
| casinos, but I made the RNG reports available daily for
| analysis (without explaining the whole infrastructure,
| obviously).
|
| [edit] I just want to say that yes, you're obviously
| right, and yeah, I ran a casino from the IoM... without
| anyone knowing if that was okay or not... and it was just
| a moment in my life. of which I'm proud, I guess. I was
| living illegally in a small apartment in Alhama de
| Granada after violating my EU visa. hah. It was a great,
| great piece of software and I don't know if I'll ever
| write anything that good again. But it didn't really
| change my life or anything.
| skrebbel wrote:
| Fwiw I think this is a great response and I'm happy you
| did not feel attacked by my comment because that was not
| my intent. Thanks for sharing & all the best!
| kapep wrote:
| > But the letter of the law was that gambling occurs _in
| the jurisdiction where_ the random chance takes place
|
| "where the random chance takes place" could easily be
| interpreted as where the random number is _used_ and not
| where it has been created. Creating random numbers is not
| "chance" per se (in this context). Using random numbers
| to e.g. determining a winner would be the chance in my
| opinion.
| dcposch wrote:
| Are you actually an LA taxi driver, like it says in your
| bio?
|
| What a story.
| noduerme wrote:
| I used to be. I got a web design job in San Francisco
| when I was 18 out of high school but I burned out and
| quit when I was 21. Drove a taxi so I could write and
| play music. Did it for a couple years. It was a good
| education. I feel old. Taxis don't even exist anymore.
| noduerme wrote:
| @kapep don't know why I can't respond directly..
|
| >>> where the random number is _used_
|
| This was my main concern, and it was exactly what I
| needed a lawyer to sign off on before I set up a rig
| there. I was told that the gambling laws applied to where
| the chance took place, not where the money is
| distributed... after all, the whole thing with the IoM
| and the reason it's allowed to be a tax haven is that
| lots of people need to move money around without a lot of
| questions. But they defined gambling in this specific way
| and if only the money moved but the dice roll didn't take
| place on their shores, then it wasn't gambling under
| their jurisdiction. What you bring up was _the_
| conversation I had before locating there.
| skrebbel wrote:
| > @kapep don't know why I can't respond directly..
|
| HN has a silly but effective piece of anti-flamewar UX
| which is that it hides the reply link in certain cases
| (some function of thread depth + amount of comments by
| you i think). However you can still reply by opening the
| comment on question (click on the timestamp, ie the "1
| hour ago" link). Maybe you hit that.
| krzrak wrote:
| > Please write a book. Or an article, at least.
| skinkestek wrote:
| > Come on, cURL'ing to a foreign server to get a random
| number and not just reading /dev/urandom is logically
| identical. It's a hack, just like calling into GPL'ed
| code over HTTP is a hack to avoid "linking" the GPL'ed
| code. It doesn't really suddenly turn a site from
| gambling site into a non-gambling site.
|
| Around here (and probably elsewhere) bars aren't allowed
| to make wine stronger by adding spirit.
|
| So if you mix a drink from wine (or similar) and spirit
| in that order you might lose your license.
|
| Put the spirit in the glass first and all is ok.
|
| I guess at this point it is just a shibboleth that
| inspectors use to see if the bar has read the rules at
| all, kind of like the no brown m&ms.
|
| Point is though: rules matter, you can lose your license
| over it.
| hnbad wrote:
| Sure, you're not breaking any laws, just intentionally
| violating the spirit of the law for personal gain.
|
| This isn't a court of law. I can understand trying to
| avoid language that makes it sound like you may have been
| in violation of the law to a judge or jury, but you
| literally described that what you did was intended to
| keep operating a service that had been banned by using an
| absurd technicality in the definition of the ban.
|
| I'm honestly surprised it worked (though having a former
| MP of the tiny nation you were operating in as a lawyer
| might have helped) considering that your service still
| facilitated online gambling directly and was advertised
| as such, despite the randomness source being on a remote
| server rather than local.
|
| In other words, using a non-local randomness source (like
| a remote server you cURL into or a webcam pointed at a
| bunch of lava lamps) is functionally indistinguishable
| from a local dice roll or other source of entropy. This
| "hack" is so flimsy it likely wouldn't hold up in court
| in a nation that is actually interested in pursuing such
| violations that has a population larger than a small
| city.
| Jamieee wrote:
| Corporations do not so dissimilar things with teams of
| accountants and lawyers. Via tax planning, international
| subsidiaries and all other sorts of loopholes.
| hnbad wrote:
| Are you trying to make an argument?
|
| I'm equally hostile to corporations doing that. I don't
| recall the HN comment threats about Google doing the kind
| of things you describe being full of replies
| congratulating their ingenuity.
|
| Big corporation doing bad thing is bad doesn't mean much
| smaller corporation doing bad thing is okay, it means we
| should work on preventing that bad thing and if we
| seemingly can't we should reconsider the underlying
| systemic conditions that enable it.
| noduerme wrote:
| I suspect your anger derives from the fact that this
| would not be possible to get away with now, in any way
| shape or form, only ten years later. And truly the world
| is way more locked down now than it was then when people
| were like, "Bitcoin? What's that? you want to pay me to
| write a legal brief?" so, yeah. I feel sorry for kids ten
| years younger than me.
|
| When I did it, the only thing I was really afraid of was
| getting arrested if/when I stepped back on American soil.
| There was redundancy so I could run the whole thing in
| Costa Rica if I had to cold shutdown the IoM servers. And
| the coin was in private wallets, mostly on my laptop. But
| I was very concerned about breaking any laws, anywhere. I
| was the only one to implement ID verification and fully
| block American players.
|
| Call it a hack or whatever, they wanted my business and I
| needed their servers, and I split up my code so it would
| be legal according to their laws. Not too different from
| what a lot of companies do.
| noduerme wrote:
| >>> I'm not angry, I'm hostile.
|
| Hah! This made me laugh. Ok so not FOMO..(trust me,
| wasn't worth it except for the thrills).. why hostile? I
| was born in a Vegas family. My uncles all worked as
| blackjack dealers and pit bosses. When I was 7 they used
| to leave me in a corner of the casino for hours and tell
| me to stand there while my parents went and gambled. I
| taught myself to code there on a TRS-80 Model 100 in
| basic and practically the first thing I wrote was a slot
| machine. My view is that adults want to go gamble and
| that's their decision. I never took a dime from anyone I
| saw with a gambling problem... I would ban them from my
| site if they seemed addicted. I like to gamble myself. I
| count cards. Like everyone on my site... because my decks
| were single shuffle. So don't be so judgmental. I didn't
| do it for the money. I did it because I love the games.
| the_absurdist wrote:
| It's not anger. It's jealousy.
|
| The same jealousy that's called "opinionated" in software
| development, but really means "Doing things differently
| than how I do them threatens me, because my sense of
| superiority is rooted in how I do things."
| hnbad wrote:
| I'm not angry, I'm hostile. That you think the only
| possible position from which to take issue with what you
| did is FOMO speaks volumes.
|
| I'm not arguing that you violated any laws. You made it
| very clear that you went to great lengths to avoid doing
| anything that could have resulted in consequences to
| yourself.
|
| EDIT: Since HN's rate limiting won't let me reply for a
| few hours, I'll just address the replies inline:
|
| I'm not jealous. I'm sure noduerme made a sizable chunk
| of money with the whole operation at the time but their
| profile says they're now working as a taxi driver and
| sold all their bitcoin before the peak. They probably
| have a lot of other interesting stories to tell and
| that's nice. But dismissing any hostility or criticism as
| jealousy is thought terminating and frankly below even
| HN's standards.
|
| Based on their backstory in the replies, I can see where
| their attitude comes from, but they severely
| underestimate how big of a problem gambling addiction is
| and how much of the profit of the gambling industry
| relies on it.
|
| It's nice if the casino their parents worked at turned
| away obvious addicts but the word "obvious" is doing a
| lot of work here and there are also clear business
| reasons you don't want obvious addicts in your
| establishment the same way bars will be happy to have
| repeat customers buying drinks for five hours every day
| but will turn them away if they get blackout drunk or
| unsightly. "Not doing it for the money" may give you a
| clean conscience but it doesn't change the consequences
| of your actions.
|
| It's also important to point out that online gambling is
| by its nature functionally anonymous for the gambler
| (even if you record IDs for legal reasons). The online
| casino isn't going to turn away the addict until they can
| no longer pay or have to resort to fraud to keep up the
| habit. And even if the casino implements limits, the
| proliferation of online casinos makes it considerably
| easier to go hopping than if you have to physically drive
| somewhere.
|
| Gambling addiction not only ruins the lives of the addict
| but also impacts their friends and family, not just
| financially. It's true that not every person who gambles
| is an addict but the line between an expensive hobby and
| a managed addiction is hard to draw until you undeniably
| cross it.
|
| But if you need a comment on HN to explain to you why
| gambling and especially online gambling is bad, a comment
| on HN isn't going to be enough to convince you.
| noduerme wrote:
| Ok, I respectfully understand where you're coming from,
| and I've struggled with gambling addiction myself. I
| personally do not think it's immoral to offer games of
| chance to people, as long as they understand the odds and
| you're not cheating them. And believe me, running a small
| online casino mostly by myself with my own bankroll was
| literally setting alarms all night waking me up when
| someone was killing the tables and potentially going to
| bankrupt me. I got to know my big players (most of whom
| went on to become Bitcoin millionaires, since the early
| adopters were the only ones gambling on Bitcoin casinos
| in 2011)... but beyond that, I really don't think
| offering gambling is immoral as long as everyone knows
| what they're signing up for. I've dealt cards for a
| living, too. I don't drive a taxi anymore. But I've seen
| all sides of life. I don't think you can judge so easily.
| Yeah, big corporations suck and they screw people into
| debt, and gambling addiction does ruin peoples' lives,
| but I knew my players, and I don't think what I did
| personally hurt anybody. They came together to enjoy
| games, and yeah it was for real money, but it was also a
| community and they were there for fun - they could have
| gambled for a lot more money on other sites. One player
| built a puzzle in his escape room in Amsterdam in honor
| of / based on a game I designed. Like a bar, this is
| something people do for enjoyment, and you don't need to
| stand out there with a sign saying we're all going to
| hell. But I do get it and I'm not a fan of the companies
| that take advantage of human weakness and shake the dimes
| out of people's pockets. I was just trying to have a good
| time and give other people a good time.
|
| Total profit from 2 years running the site? About $50k.
| It was a hobby. I never quit my job. I also turned away
| 95% of the hits because they were coming from America.
|
| [edit] I should add that I strongly advised other BTC
| site owners, especially casino owners, to follow certain
| guidelines, and watched one of them who I had told to be
| careful launch, make about $1M with one game on a crappy
| website, and get jailed within a year. That wasn't the
| trajectory I was interested in.
| rualca wrote:
| > (...) but this whole story demonstrates that I went to
| extreme lengths to never break any laws,
|
| No, not really. It shows that you went to great lengths
| to find ways to exploit a loophole where, even though you
| are clearly breaking the spirit of the law, you argue
| that it doesn't break the letter of the law.
|
| I get it that you have a vested interest in keeping up
| the plausible deniability thing, but you know it and
| everyone knows it that you went through great lengths to
| put up a tech infrastructure which meets absolutely no
| requirement other than exploiting a loophole.
|
| I mean, you explicitly expressed your personal concerns
| in this very discussion regarding what you personally
| chose to describe as testing "legally grey areas". Who do
| you expect to fool?
| noduerme wrote:
| I just really wanted to launch my software. Which when I
| began coding it, seemed completely legal and possible in
| Costa Rica. As the laws started to change - and even
| before Bitcoin came on the scene - I looked for how to do
| it without running afoul of anything. So it's not like I
| set out with a plan to exploit all the legal loopholes in
| the world, I just adapted my code and split it apart as
| necessary. I never even meant to take Bitcoin, let alone
| make it the only currency in the casino. It was just the
| only option if I wanted to launch. I had very little
| money and had written a giant gaming platform. I wanted
| it to see the light of day.
| rualca wrote:
| > I just really wanted to launch my software.
|
| Come on. Cut the bullshit.
|
| > Which when I began coding it, seemed completely legal
| and possible in Costa Rica. As the laws started to change
| - and even before Bitcoin came on the scene - I looked
| for how to do it without running afoul of anything. So
| it's not like I set out with a plan to exploit all the
| legal loopholes in the world, I just adapted my code and
| split it apart as necessary.
|
| You are clearly and unequivocally stating that you set to
| exploit all the legal loopholes once your "grey area" was
| made black and white in Costa Rica.
|
| Please, spare the thread from all that nonsense. You're
| not fooling anyone.
| noduerme wrote:
| What you say I stated "unequivocally" is precisely the
| opposite of what I stated. I was quite clear - what I
| said was that _I didn 't want to pursue the business into
| a legal gray area._ Therefore, I did what I had to do to
| keep it legal, including turning away 95% of the hits,
| retaining lawyers, and not violating any local laws.
| Moreover, it would have been perfectly legal to run the
| whole thing from Costa Rican servers at any point in
| time. I just didn't see a future in it, because they
| didn't offer full licensing, and the credit card
| companies pulled out while the software was in alpha. The
| IoM paper was intended as a step on the road to
| licensing.
|
| I didn't exploit anything. I worked within the legal
| options that were available. In any case, I don't
| understand the accusation.
|
| My only regret is that I didn't have the capital to buy a
| full license in the IoM or Malta outright. But the truth
| is, I wrote the whole thing from scratch and I was
| determined to launch it. You're free to your opinions,
| but you ought to avoid judging people's intentions while
| misreading their words.
| void_mint wrote:
| Stop speaking for other people.
| void_mint wrote:
| Is your assertion that, because you deem his actions
| immoral, they did not require intelligence?
| mellavora wrote:
| I respect both the legality and the morality of it.
|
| First, he did comply with all applicable law. No laws were
| broken.
|
| Second, he did not break the spirit of the law. The law
| clearly allows gambling from the Isle of Man.
|
| Third, he did not conflate the law with morality. What is
| the morality of a 400,000 GBP 'licensing' fee? Laws around
| licensing are weird. Another poster mentioned that pouring
| wine than liquor into a glass is illegal, but liquor then
| wine is fine. Not much moral sense in that reg.
| noduerme wrote:
| Thank you. And yes, it's incredible the morally bankrupt
| things you see under color of law if you try getting into
| that business. Side story, I was once invited to be a
| guest of the now deposed dictator of Ghana when I
| casually floated to his lawyer who I met in a casino in
| Prague the idea of making them the next offshore gambling
| capital of the world. I did a little research on the
| country and then respectfully declined. In my view, I
| never broke a law _or_ did anything immoral (since I was
| extremely transparent about the odds of every game and I
| made sure to only take adults from countries where gaming
| online was legal... who wanted to play, and knew the
| rules). And not conflating morality with legality --
| while respectfully considering both -- is just a
| precondition to being an individual in a complicated
| world.
| Cthulhu_ wrote:
| No laws were broken, else they'd be in jail. It's how the
| big companies that HN worships get away with not paying
| taxes.
|
| Morally dubious? For sure.
| maccard wrote:
| > No laws were broken, else they'd be in jail.
|
| Just because OP isn't in jail doesn't mean they didn't
| break the law. Laws are broken all the time.
| noduerme wrote:
| No. I'm not in jail because I did everything by the book.
| Everyone else thought it was the wild west.
| void_mint wrote:
| > No laws were broken, else they'd be in jail.
|
| Generous to assume that all that break laws go to jail.
| noduerme wrote:
| Sarcasm much? I'm a full blown idiot. I was just solving one
| problem at a time.
| Gepsens wrote:
| Erm no, I really liked your post
| noduerme wrote:
| thanks.
| nickthemagicman wrote:
| Same. It was very interesting.
| hdjjhhvvhga wrote:
| > we didn't perform random calculations on their server, thus
| weren't a gambling site under IoM law.
|
| Does it really matter if you get your random number from
| /dev/urandom or a server in Switzerland?
| the_absurdist wrote:
| Why doesn't it matter?
| withinboredom wrote:
| I don't think it'd hold up in court. This is the same as
| saying: "I bet you ten bucks when I mail my friend in
| Switzerland, he will mail back a ten." The random part is
| being done in another country, but the betting is still being
| done wherever you and your friend are.
| ahoka wrote:
| Isn't it like betting on games happening in a different
| country? I'm pretty sure that is still gambling, even if
| entropy is generated in a different place.
| withinboredom wrote:
| Maybe the laws are written that way in IoM just so you
| don't have to be registered to bet on Kentucky horse
| races or sports or whatever? Could be that way on
| purpose. Interesting loop-hole.
| johnklos wrote:
| This is a good, simple way to show how much can be done with
| modest resources.
|
| Sometimes we see people fetishizing bigger and faster, then
| gatekeeping when people want to do the same work with modest
| means, whether it a four quid a month hosting service or a first
| generation Raspberry Pi. Not everyone has the money or desire for
| bigger & faster, and it's nice to see that here.
| danjac wrote:
| I'm not sure it's so much about fetishizing, and more about
| realistic expectations around software development in larger
| teams.
|
| If you are the sole developer working on your own site - be it
| a side project/hobby/labour of love or your source of income -
| you have complete control up and down the stack and have the
| leeway to tweak performance wherever needed - whether that's
| indexing and optimizing queries in the backend, reducing the
| size of your static assets, caching, whatever. You can even
| yank whole features if you feel their inherent complexity and
| load outweighs their usefulness.
|
| In anything including and above a medium sized company, a
| single developer will rarely have the leeway to do anything
| beyond tinker with their small slice of the stack. They might
| spend some hours carefully optimizing a query, but it's for
| naught because the frontend team have screwed up the webpack
| settings and the JS load runs into many MB. Or you have both
| done your jobs but the PM wants a ton of analytics on every
| page. And the CEO's pet feature is a maintenance and
| performance nightmare but nobody has the clout to have it
| removed or even simplified. Nobody wants to waste sprints on
| paying down tech debt in a feature factory, so it becomes
| progressively harder to fix performance issues.
|
| At that point, the cheaper and politically easier option is to
| just fire the money cannon at expensive cloud services and hope
| the extra spend squeezes out some performance gains.
| mark_mcnally_je wrote:
| That's why I made this post, was very happy to see how much it
| could theoretically handle.
| sam0x17 wrote:
| Impressed that it is front page of HN and it isn't "This page
| cannot be displayed" with that kind of premise. Props.
| 1432132143 wrote:
| ROTFL Wikipedia 2010 handle 7mln request per seconds suqids in
| USA Europe in Japan. Some random dud who do benchmark with
| 0.00001 seconds TCP on localhost "HEllo World" only ycombinator
| lamerers can believe this without asking questions.
| modeless wrote:
| You don't need PS4. As long as you can structure your site as a
| set of static files with interactivity done client side, even if
| some of the files change every minute or two, you can serve
| everything for $0 with Cloudflare in front of any free host. I've
| served 1M pageviews a day for $0 with Cloudflare + App Engine
| free tier and there's no reason it wouldn't scale to 100M or
| beyond.
| goodpoint wrote:
| No thanks. Cloudflare is very harmful to Tor and to user's
| freedom and privacy in general.
| jnieminen wrote:
| You could also use Github pages or Cloudflare pages to host.
| modeless wrote:
| The nice thing about using Cloudflare in front of a real host
| is that you can still do dynamic pages. You can instantly
| purge any file from the Cloudflare cache, so all you have to
| do is purge anything that changes and your users see the
| update instantly, while your backend only sees a couple of
| extra requests.
|
| My site updates some data every 10 minutes or so, I don't
| think that would work with GitHub Pages. Maybe you could do
| something with Cloudflare Pages combined with Workers, but
| Workers have a limited free tier. The normal Cloudflare CDN
| scales to infinity for free.
| jonplackett wrote:
| #1 on HN and still up. That speaks for itself.
| mark_mcnally_je wrote:
| :)
| jonplackett wrote:
| Are your starting to feel the pressure yet?
| mark_mcnally_je wrote:
| My blog is fine, my self hosted analytics (on another
| server) not so much!
| jonplackett wrote:
| Do you know what the peak requests per second was?
| idworks1 wrote:
| One of my proudest moment in my career is when I lowered our app
| processing time from ~8hrs to 17 minutes. When I deployed my
| first update, it reduced it to 2 hours. The sysadmin immediately
| contacted me that there was something unusual. I confirmed the
| results but he was skeptical.
|
| Then with my second update, he told me that the app must be
| broken or that the script must be dying. There is no way it could
| complete this fast.
|
| What was the issue? We processed terabytes of data. Each and
| every single line processed created a new connection to the
| database and left it hanging. A try catch was added when the
| connections failed and restarted the process. Removing the
| connection from the for loop and properly handling it reduced the
| time drastically.
|
| And... why would you loop through millions of records when you
| can use batches? Also this was a phperlbashton* script. I turned
| it into a single PHP script and called it a day.
|
| As a consequence, backup time was reduced to 2 hours as opposed
| to 12 hours (no one was allowed on the website until the back up
| was done).
|
| Modern machines are incredibly fast.
|
| * PHP/Perl/Bash/Python
| globular-toast wrote:
| I have a similar story where I reduced the memory used by a
| script from 1TB (yes, TB) to a few megabytes. The runtime was
| massively reduced too, from something like 1 day to a few
| minutes.
|
| This was for a genomics project and they ran it on a
| supercomputer. When I looked into it, they were reading the
| entire input into a giant array before doing one pass and
| dumping the result out to disk. I made a tiny change (it was a
| Perl script) to make it stream the I/O instead.
|
| This is the most extreme example I've come across of people
| using computing power just because it's there. Nobody
| questioned why the script took so long to run because the data
| really was in the TBs and other stuff also took that long to
| run. Waiting a day for the results was considered normal. I see
| the same thing on desktop apps etc., on a much smaller scale,
| of course. When I run an electron app it takes several hundred
| milleseconds to do anything at all. But nobody questions
| whether it should because everything takes several hundred
| milliseconds.
| darylteo wrote:
| I had a similar experience as a intern years ago! Part of my
| daily responsibility was to manually enable/disable API
| integrators who had increased levels of traffic as it would
| take the API down for all partners. Pretty bad. Thousands of
| requests at peak would bring the server to its knees.
|
| Until I worked out during some minor maintenance task that
| every request was logged to a flat file. Appended. Every
| request. The file was probably 100gb by the time I found it and
| every request log would lock the logging file. The server had
| been running for a couple of years by that time.
|
| Of course I screwed up more than I fixed. :D
| nicbou wrote:
| I've done something like that at my last university internship.
| I wrote about it here:
| https://nicolasbouliane.com/projects/pratt-whitney-redesign
|
| It's the same story as yours, but with human effort. I was
| about to cut the human out entirely.
| devortel wrote:
| > no one was allowed on the website until the back up was done
|
| I'm assuming this was an internal website and backups were
| scheduled for evenings/weekends?
| nicbou wrote:
| I've done something like that at my last university internship.
| I wrote about it here:
| https://nicolasbouliane.com/projects/pratt-whitney-redesign
|
| It's the same story as yours, but with human effort. I was
| about to cut the human out entirely, and fix a ton of errors in
| the process.
| great-potential wrote:
| omg what is this, we need some Brendan Gregg posts to elevate
| the level here, this is HN not reddit :D
| BatteryMountain wrote:
| Mine was when I rewrote some application code to be processed
| within postgres to send a few thousand sms's (the logic to
| decide who should get them was the slow part + amount of data
| involved). It went from about 45 minutes to less than 1 minute.
| Was an amazing feeling to see the sms table filled up with the
| correct data - I also thought something was broken since we
| just accepted that those runs were normally supposed to be
| slow.
| vmception wrote:
| and then you were fired because your job was dependent on this
| taking forever
| jaymzcampbell wrote:
| I'm not sure there was ever an argument saying otherwise. The
| ease of processing X million requests is heavily dependant on
| what those requests actually _do_. Trivial use cases shouldn 't
| be a surprise to have high throughput.
| welder wrote:
| This one's even cheaper:
|
| My PS0 a month server can handle 4.2M requests a day [1]
|
| [1] https://ahamlett.com/blog/post/My-%C2%A30-a-month-server-
| can...
| fbrchps wrote:
| I understand your excitement for being able to handle a decent
| amount of requests on such a small server, but just like many
| other websites that get on the frontpage of HN, your site is
| taking multiple seconds to load for me, depending on when I
| refresh.
|
| As you said in your post, adding caching to your site increased
| your throughput by ~20% (or +10/req/sec). What you and other
| sites seem to lack is a more distributed caching, a la
| CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only
| really work well for a static site, however as mentioned in your
| post that's essentially what you're serving.
|
| While I'm all for having a free-as-in-freedom hosting solution
| and keeping things lean, the internet is a fickle beast, and
| nothing looks worse for a company who posts on HN when their
| technology-oriented site can't handle a few thousand requests per
| minute. (Or in this case, when a blog claims to handle 4.2M
| requests a day -- 2.9k req/min)
| jeroenhd wrote:
| Are you sure that's related to the server itself? The page
| loads instantly for me and the DNS is still resolving to an OVH
| IP address.
|
| Timing info from Firefox: Blocked: 0ms DNS resolution: 8ms
| Connecting: 9ms TLS setup: 12ms Sending: 0ms Waiting: 30ms
|
| The very last resource (favicon.ico) loaded after 466ms and
| that's mostly because of the other files being requested only
| after the CSS has come in (after about 195ms). All in all the
| entire site (without the Matomo tracking JS) loaded in half a
| second.
|
| Maybe the website has switched hosts in the last ten minutes, I
| guess, but I doubt it. I think this is more likely to be a
| problem related to distance to origin and saturation of the
| underlying connection.
| mark_mcnally_je wrote:
| Nope, website has not changed at all - speaking of the Matomo
| tracking; that is running on a separate server and has
| actually crashed!
| _fizz_buzz_ wrote:
| Weird, this site loads really fast for me actually. Much faster
| than most sites that I visit.
| mark_mcnally_je wrote:
| Thanks! Just goes to show that it runs quickly even when it
| hits #1 on HN ;)
| fbrchps wrote:
| Hey OP, in case it wasn't clear from my original comment, I
| am impressed with how stable your site is! "A few seconds"
| is wonderful for a #1 post, and far better than what
| usually happens to lightly-hosted sites that get to this
| point. Those are usually timeouts or outright failures to
| connect.
|
| I just figured I'd start some conversation on the post,
| since there wasn't any comments when I initially looked.
| For better or for worse it seems like I got people talking.
| spicybright wrote:
| Working extremely fast for me right now. Your post was about 20
| minutes ago. I don't know how HN traffic fluctuates, but it
| seems really solid compared to most sites.
| umvi wrote:
| Loaded instantly for me
| tiew9Vii wrote:
| It sounds like they are caching database queries.
|
| > Parts of the blog posts are cached using memcached for 10
| mins
|
| That means Django needs to accept the request, route it, pull
| the data from memcached, render the template.
|
| For such a site I'd just set the `Cache-Control` headers and
| stick Varnish in-front of it acting as a reverse proxy. That'd
| likely increase the page load times significantly and make the
| backend simpler not worrying about manually caching in
| memcached and just setting the correct `Cache-Control` http
| header.
|
| As it's budget hosting i'd probably not even bother with
| Varnish and outsource that to Cloudflares generous free tier,
| it's cheating as your server (Origin) isn't doing 4.2m requests
| but the practicality is really convenient.
| crazy_horse wrote:
| 200 rps is not great.
| fridif wrote:
| Wikipedia as a whole has 8k rps, and that's with multiple
| racks in multiple data centers.
|
| I haven't read recently, but they were only doing 200 rps per
| server.
| floren wrote:
| Looks fine over here, and he doesn't have to route through a
| fucking Internet gatekeeper like Cloudflare or Amazon... let's
| enjoy this golden era before Chrome starts flagging any site
| which isn't fronted by a "reputable" cache like Cloudflare,
| Amazon, or whatever Google decides to introduce.
| fbrchps wrote:
| It would also be possible for OP to spin up their own Redis
| cache, and have multiple POPs near their target audience, and
| handle DoS type attacks against their site if need be, and
| easily be able to brush aside bot traffic, and...
|
| Not all the above apply to a hobby-blog style site, but I
| wasn't referring only to OP's site in my original comment. I
| understand that not everyone needs to feed into "fucking
| Internet gatekeeper"s as you described, but the fact that
| they provide valuable services is undeniable. They make a
| complex operation -- one that could mean the difference
| between a company being able to sell their product or not --
| simple.
| antihero wrote:
| Could just install Varnish locally.
| mark_mcnally_je wrote:
| Yeah a varnish install locally is the next step up from
| what I am doing I guess. Spinning up multiple POPs +
| cloudflare is way to much IMO
| fbrchps wrote:
| Haven't used Varnish myself directly, but yeah that would
| also work.
|
| For OP, I'd also be interested to see the benchmarks
| between this PS4 server, and a PS8 or PS10 one, same
| stack.
| remram wrote:
| Google Cloud CDN? https://cloud.google.com/cdn/
| floren wrote:
| I figured they had to have something but I wasn't aware of
| the exact product. Thanks.
|
| Google Cloud CDN: Give us time, and we'll do to HTTP what
| we did to SMTP
| chii wrote:
| > we'll do to HTTP what we did to SMTP
|
| they already did it - it's called chrome.
| ezfe wrote:
| Oh I hate Google as much as the next guy, but that's not
| something they've shown any interest in doing.
| shtps wrote:
| Google AMP comes to mind.
| JohnWhigham wrote:
| People like you also probably thought the "Don't be evil"
| slogan would be around forever too, huh?
| mark_mcnally_je wrote:
| Yeah if I was building a business website I would want
| distributed caching/a CDN, mainly to support spikes, like what
| is happening now!
| fbrchps wrote:
| Working in the space, that's one of the more frustrating
| things to see on HN/Reddit/etc. It's not a complex or niche
| thing, and especially for sites that only make profit when
| people can actually visit them, it's kind of a necessity to
| stay up as much as possible.
|
| (Obviously the sales thing doesn't apply to OP)
| tomrod wrote:
| For those of us that are new to the space, lack background,
| or wear 50 hats in a startup, can you point us to best
| practices here?
| tick_tock_tick wrote:
| Just throw cloudflare in front it's free.
| fbrchps wrote:
| I don't have a "Do all These for a Fast Website" list
| handy, but here are some key points I've found can be
| applied to most sites:
|
| - Make sites that are fast by default: Small bytes sent
| over the wire, beyond just initial page load, too. Yes,
| that does mean that your giant Google Tag
| Manager/Analytics/3rd party script is bloated. Reach out
| to 3rd parties about reducing their payload size, it's
| saved me several MB over the years. Also, not writing
| efficient CSS is a huge killer when it comes to byte
| site. Devs shouldn't "leave it just in case" when it
| comes to code, you have version control for a reason. And
| when a new feature comes out, clear out the old cruft.
|
| - Avoid unnecessary DB calls: Obviously you need to get
| the data onto the page somehow, but if you can server-
| side render, then cache that result, you're reducing the
| overall calls to the DB. Also, optimizing queries to
| return only-what-you-need responses helps reducing total
| bytes over the wire
|
| - Balance between Server and Client side: Not only are
| servers getting more powerful, so are client devices.
| Some logic can be offloaded to clients in most cases, but
| there needs to be a balance. Business-critical logic
| should probably be done server side, but things like
| pagination & sorting -- so long as they client will
| likely see or use all the data -- is fine in my book.
| Having 2000 rows of JSON in memory is totally OK, but
| rendering 2000 at once might cause some issues. Again,
| balance
|
| - Hopping on the latest-and-greatest bandwagon isn't the
| best: Devs hate re-writing the site every 6 months, and
| really the newest framework might not be the best for
| your use case. Keep up to date with new technology, but
| saying "not for me" is fine.
|
| - Don't let (non technical) managers make technology
| decision: See above. More often than not, C-levels want
| to use shiny new things they read an article about on
| LinkedIn once, no matter if it fits the needs of the
| company or not. Thankfully I've only been at one place
| that was like that, but while I was there it was hell.
| Current VP was an original developer on the site back in
| the early 00's, so he knows how to deflect BS for us.
| That VP also knows that he's outdated in his knowledge by
| now, so he trust the Devs to make technical decisions
| that are best for the company.
| blfr wrote:
| _your site is taking multiple seconds to load for me, depending
| on when I refresh._
|
| Barely over a second here. Much better than vast majority of
| "webscale" services.
| fbrchps wrote:
| For sure, OP's site is handling this much better than most.
| And like I said, it's not every time that it takes multiple
| seconds. Some websites featured on HN/Reddit don't load at
| all when under load. However I was able to get it to take
| ~30s to load multiple times, over a period of around 10
| minutes.
| Tenoke wrote:
| In their defence the sites that fail to load or take too
| long are usually full webapps that do a lot of work rather
| than just static sites.
| fabian2k wrote:
| The important message there is that if you can change
| your problem from serving slow dynamic content to serving
| static content you can gain enormous performance
| benefits.
|
| Whether that means actually using static sites for stuff
| that can be static or just properly caching expensive
| things. Even dynamic content doesn't have to be slow, but
| many CMS are seriously inefficient without a cache. I'm
| not really blaming the CMSes entirely here, part of that
| is because they need to be extremely flexible, but once
| you need dozens of DB queries per page it'll fall over
| quickly on small hardware.
| fbrchps wrote:
| We're using Next.js at my current company with a custom
| MongoDB based CMS.
|
| Next has a thing called Incremental Static
| Regeneration[0] which allows us to grab the top ~100
| pages from the CMS at build time, generate the pages,
| then cache them for however long we want. The rest of the
| pages are grabbed when requested, then are cached for the
| same amount of time. After the time, they're re-grabbed
| from the DB, then re-cached. Overall I think we're down
| to around 5-10% of the way things were done before, which
| was -- you guessed it -- hit the DB on every page load
| _just in case_.
|
| Sit the Next.js site behind CloudFlare, and then we also
| don't really pay data transfer costs. Our servers are
| just low-tier GKE nodes, and we run around 3k/visitors at
| any given time, sometimes spiking up to 8k concurrent.
|
| [0] https://nextjs.org/docs/basic-features/data-
| fetching#increme...
| fabian2k wrote:
| Even database queries aren't that slow on reasonable
| hardware, as long as the queries are simple. The problem
| appears once you have dozens of DB queries per page. It's
| really not a fair comparison to the site this topic is
| about, but for trivial queries you can easily get a few
| thousand requests per second out of Postgres on desktop
| hardware without any real tuning as long as the DB fits
| into memory.
|
| But static content is of course still much faster and
| also much simpler.
| Rd6n6 wrote:
| Re: benchmarking, sometimes the bottleneck is the machine or
| server that issues the requests, not the receiver that you are
| testing. To figure out your actual capacity, you sometimes need
| multiple request servers or a more powerful request server. This
| was the case for a project I did a few years ago. Not a critique
| of the blog post, just remembering something out loud
|
| His site, https://peepopoll.com/, took about 10s to load for me.
| It's also good to chart other metrics like response times while
| you benchmark. Requests per second isn't the same as a low
| response time
| hu3 wrote:
| Indeed. Recently a client needed to bench raw req/s processing
| power of their application server and I had to ask for a
| powerful server running on the same DC in order to discard any
| potential routing issues.
| tristor wrote:
| The comments in this thread surprise me quite a lot. I suppose it
| shouldn't, but it does. This post + the response calls to the
| surface how badly basic system operations knowledge is needed in
| the industry and how much of it is missing from the toolkit of
| most developers.
| vymague wrote:
| > basic system operations knowledge
|
| Perhaps you can suggest a book or roadmap to learn it?
| tristor wrote:
| I replied to your sibling comment with something approaching
| a book list.
| planet-and-halo wrote:
| Any recommended reading? I've bought a few things in that vein
| but I'm self-taught and always looking to improve on the
| ops/perf side.
| tristor wrote:
| I don't have any one complete book that I can recommend, and
| I don't even really have a great reading list for this. But
| I'll make an attempt to share what I think is useful as a
| starting point.
|
| 1. Systems Operations is first and foremost about
| understanding systems, in all of their complexity, which
| means understanding the internals of your OS primarily.
|
| 2. Performance and networking, in particular, are super
| important areas to focus on understanding when it comes to
| learning the topic to help with software development.
|
| 3. A lot of it is about understanding concepts in abstract
| and being able to extrapolate to other situations and apply
| these concepts, so there's actually quite a lot of useful
| information that can be learned on one OS and still applied
| to another OS (or on one game engine and applied to another,
| et al).
|
| Here's a few books I think are worth reading, not in any
| particular order of prevalence, but loosely categorized
|
| Databases:
|
| High Performance MySQL:
| https://www.amazon.com/gp/product/1449314287/
|
| SQL Queries for Mere Mortals:
| https://www.amazon.com/gp/product/0321992474/
|
| The Art of SQL: https://www.amazon.com/gp/product/0596008945/
|
| Networking:
|
| TCP/IP Illustrated: https://www.amazon.com/exec/obidos/ISBN=0
| 201633469/wrichards... (updates on author's site at
| http://www.kohala.com/start/tcpipiv1.html)
|
| The TCP/IP Guide: https://www.amazon.com/TCP-Guide-
| Comprehensive-Illustrated-P...
|
| UNIX Network Programming:
| https://www.amazon.com/dp/0131411551
|
| Beej's Guide to Network Programming:
| http://beej.us/guide/bgnet/
|
| Operating Systems:
|
| Operating Systems Concepts: https://www.amazon.com/Operating-
| System-Concepts-Abraham-Sil... (various editions, I have the
| 7th edition... I recommend you find the latest)
|
| Modern Operating Systems: https://www.amazon.com/Modern-
| Operating-Systems-Andrew-Tanen... (the "Tanenbaum Book")
|
| Operating Systems Design and Implementation:
| https://www.amazon.com/Operating-Systems-Design-
| Implementat-... (the other one, the "MINIX Book")
|
| Windows Internals:
|
| Part 1: https://www.amazon.com/Windows-Internals-Part-
| architecture-m...
|
| Part 2: https://www.amazon.com/Windows-Internals-
| Part-2-7th/dp/01354... (I had the pleasure of being taught
| from this book by Mark Russinovich and David Solomon at a
| previous employer, was an amazing class and these books are
| incredible resources even applied outside of Windows, we used
| 5th edition, I linked 7th, which has the 2nd part pending
| publication).
|
| MacOS Internals:
|
| Part 1: https://www.amazon.com/MacOS-iOS-Internals-User-
| Mode/dp/0991...
|
| Part 2: https://www.amazon.com/MacOS-iOS-Internals-II-
| Kernel/dp/0991...
|
| Part 3: https://www.amazon.com/MacOS-iOS-Internals-III-
| Insecurity/dp...
|
| Linux Kernel Programming:
|
| Part 1: https://www.amazon.com/Linux-Kernel-Development-
| Cookbook-pro...
|
| Part 2: https://www.amazon.com/Linux-Kernel-Programming-Part-
| Synchro...
|
| The Linux Programming Interface:
| https://www.amazon.com/Linux-Programming-Interface-System-
| Ha...
|
| General Systems Administration:
|
| Essential Systems Administration:
| https://www.amazon.com/gp/product/0596003439/
|
| UNIX and Linux Systems Administration Handbook:
| https://www.amazon.com/UNIX-Linux-System-Administration-
| Hand...
|
| The Linux Command Line and Shell Scripting Bible:
| https://www.amazon.com/Linux-Command-Shell-Scripting-
| Bible/d...
|
| UNIX Shell Programming: https://www.amazon.com/Unix-Shell-
| Programming-Stephen-Kochan...
|
| BASH Hackers Wiki: https://wiki.bash-hackers.org/
|
| TLDP Advanced BASH Scripting Guide:
| https://tldp.org/LDP/abs/html/
|
| The Debian Administrator's Handbook: https://debian-
| handbook.info/browse/stable/
|
| TLDP Linux System Administrator's Guide:
| https://tldp.org/LDP/sag/html/index.html
|
| Performance & Benchmarking:
|
| Systems Performance: https://www.amazon.com/Systems-
| Performance-Brendan-Gregg-dp-... (this is Brendan Gregg's
| book where you learn about the magic of dtrace)
|
| BPF Performance Tools: https://www.amazon.com/Performance-
| Tools-Addison-Wesley-Prof... (the newer Brendan Gregg book
| about BPF, stellar)
|
| The Art of Computer Systems Performance Analysis:
| https://www.cse.wustl.edu/~jain/books/perfbook.htm (no longer
| available from Amazon, but is available direct from
| publisher. This is basically the one book you should read
| about creating and structuring benchmarks or performance
| tests)
|
| I guess that's a "reading list", but this is just a small
| part of what you need to know to excel in systems operations.
|
| I would say for the typical software developer writing web
| applications, the most important thing to know is how
| databases work and how networking works, since these are
| going to be the primary items affecting your application
| performance. But there's obviously topics not included in
| this list that are also worth understanding, such as
| browser/DOM internals, how caching and CDNs work, and web-
| specific optimizations that can be achievable with HTTP/2 or
| QUIC.
|
| For the average software developer writing desktop
| applications, I'd say make sure you /really/ understand OS
| internals... at the base everything you do on a computer
| system is based on what the OS provides to you. Even though
| you are abstracted (possibly many layers) away from this,
| being able to peel back the layers and understand what's
| /really/ happening is essential to writing high-quality
| application code that is performant and secure, as well as
| making you a champ at debugging issues.
|
| If you're trying to get into systems operations as a field,
| this is just a brush over the top surface and there's a lot
| deeper diving required.
| [deleted]
| [deleted]
| wyager wrote:
| When my stuff has been shared on HN and Reddit, I've seen peaks
| of around 500 rps (according to google analytics), so if you can
| hit 1k rps you're almost certainly ready to weather any kind of
| viral sharing that a blog might experience. Several krps on cheap
| VPS hardware is easy with a good compiled language backend
| (Haskell's Warp is a good one). If you use a
| node/python/ruby/other interpreted backend, you will need
| aggressive caching through a reverse proxy.
| nicoburns wrote:
| So 50req/sec. I'd hope it could handle a lot more than that!
| nostrademons wrote:
| People tend to severely underestimate how fast modern machines
| are and overestimate how much you need to spend on hardware.
|
| Back in my last startup, I was doing a crypto market intelligence
| website that subscribed to full trade & order book feeds from the
| top 10 exchanges. It handled about 3K incoming messages/second
| (~260M per day), including all of the message parsing, order book
| update, processing, streaming to websocket connections on any
| connected client, and archival to PostGres for historical
| processing. Total hardware required was 1 m4.large + 1 r5.large
| AWS instances, for a bit under $200/month, and the boxes would
| regularly run at about 50% CPU.
| lewisjoe wrote:
| I'm running two different projects on a single instance of
| cheap VM too. Both of them runs on a not-so-memory-efficient
| programming runtime, yet the VM handles the load just fine.
|
| Of course, a lot of it depends on what your app does for each
| request but most apps are simple enough and can live with being
| a monolith / single fat binary running on a single instance.
|
| The problem with today's DevOps culture is that they present
| K8's as answers for everything. Instead of defining a clear
| line on when to use them and when not to.
| rattray wrote:
| What were the most important aspects of the technology stack
| which enabled that?
| danudey wrote:
| A lot of that is due to absolutely lousy code.
|
| We had a system management backend at my last company. Loading
| the users list was unbearably slow; 10+ seconds on a warm
| cache. Not too terrible, except that most user management tasks
| required a page reload, so it was just wildly infuriating.
|
| Eventually I took a look at the code for the page, which
| queried LDAP for user data and the database for permissions
| data. It did: get list of users
| foreach user: get list of all permissions
| filter down to the ones assigned directly to the user
| foreach user: get list of all groups
| foreach group: get list of all permissions
| filter down to the ones assigned to the group
| filter down to the ones the user has
|
| I'm no algorithm genius, but I'm pretty sure O(n^2+n^3) is not
| an efficient one.
|
| I replaced it with get list of all users
| get list of all groups get list of all permissions
| <filter accordingly>
|
| Suffice to say, it was a lot more responsive.
|
| Also worth noting was that fetching the user list required
| shelling out to a command (a python script) which shelled out
| to a command (ldapsearch), and the whole system was a
| nightmare. There were also dozens of pages where almost no
| processing was done in the view, but a bunch of objects with
| lazy-loaded properties were passed into the template and always
| used, so when benchmarking you'd get 0.01 seconds for the
| entire function and then 233 seconds for "return render(...)'
| because for every single row in the database (dozens or
| hundreds) the template would access a property that would
| trigger another SQL call to the backend, rather than just doing
| one giant "SELECT ALL THE THINGS" and hammering it out that
| way.
|
| Note that we also weren't using Django's foreign keys support,
| so we couldn't even tell Django to "fetch everything non-
| lazily" because it had no idea.
|
| If that app were written right it could have run on a Raspberry
| Pi 2, but instead there was no amount of cores that could have
| sped it up.
| onlyrealcuzzo wrote:
| Yeah - you likely want to do this a single simple query -
| which you can optimize if necessary. an O(N+1) query is bad.
| An O(N^2) query is something I have rarely seen. Congrats!
| benhoyt wrote:
| Yeah, I see this a lot. I think it's especially easy to
| introduce this kind of "accidentally quadratic" behaviour
| using magical ORMs like Django's, where an innocent-looking
| attribute access like user.groups can trigger a database
| query ... access user.groups inside a loop and things get bad
| quickly.
|
| In the case of groups and permissions there's probably only a
| few of each, so fetching all of them is probably fine. But
| depending on your data -- say you're fetching comments
| written by a subset of users, you can tweak the above to use
| IN filtering, something like this Python-ish code:
| users = select('SELECT id, name FROM users WHERE id IN $1',
| user_ids) comments = select('SELECT user_id, text FROM
| comments WHERE user_id IN $1', user_ids)
| comments_by_user_id = defaultdict(list) for c in
| comments: comments_by_user_id[c.user_id].append(c)
| for u in users: u.comments =
| comments_by_user_id[u.id]
|
| Only two queries, and O(users + comments).
|
| For development, we had a ?queries=1 query parameter you
| could add to the URL to show the number of SQL queries and
| their total time at the bottom of the page. Very helpful when
| trying to optimize this stuff. "Why is this page doing 350
| queries totalling 5 seconds? Oops, I must have an N+1 query
| issue!"
| mschuster91 wrote:
| > For development, we had a ?queries=1 query parameter you
| could add to the URL to show the number of SQL queries and
| their total time at the bottom of the page. Very helpful
| when trying to optimize this stuff. "Why is this page doing
| 350 queries totalling 5 seconds? Oops, I must have an N+1
| query issue!"
|
| I'm so glad Symfony (PHP framework) has a built-in profiler
| and analysis tooling there...
| kbenson wrote:
| When I was doing a lot of DBIx::Class work, I contemplated
| writing a plugin to allow me to lock and and unlock
| automatic relation fetches and die (throw) if one was
| attempted when locked. I would carefully prefetch my
| queries (auto-fill relations from a larger joined query)
| and if something changed or there was an edge case I missed
| that caused fetching deep in a loop, it might kill
| performance, but be hard to track down if not given the
| best directions on what caused it other than "this is
| slow".
|
| It's one of those things that in the long run would have
| been much more time effective to write, but the debugging
| never quite took long enough each time to make me take the
| time.
| theptip wrote:
| I'm coming around to considering Django-seal to be
| mandatory for non-trivial applications. It will "seal" a
| query set so that you can't make accidental DB lookups
| after the initial fetch. That way you can be more confident
| that you are doing the right joins in the initial query,
| and you are safe from the dreaded O(N) ORM issue.
|
| SqlAlchemy has this as part of the ORM, it should really be
| part of Django IMO.
| [deleted]
| c17r wrote:
| I've never heard of django-seal before, thanks!
| simonw wrote:
| Django's prefetch_related mechanism essentially implements
| the pattern you describe here for you: https://docs.djangop
| roject.com/en/3.2/ref/models/querysets/#...
| petters wrote:
| select_related and prefetch_related are absolutely
| essential when coding Django.
| benhoyt wrote:
| Thanks. Yeah, I think I used that years ago when I first
| ran into this problem, and it worked well. Whether one
| uses an ORM or not, one needs to know how to use one's
| tools. My problem (not just with Django, but with ORMs in
| general) is how they make bad code look good. Like the
| following (I don't know Django well anymore, but
| something like this): users =
| User.objects.all() for u in user:
| print(u.name, len(u.comments))
|
| To someone who doesn't know the ORM (or Python) well,
| u.comments looks cheap ("good"), but it's actually doing
| a db query under the hood each time around the loop
| ("bad"). Not to mention it's fetching all the comments
| when we're only using their count. Whereas if you did
| that in a more direct-SQL way: users =
| select('SELECT id, name FROM users WHERE id IN $1',
| user_ids) for u in user: num_comments =
| get('SELECT COUNT(*) FROM comments WHERE user_id = $1',
| u.id) print(u.name, num_comments)
|
| This makes the bad pattern _look_ bad. "Oops, I'm doing
| an SQL query every loop!"
|
| The other thing I don't like about (most) ORMs is they
| fetch all the columns by default, even if the code only
| uses one or two of them. I know most ORMs provide a way
| to explicitly specify the columns you want, but the
| easy/simple default is to fetch them all.
|
| I get the value ORMs provide: save a lot of boilerplate
| and give you nice classes with methods for your tables. I
| wonder if there's a middle ground where you couldn't do
| obviously bad things with the ORM without explicitly
| opting into them. Or even just a heuristic mode for
| development where it yelled loudly if it detected what
| looked like an N+1 issue or other query inside a loop.
| bradstewart wrote:
| > Or even just a heuristic mode for development where it
| yelled loudly if it detected what looked like an N+1
| issue or other query inside a loop.
|
| Fwiw, ActiveRecord with the Bullet gem does exactly that.
| I'd guess there's an equivalent for Django.
| jbboehr wrote:
| My SQL is a bit rusty, but, to reduce the queries even
| further, I think you can do:
|
| > SELECT user_id, COUNT(*) FROM comments WHERE user_id IN
| $1 GROUP BY user_id
| danjac wrote:
| A good habit when writing unit tests is to use
| _assertNumQueries_ especially with views. It 's very easy
| even for an experienced developer to inadvertently add an
| extra query per row: for example if you have a query
| using _only()_ to restrict the columns you return, and in
| a template you refer to another column not covered by
| _only()_ , Django will do another query to fetch that
| column value (not 100% if that's still the case, but that
| was the behavior last time I looked). The developer might
| be just fixing what they think is a template issue and
| won't know they've just stepped on a performance
| landmine.
| benhoyt wrote:
| Sounds like a good habit, yeah. But the fact that an
| attribute access _in a template_ (which might be quite
| far removed from the view code) can kick off a db query
| is an indication there 's something a bit too magical
| about the design. Often front-end developers without much
| knowledge of the data model or the SQL schema are writing
| templates, so this is a bit of a foot-gun.
| simonw wrote:
| I've wanted a Django mechanism in the past that says "any
| SQL query triggered from a template is a hard error" -
| that way you could at least prevent accidental template
| edits from adding more SQL load.
| cuu508 wrote:
| users = User.objects.all() for u in user:
| print(u.name, len(u.comments))
|
| This is fine if you are working with a small data set. It
| is inefficient, but if it's quick enough, readability
| trumps efficiency IMHO.
|
| Django ORM has a succinct way of doing the "SELECT
| COUNT(*)" pattern: users =
| User.objects.all() for u in user:
| print(u.name, u.comments.count())
|
| And you can use query annotations to get rid of the N+1
| query issue altogether: users =
| User.objects.annotate(n_comments=Count("comments"))
| for u in user: print(u.name, u.n_comments)
| tinus_hn wrote:
| > This is fine if you are working with a small data set.
| It is inefficient, but if it's quick enough, readability
| trumps efficiency IMHO.
|
| And this is how you end up with the problems the parent
| is describing. During testing and when you setup the
| system you always have a small dataset so it appears to
| work fine. But when it's real work the system collapses.
| simonw wrote:
| I agree that this stuff can definitely be handled better.
|
| https://github.com/django-query-profiler/django-query-
| profil... has a neat option for detecting likely N+1
| queries. I usually use the Django Debug Toolbar for this.
|
| Django's ".only()" method lets you specify just the
| columns you want to retrieve - with the downside that any
| additional property access can trigger another SQL query.
| I thought I'd seen code somewhere that can turn those
| into errors but I'm failing to dig it up again now.
|
| I've used the assertNumQueries() assertion in tests to
| guard against future changes that accidentally increase
| the number of queries being made without me intending
| that.
|
| The points you raise are valid, but there are various
| levels of mitigations for them. Always room for
| improvement though!
| benhoyt wrote:
| Thanks for the thoughtful responses and the link --
| Django Query Profiler looks nice!
| VintageCool wrote:
| I did some work to improve performance on a dashboard several
| years ago. The way the statistics were queried was generally
| terrible, so I spent some time setting up aggregations and
| cleaning that up, but then... the performance was still
| terrible.
|
| It turned out that the dashboard had been built on top of
| Wordpress. The way that it checked if the user had permission
| to access the dashboard was to query all users, join the meta
| table which held the permission as a serialized object, run a
| full text search to check which users had permission to
| access this page, and return the list of all users with
| permission to access the page. Then, it checked if the
| current user was in that list.
|
| I switched it to only check permissions for the current user,
| and the page loaded instantaneously.
| dimitrios1 wrote:
| If I look at traces of all the service calls at my company
| within our microservices environment, the "meat" of each
| service is a fraction of the latency -- the part that's
| actually fetching the data from a database, or running an
| intense calculation. Often times its between 20-40ms
|
| Everything else are network hops and what I call
| "distributed clutter", including authorizing via a third
| party like Auth0 multiple times for machine-to-machine
| token (because "zero trust"!), multiple parameter store
| calls, hitting a dcache, if interacting with a serverless
| function, cold starts, API gateway latency, etc...
|
| So for the meat of a 20-40 ms call, we get about a 400ms-2s
| backend response time.
|
| Then if you are loading a front end SPA with
| javascript...fugetaboutit it
|
| But DevOps will say "but my managed services and infinite
| scalability!"
| scns wrote:
| Like this? https://www.youtube.com/watch?v=y8OnoxKotPQ
| dimitrios1 wrote:
| Exactly like that.
| chenxiaolong wrote:
| I had to fix a similar thing in our internal password reset
| email sender last year. The code was doing something like:
| for each user in (get_freeipa_users | grep_attribute uid):
| email = (get_freeipa_users | client_side_find user |
| grep_attribute email) last_change =
| (get_freeipa_users | client_side_find user | grep_attribute
| krblastpwdchange) expiration = (get_freeipa_users
| | client_side_find user | grep_attribute
| krbpasswordexpiration) # Some slightly
| incorrect date math... send_email
|
| I changed it to a single LDAP query for every user that
| requests only the needed attributes. It cut that Jenkins
| job's runtime from 45 minutes to 0.2 seconds.
| Thaxll wrote:
| Filter in app is rarely the right solution, your data should
| be organized in a way that you can get what you need in a
| single query. Reasons:
|
| - it's memory efficient
|
| - it's atomic
|
| - it's faster
|
| Also doesn't LDAP support filtering in query?
| zetsurin wrote:
| Given the context, it probably took the poster very little
| time to implement that fix, without digging into
| ldapsearch. With massive speedup, for their likely smallish
| ldap install. Seems like not a bad call at all.
| codebolt wrote:
| I'd also add that groups and permissions are probably
| constant and can be cached with a long timeout.
| unclebucknasty wrote:
| Not sure what the exact use case was (i.e. the output of the
| filtering) but--from reading the first algo--seems to be
| something to do with determining group membership and
| permissions for a user.
|
| In that case, was there a reason joins couldn't be used? As
| it still seems pretty wasteful (and less performant) to load
| all of this data in memory and post-process; whereas a well-
| indexed database could possibly do it faster and with less-
| memory usage.
| afrodc_ wrote:
| Is there a reason you shelled out with a subprocess versus
| using a library like ldap3? Just curious
| JJMcJ wrote:
| Your pattern is quite powerful: get data from several sources
| and do the rearranging on the client (which might be a web
| server), instead of multiple interactions for each data item.
|
| For SQL you can also do a stored procedure. Sometimes that
| works well if you are good at your DBMS's procedure language
| and the schema is good.
| spfzero wrote:
| Though stored procs will always execute on the server that
| is probably already the system bottleneck.
| funnybeam wrote:
| Sending multiple queries from your web server will likely
| put more load on the database server than using a stored
| procedure.
|
| With either technique you are still pulling all the data
| you need from the DB but with multiple queries instead of
| a stored procedure you are usually pulling more data than
| you need with each query and then dropping any rows or
| fields you're not interested in. Together with multiple
| calls over the network to the DB server and (often)
| multiple SQL connection setups this is much worse for
| performance on both the web and database servers
| bigbizisverywyz wrote:
| >Sending multiple queries from your web server will
| likely put more load on the database server than using a
| stored procedure.
|
| Lots of people seem to not realize that db roundtrips are
| expensive, and should be avoided whenever possible.
|
| One of the best illustrations of this I've found is in
| Transaction Processing book by Jim Gray and Andreas
| Reuters where they illustrate the relative cost of
| getting data from CPU vs CPU cache vs RAM vs cross host
| query.
| JJMcJ wrote:
| Except for open and writing to files, it's about the
| heaviest action on a backend server.
| hnbad wrote:
| In defense of whoever wrote the original code: it probably
| would have been reasonably fast if it had been a database
| query with proper indexes. The filters would have whittled
| the selection down to only the relevant data, whereas
| returning basically three entire tables of data to then throw
| away most of it would have been extremely inefficient.
|
| The mistake of course was not thinking about why this
| approach is faster in a database query and that it doesn't
| work that way when you already need to get all the data out
| of LDAP to do anything with it.
| IfOnlyYouKnew wrote:
| I believe the parent's point was that code tends to be
| _faster_ than what people expect, not slower.
| namenotrequired wrote:
| I think the child's point is that people expect code to be
| slower than it is because they have seen code be slow far
| more than necessary.
| tra3 wrote:
| This is an example of N+1 problem [0]. It should be a
| FizzBuzz for anyone doing any CRUD apps.
|
| [0]: https://stackoverflow.com/questions/97197/what-is-
| the-n1-sel...
| fizwhiz wrote:
| Would you mind describing your stack in more detail? Did you
| use gRPC with Go?
| nostrademons wrote:
| Sure, startup is defunct now and I think arbitrage & data on
| centralized exchanges is a dead market now. Wall Street HFTs
| got into the arbitrage game, and the data sites laypeople
| actually visit are the ones started in 2014.
|
| Codebase was pure server-side Kotlin running on the JVM.
| Jackson for JSON parsing, when the exchange didn't provide
| their own client library (I used the native client libraries
| when they did). Think I used Undertow for exchange
| websockets, and Jetty for webserving & client websockets.
| Postgres for DB.
|
| The threading model was actually the biggest bottleneck, and
| took a few tries to get right. I did JSON parsing and
| conversion to a common representation on the incoming IO
| thread. Then everything would get dumped into a big
| producer/consumer queue, and picked up by a per-CPU
| threadpool. Main thread handled price normalization (many
| crypto assets don't trade in USD, so you have to convert
| through BTC/ETH/USDT to get dollar prices), order book
| update, volume computations, opportunity detection, and other
| business logic. It also compared timestamps on incoming
| messages, and each new second, it'd aggregate the messages
| for that second (I only cared about historical data on a 1s
| basis) and hand them off to a separate DB thread. DB would do
| a big bulk insert every second; this is how I kept database
| writes below Postgres's QPS limit. Client websocket
| connections were handled internally within Jetty, which I
| think uses a threadpool and NIO.
|
| Key architectural principles were 1) do everything in RAM -
| the RDS machine was the only one that touched disk, and
| writes to it were strictly throttled 2) throw away data as
| soon as you're done with it - I had a bunch of OOM issues by
| trying to put unparsed messages in the main producer/consumer
| queue rather than parsing and discarding them 3) aggregate &
| compute early - keep final requirements in mind and don't
| save raw data you don't need 4) separate blocking and non-
| blocking activities on different threads, preferring non-
| blocking whenever possible and 5) limit threads to only those
| activities that are actively doing work.
| ochoseis wrote:
| Would you use Kotlin again for the back end? Having not yet
| used it for that purpose, it seems like you'd get the
| benefit of the JVM ecosystem along with a nice language
| (but perhaps too many power-features).
| nostrademons wrote:
| Yes. Kotlin is great for both front and backend.
|
| For that particular use-case (or related financial ones)
| I'd consider Rust, which was a bit too immature when I
| was working on this but would give you some extra speed.
| HFT is winner-take-all and the bar has risen
| significantly even in the last couple years, so if I were
| putting my own money at risk now I'd want the absolute
| fastest processing possible.
| mikhmha wrote:
| i tried to do something similar recently. Your architecture
| sounds a lot like mine. I did all the thread pool
| management and JSON parsing using C++ libraries/frameworks.
|
| The real-time data was visualized on a website. Here is an
| example. https://algot.io/
| dirkg wrote:
| may be OT, but how do you subscribe to these trade feeds, is
| there a unified service or do you need to do it individually
| for each source, and how much does it cost approximately ?
|
| I'm guessing if you put all this data into Kinesis or message
| queues it would end up costing quite a bit more.
| meltedcapacitor wrote:
| Never heard of a crypto exchange that charges for data feeds,
| the norm is free and fast. One of the positive of the
| industry compared to old school finance.
|
| They're rent seeking in other ways though, no worries.
| nostrademons wrote:
| There are probably unified services that let you do it - I
| was kinda competing in this area but didn't want to deal with
| enterprise sales, and it's a bit of a hard sell anyway.
|
| If you do it individually, there are public developer docs
| for each exchange that explain how their API works. It's
| generally free as long as you're not making a large number of
| active trades.
| aoms wrote:
| How much bandwidth did this use daily/monthly?
| nostrademons wrote:
| I don't remember offhand. I think bandwidth costs averaged
| about $40-50 out of that ~$200/month.
| Gepsens wrote:
| I'm running a crypto trading platform I'm developing on 30$ on
| DigitalOcean. I coded exclusively in Rust and recently added a
| dynamic interface to python. Today during the BTC crash it
| spiked at 20k events/s, and that's only incoming data.
| giancarlostoro wrote:
| is that $30 for a single droplet or is it spread out between
| a few different services? I'm kind of curious since I use DO
| for small projects myself.
| Gepsens wrote:
| It's not a single droplet, it's 3 + data
| Gepsens wrote:
| I use terraform and ansible, one droplet has monitoring
| (prom and grafana), 2 run services that currently run
| without any other infrastructure The rest of the cost is a
| 250 gb space on DO
| bochoh wrote:
| I'm also curious on this. My stack currently is a SQLite
| file -> ORM -> .net core on Linux on a single box
| danudey wrote:
| > I coded exclusively in Rust
|
| This reminds me of back in 2003, a friend of mine worked for
| an online casino vendor; basically, if you wanted to run an
| online casino, you'd buy the software from a company and
| customize it to fit your theme.
|
| They were often written in Java, ASP.NET, and so on. They
| were extremely heavyweight. They'd need 8-10 servers for 10k
| users. They hogged huge amounts of RAM.
|
| My friend wrote the one this company was selling in C. Not
| even C++, mind you, just C. The game modules were chosen at
| compile time, so unwanted games didn't exist. The entire
| binary (as in, 100% of the code) compiled to just over 3 MB
| when stripped. He could handle 10k concurrent users on one
| single-core server.
|
| I'm never gonna stop writing things in Python, but it still
| amazes me what can happen when you get down close to the
| metal.
| Karrot_Kream wrote:
| OpenResty [1] is a good mix of these concepts. It serves
| requests through nginx (which is at its core just a
| lightweight event loop) and then serves pages through
| LuaJIT. If you need more speed you could always write an
| nginx module in C (or in some other language viz the C
| ABI).
|
| [1]: https://openresty.org/en/
| Gepsens wrote:
| Yeah currently for me there's no tool like Rust for
| performance, the borrow-checker isn't that complicated.
| However, last month I started adding a python wrapper
| around the public api so I'm slowly going your way ;)
| Andrew_nenakhov wrote:
| Probably, Erlang would be a good fit for your task.
| Gepsens wrote:
| And you are totally right
| Gepsens wrote:
| In fact you aren't just perspicacious, the entire system
| uses actors in Rust.
| fukmbas wrote:
| Turns out cloud iaas is severely overpriced and it will always
| be cheaper to run on prem...let's see how long this lasts
| ingas wrote:
| In 2001 we started a GSM-operator on Compaq Server (it was
| before they were bought by HP) with whole 1Gb(!) of RAM and
| 2x10Gb SCSI disks.
|
| It served up to 70K of subscribers, call center with 30-40
| employees, payment systems integration, everything.
|
| Next was 8 socket Intel server. We were never able to saturate
| it's CPUs - 300 Mhz (or was it 400 ?) bus was a stopper. It
| served 350-400K of subscribers.
|
| And next: we changed architecture and used 2 servers with 2
| socket Intel CPUs again but that was time when Ghz frequencies
| appeared on market. We dreamed about 4xAMD server. We came to
| ~1 mln of active subscribers.
|
| Nowadays: every phone has more power than it was those servers.
| Typical react application consumes more resources than billing
| system. Gigabyte here, gigabyte there - nobody counts them.
|
| /grumpy oldster mode
| blacklion wrote:
| Crypto markets are very small :-)
|
| I'm working and company which process "real" exchanges, like
| NASDAQ, LSE, and, especially, OPRA feed.
|
| We've added 20+ crypto exchanges in our portfolio this year,
| and all of them are processed on one old server which is unable
| to process NASDAQ Total View in real-time anymore.
|
| On the other hand, whole OPRA feed (more than 5Gbit/s or
| 65B/day, yes, it is billions, messages of very optimized binary
| protocol, not this crappy JSON) is processed by our code on one
| modern server. Nothing special, two sockets of Intel Xeons (not
| even Platinums).
| joering2 wrote:
| Could you share some more about that very optimized binary
| protocol? I know there are ways to be more efficient than
| JSON but since you call it crappy, your solution must be much
| much better. Honestly interested to readup more.
| blacklion wrote:
| It is not "our" protocol, it is protocol designed by
| exchange and we need to support it, as we can not change it
| :). Simple binary messages, with binary encoded numbers,
| etc. No string parsing, no syntax, nothing like this, only
| bytes and offsets. Think about TCP header, for example.
|
| JSON is very inefficient both in bytes (32 bit price is 4
| bytes in binary and could be 7+ bytes as string, think
| "1299.99" for example) and CPU: to parse "1299.99" you need
| burn a lot of cycles, and if it is number of cents stored
| as native 4-byte number you need 3 shifts and 4 binary ors
| at most, if you need to change endianness, and in most
| cases it is simple memory copy of 4 bytes, 1-2 CPU cycle.
|
| When you have binary protocol, you could skip fields which
| you are not interested in as simple as "offset = offset +
| <filed-size>" (where <filed-size> is compile-time
| constant!) and in JSON you need to parse whole thing
| anyway.
|
| Difference between converting binary packet to internal
| data structure and parsing JSON with same data to same
| structure could be ten-fold easily, and you need to be very
| creative to parse JSON without additional memory
| allocations (it is possible, but code becomes very dirty
| and fragile), and memory allocation and/or deallocation
| costs a lot, both in GC languages and languages with manual
| memory management.
| joering2 wrote:
| Thanks for the writeup!
| localhost wrote:
| Curious if the binary protocol uses floating point or a
| fixed point representation? Or is floating point with its
| rounding issues sufficient for the protocol's needs?
| andylynch wrote:
| No GP but familiar with these protocols. They use fixed
| point extensively; I can't even thing of an exchange
| protocol which would use floating point since the
| rounding issues would cause unnecessary and expensive
| problems.
|
| This is typical (from NASDAQ http://www.nasdaqtrader.com/
| content/technicalsupport/specifi... ):
|
| Prices are integer fields. When converted to a decimal
| format, prices are in fixed point format with 6 whole
| number places followed by 4 decimal digits. The maximum
| price in OUCH 4.2 is $199,999.9900 (decimal, 7735939C
| hex). When entering market orders for a cross, use the
| special price of $214,748.3647 (decimal, 7FFFFFFF hex).
| mschuster91 wrote:
| > The maximum price in OUCH 4.2 is $199,999.9900
|
| For NASDAQ it seems to have been something around 430k /
| share... Buffett's BRK shares threatened to hit that
| limit a couple months ago:
| https://news.ycombinator.com/item?id=27044044
| nly wrote:
| Many exchanges use floating point, even I the Nasdaq
| technology stack.
|
| X-Stream feeds do for example
| andylynch wrote:
| Have a look at NASDAQ ITCH, OUCH, and RASH (these are the
| real names, the story I heard is the original author of
| didn't like the usual corporate style brand names and
| wanted certain people to squirm when talking about them).
| paraph1n wrote:
| There are many binary encoding protocols. A popular one is
| protobufs[1], which is used by gRPC.
|
| [1]: https://developers.google.com/protocol-buffers
| blacklion wrote:
| "old skool" exchanges uses either FIX (old and really
| vernose), FAST (binary encoding for FIX) or custom fixed-
| layout protocols.
|
| Most big USA exchanges uses custom fixed-layout
| protocols, where each message is described in
| documentation, but not in machine-readable way. European
| ones still use FAST.
|
| I didn't seen FIX in the wild for data feeds, but it is
| used for brokers, to submit orders to exchange (our
| company didn't do this part, we only consume feeds).
|
| I don't know why, but all Crypto Exchanges use JSON, not
| protobufs or something like this, and didn't publish any
| formal schemes.
|
| Fun fact: one crypto exchange put GZIP'ed and base64'ed
| JSON data into JSON which pushed to websocket, to save
| bandwidth. IMHO, it is peak of bad design.
| blibble wrote:
| there's still ascii FIX floating around on the market
| data side for a few esoteric venues
|
| FAST is not particularly common in Europe
|
| the large European venues use fixed-width binary encoding
| (LSE group, Euronext, CBOE Europe)
| blacklion wrote:
| Eurex uses FAST for sure, but I can be wrong about
| "common".
| gpderetta wrote:
| Eurex/XETRA EOBI no longer uses FAST. It is about 8 year
| old t this point, but IIRC some products are still on the
| older protocol.
| nly wrote:
| Euronext uses SBE specifically.
| rewq4321 wrote:
| And msgpack if you want an order of magnitude faster
| serialization/deserialisation and can put up with worse
| compression (I think mainly due to schema overhead since
| protobuf files don't store the schema?)
|
| https://msgpack.org/index.html
|
| Good protobuf vs msgpack comparison:
| https://medium.com/@hugovs/the-need-for-speed-
| experimenting-...
| wffurr wrote:
| https://www.opraplan.com/datafeeds
| nostrademons wrote:
| Some Googling turned up this protocol descriptor:
|
| https://uploads-
| ssl.webflow.com/5ba40927ac854d8c97bc92d7/5bf...
|
| If you're optimizing for latency JSON is pretty terrible,
| but most people who use it are optimizing for
| interoperability and ease of development. It works just
| fine for that, and you can recover decent bandwidth just by
| compressing it.
| [deleted]
| secondcoming wrote:
| It depends on what your server does with each request though;
| '65B a day' means little. If all it does is write it to a log
| then I'm surprised you're not using a rPI.
| Foomf wrote:
| I've read your few posts a few times and I'm still not sure
| why you made your post. You're telling the person that you
| handle more data than them and thus need more resources than
| them. Was your goal to smugly belittle them? It's not like
| they said any problem can be solved on their specific
| resources.
| Aeolun wrote:
| I feel like he's saying two things.
|
| One, that he's surprised by how small crypto markets are.
|
| Two, that this one server (or very few server processing)
| thing scales quite well to billions of messages a day.
|
| I didn't find any element of smugness here, but maybe I
| misread the tone.
| lost-found wrote:
| They are just pointing out the size of crypto market data
| vs normal market data. I found it interesting, not their
| fault you didn't.
| blacklion wrote:
| Nope, I want to say, that even much more data could be
| processed on very limited hardware, and that it is
| additional confirmation that current hardware is immensely
| powerful and very under-estimated.
| Cthulhu_ wrote:
| And that go-to solutions / golden hammers like REST /
| JSON are very much suboptimal.
|
| I mean I'm still going to use it for client/server
| communication and the like because I don't have serious
| performance constraints enough to warrant something that
| will be more difficult to develop for etc, but still.
| nostrademons wrote:
| I'd initially responded, but I found that my response had
| an element of a pissing match over whose data is bigger, so
| I deleted it.
|
| The thing is - when two engineers get smug, oftentimes lots
| of fairly interesting technical details get exchanged, so
| such discussions aren't really useless to bystanders.
| arjie wrote:
| How do I get in touch with you? Definitely using more resources
| than this to process fewer integrations. I'm curious what trade
| offs you made to enable this.
|
| If you're anywhere in the US, let me know.
| mynameisash wrote:
| What Andy giveth, Bill taketh away.[0]
|
| I'm more than a little annoyed that so much data engineering is
| still done in Scala Spark or PySpark. Both suffer from pretty
| high memory overhead, which leads to suboptimal resource
| utilization. I've worked with a few different systems that
| compile their queries into C/C++ (which is transparent to the
| developer). Those tend to be significantly faster or can use
| fewer nodes to process.
|
| I get that quick & dirty scripts for exploration don't need to
| be super optimized, and that throwing more hardware at the
| problem _can_ be cheaper than engineering time, but in my
| experience, the latter ends up costing my org tens of millions
| of dollars annually -- just write some code and allocate a ton
| of resources to make it work in a reasonable amount of time.
|
| I'm hopeful that Ballista[1], for example, will see uptake and
| improve this.
|
| [0] https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law
|
| [1] https://github.com/apache/arrow-
| datafusion/tree/master/balli...
| Waterluvian wrote:
| What reminded me of this the other day is how MacOS will grow
| your cursor if you "shake" it to help you find it on a big
| screen.
|
| I was thinking about how they must have a routine that's
| constantly taking mouse input, buffering history, and running
| some algorithm to determine when user input is a mouse
| "shake".
|
| And how many features like this add up to eat up a nontrivial
| amount of resources.
| hypertele-Xii wrote:
| Simply moving my USB mouse consumes 10% of my CPU.
| Computers these days...
| trisiak wrote:
| What's behind the mouse cursor while you're doing it?
| Could it be the UX/UI layer keeping state up-to-date?
|
| Other possibility, do you have a gaming mouse with 1000Hz
| polling rate configured?
| hypertele-Xii wrote:
| Yeah, it's a high quality mouse. But the only excuse for
| this is it's slightly cheaper to make everything USB.
| PS/2 worked much better. It was limited to 200Hz but
| needed no polling. Motherboards just stopped providing
| the port.
| doublepg23 wrote:
| Gaming or off the shelf prosumer mobos still have PS/2 on
| them occasionally. Although, it's probably just using a
| converter to USB anyway.
| 6510 wrote:
| If the computer has to do anything at all it ads to
| complexity and it isn't doing other things. One could do
| something a bit like blue screening and add the mouse
| pointer to the video signal in the monitor. For basic
| functionality the computer only needs to know the x and y
| of clicks. (it could for laughs also report the colors in
| the area) Hover and other effects could be activated when
| [really] needed. As a bonus one could replace the hideous
| monitor configuration menus with a point and click
| interface.
| voltagex_ wrote:
| Your report rate may be too high.
| Cthulhu_ wrote:
| Back in the day, moving your mouse made things look like
| they were processing faster.
| codesections wrote:
| Not just _look like_ -- on Windows 95, moving the mouse
| _actually_ made things process faster, for fairly bizarre
| reasons.
|
| Source: https://retrocomputing.stackexchange.com/question
| s/11533/why...
| nostrademons wrote:
| That particular example seems like something that's
| probably a lot cheaper than you'd initially think. The OS
| has to constantly take mouse input anyway to move the
| pointer and dispatch events to userspace. It also needs to
| record the current and new position of the mouse pointer to
| dispatch the events. Detecting whether the mouse is being
| "shaken" can be done with a ring buffer of mouse velocities
| over the last second or two of ticks. At 60 fps, that's
| about 120 ints = 480 bytes. Since you don't need to be
| precise, you can take Manhattan distance (x + y) rather
| than Euclidean distance (sqrt(x^2 + y^2)), which is a
| basically negligible computation. Add up the running total
| of the ring buffer - and you don't even need to visit each
| element, just keep a running total in a variable, add the
| new velocity, subtract the velocity that's about to be
| overwritten - and if this passes a threshold that's say 1-2
| screen widths, the mouse is being "shaken" and the pointer
| should enlarge. In total you're looking at < 500 bytes and
| a few dozen CPU cycles per tick for this feature.
| metafunctor wrote:
| If you shake the cursor up and down, it doesn't grow.
|
| Looks like it's measuring something like number of
| direction changes in relation to distance traveled;
| ignoring the y axis completely.
| headmelted wrote:
| Or, alternatively, the engineer that worked on this at
| Apple has just read the above as another way of solving
| the problem and is throwing this on their backlog for
| tomorrow..
| WanderPanda wrote:
| Why not use an exponential filter that only uses a single
| variable?
| Karrot_Kream wrote:
| Agreed, a single-pole IIR filter would be cheaper and
| have a better frequency domain.
| kragen wrote:
| The running-sum-difference approach suggested above is a
| box filter, which has the best possible noise suppression
| for a given step-function delay, although in the
| frequency domain it looks appalling. It uses more RAM,
| but not that much. The single-pole RC filter you're
| suggesting is much nicer in the frequency domain, but in
| the time domain it's far worse.
| WanderPanda wrote:
| Is it ,,far worse in the time domain" due to its infinite
| impulse response?
| kragen wrote:
| Not really? Sort of? I don't really have a good answer
| here. It depends on what you mean by "due to". It's
| certainly due to the impulse response, since in the sense
| I meant "far worse" the impulse response is the only
| thing that matters.
|
| Truncating the impulse response after five time constants
| wouldn't really change its output noticeably, and even if
| you truncated it after two or three time constants it
| would still be inferior to the box filter for this
| application, though less bad. So in that sense the
| problem isn't that it's infinite.
|
| Likewise, you could certainly design a direct-form IIR
| filter that did a perfectly adequate job of approximating
| a box filter for this sort of application, and that might
| actually be a reasonable thing to do if you wanted to do
| something like this with a bunch of op-amps or microwave
| passives instead of code.
|
| So the fact that the impulse response is infinite is
| neither necessary nor sufficient for the problem.
|
| The problem with the simple single-pole filter is that by
| putting so much weight on very recent samples, you sort
| of throw away some information about samples that aren't
| quite so recent and become more vulnerable to false
| triggering from a single rapid mouse movement, so you
| have to set the threshold higher to compensate.
| hoseja wrote:
| Please elaborate.
| Waterluvian wrote:
| Thanks for the thoughtful analysis and napkin math. You
| may very well be right. I wonder if this is true in
| practice or if they suffer from any interface
| abstractions and whatnot.
| rollcat wrote:
| On every modern (past few decades) platform, the mouse
| cursor is a hardware sprite with a dedicated, optimized,
| *fast* path thru every layer of the stack, just to shave
| off a few ms of user-perceived latency. Grab a window and
| shake it violently, you'll notice it lags a few pixels
| behind the cursor - that's the magic in action.
|
| In some places there's no room left for unnecessary
| abstractions, I can imagine most of the code touching
| mouse / cursor handling is in that category.
| spfzero wrote:
| That's some actual engineering there, understanding the
| problem you're trying to solve, instead of solving
| problems you don't even have. Nice.
| Const-me wrote:
| > they must have a routine that's constantly taking mouse
| input
|
| Possible but unlikely. Well-written desktop software never
| constantly taking input, it's sleeping on OS kernel
| primitives like poll/epoll/IOCP/etc waiting for these
| inputs.
|
| Operating systems don't generate mouse events at 1kHz
| unless you actually move the mouse.
| lilyball wrote:
| "Constantly taking" is not the same thing as "constantly
| polling". The ring buffer approach works identically in
| the event-driven approach, you just need to calculate the
| number of "skipped" ticks and zero them out in the ring
| buffer.
| softveda wrote:
| And yet MacOS doesn't allow to change the cursor color. On
| my Windows 10 desktop I set the cursor color to a slightly
| larger size and yellow color. So much easier to work with.
| Grustaf wrote:
| That seems like an incredibly cheap feature. The mouse
| pointer png probably costs a lot more than the shake
| detector.
| iaw wrote:
| Could you elaborate on other systems besides Ballista? (which
| looks great btw, thank you for sharing)
| mynameisash wrote:
| U-SQL on Azure Data Lake Analytics[0] and ECL from HPCC
| Systems[1] are two I have fairly extensive experience with.
| There may be other such platforms, but those and Ballista
| are the three on my radar.
|
| [0] https://github.com/MicrosoftDocs/azure-
| docs/blob/master/arti...
|
| [1] https://wiki.hpccsystems.com/pages/viewpage.action?page
| Id=28...
| disgruntledphd2 wrote:
| ECL is the devil. There's so little documentation on it,
| that you basically need to work for LNRS to actually get
| any experience with the damn thing.
|
| If it had an SQL layer, I'd spend time evangelising it,
| but it's not worth learning another query language for.
|
| There exists a world where it got open-sourced before
| Hadoop was built, and in that world it's probably
| everywhere.
| SNosTrAnDbLe wrote:
| I work in the Analytics space and been mostly on Java and I
| am so glad that other people feel the same. At this point,
| people have become afraid of suggesting something other than
| Spark. I see something written in Rust to be much better at
| problems like this. I love the JVM but it works well with
| transactional workloads and starts showing its age when its
| dealing with analytical loads. The worst thing is then people
| start doing weak references and weird off the heap processing
| usually by a senior engineer but really defeats the purpose
| of the JVM
| wiz21c wrote:
| I guess your company is running on Java and running
| something else would cost a lot in training, recruiting,
| understanding, etc. But down the line, defeating the JVM
| will be understood only by the guy who did it... Then that
| guy will leave... Then the newcomers will rewrite the thing
| in Spark 'cos it will feel safer. Rinse-repeat...
|
| (I'm totally speculating, but your story seems so true that
| it inspired me :-)
| SNosTrAnDbLe wrote:
| Some of it is what you mentioned about training and
| hiring costs that but mostly its this creation of the
| narrative that it will scale someday in the future. This
| is usually done by that engineer(s) and they are good at
| selling so a different opinion is ignored or frowned upon
| by the leadership.
|
| I have now seen this anti pattern in multiple places now
| robertlagrant wrote:
| > I love the JVM but it works well with transactional
| workloads and starts showing its age when its dealing with
| analytical loads.
|
| This is interesting. Can you elaborate a bit?
| SNosTrAnDbLe wrote:
| Analytical loads deal with very large datasets in the
| order of terabytes even after you compress them. These
| workloads dont change much so keeping them in the heap
| eventually results in long processing pauses because JVM
| tries to recover this memory. However, in most cases,
| this data is not meant to be garbage collected. For
| transactions, once you have persisted the data, it should
| be garbage collected so the pattern works. There are a
| lot of other aspects that I can probably think of but the
| above one is the most important in my mind.
| willvarfar wrote:
| Yes, the whole idea of sending "agents" to do processing is
| poor performing and things like snowflake and Trino, where
| queries go to already deployed code, run rings around it.
|
| Furthermore, pyspark is by far the most popular and used
| spark, and it's also got the absolute world-worst atrocious
| mechanical sympathy. Why?
|
| Developer velocity trumps compute velocity any day?
|
| (I want the niceness of python and the performance of eg
| firebolt. Why must I pick?)
|
| (There is a general thing to get spark "off heap" and use
| generic query compute on the spark sql space, but it is miles
| behind those who start off there)
| b9a2cab5 wrote:
| Abstractions almost always end up leaky. Spark SQL, for
| example, does whole-stage codegen which collapses multiple
| project/filter stages into a single compiled stage, but your
| underlying data format still needs to be memory friendly
| (i.e. linear accesses, low branching, etc.). The codegen is
| very naive and the JVM JIT can only do so much.
|
| What I've seen is that you need people who deeply understand
| the system (e.g. Spark) to be able to tune for these edge
| cases (e.g. see [1] for examples of some of the tradeoffs
| between different processing schemes). Those people are
| expensive (think $500k+ annual salaries) and are really only
| cost effective when your compute spend is in the tens of
| millions or higher annually. Everyone else is using open
| source and throwing more compute at the problem or relying on
| their data scientists/data engineers to figure out what magic
| knob to turn.
|
| [1]: https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf
| paulluuk wrote:
| What's wrong with relying on data engineers for data
| engineering?
| disgruntledphd2 wrote:
| Spark is very very odd to tune. Like, it seems (from my
| limited experience) to have the problems common to
| distributed data processing (skew, it's almost always
| skew) but because it's lazy, people end up really
| confused as to what _actually_ drives the performance
| problems.
|
| That being said, Spark is literally the only (relatively)
| easy way to run distributed ML that's open source. The
| competitors are GPU's (if you have a GPU friendly
| problem) and running multiple Python processes across the
| network.
|
| (I'm really hoping that people will now school me, and
| I'll discover a much better way in the comments).
| Spooky23 wrote:
| I get a kick out of stuff like this - I'm mostly an exec
| these days, but I recently prototyped a small database system
| to feed a business process in SQLite on my laptop.
|
| To my amusement, my little SQLite prototype smoked the
| "enterprise" database. Turns out that a MacBook Pro SSD
| performs better than the SAN, and the query planner needs
| more tlc. We ended up running the queries off my laptop for a
| few days while the DBAs did their thing.
| KptMarchewa wrote:
| Who the fuck thinks it's a good idea to run database on
| networked storage?
| Spooky23 wrote:
| You might have heard of this little company in Seattle
| called "Amazon".
| nexuist wrote:
| Like the river? They're gonna need to fight hard for that
| SEO.
| panzagl wrote:
| The same people who will run a clustered database on VMs.
| skrtskrt wrote:
| Isn't that what's happening if you use any managed
| database product? They have probably colocated everything
| as much as possible and used various proprietary tricks
| to cut latency, but still.
| PhilipTai wrote:
| Right. Local storage is much more performant and cost
| effective than network storage. I tried to run some iops
| sensitive workload on cloud. It turns out I need to pay
| several thousands dollar per month for the performance I
| can get on a $100 nvme ssd.
| noduerme wrote:
| I'm working on a web app right now that does a lot of
| heavy/live/realtime processing with workers. The original
| thought was to run those workers on the node servers and
| stream the results over a socket, charging by the
| CPU/minute. But it surprised me that the performance
| looks a lot better up to about 16 workers if the user
| just turns over a few cores to local web workers and runs
| it on their own box. As long as they don't need 64 cores
| or something, it's faster to run locally, even on an
| older machine. Thread transport is slow but sockets are
| slower at a distance; the bottlenecks are in the main
| thread anyway. So I've been making sure parts are
| interchangeable between web workers and "remote socket
| web workers" assigned to each instance of the app
| remotely.
| fhood wrote:
| People may underestimate how fast modern machines are, but that
| is probably in part because, at least in my fairly relevant
| experience, I have literally never seen a CPU bottleneck under
| normal circumstances. Memory pressure is nearly always the
| driving issue.
| foobarbazetc wrote:
| Yeah. Now that CPUs are insanely powerful and you have NVMe
| SSDs etc the bottleneck is always memory.
| xfitm3 wrote:
| In my experience disk i/o is the biggest bottleneck. It
| used to be sync()ing writes to disk for strict consistency
| but that's been pushed down to the DB now. I just looked at
| my DB systems and CPU is low but disk is nearly pegged.
|
| My data sets are far too big to fit into memory/cache. Disk
| pressure can be alleviated by optimizing queries but it's a
| game of whack-a-mole.
|
| I have exhausted EBS i/o and been forced to resort to dirty
| tricks. With RDS you can just pay more but that only scales
| to a point - normally the budget.
| FridgeSeal wrote:
| Does anyone know if any of the current crop of standard
| databases make use of things like Direct IO (where
| available on Linux), I've seen some write-ups that
| indicate you can get eye-wateringly fast performance when
| combined with an nvme drive.
| rapsey wrote:
| Scylla is built arround direct io.
| zozbot234 wrote:
| Sure, but as far as software is concerned, optimizing for
| memory bandwidth (the typical bottleneck in modern systems)
| is not so different from optimizing for CPU.
| nostrademons wrote:
| It's also amazing how much you can fit in RAM if you're
| careful. I remember ~2007 people were aghast at Facebook's
| 4T memcached deployment that stored basically everyone's
| social network posts; now you can get single servers for
| ~$4K with 4T of RAM.
|
| The trick is basically that you have to eschew the last 15
| years of "productivity" enhancements. Pretty much any
| dynamic language is out; if you must use the JVM or .NET,
| store as much as possible in flat buffers of primitive
| types. I ended up converting order books from the obvious
| representation (hashtable mapping prices to a list of Order
| structs) to a pair of SortedMaps from FastUtils, which
| provides an unboxed float representation with no pointers.
| That change ended up reducing memory usage by about 4x.
|
| You can fit _a lot_ of ints and floats in today 's 100G+
| machines, way more than needed to represent the entire
| cryptocurrency market. You just can't do that when you're
| chasing 3 pointers, each with their associated object
| headers, to store 4 bytes.
| NetToolKit wrote:
| > now you can get single servers for ~$4K with 4T of RAM
|
| Does the $4K include the cost of the RAM? Where can I
| find these servers? Thanks!
| Kranar wrote:
| The more I read comments on subjects I am intimately
| familiar with, the more I realize most people who comment
| on HN don't really know what they're talking about and
| mostly make things up.
|
| To answer your question, you can't find these servers
| because they don't exist. A server with 4T of RAM will
| cost you at a minimum $20,000 and that will be for some
| really crappy low-grade RAM. Realistically for an actual
| server that one would use in an actual semi-production
| setting, you're looking at a minimum of $35,000 for 4TB
| of RAM and that's just for the RAM alone, although to be
| fair that 35k ends up dominating the cost of the entire
| system.
| NetToolKit wrote:
| Thanks for your response. Your numbers are better than
| what I came up with, but I thought I'd check just in case
| I was missing a great deal somewhere.
| cvwright wrote:
| For $4k you could get an (old) refurbished R820 with
| 1.5TB of (old) DDR3 RAM.
|
| Which is a far cry from the claimed 4TB, but still, damn.
| panzagl wrote:
| Maybe they downloaded more RAM...
| adventured wrote:
| > the more I realize most people who comment on HN don't
| really know what they're talking about and mostly make
| things up.
|
| I don't think they're typically making things up. It's
| what I prefer to call Reddit knowledge. They saw someone
| else claim it somewhere, they believed it, and so they're
| repeating it so they can be part of the conversation
| (people like to belong, like to be part of, like to
| join). It's an extremely common process on Reddit and
| most forums, and HN isn't immune to it. Most people don't
| read much and don't acquire most of their knowledge from
| high quality sources, their (thought to be correct) wider
| knowledge - on diverse topics they have no specialization
| on - is frequently acquired from what other people say
| and that they believe. So they flip around on Reddit or
| Twitter for a bit, digest a few nuggets of questionable
| 'knowledge' and then regurgitate it at some later point,
| in a process of wanting to participate and belong
| socially. It's how political talking points function for
| example, passed down to mimic distributors that spread
| the gospel to other mimic followers (usually without
| questioning). It's how religion functions. And it's how
| most teachers / teaching functions, the teachers are
| distribution mimics (mimics with a bullhorn, granted
| authority by other mimics to keep the system going, to
| clone).
|
| It's because some very high percentage of all of humans
| are mimics. It's not something Reddit caused of course,
| it's biology, it's a behavior that has always been part
| of humanity. It's an increased odds of success method of
| optimizing for survival of the species, successful
| outcomes, meets the Internet age. It's why most people
| are inherent followers, and can never be (nor desire to
| be) leaders. It's why few people create anything original
| or even attempt to across a lifetime. It's why such a
| small fraction of the population are very artistic,
| particularly drawn to that level of creative expression.
| If you're a mimic biologically it's very difficult to be
| the opposite. This seems to be viewed by most people as
| an insult (understandably, as mimics are the vast
| majority of the population and control the vote), however
| it's not, it's simply how most living things function,
| system wise, by mimicry (or even more direct forms of
| copying). Humans aren't _that_ special, we 're not
| entirely distinct from all the other systems of animal
| behavior.
|
| That saying, safety in numbers? That's what that is all
| about. Mimicry. Don't stand out.
|
| The reason most Wall Street money managers can't beat the
| S&P 500? It's because they're particularly aggressive
| mimics, they intentionally copy eachother toward safe,
| very prosperous, gentle mediocrity. They play a game of
| follow, with popular trends (each decade or era on Wall
| Street has popular trends/fads). Don't drift too far
| below the other mimics and it's a golden ticket.
|
| Nobody got fired for buying IBM? Same thing. Mimic what
| has worked well for many others is biologically typically
| a high success outcome pattern (although amusingly not
| always, it can also in rare occasions lead off a cliff).
|
| The Taliban? The Soviet Union? Nazism? Genocide? Multi
| generational patterns of mistake repetition passed down
| from parental units? That's how you get that. People
| mimic (perhaps especially parental units; biology very
| much in action), even in cases where it's an
| unsuccessful/negative pattern. All bad (and good)
| ideologies have mimic distributors and mimic followers,
| the followers do what they're told and implement as
| they're told. And usually there are only a very small
| number of originators, which is where the mimic
| distributors get their material.
|
| The concept of positive role models? It's about mimicry
| toward successful outcomes.
| saagarjha wrote:
| Being a mimic and being a leader/creator are not
| exclusive; you can do both in various fields. One can
| even mimic and eventually start creating their own things
| on the back of that: "fake it until you make it" is a
| real thing.
| da_chicken wrote:
| > The more I read comments on subjects I am intimately
| familiar with, the more I realize most people who comment
| on HN don't really know what they're talking about and
| mostly make things up.
|
| HN doesn't look exactly like SlashDot, but it's
| absolutely just like SlashDot.
| mPReDiToR wrote:
| It's mostly the good bits of old /. for me.
|
| Intelligent discourse by knowledgeable persons.
|
| The omission of GNAA and "frosty" posts are a massive
| boon.
| nostrademons wrote:
| I just Googled [servers with 4T of RAM], but apparently
| no, it does not include the RAM itself. Came up with this
| (about $4K base, another $44K to max it out with RAM):
|
| https://www.broadberry.com/dual-amd-epyc-rackmount-
| servers/a...
| NetToolKit wrote:
| Yeah, I've been eyeing the EPYC servers, but RAM pricing
| seems to very roughly be in the ballpark of $1,000 for
| 128GB, so $4k for 4T sounded very attractive and I wanted
| to make sure that I wasn't missing anything. As cvwright
| pointed out, you can get old DDR3 RAM for less, but I
| haven't found servers that can fit that many DIMMs.
| Thanks for your response.
| walrus01 wrote:
| I'm fairly sure I know where I can buy used/refurb Dell
| R820 at good prices with 512 or 1024GB of RAM, but I
| don't think I could accomplish the same for 4TB of RAM
| and $4000. Certainly not in one machine. And not with two
| $2000 machines each with 2048MB. We're close, but it's
| not _that_ cheap yet.
|
| Looking on ebay I can find some pretty decent R820 with
| 512GB each for right around $1500 a piece. Not counting
| any storage, even if they come used with some spinning
| hard drives, would end up replacing with SSDs. So more
| like three servers, 1.5TB of RAM, for $4500.
| Karrot_Kream wrote:
| Yeah I run mirrors for certain sites in different regions
| of the world and a trick I often do is scrape the target
| site and then hold the entire site in memory. Even the
| top 500 posts of HN and all of the associated comments
| can fit in < 16 MB of RAM. If you want to serve that up
| over a socket, it's really fast to grab a pointer to the
| memory and just write it to the socket. You can get
| response times which are dwarfed by network latency that
| way.
| nine_k wrote:
| The CPU is rarely used up to 100% because most code fails to
| utilize several cores efficiently.
|
| OTOH a service loading the single core with the main thread
| is a frequent sight :( Interpreted languages like Python can
| easily spend 30% of time just on the deserialization
| overhead, converting the data from a DB into a result set,
| and then into ORM instances.
| OneEyedRobot wrote:
| I've seen exactly the opposite although you certainly can't
| ignore memory speed.
| SilverRed wrote:
| In my experience it is almost always the database holding
| things up. If your app does not use a database or it makes
| very simple use of it, then I'm not surprised it is blazing
| fast. As soon as you need to start joining tables and
| applying permissions to queries, it all gets slow.
| Cthulhu_ wrote:
| They really do, and because of that they reach for over-
| engineered infrastructure solutions. I mean I get that you'd
| like to have some redundancy for your webserver and database,
| maybe some DDOS mitigation, off-site backup, etc, but you don't
| need an overblown microservices architecture for a low-traffic
| CRUD application. That just creates problems, instead of
| solving problems you WISH you had, and it's slower than
| starting simple.
| [deleted]
| iambozdar wrote:
| You have clearly stated in your footnote that if anything goes
| wrong then the given numbers can go down. To be honest, that's
| what happens all the time. Developers do something wrong and
| number of requests just drop down to the floor.
|
| Only static websites are the one which handle large amount of
| requests at low cost. Web hosting providers don't make money out
| of those clients, so they run shared plans.
| zhuzhu wrote:
| My website https://www.v2ph.com
|
| $6 VPS can handle 500,000 requests daily
|
| On this server, I have PHP-fpm workers, nginx and MariaDB
|
| The average CPU usage about 30%, load average is about 0.5
| Shadonototro wrote:
| i'm running my SaaS for 10 years already, and still using the
| same $5 plan a month
|
| HTTPS and certificates? i have no clue how to setup that, i use
| dns from cloudflare and they have it all automatic for free
|
| if your employees are asking you to pay ton of money for your
| services, hire someone else
| mkl95 wrote:
| Nginx can handle ~250M requests a day out of the box, and ~600M
| by tuning a few parameters. *
|
| * https://www.cloudbees.com/blog/tuning-nginx
| bob229 wrote:
| Pepe ftw
| seandong wrote:
| All these examples show even more why software engineer(not
| developer) is a discipline where 10x salary difference can be
| seen between the best and the worst. For the 10x you are paying,
| you are getting a (n^2 - logN) times performance gain, especially
| when you are dealing with problems with large amount of data.
|
| However, relying on people themselves is often not the best
| stable solution. I am wondering if all these N^2 mistakes people
| made can be prevented by innovative means like language features,
| framework improvements, tooling and etc. And I'm talking about
| prevention, not the post mortem perf measure and fix kind
| olingern wrote:
| I might get downvoted for not getting on the 2000s style of
| development bandwagon, but do you really need a web server to
| serve static text?
| bellyfullofbac wrote:
| What do you suggest, Gopher?
| pctf wrote:
| S3 (+ optionally a CDN)
| crazy_horse wrote:
| Nginx.
| Zababa wrote:
| Nginx is a web server though. You're assuming that by "web
| server" the first comment meant "web framework"?
| SilverRed wrote:
| Github/Gitlab pages is my choice for being free, managing
| https and just generally being pretty good. You can even use
| the CI to compile the site for you.
| bellyfullofbac wrote:
| Ah, kids these days. So I guess by "webserver" grandparent
| meant "A server you actually control", and by "not
| webserver" people mean "It's on the cloud somewhere".
|
| GH/GL-pages still respond to HTTP/HTTPS requests, in my
| dictionary that's a webserver, but I guess for the
| millennials (I say with snark) it means "I don't have to
| think about what happens, it just serves my content for
| me."
| Gigachad wrote:
| I read "do you need a webserver" like "do you need one
| yourself" and freeloading off someone elses server like
| with GL pages counts as not having a webserver. Otherwise
| how else would you serve content?
| CyanLite2 wrote:
| Cloudflare
| mr_toad wrote:
| Last I checked cloudflare was running nginx.
| sigg3 wrote:
| Put Wordpress on it, and do a new battery of TTFB tests ;)
| celsoazevedo wrote:
| Even WordPress would work fine if we use a plugin like WP Super
| Cache (no idea why they don't cache things by default). It
| wouldn't beat a simple static page, but WordPress + Cache
| plugin + cheap VPS can easily handle #1 on HN.
| kjgkjhfkjf wrote:
| That's less than 50qps. I can't count that low.
| jka wrote:
| There are a few comments in here that predictably suggest that
| simple static sites can handle large request rates easily.
|
| Sure, that's true - but to try to progress the conversation: how
| would you measure the complexity of serving web requests, in
| order to perform more advanced cost comparisons?
|
| (bandwidth wouldn't be quite right.. or at least not sufficient -
| maybe something like I/O, memory and compute resource used?)
| throwaway20371 wrote:
| You can also handle 50 requests per second on a 66MHz 486DX2 with
| 16MB of RAM and a 10Mbit/s network card. Not with modern "I have
| infinite resources" software, but we used to handle more than
| that traffic regularly in the early 90s.
| muzani wrote:
| I was skeptical that it would fail under real situations (the
| calculation says it's 50 requests/second). But it looks like
| reaching the top of HN didn't crash it.
| fabian2k wrote:
| 50 rps not that much, though of course easily sufficient for many
| situations. This is also Django which certainly isn't the fastest
| choice. I played around with it a long time ago and liked it
| quite a bit, but you don't choose Django for performance but for
| the other benefits.
|
| I'm really more surprised that static serving is so slow at 180
| rps. This should be able to easily saturate the network,
| statically serving files is very, very fast. From what I see in
| the blog I doubt that the files are very large, so there is
| probably some other bottleneck or I'm missing something here.
| tambourine_man wrote:
| > These benchmarks show that a very cheap server can easily
| handle 50 requests a _minute_
|
| I think you mean a _second_ , but yeah, old tech is fast.
|
| I find it funny when I read "raw html" emphatically, as if it was
| akin to writing assembly.
| janmo wrote:
| There is a difference between being able to handle 4.2M requests
| a day, and handling 4.2M requests per day.
|
| Visitors don't come neatly one after the other. You might only
| have 1M requests a day but get random spikes with 100 requests at
| the same time.
| bArray wrote:
| Very true. It also makes a difference as to which resource is
| being pulled, whether it is cached, what transport is being
| requested (SSL, compression, etc).
|
| I really suspect the website would fall long before it hits
| anything close to 4.2 million requests (which the author also
| seems to except).
|
| That all said - long live tiny web servers!
| brokencode wrote:
| It's the database part that gets expensive for web applications.
| Serving up static web pages is absolutely trivial for modern
| servers.
|
| The database is also the part that doesn't easily scale, unless
| you pick a highly scalable database from the outset, and those
| have their own complexity and tradeoffs as well.
|
| That's why I believe every project should start with a
| bulletproof model of how the database will work first, then fill
| in the other details from there.
|
| It's not always as easy as picking Postgres and calling it a day,
| unfortunately.
| great-potential wrote:
| Dont mean to be the negative Joe but you dont need a webserver if
| you're serving a static-able website.
| nayuki wrote:
| You mean use someone else's web server instead?
| great-potential wrote:
| I mean this is just basic stuff, for me it just sounds like a
| developer putting a site in production, no auto-scaling,
| patching ... might as well outsource this to a CDN since
| there is no database/redis/varnish ...
| sleepyhead wrote:
| So...someone else's web server instead?
| great-potential wrote:
| No offense but I think you've missed the point.
| icehawk wrote:
| A CDN is just a web server somewhere else.
| bruce343434 wrote:
| How are you supposed to _serve_ that static _web_site then?
| [deleted]
| fnord77 wrote:
| apache + wsgi isn't even close to being the most performant
| webapp server software, either. Bet he'd get 5x the performance
| out of nginx + lua on the same virtual hardware.
| fimdomeio wrote:
| Most of the times the more interesting question is not really how
| to make a server that can handle 4.2M requests a day, but how to
| make something so useful that it gets more than 100 pageviews a
| day.
| danielsamuels wrote:
| Can it?
|
| > Service Unavailable
|
| > The server is temporarily unable to service your request due to
| maintenance downtime or capacity problems. Please try again
| later.
| pluc wrote:
| > Not taking into account any issues that may occur around
| CPU/RAM/Disk IO due to sustained levels of traffic as well as
| bandwidth issues
|
| congrats?
| baby wrote:
| I used to be on the top 500 of Alexa with a shared hosting server
| running PHP. Never had any issue. Mind you, I wasn't doing
| anything complicated, but still.
| anaganisk wrote:
| But can it run k8s /s
___________________________________________________________________
(page generated 2021-09-08 23:02 UTC)