[HN Gopher] Tower Defense: Cache Control
___________________________________________________________________
Tower Defense: Cache Control
Author : jasonthorsness
Score : 50 points
Date : 2025-05-13 12:59 UTC (3 days ago)
(HTM) web link (www.jasonthorsness.com)
(TXT) w3m dump (www.jasonthorsness.com)
| ec109685 wrote:
| Regarding stale-while-revalidate, the author calculates maximum
| load like this: "With the options available on hn.unlurker.com,
| there are only 10 * 12 * 8 * 2 or 1920 possible combinations,
| refreshed at most once every 15 seconds"
|
| That math works until it doesn't. If for some reason there's a
| greater than 15 second response, the cache will open the
| floodgate and won't return any cached response until the cache is
| full.
|
| Similarly, errors need to be accounted for. If the server returns
| a 500, is it configured to be cached? Is stale-while-error
| configured so the error state doesn't take the server down?
| ndriscoll wrote:
| Put nginx in front with proxy_cache_lock with a sufficiently
| high lock timeout. Even if your service can somehow be that
| slow, now you have bounded requests again.
| tshaddox wrote:
| Also, wouldn't each CDN node maintain its own cache and thus
| its own state-while-revalidate timer? Or is the implication
| that this is using a tiered cache where there's only a single
| CDN node that connects to the origin server?
| jasonthorsness wrote:
| Good point. I guess this depends on Vercel's cache topology
| and whether it uses multiple layers. I would have thought
| they would cache the output of the origin function in a cache
| right at the origin but I can't find anything about this in
| the docs so likely not. As an alternative, they do allow
| caching the fetch to the API in their "data cache" so I'll
| probably need to enable that to protect the VPS in the way I
| intend.
| nisten wrote:
| Where's the fucking game, I was teased with a potential of a
| vibecoded game!
| jerf wrote:
| Now I really want to somehow work out a way to drop that in to
| my next architecture diagram wholesale:
| https://www.jasonthorsness.com/_next/image?url=%2F_next%2Fst...
|
| "Why did you choose Go for your backend?"
|
| "Well, the lightning strikes it implements deliver critical
| elec-damage against the customers that get past the ice blast
| from our DATA CACHE-powered FUNC, striking them in their
| frozen-status for x3 before they can get to our vital CPURAM
| resources. The most important thing to have in a cloud-based
| deployment is a diverse mix of elemental attacks and crit-
| stacking bonuses."
| bee_rider wrote:
| Now we have to figure out which game will, if you train an a
| dnn on it, will result in a network that solved technical
| problems best. Maybe a facorio-playing AI could design a good
| cpu, haha.
| harrall wrote:
| Read your CDN docs very carefully on how it processes requests.
| Make sure that your CDN does request coalescing.
|
| At my previous job, I became responsible for handling >350,000
| requests/second and I dug so deep.
| grep_it wrote:
| > Why Not Redis? I have a single VPS, so I can get by with a
| simple SQLite database. If I had many instances of my API on
| separate servers
|
| Just to push against this a bit. Redis can be very low memory
| cost and is very easy to run (I give it 5mb). I have a small
| single server with a few instances of my API that let's me cache
| pretty much everything I need.
| jasonthorsness wrote:
| Huh, I never thought of running it just on the one same node. I
| guess that would better prepare for a scale-up later.
| johnmaguire wrote:
| You may even be able to scale horizontally with many
| nginx+redis nodes, if the cache does not need to be shared.
| hangonhn wrote:
| It's so obvious once I've heard someone else say it but wow
| that's really clever actually.
| arp242 wrote:
| When I last benchmarked Redis vs. PostgreSQL for a simple k/v
| cache it was about ~1ms for PostgreSQL to fetch a key, and
| ~0.5ms for Redis. Faster, but not really noticeably so. I
| haven't benchmarked SQLite, but I would be surprised if the
| numbers are substantially different.
|
| Of course Redis can do other things than just a k/v cache, and
| at scale you just want to offload some load from your main SQL
| server. But for "small" use cases my conclusion was that Redis
| doesn't really add anything. OTOH it's also not especially
| difficult to run so it also not a big problem to use it, but by
| and large it seems superfluous.
| nixpulvis wrote:
| I've always wondered if it would be worth the effort to display
| information to the user to indicate that their request was stale
| and that they'll need to refresh again? 15s is pretty quick, but
| how do caching schemes work when users need to know when to use
| their time refreshing and checking the results. Alternatively,
| I'd be interested to read about the architecture of a modern
| always fresh caching scheme.
| tikotus wrote:
| I've been running a handful of low traffic services on a $10 VPS
| from DigitalOcean for years. I'm currently in a situation where a
| thing I've made might blow up, and I had to look into CDNs just
| in case. It's just static content, but updates once per day (it's
| a daily logic puzzle).
|
| I must admit I had no idea what goes into runnig a CDN. First I
| had a look at DO's spaces object storage which has CDN support.
| But it wasn't exactly the right tool for serving a static website
| apparently, but rather for serving large files. For example I
| couldn't make it serve css files with the correct mime type
| without some voodoo, so I had to conclude I wasn't doing the
| right thing.
|
| Then I looked at DO's app platform. But that seemed like an
| overkill for just sharing some static content. It wants me to
| rely on an external service like GitHub to reliably serve the
| content. I already rely on DO, I don't want to additionally rely
| on something else too. Seems like I could also use DO's docker
| registry. What? To serve static content on CDN I need to create a
| whole container running the whole server? And what do I need to
| take into consideration when I want to update the content once
| per day simultaneously for all users? It's easy when it's all on
| my single VPS (with caching disabled for that url) but I actually
| have no idea what happens with the docker image once it's live on
| the "app platform". This is getting way more complex than I was
| hoping for. Should I go back to the spaces solution?
|
| Right now I'm in a limbo. On one hand I want to be prepared in
| case I get lucky and my thing goes "viral". On the other hand my
| tiny VPS is running on 2% CPU usage with already quite a few
| users. And if I do get lucky, I should afford doubling my VPS
| capacity. But what about protection from DDoS? Anything else I
| should worry about? Why is everyone else using CDN?
|
| And I don't even have caching! An article like this puts my
| problem into shame. I just want to serve a couple of plain web
| files and I can't choose what I should do. This article really
| shows how quickly the problem starts ballooning.
| _QrE wrote:
| It sounds like you might be looking at the wrong place. There's
| services like bunny.net and cloudflare CDN (I'm not affiliated
| with either, but I use the former) that are really easy to set
| up and configure, if you've built your site properly (edit for
| clarification: if you have clearly defined static content,
| and/or you're using some build system). You don't want to 'run'
| a CDN, you want to use one.
|
| Configuration depends a lot on the specifics of your stack. For
| Svelte, you can use an adapter
| (https://svelte.dev/docs/kit/adapters) that will handle
| pointing to the CDN for you.
|
| Cloudflare's offering is free, bunny.net is also probably going
| to be free for you if you don't have much traffic. CDNs are
| great insurance for static sites, because they can handle all
| the traffic you could possibly get without breaking a sweat.
| amenghra wrote:
| If it's updated daily, maybe host it on GitHub pages (public
| repo) and have a job push an update on a daily basis?
|
| If you want to be more fancy, you could do private GitHub repo+
| Cloudflare pages enabling you to prepare puzzles in advance.
|
| You can get quite far with both service's free offering or
| Cloudflare's $5/mo stuff.
|
| Full disclosure: I work for neither companies but I have
| regularly been a happy paid customer of both products.
| arjie wrote:
| My blog is on a totally overprovisioned VPS and I use Cloudflare
| to cache responses for non-logged-in users. I'm using Mediawiki
| because I want people to be able to edit it and sometimes it can
| be extraordinarily slow but with Cloudflare it loads instantly if
| someone visited it some time recently. I really like Cloudflare's
| caching control.
| kawsper wrote:
| > stale-while-revalidate is a relatively recent cache-control
| option
|
| Varnish Cache have supported that since version 4.1, it was
| released 2015-09-30:
|
| > Varnish will now use the stale-while-revalidate defined in
| RFC5861 to set object grace time.
|
| https://varnish-cache.org/docs/trunk/whats-new/changes-4.1.h...
___________________________________________________________________
(page generated 2025-05-16 23:01 UTC)