[HN Gopher] OpenFreeMap survived 100k requests per second
___________________________________________________________________
OpenFreeMap survived 100k requests per second
Author : hyperknot
Score : 572 points
Date : 2025-08-09 13:31 UTC (1 days ago)
(HTM) web link (blog.hyperknot.com)
(TXT) w3m dump (blog.hyperknot.com)
| colinbartlett wrote:
| Thank you for this breakdown and for this level of transparency.
| We have been thinking of moving from MapTiler to OpenFreeMap for
| StatusGator's outage maps.
| hyperknot wrote:
| Feel free to migrate. If you ever worry about High
| Availability, self-hosting is always an option. But I'm working
| hard on making the public instance as reliable as possible.
| v5v3 wrote:
| The article mentions Cloudflare, so how much of this was cached
| by them?
| do_anh_tu wrote:
| Do you even read the article?
| keketi wrote:
| Are you new? Nobody actually reads the articles.
| LorenDB wrote:
| False. I almost never upvote an article without reading it,
| and half of those upvotes are because I already read
| something similar recently that gave me the same
| information.
| eszed wrote:
| I'll submit in the second case (already read something
| similar) that, _properly speaking_ , we should read both,
| and upvote (or submit, if not already here) the better of
| the articles.
|
| Not that, you know, I often take the time to do that,
| either - but it would improve the site and the
| discussions if we all did.
| jwilk wrote:
| From the HN Guidelines
| <https://news.ycombinator.com/newsguidelines.html>:
|
| > _Please don 't comment on whether someone read an article.
| "Did you even read the article? It mentions that" can be
| shortened to "The article mentions that"._
| RandomBacon wrote:
| That guideline is decent I guess.
|
| I am disappointed that they edited another guideline for
| the worse:
|
| > Please don't comment about the voting on comments. It
| never does any good, and it makes boring reading.
|
| It used to just say, don't complain about voting.
|
| If the number of votes are so taboo, why do they even show
| us the number or user karma (and have a top list)?
| RandomBacon wrote:
| We can't even talk about the guidelines?
| alessandroberna wrote:
| 99.38%
| fnord77 wrote:
| sounds like they survived 1,000 reqs/sec and the cloudflare CDN
| survived 99,000 reqs/sec
| LoganDark wrote:
| > I believe what is happening is that those images are being
| drawn by some script-kiddies.
|
| Oh absolutely not. I've seen so many autistic people literally
| just nolifing and also collaborating on huge arts on wplace. It
| is absolutely not just script kiddies.
|
| > 3 billion requests / 2 million users is an average of 1,500
| req/user. A normal user might make 10-20 requests when loading a
| map, so these are extremely high, scripted use cases.
|
| I don't know about that either. Users don't just load a map, they
| look all around the place to search for and see a bunch of the
| art others have made. I don't know how many requests is typical
| for "exploring a map for hours on end" but I imagine a lot of
| people are doing just that.
|
| I wouldn't completely discount automation but these usage
| patterns seem by far not impossible. Especially since wplace
| didn't expect sudden popularity so they may not have optimized
| their traffic patterns as much as they could have.
| nemomarx wrote:
| There are some user scripts to overlay templates on the map and
| coordinate working together, but I can't imagine that increases
| the load much. What might is that wplace has been struggling
| under the load and you have to refresh to see your pixels
| placed or any changes and that could be causing more calls an
| hour maybe?
| Karliss wrote:
| Just scrolled around a little bit 2-3minutes with network
| monitor open. That already resulted in 500requests, 5MB
| transferred (after filtering by vector tile data). Not sure how
| many of those got cached by browser with no actual requests,
| cached by browser exchanging only headers or cached by
| cloudflare. I am guessing that the typical 10-20 requests/user
| case is for embedded map fragment like those commonly found in
| contact page where most users don't scroll at all or at most
| slightly zoom out to better see rest of city.
| charcircuit wrote:
| >Nice idea, interesting project, next time please contact me
| before.
|
| It's impossible to predict that one's project may go viral.
|
| >As a single user, you broke the service for everyone.
|
| Or you did by not having a high enough fd limit. Blaming sites
| when using it too much when you advertise there is no limit is
| not cool. It's not like wplace themselves were maliciously
| hammering the API.
| columb wrote:
| You are so entitled... Because of you most nice things have "no
| limits but...". Not cool stress testing someone's
| infrastructure. Not cool. The author of this post is more than
| understanding, tried to fix it and offered a solution even
| after blocking them. On a free service.
|
| Show us what you have done.
| charcircuit wrote:
| >You are so entitled
|
| That's how agreements work. If someone says they will sell a
| hamburger for $5, and another person pays $5 for a hamburger,
| then they are entitled to a hamburger.
|
| >On a free service.
|
| It's up to the owner to price the service. Being overwhelmed
| by traffic when there are no limits is not a problem limited
| only to free services.
| perching_aix wrote:
| > Do you offer support and SLA guarantees?
|
| >
|
| > At the moment, I don't offer SLA guarantees or
| personalized support.
|
| From the website.
| eszed wrote:
| Sure, and if you bulk-order 5k hamburgers the restaurant
| will honor the price, but they'll also tell you "we're
| going to need some notice to handle that much product".
| Perfect analogy, really. This guy handled the situation
| perfectly, imo.
| charcircuit wrote:
| Except in this case the restauraut would have been able
| to handle the 5k orders if they didn't arbitrarily have
| their workers work with their hands tied behind their
| back. And instead of untieing their workers and
| appreciating the realization they were accidently
| bottlenecking themselves they blame the nearby event who
| caused a spike in foot traffic.
|
| Publicly attacking your users instead of celebrating
| their success and your new learnings is not what I would
| call handling it perfectly. I think going for a halo
| effect strategy where you celebrate how people are using
| your platform to accomplish their goals will help people
| understand how what is being done is valuable and want
| people to adopt it or financially support it. On the
| other hand attacking people who use your platform
| publicly can make people apprehensive in using it fearing
| that they will be criticized too.
| austhrow743 wrote:
| Hamburger situation is not comparable. It's a trade.
|
| This is just someone being not very specific in a text file
| on their computer. I have many such notes, some of them
| publicly viewable.
| 010101010101 wrote:
| Do you expect him just to let the service remain broken or to
| scale up to infinite cost to himself on this volunteer project?
| He worked with the project author to find a solution that works
| for both and does not degrade service for every other user,
| under literally no obligation to do anything at all. This isn't
| Anthropic deciding to throttle users paying hundreds of dollars
| a month for a subscription. Constructive criticism is one
| thing, but entitlement to something run by an individual
| volunteer for free is absurd.
| charcircuit wrote:
| We are talking about hosting a fixed amount of static files.
| This should be a solved problem. This is nothing like running
| large AI models for people.
| 010101010101 wrote:
| The nature of the service is completely irrelevant.
| charcircuit wrote:
| Running a no limit service for free definitely depends on
| the marginal cost of serving a single request.
| toast0 wrote:
| The project page kind of suggests he might scale up to
| infinite cost...
|
| > Financially, the plan is to keep renting servers until they
| cover the bandwidth. I believe it can be self-sustainable if
| enough people subscribe to the support plans.
|
| Especially since he said Cloudflare is providing the CDN for
| free... Yes, running the origins costs money, but in most
| cases, default fd limits are low, and you can push them a lot
| higher. At some point you'll run into i/o limits, but I think
| the I/O at the origin seems pretty managable if my napkin
| math was right.
|
| If the files are all tiny, and the fd limit is the actual
| bottleneck, there's ways to make that work better too. IMHO,
| it doesn't make sense to accept a inbound connection if you
| can't get a fd to read a file for it, so better to limit the
| concurrent connections and let connections sit in the listen
| queue and have a short keepalive time out to make sure you're
| not wasting your fds on idle connections. With no other
| knowledge, I'd put the connection limit at half the FD limit,
| assuming the origin server is dedicated for this and serves
| static files exclusively. But, to be honest, if I set up
| something like this, I probably wouldn't have thought about
| FD limits until they got hit, so no big deal ... hopefully
| whatever I used to monitor would include available fds by
| default and I'd have noticed, but it's not a default output
| everywhere.
| rikafurude21 wrote:
| the funny part is that his service didnt break- cloudflares
| cache caught 99% of the requests. just wanted to feel powerful
| and break the latest viral trend.
| feverzsj wrote:
| So, OFM was hit by another Million Dollar Homepage for kids.
| eggbrain wrote:
| Limiting by referrer seems strange -- if you know a normal user
| makes 10-20 requests (let's assume per minute), can't you just
| rate limit requests to 100 requests per minute per IP (5x the
| average load) and still block the majority of these cases?
|
| Or, if it's just a few bad actors, block based on JA4/JA3
| fingerprint?
| hyperknot wrote:
| What if one user really wants to browse around the world and
| explore the map. I remember spending half an hour in Google
| Earth desktop, just exploring around interesting places.
|
| I think referer based limits are better, this way I can ask
| high users to please choose self-hosting instead of the public
| instance.
| toast0 wrote:
| Limiting by referrer is probably the right first step. (And
| changing the front page text)
|
| You want to track usage by the site, not the person, because
| you can ask a site to change usage patterns in a way you can't
| really ask a site's users. Maybe a per IP limit makes sense
| too, but you wouldn't want them low enough that it would be
| effective for something like this.
| jspiner wrote:
| The cache hit rate is amazing. Is there something you implemented
| specifically for this?
| hyperknot wrote:
| Yes, I designed the whole path structure / location blocks with
| caching in mind. Here is the generated nginx.conf, if you are
| interested:
|
| https://github.com/hyperknot/openfreemap/blob/main/docs/asse...
| rtaylorgarlock wrote:
| Is it always/only 'laziness' (derogatory, i know) when caching
| isn't implemented by a site like wplace.live ? Why wouldn't they
| save openfreemap all the traffic when a caching server on their
| side presumably could serve tiles almost as fast or faster than
| openfreemap?
| VladVladikoff wrote:
| I actually have a direct answer for this: priorities. I run a
| fairly popular auction website and we have map tiles via stadia
| maps. We spend about $80/month on this service for our volume.
| We definitely could get this cost down to a lower tier by
| caching the tiles and serving them from our proxy. However we
| simply haven't yet had the time to work on this, as there is
| always some other task which is higher priority.
| latchkey wrote:
| Like reading and commenting on HN articles! ;-)
| markerz wrote:
| It looks like a fun website, not a for-profit website. The
| expectations and focus of fun websites is more to just get it
| working than to handle the scale. It sounds like their user
| base exploded overnight, doubling every 14 hours or so. It also
| sounds like it's other a solo dev or a small group based on the
| maintainers wording.
| hyperknot wrote:
| We are talking about an insane amount of data here. It was 56
| Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not
| something a "caching server" could handle. We are talking on
| the order of CDN networks, like Cloudflare, to be able to
| handle this.
| wyager wrote:
| > or 56 x 1 Gbit servers 100% saturated
|
| Presumably a caching server would be 10GbE, 40GbE, or 100GbE
|
| 56Gbit/sec of pre-generated data is definitely something that
| you can handle from 1 or 2 decent servers, assuming each
| request doesn't generate a huge number of random disk reads
| or something
| ndriscoll wrote:
| I'd be somewhat surprised if nginx couldn't saturate a 10Gbit
| link with an n150 serving static files, so I'd expect 6x $200
| minipcs to handle it. I'd think the expensive part would be
| the hosting/connection.
| Sesse__ wrote:
| > We are talking about an insane amount of data here. It was
| 56 Gbit/s. This is not something a "caching server" could
| handle.
|
| You are not talking about an insane amount of data if it's 56
| Gbit/s. Of course a caching server could handle that.
|
| Source: Has written servers that saturated 40gig (with TLS)
| on an old quadcore.
| hyperknot wrote:
| OK, technically there might exist such server, I guess
| Netflix and friends are using those. But we are talking
| about a community supported, free service here. Hetzner
| servers are my only options, because of their unmetered
| bandwidth.
| Sesse__ wrote:
| It really depends on the size of the active set. If it
| fits into RAM of whatever server you are using, then it's
| not a problem at all, even with completely off-the-shelf
| hardware and software. Slap two 40gig NICs in it, install
| Varnish or whatever and you're good to go. (This is, of
| course, assuming that you have someone willing to pay for
| the bandwidth out to your users!)
|
| If you need to go to disk to serve large parts of it,
| it's a different beast. But then again, Netflix was doing
| 800gig already three years ago (in large part from disk)
| and they are handicapping themselves by choosing an OS
| where they need to do significant amounts of the scaling
| work themselves.
| hyperknot wrote:
| I'm sure the server hardware is not a problem. The full
| dataset is 150 GB and the server has 64 GB RAM, most of
| which will be never requested. So I'm sure that the used
| tiles would actually get served from OS cache. If not,
| it's on a RAID 0 NVME SSD, connected locally.
|
| What I've been referring to is the fact that even
| unlimited 1 Gbps connections can be quite expensive, now
| try to find a 2x40 gig connection for a reasonable money.
| That one user generated 200 TB in 24 hours! I have no
| idea about bandwidth pricing, but I bet it ain't cheap to
| serve that.
| Sesse__ wrote:
| Well, "bandwidth is expensive" is a true claim, but it's
| also a very different claim from "a [normal] caching
| server couldn't handle 56 Gbit/sec"...?
| hyperknot wrote:
| You are correct. I was putting "a caching server on their
| side" in the context of their side being a single dev
| hobby project running on a VPS, exploding on the weekend.
| I agree that these servers do exist and some companies do
| pay for this bandwidth as part of their normal
| operations.
| Aeolun wrote:
| 56 Gbit/sec costs you about EUR590/day even on Hetzner.
| bigstrat2003 wrote:
| I realize that what constitutes "insane" is a subjective
| judgement. But, uh... I _most certainly_ would call 56 Gbps
| insane. Which is not to say that hardware which handles it
| doesn 't exist. It might not even be especially insane
| hardware. But that is a pretty insane data rate in my book.
| toast0 wrote:
| Why should they when openfreemap is behind a CDN and their home
| page says things like:
|
| > Using our public instance is completely free: there are no
| limits on the number of map views or requests. There's no
| registration, no user database, no API keys, and no cookies. We
| aim to cover the running costs of our public instance through
| donations.
|
| > Is commercial usage allowed?
|
| > Yes.
|
| IMHO, reading this and then just using it, makes a lot of
| sense. Yeah, you could put a cache infront of their CDN, but
| why, when they said it's all good, no limits, for free?
|
| I might wonder a bit, if I knew the bandwidth it was using, but
| I might be busy with other stuff if my site went unexpectedly
| viral.
| Aeolun wrote:
| I think, when you read that, you should be reassured that
| nobody is going to suddenly tell you to pay, and then still
| implement caching on your own side to preserve the free
| offering for everyone else.
|
| Seriously, whose first thought on reading that is "oh great,
| I can exploit this".
| naniwaduni wrote:
| You don't need to be thinking "I can't exploit this" when
| you can just stop thinking about it.
| radu_floricica wrote:
| Politeness? Also speed.
|
| But it depends on the project architecture. If the tiles are
| needed only client-side, then there's really little reason to
| cache things on _my_ server. That would imply I'm better at
| caching openstreetmap tiles than... openstreetmap. Plus
| you're just making the system needlessly more complicated.
|
| And there's little reason for openstreet map to be upset,
| since it's not like _I_ am making 2 million requests - there
| are 2 million separate users of osm+. It's aligned to their
| incentive of having more people use osm and derived works.
| All is well.
| willsmith72 wrote:
| so 96% availability = "survived" now?
|
| but interesting write-up. If I were a consumer of OpenFreeMap, I
| would be concerned that such an availability drop was only
| detected by user reports
| ndriscoll wrote:
| If I were a consumer of a free service from someone who will
| not take your money to offer support or an SLA (i.e. is not
| trying to run a business), I would assume there's little to no
| monitoring at all.
| timmg wrote:
| 96% during a unique event. I think you would typically consider
| long term in a stat like that.
|
| Assuming it was close to 100% the rest of the year, that works
| out to 99.97% over 12 months.
| ch33zer wrote:
| Since the limit you ran into was number of open files could you
| just raise that limit? I get blocking the spammy traffic but
| theoretically could you have handled more if that limit was
| upped?
| hyperknot wrote:
| I've just written my question to the nginx community forum,
| after a lengthy debugging session with multiple LLMs. Right
| now, I believe it was the combination of multi_accept +
| open_file_cache > worker_rlimit_nofile.
|
| https://community.nginx.org/t/too-many-open-files-at-1000-re...
|
| Also, the servers were doing 200 Mbps, so I couldn't have kept
| up _much_ longer, no matter the limits.
| ndriscoll wrote:
| One thing that might work for you is to actually make the
| empty tile file, and hard link it everywhere it needs to be.
| Then you don't need to special case it at runtime, but
| instead at generation time.
|
| NVMe disks are incredibly fast and 1k rps is not a lot (IIRC
| my n100 seems to be capable of ~40k if not for the 1 Gbit NIC
| bottlenecking). I'd try benchmarking without the tuning
| options you've got. Like do you actually get 40k concurrent
| connections from cloudflare? If you have connections to your
| upstream kept alive (so no constant slow starts), ideally you
| have numCores workers and they each do one thing at a time,
| and that's enough to max out your NIC. You only add
| concurrency if latency prevents you from maxing bandwidth.
| hyperknot wrote:
| Yes, that's a good idea. But we are talking about 90+% of
| the titles being empty (I might be wrong on that), that's a
| lot of hard links. I think the nginx config just need to be
| fixed, I hope I'll receive some help on their forum.
| ndriscoll wrote:
| You could also try turning off the file descriptor cache.
| Keep in mind that nvme ssds can do ~30-50k random
| reads/second with no concurrency, or at least hundreds of
| thousands with concurrency, so even if every request hit
| disk 10 times it should be fine. There's also kernel
| caching which I think includes some of what you'd get
| from nginx's metadata cache?
| toast0 wrote:
| I'm pretty sure your open file cache is way too large. If
| you're doing 1k/sec, and you cache file descriptors for 60
| minutes, assuming those are all unique, that's asking for 3
| million FDs to be cached, when you've only got 1 million
| available. I've never used nginx or open_file_cache[1], but I
| would tune it way down and see if you even notice a
| difference in performance in normal operation. Maybe 10k
| files, 60s timeout.
|
| > Also, the servers were doing 200 Mbps, so I couldn't have
| kept up _much_ longer, no matter the limits.
|
| For cost reasons or system overload?
|
| If system overload ... What kind of storage? Are you
| monitoring disk i/o? What kind of CPU do you have in your
| system? I used to push almost 10GBps with https on dual
| E5-2690 [2], but it was a larger file. 2690s were high end,
| but something more modern will have much better AES
| acceleration and should do better than 200 Mbps almost
| regardless of what it is.
|
| [1] to be honest, I'm not sure I understand the intent of
| open_file_cache... Opening files is usually not that
| expensive; maybe at hundreds of thousands of rps or if you
| have a very complex filesystem. PS don't put tens of
| thousands of files in a directory. Everything works better if
| you take your ten thousand files and put one hundred files
| into each of one hundred directories. You can experiment to
| see what works best with your load, but a tree where you've
| got N layers of M directories and the last layer has M files
| is a good plan, 64 <= M <= 256. The goal is keeping the
| directories compact so searching and editing is effective.
|
| [2] https://www.intel.com/content/www/us/en/products/sku/6459
| 6/i...
| Aeolun wrote:
| If you do 200Mbps on a hetzner server _after_ cloudflare
| caching, you are going to run out of traffic pretty
| rapidly. The limit is 20TB / month (which you'd reach in
| roughly 9 days).
| johnisgood wrote:
| One would think services like these do not have to rely
| on online services and have their own rack of servers. Or
| is this so alien these days?
| MaKey wrote:
| Small addition: That limit applies to Hetzner Cloud
| servers, their dedicated servers have unlimited traffic.
| Aeolun wrote:
| Depends on your connection I think. Mine do 1Gbit/sec but
| have a 20TB limit. The 100Mbit ones are unlimited (last I
| checked)
| justinclift wrote:
| > so I couldn't have kept up _much_ longer, no matter the
| limits.
|
| Why would that kind of rate cause a problem over time?
| perching_aix wrote:
| Haven't worked with Cloudflare yet first hand, and I'm not
| familiar with web map tech. But if the site really is pretty much
| just serving lots of static files, why is Hetzner in the loop?
| Wouldn't fully migrating to Cloudflare Pages be possible?
| internetter wrote:
| The tiles need to be rendered. Yes frequent tiles can be cached
| but you already have a cache... it's Cloudflare. Theoretically
| you could port the tileserver to Cloudflare pages but then
| you'd need to... port it... and it probably wouldn't be cheaper
| perching_aix wrote:
| Oh interesting, okay. For some reason I had the impression
| that the tiles were static and rendered offline.
| hyperknot wrote:
| They are actually static files. There is just too many of
| them, about 300 million. You cannot put that in Pages.
| jonathanlydall wrote:
| Is CloudFlare's R2 an option for you?
| hoppp wrote:
| It would cost a lot. Hetzner is hardware and them you can
| hammer it, free bandwidth. You get a very good server for
| cheap.
|
| cloudflare would be pay per request, a hefty sum if ddos
| happens
| perching_aix wrote:
| Did a quick cost calc (with the help of gpt5, so might be
| wrong) when I read their comment about Pages not being
| suitable for this many files.
|
| They say they're receiving $500/mo in donos and that it's
| currently just enough to cover their infra costs. Given 300
| million 70 KB files, R2 + high cache hit ratio would work
| out to about $300 in storage-months + request costs, or
| $600/mo with Cache Reserve and then they'd always hit cache
| if I understand the project right: meaning the costs
| shouldn't blow up beyond that, and that request count would
| essentially just not matter.
| hoppp wrote:
| Yea but the cost is not a fixed monthly sum and things
| can go wrong as we can see from the blog post. An
| accident could bankrupt the dev.
|
| A dedicated server will always cost the same so you
| always know how much you pay.
|
| It will cost 40 Euro/month to have 6 cores/12
| threads,64gb of ram and 1Tb of ssd.
|
| Dirt cheap compared to any other alternative
| cuu508 wrote:
| Factor in that you also need resources to generate and
| upload the tiles weekly.
| ohdeargodno wrote:
| Noone renders tiles on servers anymore. The vast majority of
| services have moved on to sending out vector tiles.
| andai wrote:
| From the screenshot I wanted to say, couldn't this be done on a
| single VPS? Seemed over engineered to me. Then I realized the
| silly pixels are on top of _a map of the entire earth._ Dang!
|
| I'm curious what the peak req/s is like. I think it might be just
| barely within the range supported by benchmark-friendly web
| servers.
|
| Unless there's some kind of order of magnitude slowdowns due to
| the nature of the application.
|
| Edit: Looks like about 64 pixels per km (4096 per km^2). At full
| color uncompressed that's about 8TB to cover the entire earth
| (thinking long-term!). 10TB box is EUR20/month from Hetzner.
| You'd definitely want some caching though ;)
|
| Edit 2: wplace uses 1000x1000 px pngs for the drawing layer. The
| drawings load instantly, while the map itself is currently very
| laggy, and some chunks permanently missing.
| TylerE wrote:
| "EUR20/month from Hetzner" is great until you actually need it
| to be up and working when you need it.
| immibis wrote:
| IME Hetzner's not unreliable. I don't think you could serve
| 100k requests per second on a single VPS though. (And with
| dedicated, you're on the hook for redundancy yourself, same
| as any dedicated.)
| cyberpunk wrote:
| They're unreliable as soon as you have to deal with their
| support who have the technical knowledge of a brick.
|
| And as soon as you have to do ant business / deal with the
| german side of the business expect everything to slow down
| to 2 weeks for response which will still be incorrect.
|
| They are simply not worth the hassle. Go with a competent
| host.
| Aeolun wrote:
| > They're unreliable as soon as you have to deal with
| their support who have the technical knowledge of a
| brick.
|
| Since I never have to, that's perfect isn't it? If you
| need support from Hetzner you are using the wrong host.
| cyberpunk wrote:
| So they've never decided to blackhole your prod traffic 2
| days after migrating to them? lucky.
| Aeolun wrote:
| I don't think I've ever migrated anywhere and not kep the
| previous thing running for a month unless I wasn't
| worried about exactly that thing happening (to be fair,
| not prod traffic blackhole, but similar effect)
| ranger_danger wrote:
| Simply running "ipfs daemon" is enough to get hetzner to
| threaten canceling your service.
|
| They send nasty letters with pcap dumps of your "hacking"
| attempts and it doesn't matter how wrong you think they
| are.
| celsoazevedo wrote:
| Ideally, we'd all be using very good and competent
| hosting companies, but they're not ideal for free/low
| revenue projects. It's better to have some down time than
| having to shut down the service because you can't afford
| to keep it running.
|
| I think that in both cases here (OpenFreeMap and wplace),
| Hetzner/OVH/Scaleway is the way to go. Depending on what
| we're doing, the cost savings can even allow us to have
| redundancy at another cheap provider just in case
| something goes wrong.
| motorest wrote:
| > "EUR20/month from Hetzner" is great until you actually need
| it to be up and working when you need it.
|
| I managed a few Hetzner cloud instances, and some report
| perfect uptime for over a year. The ones that don't, I was
| the root cause.
|
| What exactly leads you to make this sort of claim? Do you
| actually have any data or are you just running your mouth
| off?
| slacktivism123 wrote:
| >What exactly leads you to make this sort of claim?
|
| https://news.ycombinator.com/item?id=29651993
|
| https://news.ycombinator.com/item?id=42365295
|
| https://news.ycombinator.com/item?id=44038591
|
| >are you just running your mouth off?
|
| _Don 't be snarky. Edit out swipes._
|
| https://news.ycombinator.com/newsguidelines.html
| Aeolun wrote:
| This all seems to be Hetzner Cloud offerings?
| eitland wrote:
| I've just checked all three, and it appears to be rather
| more nuanced than you make it out to be.
| Starlevel004 wrote:
| > I believe what is happening is that those images are being
| drawn by some script-kiddies. If I understand correctly, the
| website limited everyone to 1 pixel per 30 seconds, so I guess
| everyone was just scripting Puppeteer/Chromium to start a new
| browser, click a pixel, and close the browser, possibly with IP
| address rotation, but maybe that wasn't even needed.
|
| I think you perhaps underestimate just how big of a thing this
| became basically overnight. I mentioned a drawing over my house
| to a few people and literally everyone instantly knew what I
| meant without even saying the website. People _love_ /r/place
| style things every few years, and this having such a big canvas
| and being on a world map means that there is a lot of space for
| everyone to draw literally where they live.
| Aurornis wrote:
| > I think you perhaps underestimate just how big of a thing
| this became basically overnight.
|
| They don't need to estimate because in the article they talked
| to the site and got their traffic numbers: An estimated 2
| million users.
|
| That's 1500 requests per user, which implies a lot of scripting
| is going on.
| zahlman wrote:
| > I think you perhaps underestimate just how big of a thing
| this became basically overnight. I mentioned a drawing over my
| house to a few people and literally everyone instantly knew
| what I meant without even saying the website.
|
| On the other hand, this is the first I've heard of this thing.
| johnisgood wrote:
| I have known about this kind of pixel drawing but it was on
| empty canvas.
| yifanl wrote:
| They have the user count from the dev, 2 million daily users
| shouldn't be generating billions of requests unless a good
| portion of them are botting.
| zamadatix wrote:
| Why not? This is tile requests right, not login requests or
| something, so shouldn't a single user be expected to consume
| a few thousand zipping around the map while looking at
| drawings overlaid?
|
| I'm sure there is some botting, it's basically guaranteed,
| but I wouldn't be surprised if nearly half the traffic was
| "legitimate". The bots don't normally need to reload (or even
| load) the map tiles anyways.
| indrora wrote:
| It's also way more than 1px/30s -- Its like 20px/30s and you
| have a "tank" of them, which you can expand to however big you
| want.
|
| Placing pixels gives you points, which you can turn into more
| pixels or a bigger bag of pixels over time. I've seen people
| who have done enough pixel pushing that they get 3-4K pixels at
| a time.
| hoppp wrote:
| Cool... You did well to ban them.
|
| Its a ddos attack, lucky you dont have to pay for the brandwidth,
| then its a denial of wallet
| biker142541 wrote:
| Curious how this would have compared to a static pmtiles file
| being read directly by maplibre. I've had good luck with
| virtually equal latency to served tiles when consuming pmtiles
| via range requests on Bunnycdn.
| hyperknot wrote:
| Yes, wplace could solve their whole need by a single, custom-
| built static pmtile. No need to serve 150 GB of OSM data for
| their use-case.
| nielsole wrote:
| https://github.com/hyperknot/openfreemap?tab=readme-ov-file#...
| biker142541 wrote:
| Interesting, I should benchmark this. I have only used
| Bunnycdn so far and latency seemed similar to most tile
| providers like Maptiler and others (but a very limited test).
| This was using the full planet pmtiles file. Bunnycdn also
| makes it easy to prevent downloading the entire file, either
| in case you care about anyone using it or just want to
| prevent surprise downloads for those exploring network tab.
| biker142541 wrote:
| Quick benchmark of pmtiles directly in maplibre vs served
| tiles, both via Bunnycdn and 5 areas sampled using same
| style. Total impact on page end to end load time: 39ms
| longer with cached range requests from pmtiles than cached
| tiles. Individual requests are comparable in the 20-35ms
| range, so the slight extra time seems to be from the
| additional round trip for range headers (makes sense).
| Ericson2314 wrote:
| Oh wow, TIL there is finally a simple way to actually _view_
| OpenStreetMap! Gosh, that 's overdue. Glad it's done though!
| bravesoul2 wrote:
| ... And then it became 1M rps!
| bspammer wrote:
| What was wrong with the main site? Genuine question
| https://www.openstreetmap.org
| drewda wrote:
| The OSM Foundation has been serving raster tiles for years and
| years (that's what's visible by default on the slippy map at
| www.openstreetmap.org):
| https://wiki.openstreetmap.org/wiki/OpenStreetMap_Carto
|
| After on and off experimentation by various contributors, OSMF
| just released vector tiles as well:
| https://operations.osmfoundation.org/policies/vector/
| bravesoul2 wrote:
| 429 is your friend... but well done for handling the load!
| parhamn wrote:
| Since cloudflare is already sponsoring it, I do wonder how much
| of this type of service can be implemented all on cloudflare.
| Their stack could be great for tile serving.
| leobuskin wrote:
| I'm also surprised to see nginx and hetzner in this project.
| Why not entirely Cloudflare: workers, R2, and cache
| conradfr wrote:
| You can get cheap dedicated server on Hetzner with unlimited
| bandwidth, would the cost be similar with CF?
| PUSH_AX wrote:
| You can run containers now on CF, I think this was one of the
| last barriers that might have prevented a lot of software being
| migrated without a re-architecting.
|
| Most stuff could run there now.
| wiradikusuma wrote:
| "Wplace.live happened. Out of the blue, a new collaborative
| drawing website appeared, built from scratch using OpenFreeMap."
| -- as a founder, you know you're working on the wrong thing when
| there's a "fun project" getting daily traffic more than what
| you'd get in a lifetime :)
| ivanjermakov wrote:
| > Nice idea, interesting project, next time please contact me
| before.
|
| I understand that my popular service might bring your less
| popular one to the halt, but please configure it on your end so I
| know _programmatically_ what its capabilities are.
|
| I host no API without rate-limiting. Additionally, clearly
| listing usage limits might be a good idea.
| Aeolun wrote:
| I think it's reasonable to assume that a free service is not
| going to deal gracefully with your 100k rps hug of death. The
| fact that it actually did is an exception, not the rule.
|
| If you are hitting anything free with more than 10rps
| (temporarily) you are an taking advantage in my opinion.
| Aurornis wrote:
| > I understand that my popular service might bring your less
| popular one to the halt, but please configure it on your end so
| I know _programmatically_ what its capabilities are.
|
| Quite entitled expectations for someone using a free and open
| service to underpin their project.
|
| The requests were coming from distributed clients, not a
| central API gateway that could respond to rate limiting
| requests
|
| > I host no API without rate-limiting. Additionally, clearly
| listing usage limits might be a good idea.
|
| Again, this wasn't a central, well-behaved client hitting the
| API from a couple of IPs or with a known API key.
|
| They calculate that per every 1 user of the wlive.place
| website, they were getting 1500 requests. This implies a lot of
| botting and scripting.
|
| This is basically load testing the web site at DDoS scale.
| ivanjermakov wrote:
| Ugh, then I agree. This way it's indistinguishable from DDoS
| attack.
| zamadatix wrote:
| > The requests were coming from distributed clients, not a
| central API gateway that could respond to rate limiting
| requests
|
| The block was done based on URL origin rather than
| client/token, why wouldn't a rate limiter solution consider
| the same? For this case (a site which uses the API) it would
| work perfectly fine. Especially since the bots don't even
| care about the information from this API so non-site based
| bots aren't even going to bother to pull the OpenFreeMap
| tiles.
| CommanderData wrote:
| I love sites like wplace can still go viral and blow up, in a age
| of an increasingly centralised web. Woot
| arend321 wrote:
| I'm using OpenFreeMap commercially, fantastic and stable service.
| cube00 wrote:
| Hopefully you've been able to contribute back financially if
| you're making money from this service.
| arend321 wrote:
| For sure, monthly sponsor.
| cube00 wrote:
| It's really surprising that no CDNs or cloud storage providers
| offer even the single PMTiles file in some sort of shared library
| customers can use.
|
| I guess they'd all rather their customers each upload the 120GB
| file and then charge them all individually.
|
| If they're crafty they'll have their storage configured so
| there's only one actual copy on the underlying infra so every
| other shadow copy is pure profit.
___________________________________________________________________
(page generated 2025-08-10 23:02 UTC)