[HN Gopher] Replacing Mapbox with open source solutions
___________________________________________________________________
Replacing Mapbox with open source solutions
Author : gaws
Score : 563 points
Date : 2023-02-17 16:27 UTC (1 days ago)
(HTM) web link (www.kschaul.com)
(TXT) w3m dump (www.kschaul.com)
| kabes wrote:
| Worth mentioning that openlayers is also really extending their
| webgl rendering capabilities lately.
| lukeqsee wrote:
| There's also support for MapLibre GL JS style specs in the
| works! Excited to see another stable vector tile & style
| renderer added to the ecosystem.
| [deleted]
| __turbobrew__ wrote:
| Titiler works pretty well for raster data in my experience. It is
| a step up from using an out of the box solution and involves a
| bit more development work but pretty easily allows customization.
| Using titiler on lambda with COGs stored on S3 we could get about
| 200ms load of a retina resolution map.
|
| We wanted to do things even faster so we started migrating to ECS
| which when using a network optimized instance we could render a
| full page map in sub 100ms.
|
| The author states that having all the data in a single file for
| PMTiles is advantageous, but it is a tradeoff. First of all you
| cannot modify parts of a S3 file so if you want to modify/update
| your dataset you need to re-process and re-upload the entire
| dataset. This can be a real pain if you are working with many TB
| datasets and you just want to update a subset of the data. Also,
| I thought that there is a global throughput limit on S3 files so
| wouldn't putting all data in a single S3 file potentially hit
| bottlenecks?
| bdon wrote:
| Yes, PMTiles is a tradeoff that isn't appropriate for
| transactional use cases. SQLite is pretty good for that
| already.
|
| There is a throughput limit on S3 files of approximately 5500
| GETs/sec per key. Bare archives on S3 is an appropriate choice
| for small-scale, zero maintenance deployments. If your
| application demands any thing close to that level of
| throughput, you're probably either:
|
| * Serving individual tiles over the internet: you should use
| the CDN integration http://protomaps.com/docs/cdn ; most tile
| requests will be cached and only misses will interact with the
| S3 bottleneck.
|
| * Bulk accessing a spatial subset of tiles: You shouldn't be
| requesting HTTP GETs for single tiles, but instead entire
| subsets of tiles with a single Range request made possible by
| the internal Hilbert curve ordering. This is still WIP here:
| https://github.com/protomaps/go-pmtiles/issues/31
| neuronexmachina wrote:
| > Using titiler on lambda with COGs stored on S3 we could get
| about 200ms load of a retina resolution map.
|
| Since Lambda presumably has a separate GDAL RAM cache for each
| request, I'm guessing this wasn't able to take advantage of any
| of GDAL's header or block caching? It's pretty impressive you
| were able to get latencies that low with that approach.
| 0xDEF wrote:
| Leaflet is also an interesting solution in this space:
|
| https://github.com/Leaflet/Leaflet
| lights0123 wrote:
| Although I used to exclusively use it, Canvas-based maps just
| run so much smoother and support things like rotation that are
| really helpful for changing perspective.
| aendruk wrote:
| Vector tiles?
| lukeqsee wrote:
| https://github.com/maplibre/maplibre-gl-leaflet exists!
|
| It's not perfect, and you don't see the full benefit of a
| WebGL renderer, but if you want to keep using a Leaflet API,
| it's great.
| time4tea wrote:
| I think you can do this quite easily. Use docker compose to start
| an nginx and maptiler/tileserver-gl. Use an nginx config to send
| requests upstream to tileserver and cache.
|
| Download a region file from maptiler, or make one.
|
| That's it.
|
| It's about 2h work, using certbot for certificates.
|
| If you want to create your own styles, it's slightly more fiddly,
| but essentially it's https://maputnik.github.io/editor/#0.41/0/0
|
| Host on hetzner for EUR3/month.
| flakeoil wrote:
| It's weird, 10 years ago I implemented using Google Maps API a
| website with dynamic maps, distance matrix, geolocation, map
| overlays and address autocomplete completely free also at high
| traffic. It was also easy to use.
|
| Five years ago Google started to charge an outrageous amount >
| $1000/month and we could not continue so we switched to something
| that cost $100/month. And that was for only the maps. We had to
| scrap the distance matrix and address autocomplete.
|
| Today it's still the same status, and if you want something
| cheaper then you have to work on putting pieces together like the
| Washington Post does.
| lukeqsee wrote:
| We offer a very competitive solution that is quickly becoming a
| one-stop shop for what you mentioned: https://stadiamaps.com.
|
| All that functionality for a lot less than $1000 / month.
| countvonbalzac wrote:
| you should put a price comparison for stadia vs. google vs.
| mapbox
| lukeqsee wrote:
| Yes. We should, and we're literally working on it right
| now!
| notatoad wrote:
| and as a bonus you've got alidade smooth, the ideal map tile
| for overlaying colorful data on top of:
| https://stadiamaps.com/themes/
| lukeqsee wrote:
| Thanks for the kind words! :)
| vinibrito wrote:
| Hey guys not affiliated with Stadia but I can vouch for them,
| I use it in one of my products for a few years now and never
| heard a complaint, only nice things. You can't go wrong with
| them.
| lukeqsee wrote:
| Thanks for the kind words!
|
| No idea which customer you are :), but if you ever want to
| have a chat or have feedback, ping me!
| keltex wrote:
| Same here with our website https://www.meetways.com. This
| product was created around the time google created their maps
| API. In fact we worked extensively with google to help them
| iron out some of their API issues. They even discuss our
| product on their blog:
| https://mapsplatform.googleblog.com/2008/10/geocoding-in-rev...
|
| This site made a little bit of money, but not that much. But
| then 5 years ago google raised their prices so much that we
| were suddenly in the red. We had to drop the places API
| (replaced with Yelp) and the autocomplete API (we don't even
| use that anymore). Recently we've been moving to Apple Maps API
| which has a pretty generous free tier. Stadia Maps also looks
| interesting.
| Reason077 wrote:
| > _"Stadia Maps"_
|
| Interesting name, considering Stadia was the brand of a
| recently-discontinued cloud gaming platform made by...
| Google.
| ianthetechie wrote:
| Yeah, it's unfortunate as Google launched their gaming
| product after we launched our maps, and we've outlasted
| them ;)
| folli wrote:
| I'm not affiliated, but a reasonably satisfied customer of
| https://maptiler.com for my project https://cubetrek.com.
| Perhaps also worth to take a look.
| bloudermilk wrote:
| Love the aesthetics you achieved with your map. Did you
| blog about the design process anywhere?
| folli wrote:
| Thanks! No, but let me know if you have any questions.
| You can send me an email at contact@cubetrek.com
| lukeqsee wrote:
| Would love to chat to see if we can help you! If you want,
| give me a shout at luke <at> stadiamaps.com
| didip wrote:
| Is Mapbox dead now? It used to be awesome many years ago.
| pierotofy wrote:
| OpenMapTiles is open source just as much as OpenAI is open
| source.
| T3RMINATED wrote:
| [dead]
| jeffbee wrote:
| The Post has been doing great data journalism lately. The maps
| from the article were really well done. Yesterday they ran this
| story that has all its graphics done in D3 and provides links to
| notebooks at ObservableHQ with evidence supporting the arguments
| made and conclusions drawn in the article.
| https://www.washingtonpost.com/climate-environment/interacti...
| wipfli wrote:
| If you like PMTiles you should consider supporting Brandon
| financially as a GitHub Sponsor.
|
| Brandon is a solo-developer who builds Protomaps alone and we get
| all his great stuff for free...
|
| https://github.com/sponsors/protomaps
| lukeqsee wrote:
| You can also sponsor MapLibre:
| https://github.com/sponsors/maplibre
|
| (Disclaimer: I'm on the governing board, but derive no
| financial benefit from it.)
| pella wrote:
| And https://donate.openstreetmap.org/
|
| _" OpenStreetMap is the largest open geographic database in
| the world, the data infrastructure for multitudes of mapping
| projects around the globe. Your donation to the OpenStreetMap
| Foundation will cover our core operational expenses in
| supporting the OpenStreetMap project: hardware costs, legal
| fees, administrative assistant and other expenses of our
| working groups and administration.
|
| We currently run extremely lean for an operation for a
| project the size and importance of OpenStreetMap. The
| OpenStreetMap Foundation relies on revenue from individual
| and corporate membership dues, profits generated by the
| annual State of the Map conferences, and donations. We must
| keep our income sources diversified, as these vary from year
| to year, but our modest needs stay the same. For this, we
| need your support."_
| jackson1442 wrote:
| Looks great. Been using Mapbox as my go-to for small projects
| since it's much smoother than Leaflet and has better DX in my
| opinion. This seems to capture all of that with an even better DX
| so definitely will be trying it out.
| lukeqsee wrote:
| https://maplibre.org/ definitely attempts to preserve the DX
| with free selection of tile providers.
| tacker2000 wrote:
| DX == Developer Experience? Thats a new one.
| [deleted]
| mkl wrote:
| No, it's quite old now. Well over a decade.
| preya2k wrote:
| Is there an OSS way to use vector tiles, without using Mapbox?
| rhodysurf wrote:
| Yes, just use tippecanoe to create your tiles, and serve them
| up via s3 or howerver else you want. Then client side render
| them with maplibregl
| mtmail wrote:
| The article covers that. See the "maplibre-gl-js for client-
| side rendering" section.
| Demiurge wrote:
| Check this out for backend:
| https://postgis.net/docs/ST_AsMVT.html if you can run ogr2ogr
| to import your data into PostGIS, and serve the binary data
| returned by a query, you're all set. You literally only need
| PostGIS for a dynamic vector tile service.
|
| Then mapboxgl or libremapjs on the frontend, or even
| OpenLayers.
| timwis wrote:
| Also see valhalla for an open source routing engine:
| https://github.com/valhalla/valhalla/
| lukeqsee wrote:
| Valhalla is great (I've contributed a few patches and run it
| for a while)!
|
| Also check:
|
| http://project-osrm.org/
|
| https://www.graphhopper.com/open-source/
| jokoon wrote:
| I don't want to sound annoying, but it's a bit hard to generate
| raster tiles. I mean openstreetmap data is open and free, but it
| doesn't seem they distribute any software to build the raster
| tiles they host.
|
| I don't know what they use, but it doesn't seem there is any
| software that is simple enough to use.
|
| I just want to configure a subregion, call generate-tiles.py
| planet.xml or whatever, and be done with it.
|
| I like open source, but there will always be open source software
| that is needlessly complicated because, well, it lets companies
| like mapbox and others to generate income for their expertise.
|
| That's not the best open source spirit, in my view.
| Doctor_Fegg wrote:
| The classic instructions for raster tiles are at
| https://switch2osm.org. I think there's a Docker image if that
| floats your boat.
| KronisLV wrote:
| These are some really great instructions, I wish other sites
| would be so informative and approachable.
|
| They do have more details on the Docker container approach in
| one of the pages as well: https://switch2osm.org/serving-
| tiles/using-a-docker-containe...
|
| Here's the relevant Docker Hub link:
| https://hub.docker.com/r/overv/openstreetmap-tile-server
|
| The requirements are considerable, though, given the
| workload:
|
| > Serving your own maps is a fairly intensive task. Depending
| on the size of the area you're interested in serving and the
| traffic you expect the system requirements will vary. In
| general, requirements will range from 10-20GB of storage, 4GB
| of memory, and a modern dual-core processor for a city-sized
| region to 1TB of fast storage, 24GB of memory, and a quad-
| core processor for the entire planet.
|
| Personally, for lower end mobile devices raster tiles still
| seem to perform better, when compared with at least some of
| the current vector tile implementations. I actually recorded
| a short comparison on my phone a while back:
| https://www.youtube.com/watch?v=A-IRtBGO9rM
| j_heffe wrote:
| We used https://github.com/developmentseed/titiler (and rio-
| tiler, the underlying library) at my last company which does
| dynamic tiling based on some input raster. It's an awesome
| project and there's nothing else quite like it in the
| geospatial space.
| lukeqsee wrote:
| I discovered it while looking at raster solutions, and it is
| pretty great! The raster stack is pretty bonkers. TiTiler
| uses at least three fundamental layers: GDAL, RasterIO, and
| rio-tiler. Each contribute a fairly significant improvement
| (for the use-case) over the underlying one.
| neuronexmachina wrote:
| There's a lot to like about GDAL+Rasterio, although I've
| found having all HTTP requests go through GDAL's C API does
| result in some limitations on concurrency and
| multithreading. GDAL's configuration being based entirely
| on env vars also has its downsides:
| https://github.com/developmentseed/titiler/issues/186
| [deleted]
| livinglist wrote:
| Test
| uneekname wrote:
| Open-source geospatial tools in general have developed rapidly
| over the last couple of decades. Mapbox and friends have laid
| groundwork, both good and "needlessly complicated." But there
| are just too many people who care about this to think it'll
| always be hard.
|
| I assure you, the OSS (and in this case, OSM) community will
| steadily build out better, faster software that includes more
| and more of the features a modern map needs. It's already
| looking like companies like WashPo are seeing the potential
| here.
|
| Come join us and make the world a better place :)
| [deleted]
| tylershuster wrote:
| https://github.com/maptiler/tileserver-gl makes it really easy.
| I have a pretty simple pipeline set up to download an OSM
| extract, convert it to MBTiles, host it in CloudFlare, and
| render raster tiles with this software.
| lukeqsee wrote:
| Checkout https://github.com/onthegomap/planetiler.
|
| Super easy way to generate a MBTiles, which you can then serve
| directly, or further convert to PMTiles, which can be used to
| host vector tiles for client-side rendering using MapLibre (or
| other renderers).
|
| Raster tiles are a lot harder because you have to generate them
| on the server, and that's a _lot_ more resource intensive.
| cancan wrote:
| [founder of felt, maintainer of some of the libraries mentioned]
|
| this is amazing to see! we are happy to see our OS contributions
| like tippecanoe are being used by people all over the place, and
| i personally really love to work with journalists.
|
| happy to answer any questions!
| lukeqsee wrote:
| Thanks for taking up the maintenance of Tippecanoe! I haven't
| needed to use it too much, but when I have, it fills a nice
| void.
| jillesvangurp wrote:
| We use maplibre and map tiler for the map tiles in the startup
| that I run. We switched to that from leaflet and I've been pretty
| happy with it. I might invest some time at some point in self
| hosting tiles. But honestly, map tiler is good value for money
| for now.
|
| I actually used to work with Nokia and Nokia Maps (in Berlin),
| which is now Here Maps. So, I know a thing or two about this
| market and still know some people working with various map
| companies, including at mapbox.
|
| Mapbox is increasingly an alternative to Here, Tom Tom and a few
| others that offer a wide range of services. The most simple one
| of which is showing maps. Think of indoor maps, traffic services,
| routing, geo coding, address search (surprisingly hard problem
| with open data), etc. Customers for these services tend to spend
| a lot of money on getting all sorts of bespoke solutions. Maps
| are a commodity at this point and there's very little money in
| providing just some basic maps. It's all the other stuff that's
| built on top of that that is a lot harder and more valuable.
|
| The recently announced overture collaboration could actually
| shake things up a little in this space. Openstreetmap is nice but
| also has some pretty significant limitations. Basically what the
| above mentioned companies do is creating proprietary services and
| data (with and without open data typically) that they then sell
| access to for a lot of money. The big companies behind overture
| are each big customers for this stuff. And now they are pooling
| resources to create a cheaper, more open alternative to all that.
| trynewideas wrote:
| In the tools space, cf. open source GeoJSON-to-vector tile tool
| Tippecanoe, which recently got a new non-Mapbox home at Felt:
| https://felt.com/blog/erica-fischer-tippecanoe-at-felt,
| https://github.com/felt/tippecanoe
| Doctor_Fegg wrote:
| > But for most of our use cases, we don't need the latest and
| greatest.
|
| This is the crux of this. Let's not pretend that much of this is
| anything other than "open source clones of Mapbox".
|
| PMTiles is genuinely innovative and I'm delighted to see it
| getting traction.
|
| Maplibre GL (which I use and like very much) is a fork of the
| last open source version of Mapbox GL, with not many code changes
| between now and then.
|
| Maputnik is an open source reimplementation of Mapbox Studio.
| Mapbox went on to hire its original dev.
|
| OpenMapTiles started life as basically a how-to/makefile pulling
| together the disparate, poorly documented parts of Mapbox's open
| source tile generation code. It's now mostly known for a
| reasonable, not particularly exceptional vector tile schema.
|
| It's great that people are now shipping production vector tiles
| without paying the Mapbox dollar. But, PMTiles aside, this is
| still Mapbox's world. The next challenge is to evolve the tech
| stack to something beyond what Mapbox worked up five/ten years
| ago.
| lukeqsee wrote:
| > PMTiles aside, this is still Mapbox's world.
|
| Isn't this a bit like saying we're still in Google's world
| because most maps still use Web Mercator? :)
|
| Good tech builds on what came before. Mapbox did a lot of
| ground-breaking work in building tooling around OSM, but so
| have many others. The fact that they named it _Mapbox_ Vector
| Tiles is genius in hindsight, because even though we may use
| tons of tooling they didn 't create to build and render them,
| their name is still there.
|
| > The next challenge is to evolve the tech stack to something
| beyond what Mapbox worked up five/ten years ago.
|
| Agreed, and I think we've seen a lot of iterative work in the
| open since then. The next challenge is likely building a OSS
| stack to do proper 3D: open data (including OSM) to pixels, and
| that work is already beginning across a lot of organizations:
| https://github.com/nyurik/future-mvt/discussions, Overture
| Maps, MapLibre, etc.
| twelvechairs wrote:
| I get the parent comment. There are good alteratives to the
| mapbox ecosystem though they arent always used because people
| see mapbox as default
|
| DeckGL for instance is a very innovative js library for front
| end (really developed by Uber I believe), with better 3d and
| movement, better with dealing data at scale, etc.
|
| Prettymaps is a pure python approach to rendering OSM data.
|
| Etc.
| notyourday wrote:
| Open source use we will, pay for development we won't, paywall
| for our "reporting" we have
| ARandomerDude wrote:
| Write like Yoda you do.
| margorczynski wrote:
| That's why I actually support paid licensing for commercial
| use. I'm all for helping out people but not a fan of getting
| taken advantage off - you make money off my or someones else's
| work then pay up.
| ecf wrote:
| Currently using Mapbox in an open-source project for their static
| map API and I wholeheartedly recommend against it. Everything but
| the simplest of polylines look jagged since they refuse to switch
| to POST requests with a json body and instead rely on you fitting
| in as much detail into URL query parameter length constraints.
|
| Going to take a look at replacing them with PMTiles and my own
| polyline rendering.
| lukeqsee wrote:
| There's actually a reason why most of these APIs support POST--
| most use-cases are hot-linking, where you don't control how the
| client makes the request. (Often using an <img>.)
|
| Here's one alternative, if you don't want to build from
| scratch: https://docs.stadiamaps.com/static-maps/ We don't have
| much in the way of query string limits, so have at it!
| andy_ppp wrote:
| Are there decent label placement solutions these days? It's a
| really tricky problem to do well.
| leetrout wrote:
| Subscribing for updates :)
|
| Will you be talking about this work openly somewhere?
| capitainenemo wrote:
| Rather curious about the URL query parameter length constraint.
| Excluding IE11/Edge, don't most major browsers have at least
| 64k query lengths these days? Could you just serve the low
| quality jaggy poly to few percent still on IE11/Edge and a high
| detail one to the rest?
| lukeqsee wrote:
| A lot of servers impose a constraint on URL length, so I
| suspect Mapbox does on their APIs.
| tylershuster wrote:
| I've switched over a fairly large transit client to PMTiles
| hosted on CloudFlare for their dynamic map and am switching
| another large transit client to using
| https://github.com/maptiler/tileserver-gl with MBTiles and the
| same style.json to generate static maps. It's all gone
| swimmingly and we've cut costs by about $1,000 a month.
| danpat wrote:
| You always have the option of creating a map style and using
| Mapbox's Raster Tile API - although that does mean you probably
| can't dynamically adjust your raster images in time for a
| request to be served.
|
| How are you using the Static Maps API currently where you would
| be able to send a POST request and display the image back to
| the user? Most use-cases for the Static Maps API involve the
| HTML <img> tag or something similar, which all send GET
| requests.
| tpmx wrote:
| Wasn't Mapbox once the low-cost option?
| usrusr wrote:
| And since the state of the open source map rendering art is
| consisting almost exclusively of pre-licence-change forks of
| mapbox code these days, they kind of still are.
|
| I wonder if Strava's recent acquisition of fatmap.com
| eventually boils down to them cutting ties with mapbox via
| acqui-hiring some mb-libre excellence. I could imagine Strava,
| due to the extreme map-centric nature of their product,
| bleeding quite considerable money to mapbox.
| kylebarron wrote:
| I didn't think Fatmap was based on mapbox at all? I thought
| it was their own custom renderer
| usrusr wrote:
| I took a superficial glance at what the site is loading and
| iirc there were plenty of the usual suspects in there. And
| it would be silly to do it any other way, because their
| product is _what_ they do with maps, not winning some NIH
| purity award.
| gorbypark wrote:
| I'm interpreting the comment above to mean Strava was/is a
| user of Mapbox and bought Fatmap in an effort to wean
| themselves off of Mapbox and use "inhouse" (Fatmap) mapping
| tech.
| lukeqsee wrote:
| MapLibre has made really good strides (especially on the web)
| since the fork--3d-terrain support, more tile sources, other
| DX improvements (e.g., conversion to TypeScript).
|
| https://maplibre.org/maplibre-gl-js-docs/api/
| [deleted]
| Operyl wrote:
| Yup, which disrupted the market, and now it's not.
| klysm wrote:
| 1. Subsidize low costs with venture capital and take on a ton
| of customers at a loss 2. Lock those customers into your API
| somehow. Enough code is effectively lock in for a while. 3.
| Change your pricing to be profitable. 4. Hopefully get
| acquired before all your customers leave
| tpmx wrote:
| They seem to be at step 3 at the moment.
| Nemo_bis wrote:
| Usage seems to have peaked in April 2022
| https://trends.builtwith.com/mapping/Mapbox
| gorbypark wrote:
| They used to be low-cost _and_ open source. Now they're source
| available and not interested in anything but big clients. I was
| using Mapbox for all my clients before their unopen sourcing,
| and I tried reaching out three times to ask about options for
| one of my clients. It wouldn't have been a huge amount of
| money, probably a few hundred per month, but they never, ever
| reply. That client wasn't even using any of their server
| resources, they are 100% self hosted, it's just that Mapbox now
| charges even for using their client. Anyways, I was trying to
| give them money, and they wouldn't even acknowledge I exist. I
| switched that client over to Maplibre and never looked back.
|
| The other thing that annoys me is that they've more or less
| abandoned all their open source repos without even as much of a
| notice in the README. All these amazing tools sitting there
| with open pull requests and no one responding for years. If
| they just did a clean break, put a notice on everything they
| weren't going to touch again, it would have been a lot
| better/easier for the community to fork and move on with their
| lives.
| shawn-butler wrote:
| Mapbox was born from open source tools & culture. They were
| always active and supporting at FOSS4G.
|
| I wonder what has changed?
| NelsonMinar wrote:
| 2017, Mapbox Raises $164M in Series C Financing:
| https://www.finsmes.com/2017/10/mapbox-raises-164m-in-
| series...
|
| It's gone poorly since then though. SoftBank SPAC hunts new
| merger partner as Mapbox deal falters:
| https://news.sky.com/story/softbank-spac-hunts-new-merger-
| pa...
| jonas21 wrote:
| VC funding? I think they were funded mostly by grants and
| their own revenues for the first few years. Then they
| started taking VC money (over $300M to date).
| dataviz1000 wrote:
| They were once Development Seed [0], a company that
| specialized in making geo spatial visualizations for the
| UN, NGOs and nonprofits. Development Seed was a large
| contributor of code and modules to Drupal 6 which they
| left. In the beginning, their open source mapping and geo
| spatial visualizations projects were mostly funded by
| grants from the Knight Foundation which funded news
| related projects. MapBox came from an in-house tool they
| developed to create and manage map overlay tiles. (I
| could be making this all up.)
|
| [0] https://developmentseed.org/
| [deleted]
| bpodgursky wrote:
| It's a really hard product to make profitable.
|
| The hosting and bandwidth costs are huge, and most of your
| customers (by raw count and nominal usage) are hobbyists or
| free products unwilling to pay even a few cents per user.
| Tostino wrote:
| There is distributed hosting tech for this type of thing
| at this point. Use it. You could even go old-school
| BitTorrent, it's been around for decades.
| lukeqsee wrote:
| It's hard, but certainly not impossible.
|
| Hosting and bandwidth are key inputs to your costs, but
| they are manageable.
| sgt wrote:
| Geoserver is also good for serving spatial data and tiles. A bit
| dated look but fast and stable
| wiredfool wrote:
| I find geoserver to be a bit of a headache to run, and have
| been moving more towards client side rendering using map ox
| style tools.
| lukeqsee wrote:
| This is really awesome to see.
|
| I'm involved in many of these projects (helped get MapLibre
| started, wrote a library supporting PMTiles, have used
| OpenMapTiles for years), and to see open-source, open-data win in
| the end is really satisfying.
|
| If anyone has any questions about how you could do this for
| yourself, I'm happy to answer! (I have a company offering SaaS
| maps, but we know SaaS isn't for everyone, and are happy to point
| people the right direction when self-hosting is what you want.)
| franga2000 wrote:
| Thanks for all your work on MapLibre, along with all the other
| contributors! I use MapLibre extensively in an open source and
| non-profit transit data aggregator and everyone keeps being
| blown away by how smoothly the map runs, even with tens of
| thousands of points.
|
| We burn through MapTiler free tiles on a regular basis and have
| been looking to migrate to self-hosting, but somehow PMTiles
| flew under my radar and every other approach seemed to either
| still rely on MapTiler for data (where you have to chose
| between outdated or expensive) or just be a huge pain to set
| up. Hopefully this finally solves our map issues so we can
| actually start promoting the project without fear of the map
| disappearing in the first week every month.
| lukeqsee wrote:
| My company may be able to help save you the trouble of self-
| hosting. We do have a free tier (for non-commercial / test
| usage), and our pricing structure allows you to save quite a
| bit compared to MapTiler.
| franga2000 wrote:
| Stadia Maps, right? Your pricing is better than MapTiler,
| but both have the same issues for us - there's a
| significant jump from free to paid (no pay-as-you-go
| option) and server-side caching is prohibited. Both of
| these are completely sensible business decisions, but don't
| make sense for us specifically.
|
| We have plenty of server resources from both our local
| partners and various cloud providers that have non-profit
| grants, but basically no regular budget to spend on SaaS,
| so self-hosting everything is the only thing that makes
| sense.
| lukeqsee wrote:
| Ah, that makes sense.
|
| I'd take a look at
| https://github.com/onthegomap/planetiler -- I bet that
| fits the bill for what you need if you have the servers
| but need the data. :)
| jddj wrote:
| Is there a standard folder tree of tiles -> PMTiles archiver or
| is that the wrong question to be asking?
| bdon wrote:
| There isn't a tool to do that right now. It could be a fit in
| either https://github.com/protomaps/go-pmtiles or
| https://github.com/protomaps/PMTiles/tree/main/python - the
| Go program is faster and more production ready at this point.
| I imagine if folders are working for you the quantity of
| tiles doesn't number into the millions, so the Python program
| might be sufficient.
|
| Feel free to open an issue.
| jddj wrote:
| Not in the millions no - 100k would be a reasonable order
| of magnitude.
|
| I'll take a look to see if MBTiles will get me there and if
| not I'll come back to these.
|
| Thanks heaps
| lukeqsee wrote:
| I don't think so. There's this:
| https://github.com/protomaps/go-pmtiles to get from MBTiles
| and there's a few xyz folder tree tools to get mbtiles from
| that structure (e.g., https://pypi.org/project/mbutil/). So
| you can get there, but I don't know of a good straight-line
| right now.
| jddj wrote:
| Thanks a lot. I developed and support a standalone desktop
| application which contains customer-specific tiled maps,
| and it can get unwieldy.
|
| A pivot at some point to these types of archives is an
| option, but I'd like to roll forward existing customers as
| well.
| bdon wrote:
| If you have access to SQLite in your desktop application
| then there isn't any big wins from using PMTiles over
| MBTiles; PMTiles does have some special design around
| tile-deduplication but this is possible in MBTiles too by
| using table views.
| 762236 wrote:
| If all of the tiles are coming from a single file (via PMTiles),
| how are they cached by the client? Do clients cache byte ranges,
| or do you configure things to give the appearance of serving
| separate files?
| lukeqsee wrote:
| You typically use byte-range caching in libraries--either using
| an edge worker (a la Cloudflare) to create separate GET
| requests from the client's perspective or directly in the
| client.
| cancan wrote:
| i think you can also directly call them from the maplibre lib
| itself using the pmtiles adapter
|
| https://github.com/protomaps/PMTiles
|
| https://github.com/protomaps/PMTiles/blob/main/js/examples/m.
| ..
| bdon wrote:
| The renderer used in the linked article (MapLibre) has an
| internal tile cache used when you return to a previously loaded
| area of the map.
|
| For the non-tiled parts of the PMTiles file, like metadata and
| the directories that store Z/X/Y to offset information, those
| are LRU cached by the PMTiles implementation. The most widely
| deployed right now is for TypeScript/Node. The "client" can
| either be web browser or inside a runtime like
| Lambda/Cloudflare Workers. On those the cache will persist
| across invocations using the ephemeral memory of the serverless
| function.
|
| A level deeper, browsers vary in how they cache Range requests;
| IIRC Firefox does not cache while others will treat Range as if
| it was any other HTTP header.
___________________________________________________________________
(page generated 2023-02-18 23:02 UTC)