[HN Gopher] Translating OpenStreetMap data to HTML5 Canvas with ...
___________________________________________________________________
Translating OpenStreetMap data to HTML5 Canvas with Rust and
WebAssembly
Author : todsacerdoti
Score : 294 points
Date : 2024-02-20 10:28 UTC (12 hours ago)
(HTM) web link (mary.codes)
(TXT) w3m dump (mary.codes)
| dakial1 wrote:
| Author could use that queue time and some ML to forecast waiting
| times trends and create a real time Next-Best-Ride (NBR) model
| and plot those in the map. I'd find that useful the next time I'm
| pushing my 2 kids around that place...
| maxerickson wrote:
| The mentioned Overpass API uses sets as a sort of fundamental
| concept, so the code
| way[building][!name];foreach{(._;>;);out;}
|
| can be replaced with
| way[building][!name];(._;>;);out;
|
| The "(._;>;);" stanza is sort of dense and seems opaque without
| knowing what it's doing, but it's straightforward, returning the
| union "();" of the previous result "._;" and the children of the
| previous result ">;".
|
| The change I propose above adjusts the ">;" to operate on all of
| the "way[building][!name];" results together, rather than each
| element one at the time.
| enigmo wrote:
| this is a common enough need that out geom
|
| does a similar operation in fewer characters
| maxerickson wrote:
| It doesn't do exactly the same thing, it inserts the lat/long
| into the parent instead of returning the nodes. Often doesn't
| matter, but it does if you are using a library that is
| expecting osm xml.
| simpaticoder wrote:
| What a clear, step-by-step write-up of getting this POC (proof-
| of-concept) code running. She has picked as her audience
| experienced programmers, and one gets the sense that the write-up
| is as much for herself as for us. Which is great. We get to live
| vicariously through her and solve a small-ish (but non-trivial)
| problem with Rust, WASM and an OSM api with a cool visual payoff
| at the end. It's easy to let yourself not be impressed by
| straight-forward writing to a clear audience, because it looks so
| simple. But it's absolutely not simple and I'm glad to have found
| this blog and author. Thanks again, HN!
| mxfh wrote:
| The direct plotting from geographic coordinates (e.g., WGS84) can
| lead to visual distortions, especially for parks located further
| from the equator, where circles will appear flattened.
|
| A practical solution would be to reproject the data to the
| Mercator projection or the appropriate local UTM Zone, ensuring
| accurate distances in cartesian units with no visible distortions
| of angles.
|
| Alternatively, a quite straightforward method would involve
| scaling the longitudinal axis by the cosine of the latitude (cos
| ph [1]) at the center latitude of the area of interest. This
| adjustment provides a good approximation for small extents, like
| theme parks, without delving deeply into map projections.
|
| [1]
| https://en.wikipedia.org/wiki/Longitude#Length_of_a_degree_o...
| wodenokoto wrote:
| I thought all maps were drawn using web Mercator and that
| things were simply not large enough to be really distorted.
| Does gmaps and company do projected rendering?
| sebstefan wrote:
| They do, if you zoom out they're a mercator that wraps
| around, but when they give you latitude and longitude instead
| of x and y, you're getting the real sphere stuff
| samstave wrote:
| I used to know a lot more about rendering than I do today,
| but (forgive me if this sounds stupid) - I wonder if you
| could add to a gps coordinate area (say 1 sq. km. to also
| include that patche's NORMAL from the surface of the globe
| that you're projecting your map to, and have the 1km tile's
| image adjusted based on a orthographic view from that tile's
| NORMAL, so effectively - you're projecting each 1k square
| more directly from areal imagery?
|
| Or is this already considered in modern projection methods?
| modeless wrote:
| Google Maps on web uses perspective projection, unless WebGL
| is disabled in your browser. Zoom out and see!
| lstodd wrote:
| I once experimented with something like this for OpenXcom is to
| first rotate the geographic coordinate system so that local
| equator/zero meridian intersection is at the center of the area
| of interest, then apply merkator or whatever other projection
| you like.
|
| Otherwise it's a sea of hacks which doesn't get rid of
| latitude-induced distortions anyway.
|
| After that being very annoyed with lat/long stuff I moved to
| N-vector representation.
|
| https://en.wikipedia.org/wiki/N-vector
| jillesvangurp wrote:
| Should not be an issue here as this is based on openstreetmap,
| which uses wgs84 coordinates everywhere. Changing the
| projection won't make that much of a difference for small
| features like parks or buildings.
|
| You are right that this is an issue if you start plotting e.g.
| circles using a naive algorithm as the x and y scales in
| degrees are simply not the same in meters. Which is an issue if
| you want to draw circles and rectangles on a map.
|
| The solution to that is to do it in meters and then translate
| the points from some origin using e.g. the Haversine distance.
| A related issue you have there is that distance algorithms
| aren't accurate either but it's close enough as an
| approximation over small distances.
|
| I have some algorithms for that in my jillesvangurp/geogeometry
| library. I also have some UTM coordinate conversion algorithms
| in there that I recently added.
| mxfh wrote:
| The distortion is still there, even for small features as
| seen in the OP examples of circular features of the park.
| This due to the changing length of longitudinal unit at a
| given latitude.
|
| Historically this is the reason web mapping picked up
| Mercator in the first place, since it preserves angles and
| relatives sizes (conformal) at city scales, which is the
| primary use case of those.
|
| WGS84 is a CRS first, not meant to be used as cartographic
| projection for a known local scope. (for some Plate caree is
| more of a plot than a projection)
|
| I'm not talking about converting between local CRS for
| surveying an local plate consistency. You just want the
| display to look not visibly distorted in a way that's not
| intentional. Florida is still quite forgiving here.
| Eurodisney not so much.
|
| https://commons.wikimedia.org/wiki/File:Plate_Carr%C3%A9e_wi.
| ..
| naitgacem wrote:
| I have tried to use Canvas on Android to draw some custom view
| (chart). I still have a PTSD from that experience. I have an
| exceptionally great amount of respect for people who can do this
| sort of stuff.
| liftm wrote:
| Well, if you're playing with wasm like this post, you could
| sidestep canvas and render the image on the wasm side. c.f.
| https://github.com/plotters-rs/plotters-wasm-demo
| ativzzz wrote:
| I did something similar but used SVG instead - it didn't feel
| that bad
| beeboobaa wrote:
| > While I haven't performed any benchmarks about whether Rust or
| JavaScript processing is faster, I decided to use Rust and
| WebAssembly because there will be a lot of coordinates to
| process, and Rust generally performs faster at large amounts of
| data processing.
|
| Is this really true for Rust compiled to WebAssembly in the
| browser?
| davidmurdoch wrote:
| Depends on the data types and if the WASM code can make use of
| SIMD.
| grovesNL wrote:
| It depends on the use case (e.g., how much you're interacting
| with web APIs) but I'd generally expect this to be the case for
| data processing at least.
|
| During compilation from Rust to WebAssembly you have the
| opportunity to apply ahead-of-time optimizations that would be
| too expensive or impossible to apply to JavaScript during
| execution, so this can improve performance pretty significantly
| over JavaScript.
| afavour wrote:
| It's an interesting question. When it comes to processing large
| amounts of data I'd actually be more concerned about
| JavaScript's memory usage than strict processing performance.
| Certainly you could use JS smartly with typed arrays and such
| to minimise memory impact but you'd basically be manually
| creating a lot of the convenience that Rust gives you out of
| the box.
| Solvency wrote:
| If they have a public API why did she have to crop and manually
| export stuff? What is she exporting, data? If so what is the API
| doing?
| Ajedi32 wrote:
| Looks like she wasn't actually exporting anything, just using
| the export feature as a convenient way to get the coordinates
| of the bounding box she wanted to search in.
| gerogerke wrote:
| For anyone interested, I have a similar blog-post [0], in which I
| explore rendering OSM data using WebGL.
|
| [0]: https://gero.dev/blog/webgl-render-osm-streets
| wvh wrote:
| This is a cool project; it has just enough meat on its bones to
| learn a bit about WASM and mapping data, and an attainable goal
| with a clear, visual result after some effort. I just spent half
| an hour to try to get this to work myself to see how far WASM has
| come. Not sure I'll be doing something with this specifically,
| but good to keep in the back of my head for future challenges
| along the same vein.
| butz wrote:
| Here is an idea: use Mapsforge binary format for map data. Map
| sizes are quite small, especially useful for not to huge areas.
| Implementing HTTP Range requests might be an interesting exercise
| in reducing bandwidth usage.
| coder543 wrote:
| An existing solution:
|
| > PMTiles readers use HTTP Range Requests to fetch only the
| relevant tile or metadata inside a PMTiles archive on-demand.
|
| https://docs.protomaps.com/pmtiles/
| emporas wrote:
| I was looking at that project [1] trying to understand what this
| article explains in a more simple way. There is some non-trivial
| complexity of how to put together wasm-pack, web-sys, js-sys and
| wasm-bindgen functions to work well together with javascript for
| a web page or a browser extension. Nice article!
|
| [1] https://github.com/drakerossman/hackernews-userscript
| NoPedantsThanks wrote:
| Wasn't there a post on here recently about OSM going to vector-
| based tiles? I did a quick search and it seems like this is
| already a widely-offered option. Can anyone jog our memory on
| what the change is?
| maxerickson wrote:
| OSM foundation is going to host vector tiles and build a stack
| to keep them up to date with ongoing editing.
|
| You are probably talking about
| https://news.ycombinator.com/item?id=39339182
| mikeocool wrote:
| The OSM Foundation is building out a stack to generate
| 'minutely' vector tiles -- meaning the if you edit the map,
| it'll be reflected in the vector tiles within a few minutes.
|
| Most vector tile sets today generated from OSM are hours behind
| the current state of the map data in the best case, much more
| commonly they are months behind.
| HumblyTossed wrote:
| In case you're hungry after reading this, she has a post on how
| to make creamy grits:
|
| https://mary.codes/blog/recipes/the_secret_to_the_creamiest_...
|
| I love grits, but have never tried baking soda (or even heard of
| it). Gonna have to try this.
| londons_explore wrote:
| So the data seems to come from the overpass API as XML, gets a
| bit of transformation in rust, passed back to javascript for
| drawing...
|
| To me, that seems like the wrong design for performance...
| Drawing you probably want to use webgl shaders for so you can do
| zooming and panning at 60 fps...
|
| And downloading the data as XML seems like a bad idea if you're
| drawing city-sized areas because the XML will be huge and contain
| plenty more detail than you need. Instead you need a server-side
| component to pack simplified and quantized polygon data for the
| necessary scale into a binary datastructure for sending to the
| client.
|
| And thats exactly how bing/google maps/apple maps work...
___________________________________________________________________
(page generated 2024-02-20 23:00 UTC)