[HN Gopher] Overpass Turbo: A Web Based Data Mining Tool for Ope...
___________________________________________________________________
Overpass Turbo: A Web Based Data Mining Tool for OpenStreetMap
Author : stefankuehnel
Score : 190 points
Date : 2024-01-24 03:13 UTC (19 hours ago)
(HTM) web link (overpass-turbo.eu)
(TXT) w3m dump (overpass-turbo.eu)
| dabber wrote:
| For anyone else curious:
|
| Source code for the backend here:
| https://github.com/drolbr/Overpass-API
|
| Found from this link listed in the help modal: https://overpass-
| api.de/ (also links to two additional frontends.)
| shpx wrote:
| I've been asking ChatGPT to write Overpass Turbo queries and it
| does a good job.
| maelito wrote:
| Yes. It was way harder with the other LLMs. Except Claude
| maybe.
| zarazas wrote:
| There is also this: https://overpassnl.schumann.pub/
| Raphaellll wrote:
| we systematically tested this:
| https://arxiv.org/pdf/2308.16060.pdf
| maelito wrote:
| Where can we use it ? I couldn't find a link. It is closed-
| source ?
|
| OK, I had a hard time understanding that the link given above
| is the demo of this paper.
|
| https://overpassnl.schumann.pub.
|
| Very interesting. Quite surprising too : it can find non
| trivial searches like "bars with darts" ("bars a flechette"
| in French), but not easy ones ("fromagerie" or "dermato").
|
| More complex queries did not work : "cafes close to the
| river" returned nothing, neither did "touristic site close to
| a pedestrian area" or "touristic streets".
| Raphaellll wrote:
| Yes, it's far from perfect. We currently work on augmenting
| the prompt with e.g. relevant key-value pairs extracted
| from https://wiki.openstreetmap.org/wiki/Map_features
| e-v wrote:
| Lucky you! I tried learning with examples generated by ChatGPT
| and none of them worked at all. I then learned the basics of
| the API by reading this excellent tutorial: https://osm-
| queries.ldodds.com/tutorial/, and realised that, in hindsight,
| ChatGPT's answers weren't even close to being correct.
| stevage wrote:
| Yeah, I've had that experience too. Or at least it gets close
| enough that I can make it work.
|
| The Overpass QL language is the most...bizarrely constructed
| language I've ever had to use. It's a bit hard to believe that
| the person who designed the language thought this was a
| perfectly reasonable statement: (._; >;);
| Alifatisk wrote:
| Gotta agree with you, the QL is strange. I tried using the
| wizard but it kept spinning. I tried asking Bard and ChatGPT
| but they provided invalid syntax.
|
| This is sad because overpass is extremely powerful tool.
|
| I'll give the tutorials other commenters linked a try.
| Aachen wrote:
| I read up on the syntax a few years ago and, by learning only
| a few of the symbols, it actually starts to make a lot of
| sense and seemed like a nice shorthand.
|
| I then promptly forgot everything and find the syntax very
| annoying again nowadays... seriously, it's weird remembering
| having knowledge but not having the knowledge itself anymore
| flexagoon wrote:
| Overpass syntax is one of those things that I just read the
| documentation for every single time I have to do it. Last
| time I used it things like ChatGPT didn't exist yet though,
| so I'll probably use it from now on if I ever have to write
| an overpass query
| stevage wrote:
| Yeah me too. No way is my brain going to remember any of
| that between times that I use it.
| vloewe wrote:
| I scraped the these websites
| https://wiki.openstreetmap.org/wiki/Map_features and gave it
| to OpenAI's Assistant maker -> This made great results in
| making valid queries
|
| Could even make prompts like "Can you find the buildings
| closer than 500m to a cafe?"
| dylan604 wrote:
| I really don't understand the point of shortcuts where this
| is the result.
|
| I'm a really big fan of clear and legible code. I'm not at
| In-n-Out with a secret menu. I'm trying to make
| changes/updates to code someone else wrote. I don't care how
| clever they felt they were by using esoteric shortcuts that
| might make things short and fit on one line. I want to be
| able to read the code and follow what it does in the least
| amount of time. Now, if the compiler makes different/better
| decisions when reading something like that vs much more
| legible code, we can maybe have a discussion.
| webel0 wrote:
| It does. The problem is that I have a hard time determining if
| the output is, indeed, what I was trying to query in the first
| place!
| ImaCake wrote:
| There is a QGIS plugin for this API as well which is excellent.
|
| Note that in openstreetmap.com you can zoom in and query items
| (cursor with question mark icon) to get the feature information
| that you then use in Overpass. Very useful!
| Freak_NL wrote:
| openstreetmap.org not .com (!)
| flexagoon wrote:
| openstreetmap.com just redirects to the .org domain. There's
| also just osm.org, which is much easier to type.
| curtisblaine wrote:
| The wizard isn't working for me. Just keeps spinning.
| a_gnostic wrote:
| Spinning is a nice trick.
| marklit wrote:
| I built osm_split to extract OSM features into named GPKG files.
| It makes it easy to pick features by name and drop them into
| QGIS. https://github.com/marklit/osm_split
| RicoElectrico wrote:
| Isn't the point of GPKG that you can bundle multiple named
| layers in one file?
| marklit wrote:
| Yeah, you can. The file will be huge though and QGIS won't be
| performant. Somewhere like Tokyo will be ~400 MB in OSM's PBF
| format but will contain ~1,300 different types of features.
|
| Seeing these listed in alphabetical order in a file explorer
| makes it easier to track down only what you need to build
| your map.
|
| It could be you only end up using 150 MB of GPKG files and
| QGIS, even on an old laptop, would be performant.
| vloewe wrote:
| Overpass is great!
|
| Published a ShowHN about Atlas.co yesterday, but I also just want
| to mention that we have a Overpass Turbo OSM integration that
| enables you to query data directly in a map and then download it
| as geojson, kml or shp (obv you can also style it and use it in
| analysis). Might be useful for some
| lmeyerov wrote:
| Related: If any OSM GIS engineering enthusiasts, we have a small
| OSS repo we are looking for an assist on that should be fun for
| the right person: Spin up a big box, quickly render planet-scale
| tiles at a decent level (maptiler), and restart as a more
| reasonably-sized zoom level for serving. We did the core
| infra/devops, and are looking for someone to help tweak the perf
| knobs etc. See my profile on how to reach.
| mistrial9 wrote:
| please disclose if you are asking for an unpaid intern or
| volunteer with certain qualifications
| lmeyerov wrote:
| Sponsor, even!
| Doctor_Fegg wrote:
| Do you really mean maptiler? tilemaker or planetiler will do a
| better job of "quickly rendering planet-scale tiles".
| the_g0d_f4ther wrote:
| Actually saved me in an internship where i needed a complete
| dataset of adresses. This post remindes me that i had a gdoc
| where i had i lot of ideas for side projects, thanks !
| Tempest1981 wrote:
| Mentioned here a few months ago: (was trying to remember where I
| saw it)
|
| _How to find a street in 2 minutes [video] (youtube.com)_
|
| https://news.ycombinator.com/item?id=36912880
| guwop wrote:
| Hmmmm. If i want to do larger scale extractions of OSM data into
| a layer - what's the move? I remember someone on here posted a
| way to do it
| twp wrote:
| I wrote https://github.com/twpayne/osm-extract for exactly
| this.
| ikawe wrote:
| I typically download a pbf extract, and then use the osmium cli
| for local filtering.
|
| Then I use ogr2ogr to convert to whatever output format I need.
| HumblyTossed wrote:
| Slightly off topic and maybe better as an Ask HN, but here goes.
|
| There are apps for updating OSM that will let me put in the
| information for the houses in my neighborhood, for example, but
| that gets tedious. Are there any scripts (python, shell, ?) that
| would make that easier?
| toomuchtodo wrote:
| https://wiki.openstreetmap.org/wiki/Import
|
| > Imports and automated edits should only be carried out by
| those with experience and understanding of the way the
| OpenStreetMap community creates maps, and only with careful
| planning and consultation with the local community. See
| Import/Guidelines and Automated Edits code of conduct for more
| information. Imports/automated edits which do not follow these
| guidelines might be reverted!
| Quot wrote:
| I used StreetComplete when I had an Android phone. It looks for
| missing data near you, and gives you a really nice interface to
| input that data.
|
| They are working on an iOS build, and I can't wait to start
| using it again.
|
| https://github.com/streetcomplete/StreetComplete/
|
| Edit: Also, OSM has a few wiki pages for editing software on
| different platforms.
|
| - Android_apps_that_can_upload_changes_to_OSM -
| https://wiki.openstreetmap.org/wiki/Category:Android_apps_th...
|
| - Android_apps_that_can_record_GPS_tracks -
| https://wiki.openstreetmap.org/wiki/Category:Android_apps_th...
|
| - IOS_software -
| https://wiki.openstreetmap.org/wiki/Category:IOS_software
|
| - Mobile_editors -
| https://wiki.openstreetmap.org/wiki/Category:Mobile_editors
| jjwiseman wrote:
| I've played around with Overpass and GPT a bit. Here are a few
| examples that I think demonstrate the potential:
|
| Find all buildings that straddle the boundary between Glendale
| and Burbank in California:
|
| https://twitter.com/lemonodor/status/1636849040548675584
|
| Finding the suspected origin of an explosion, by using the time
| from seeing the explosion on video to the time the sound of the
| explosion reaches the microphone:
|
| https://twitter.com/lemonodor/status/1636859983223734273
|
| Someone on twitter asked "Is anyone aware of any 4 way stop
| intersections in Australia?"
|
| https://twitter.com/lemonodor/status/1516239321094713346
|
| Finding examples of airport runways that cross highways:
|
| https://twitter.com/lemonodor/status/1709598048459132976
|
| Finding banks that might be at risk of robbery:
|
| https://twitter.com/lemonodor/status/1716153200750051332
| jjwiseman wrote:
| Bellingcat's Bellingcat OpenStreetMap search is a tool that
| wraps Overpass Turbo in an easier-to-use interface for the
| specific task of geolocation based on image and videos:
| https://osm-search.bellingcat.com
| jjwiseman wrote:
| Here's a video showing the sort of geolocation task the
| Bellingcat tool is intended for:
| https://youtu.be/GqKNKQ02pjY?si=DyAB3YZzl3gJUzT9
| WaxProlix wrote:
| This is awesome, but a lot of the queries I've tried to make
| seem to fail on OSM's data itself, which is sad. For instance,
| "coffee shops within a certain distance of fast EV chargers"
| would be really valuable, but the underlying data just doesn't
| have that EV data. So it's cool for a lot of stuff, but mostly
| explicitly streets; the other kinds of nodes all exist but
| aren't as well fleshed out as they could be (totally
| understandable).
| Doctor_Fegg wrote:
| That's because we're all cyclists ;)
___________________________________________________________________
(page generated 2024-01-24 23:01 UTC)