[HN Gopher] Peermaps - distributed, offline-friendly alternative...
___________________________________________________________________
Peermaps - distributed, offline-friendly alternative to commercial
map providers
Author : pabs3
Score : 124 points
Date : 2021-09-12 07:00 UTC (16 hours ago)
(HTM) web link (peermaps.org)
(TXT) w3m dump (peermaps.org)
| usui wrote:
| I may misunderstand how P2P at this scale works. How are map
| changes authorized and distributed if it's decentralized/P2P?
| Does this initiative focus exclusively on physical mapping and
| forego the extras like national borders/country naming (import
| your own GeoJSON)? What about street corrections? If it is a
| dynamically changing data set, is it reasonably plausible that
| the map data you get back could be poisoned?
| georgyo wrote:
| They are just distributing OSM data, which has many things
| including park, city, nation borders. And even includes more
| data then google, bing, and apple maps in some cases. However
| while business and store data is there, it is also the most
| incomplete.
|
| OSM is basically a wiki where anyone can edit. I use it for
| about 20% of my mapping needs, especially when in a park.
|
| There are plenty of great offline OSM clients, such as OsmAnd.
| I don't see any real value in the P2P here, particularly for
| mobile. However having more clients that render great maps is
| always a win.
| NoImmatureAdHom wrote:
| Organic Maps deserves a mention here
|
| https://organicmaps.app/
|
| It's a free and open-source reboot of maps.me. Probably the
| best FOSS mobile front-end for OSM to date.
| usui wrote:
| I see. I misunderstood that all of it depended on OSM. I
| thought that OSM would have been one of the possible data
| providers.
|
| Doesn't this kind of reduce the efficacy of it being P2P? If
| the goal is to reduce the load on OSM's servers, I suppose it
| makes sense, but from the standpoint of a free web, it
| doesn't seem to add much.
|
| > Peermaps is a distributed, offline-friendly alternative to
| commercial map providers such as google maps. Instead of
| fetching data from a centralized tile service, your computer
| fetches map data from other peers across the network.
|
| I think the wording is what confused me. It would be more
| descriptive to say "Instead of fetching data from a
| centralized tile service, your computer fetches OpenStreetMap
| data from other peers across the network". OSM is still
| centralized.
| FeepingCreature wrote:
| I think the idea is that distributing tile data is pretty
| expensive? Especially for embedded web map views.
| georgyo wrote:
| pre-rendered tiles is expensive, vector data less so.
|
| The entire OSM working data set for NA is ~12GB. NY State
| is ~500MB. Most countries and states are under 100MB. Daily
| deltas are usually well under 1MB.
|
| Tile servers are great for web apps, as you can pre-compute
| all the rendering and then cache the tiles. But there is no
| need to store terabytes of image data in an offline mobile
| app.
| kiwidrew wrote:
| I'm very happy with OsmAnd+ on Android. I'm not sure
| which server(s) are being used by the app when it
| downloads the map data, but the transfer rate from the
| server leaves much to be desired. It's probably a
| significant burden to host the entire collection, both in
| terms of storage space and in available bandwidth.
|
| So I can see how hosting the data as a series of torrents
| would help make this more sustainable. But it's a bit
| silly to implement an entire torrent stack inside a web
| browser just to display a few static image tiles...
| jqpabc123 wrote:
| _There are plenty of great offline OSM clients, such as
| OsmAnd_
|
| Personally, I prefer Magic Earth. I keep 7 surrounding US
| states downloaded on my phone and I get notification of
| updates (every few months).
|
| Average file size per state is about 450MB stored on SD card.
| It is how maps should be done in my opinion.
|
| The only difference here is how the distribution is handled.
| Not sure P2P is the way to go since most peers are likely to
| be mobile devices where data transfer can be disruptive and
| expensive.
| Tijdreiziger wrote:
| This app looks really great, but I have a few questions...
|
| (1) Where does the online map data come from? From Magic
| Earth servers or somewhere else?
|
| (2) Where is the 3D data sourced from?
|
| (3) Where is the satellite imagery sourced from? OSM
| definitely doesn't have this.
|
| (4) Where are transit directions sourced from? AFAIK OSM
| also can't do this.
|
| (5) Who is behind this app? The privacy policy just refers
| to them as 'Company'.
|
| All in all, the functionality looks great but I'm quite
| skeptical about the lack of transparency.
| jqpabc123 wrote:
| I suggest looking at www.magicearth.com and contacting
| their support.
| NHQ wrote:
| If there is ever a robust, truly P2P web, your peers will be
| your filters. IOW, your filters will anneal and morph according
| to the canonical filters created by your trust network--a
| delegation of peers, filtered via parameters of your choosing.
| This goes for everything from search to spam block.
|
| It is not all about scale. Much of P2P web development is
| idealistically about designing a more subjective internet.
| substack wrote:
| The website hasn't been updated in a while and the stack somewhat
| diverges from what is written there but we're very hard at work
| making all the pieces fit together. The p2p database (eyros)
| works pretty well with only some transfer size improvements left.
| The database is fully symmetric and runs fully in the browser
| with a ~400kb wasm build (will work on getting this down later).
| And the rendering stack works pretty well. The main hurdle at the
| moment which I have been working on is the ingest phase which
| consumes planet-osm.pbf and writes into the spatial database
| (eyros). I can process all the nodes in 1 hour and all the ways
| in 35 hours on a not very expensive vps but processing the
| relations I still run out of memory. Probably in the next few
| weeks I can get this ingest phase working and we will have an
| initial data release. There is still some rendering work with
| polish and label rendering but it basically works.
|
| The main initial benefit for the project is to have completely
| free embeddable web maps hosted p2p (using ipfs, hyperdrive,
| webtorrent) where you can entirely customize the rendering. Then
| later the benefits of the fully symmetric nature of the database
| will start to make more sense and the project has the potential
| to become less centralized on open street map servers and data.
| For some prior work that I and other people who work on peermaps
| have worked on check out https://mapeo.world/ and this very old
| writeup I made about an early version of the osm-p2p database.
| The experiences of working on that project and prior versions
| heavily informs how the current peermaps stack works.
|
| You can check out a more up to date version of the progress on
| this talk we gave for speakeasyjs recently:
| https://www.youtube.com/watch?v=P7X7C-door4
|
| Or here is a slightly old slightly broken version of panning
| across a processed version switzerland using the end-to-end stack
| on my laptop (it looks better now)
| https://www.youtube.com/watch?v=gHEmmQ6GnDI
| dcposch wrote:
| This is incredible. How big of a VPS would you need to
| preprocess the whole dataset now?
|
| If I use peermaps and zoom to a particular city, how does it
| find peers that have that particular part of the db?
| substack wrote:
| It's similar how with a torrent you can start seeking into a
| particular spot in a file and start playing by requesting
| particular chunks at that spot. Some clients like webtorrent
| support this behavior but it changes the dynamics of the
| network somewhat if many clients do this kind of thing. You
| can build some supplementary peer info to help the process
| along for different p2p networks depending on if they let you
| create side-channels or let you make more explicit
| connections to peers. For peermaps, the database is file and
| directory based so most of that peer tree traversal should be
| handled already by the network. And there are more ways to
| optimize the connections with additional tricks once you get
| the basics working with a somewhat slower and less
| sophisticated transfer method.
| substack wrote:
| The VPS we're running on has 60GB of RAM which should be
| plenty but the ingest program needs more work to use less
| memory so it stops crashing when denormalizing multipolygon
| relations, which involves denormalizing ways which fetch
| nodes... all referenced by ID which has not much locality
| spread across the pbf file. And if you write to temporary
| storage it can use a lot of disk and denormalization based on
| the on-disk format can get really slow. It's just all very
| tricky to get working well within reasonable time constraints
| (less than a week of processing ideally) and a reasonable
| memory footprint.
| EGreg wrote:
| Is this related to OpenStreetMap
| habi wrote:
| > [...] a fully p2p web-based map viewer to display
| OpenStreetMap data [...]
|
| So yes, seems so.
| butz wrote:
| For Android apps Mapsforge is a good solution, if you need
| detailed map of relatively small area, or less detailed map is
| enough for large areas. You can distribute map with app and have
| a decent sized app.
___________________________________________________________________
(page generated 2021-09-12 23:02 UTC)