[HN Gopher] Show HN: Maps and Splats - Mashup of 3D tile maps wi...
___________________________________________________________________
Show HN: Maps and Splats - Mashup of 3D tile maps with Gaussian
Splats
Hello HN! I'm excited to share this open-source community mashup
showing the power and possibility of combining 3D Gaussian splat
scans with traditional mesh-based 3D tiles maps in the browser. As
you've probably seen on HN a bunch lately, Gaussian splats are a
new method to allow custom 3D scans of scenes and objects [1] and
Google Maps 3D Tiles API are the same 3D buildings and street views
your familiar with in Google Earth or Google Maps, and recently
they've been made available for inclusion in web apps. Combining
splats + 3d map mixes highly detailed real-world custom micro-scans
with the macro-level 3D imagery from satellite data. This is useful
for civil engineering, transportation planning, real estate,
construction, creative production, and so much more - we're just
scratching the surface. Demo1 with manual orbit controls: -
Video: https://twitter.com/kfarr/status/1773934700878561396 - Live
Demo: https://maps-and-splats.glitch.me/ - Source:
https://glitch.com/edit/#!/maps-and-splats?path=index.html -
Instructions: zoom in / out using scrollwheel and rotate with click
and drag Demo2 with animated camera path timeline: - Video:
https://twitter.com/3dstreetapp/status/1775203540442697782 - Live
Demo: https://look-at-splat-maps.glitch.me/ - Source:
https://glitch.com/edit/#!/look-at-splat-maps - Instructions for
using timeline to make your own camera flight path:
https://www.youtube.com/watch?v=QDWKina45f4 These demos combine:
A-Frame / three.js, NYTimes three.js 3DTiles viewer component with
Google 3D Tiles, 3DStreet Gaussian splat viewer component from Luma
Labs scanned scenes, theatre.js animated camera path and timeline
editor, Glitch hosting / IDE because they're awesome for hacking
projects like this These demos are results of R&D from a larger
project: https://3dstreet.org Some of these API cost real money so
I worked to get a temporary quota of 25,000 sessions. After that
the 3dtiles won't load, you'll need to get your own API key from
Google :) [1] 2 min intro to 3D Gaussian splatting:
https://www.youtube.com/watch?v=HVv_IQKlafQ
Author : kfarr
Score : 54 points
Date : 2024-05-01 14:12 UTC (8 hours ago)
(HTM) web link (maps-and-splats.glitch.me)
(TXT) w3m dump (maps-and-splats.glitch.me)
| geuis wrote:
| On mobile web you can't dismiss the text area. It covers 80% of
| the screen.
| kfarr wrote:
| Thanks I just shrunk it a bunch more. It's a tech demo so
| apologies for the basic UI. It's also open source so anyone can
| remix it and play with the UI too
| dang wrote:
| Related: https://news.ycombinator.com/item?id=39909110 (April
| 2024)
| kfarr wrote:
| Next we are combining this work with an AR collection app for on-
| site capture and testing out workflows for "shared global
| anchors" (aka Spatial anchors or Cloud anchors) a recurring topic
| in the w3c immersive web working group:
| https://github.com/immersive-web/proposals/issues/82#issueco...
| pimlottc wrote:
| It's not immediately obvious, but you can move the viewpoint via
| panning by holding down shift while clicking and dragging (on
| desktop, at least). My natural instinct was to double-click like
| in Google Streetview.
|
| Not sure if there's a limit on the map area but I was able to
| navigate all the way to Ocean Beach!
| theultdev wrote:
| This is the best real world (heh) demo of Guassian Splats I've
| seen so far.
|
| Now to fork and enable the FPV camera controls.
|
| Great work!
| kfarr wrote:
| Yes, you can just remix and remove the orbit controls in lines
| 43 and 44 https://glitch.com/edit/#!/maps-and-
| splats?path=index.html%3... and then add `look-controls` `wasd-
| controls` to the camera entity.
| https://aframe.io/docs/1.5.0/components/wasd-controls.html
| theultdev wrote:
| Thanks!
|
| For anyone interested this is what I ended up with to fly
| around: <a-entity camera
| look-controls="pointerLockEnabled: true;" wasd-
| controls="enabled: true; fly: true; acceleration: 3000;"
| ></a-entity>
|
| Demo: https://delicious-enthusiastic-accelerator.glitch.me
| AlexAltea wrote:
| Congratulations on the amazing work/demo! An interesting
| extension of this would be integrating the "Level of Gaussians"
| approach presented at: https://zju3dv.github.io/LoG_webpage/
|
| Also interesting would be to generate gaussian splats from Google
| Street View footage, and integrate it directly into the Google
| Maps 3D data as you have done.
| jwilber wrote:
| This is great!
|
| Unrelated - does anyone know a great library for building tile
| maps for 2D scatter plots?
| spython wrote:
| Really nice! I did a quick test some weeks ago and apparently
| using google street view images to generate Gaussian splats works
| quite well: https://www.instagram.com/p/C2xozxVAdpG/
|
| (I just took screenshots of the Street View, not the API, so the
| ground is falling apart a bit because of the street names.)
| kfarr wrote:
| Agreed tons of potential here, however we may need to get our
| own geo data for this: Google was very clear during the API
| quota approval process that 3d tiles cannot be used for ML
| training or generation of derivative assets like Gaussian
| splats, at least with this specific product offering.
| spython wrote:
| Have you looked into Mapillary? Could be a viable
| alternative, even though the image quality varies a lot.
| kfarr wrote:
| Great idea... could even be generated "on-demand" only when
| the user requests high fidelity for a specific area!
| tkzed49 wrote:
| The video looks awesome! I'm on mobile and the first demo is
| loading 3D tiles but not the splats for me.
| kfarr wrote:
| I saw the splat "stall" during loading once while testing but
| couldn't reproduce. Curious if you reload after a stalled splat
| load if it works?
| ghostly_s wrote:
| > [1] 2 min intro to 3D Gaussian splatting:
| https://www.youtube.com/watch?v=HVv_IQKlafQ
|
| I don't know what the hell a "gaussian splat" is and I have to
| say this video did not help at all.
___________________________________________________________________
(page generated 2024-05-01 23:00 UTC)