[HN Gopher] Show HN: SuperSplat - open-source 3D Gaussian Splat ...
___________________________________________________________________
Show HN: SuperSplat - open-source 3D Gaussian Splat Editor
Author : ovenchips
Score : 175 points
Date : 2024-11-06 12:07 UTC (10 hours ago)
(HTM) web link (playcanvas.com)
(TXT) w3m dump (playcanvas.com)
| Retr0id wrote:
| It would be interesting to have a 3d version of a mesh warp /
| puppet warp
| HexDecOctBin wrote:
| Is there a reading guide to the maths behind Gaussian splats? All
| the resources I could find either assumed lots of knowledge
| (including what a "3D Gaussian" even is), or were written for
| complete lay-person (and probably includes some AI grift).
| dheera wrote:
| I wish all these guides used better notation. They all use the
| scariest possible Greek symbols (pi epsilon feta etc) that are
| hard to make sense of without a degree in anthropology, instead
| of nicely named variables like programmers use.
| akx wrote:
| I do hope they don't use brined white cheese as a symbol.
| Sardtok wrote:
| Oh, they do. They most definitely do. And salads. [?]= Feta
| hasn't made into Unicode yet, so I'm substituting it for
| generic cheese, but just you wait.
| speps wrote:
| The blog posts from Aras are a really good starting point:
| https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr...
| yokto wrote:
| Not a reading guide, but Computerphile have a good introductory
| video on Gaussian Splats:
| https://youtu.be/VkIJbpdTujE?si=8hoMbMx6tKuMZo2S
|
| Individualkex also has a couple videos on the high level ideas:
| https://youtu.be/GQXDjzNWuPc?si=zlAN7dO9STGATKad
| infocollector wrote:
| Impressive! Does anyone know if this is open source? Or perhaps
| can be run locally as a server?
| salviati wrote:
| I found its repository after searching google. The license is
| MIT.
|
| [1] https://github.com/playcanvas/supersplat
|
| I remember a time when it was considered unpolite to ask a
| question without googling first. Is it still the case?
| criddell wrote:
| As I recall it, the asshole move was to reply to a question
| with a LMGTFY link.
| TeMPOraL wrote:
| Those were the times when you could actually rely on Google
| to give you the right results for such a query near the
| top, and more importantly, give you the same results it
| would give to the person you're telling to Google it.
| viraptor wrote:
| Kagi allows you to share a URL to a specific search
| results page. Maybe it's time to revive the idea...
| yapyap wrote:
| > I remember a time when it was considered unpolite to ask a
| question without googling first. Is it still the case?
|
| Yes
| jonasdoesthings wrote:
| Not sure if I'm missing something, but the submission title
| says "open-source" and in the tool's help menu there's a link
| to the repo (https://github.com/playcanvas/supersplat), the
| tool runs in your browser, there's no server involved besides a
| Web-Server hosting the files.
| joch wrote:
| There's an app for Quest 3 called Gracia, which allows you to see
| these in 3D space:
|
| - https://www.meta.com/en-gb/experiences/gracia/25784099001234...
|
| - https://www.gracia.ai
| creativenolo wrote:
| Whats the performance like?
| joch wrote:
| Performance is surprisingly great, but of course it's not as
| sharp/high res as you would get on a PC.
| misterdata wrote:
| This is cool!
|
| Any tips for an app to use on iOS to capture the necessary .ply
| data?
|
| Scaniverse is a great app by Niantic that can do this on-device,
| but it isn't very customizable and can't export its raw scanning
| data (exported .plys do not have the data this editor requires).
| bhouston wrote:
| PolyCam I think is the most popular?
| bhouston wrote:
| This is neat but Splats are not really mean to be edited in this
| way.
|
| Splats are sort of like byte code, they are the compiled and
| optimized representation of reflected light as semi-transparent
| guassians.
|
| Or you can think of them as the PDF equivalent of a Google or
| Word Doc. All the logic is gone, and you just have final
| optimized results.
|
| Generally when you edit PDFs, the results are not great and you
| cannot make major edits because the layout won't reflow, etc.
|
| So while this is cool, I don't think it will take off unless
| there is another innovation in terms of either using AI to
| "reflow" the lighting and surfaces after an edit, or inferring
| more directly the underlying representations (true surface
| properties and the light sources.)
| yorwba wrote:
| There are already approaches to infer bidirectional reflectance
| distribution functions (BRDFs, the "true surface properties")
| and lights:
| https://nju-3dv.github.io/projects/Relightable3DGaussian/
| slimbuck wrote:
| I'm not really sure what you mean. Think of SuperSplat as the
| photoshop of gaussian splats?
|
| - SuperSplat dev :)
| caycecan wrote:
| Relightable gaussian splats - https://junxuan-
| li.github.io/urgca-website/
| ovenchips wrote:
| Hi Ben! I would argue that it is very useful for splats to be
| edited in this way. I couldn't have built this application
| without SuperSplat for isolating, cleaning, transforming and
| optimizing/compressing the PLY:
|
| https://playcanv.as/e/p/cLkf99ZV/
|
| Integrating AI is an interesting topic and something that
| certainly has potential.
| bhouston wrote:
| I 100% agree with:
|
| - cleaning up noisy GuassianSplats is useful. There are often
| stragglers floating around in space that need to get deleted.
|
| - compression/optimizing them is useful.
|
| This being a cleanup and compression tool makes sense, but I
| guess I don't call that an "editor."
|
| I guess I was more arguing against the idea that this is a
| viable "editor" where one can combine and manipulate in more
| radical ways Gaussian Splats. The current technological
| approach doesn't make this a feasible use case.
| ovenchips wrote:
| Coming very soon is:
|
| - Copy & Paste: e.g. delete a tree and fill the hole with a
| copied patch of grass
|
| - Color Adjustments: tinting, brightness, etc.
|
| If these aren't editing ops, I don't know what is. :) Sure,
| you _could_ go back and recapture photogrammetry or rerun
| training, but that's super costly in terms of time.
| SuperSplat lets you make simple edits quickly and easily.
| jobigoud wrote:
| In theory if you delete something you have to recompute
| global illumination and remove cast shadows in the
| immediate environment of the removed object, but that
| information is baked in the gaussian splats. I think
| that's the kind of limitation the parent comment is
| talking about.
| ovenchips wrote:
| To be as accurate as possible, yes, you need to consider
| lighting/shadows. But trust me, in many circumstances,
| you can copy+paste gaussians and it looks 'good enough'.
| It depends on the scene and the edit you want to make.
| rsp1984 wrote:
| Wow, the fade-in animation is most excellent! Mind sharing
| how you created it?
| yapyap wrote:
| great metaphor, thanks!
| pixelpoet wrote:
| Seems like 54 day old spam account from post history? A bit
| difficult to tell.
| kfarr wrote:
| I guess in theory what you say could be correct, however in
| practice this tool has been very helpful for client work of
| editing, cleaning, cropping and even slight modification of
| Gaussian splats. I could see a similar argument for raster
| images in general -- they are hard to edit as you're modifying
| individual pixels and it's not efficient, but we've seen tools
| grow from MS Paint to modern Photoshop to become very useful. I
| think the same could be said here -- it's just early and we're
| at the "bytecode" level as you say.
| bufferoverflow wrote:
| That's a ridiculous take. Generated splats almost always have
| garbage parts that need to be truncated. An editor is
| absolutely needed for that.
| trebligdivad wrote:
| I found it interesting, I'd heard of guassian splats but not
| really appreciated how they worked, but this let me play with a
| model; so I'm not saying necessarily useful but instructive.
| neomantra wrote:
| I don't like the "byte code" analogy for Gaussian splats. If
| they were like that, then we could apply compiler optimization
| and those sorts of math techniques to them. But they are
| Probability Distribution Functions with transforms, so the math
| tools we have to work with them are the similar to those in
| signal processing -- resampling, quantizing, estimating, etc.
|
| In that model, we don't compile them, we train them; we don't
| run them, we sample/rasterize them.
|
| This link came up on HN before and was a great
| refresher/expander on the math of Guassians which allow all
| this. [1].
|
| Since Gaussians can be estimated, neural networks can
| model/generate them. Researchers are using this for 4D work and
| mesh extraction. The NNs run at lower frame rate informing the
| 3DGS running at interactive rates.
|
| You are right that it is ephemeral and really a weird trick of
| the eye and we need new ways to edit/create it. Vectors/pixels
| have had a lot more time to grow tooling. People are working on
| it, just the toolbox is different. Very cool stuff will be
| coming up, I bet!
|
| [1] https://news.ycombinator.com/item?id=41912160 I've also re-
| learned Fourier transforms to appreciate similar concepts.
| jtolmar wrote:
| I don't really think this is true. Gaussian splats certainly
| came from a context where an opaque representation is expected
| and normal, but they ended up being an entirely comprehensible
| format. They're not as simple to operate on as an SDF or voxel
| representation, but I think they're on par with triangle mesh
| geometry. A transformed fuzzy sphere is about as complex as a
| triangle, and spherical harmonic colors, while more
| conceptually difficult than textures, have fewer moving parts.
| amegahed wrote:
| I could imagine this as a clean-up tool for splats. In any case,
| beautiful interface and the sample model made me smile. Thanks
| for sharing.
| antoineMoPa wrote:
| Did anyone build a text-to-splat 3D generation model? Seems like
| it would be pretty straightforward? Should make it really easy to
| generate assets for video games.
|
| EDIT: yep - https://gsgen3d.github.io/
| viraptor wrote:
| Aren't gaussian splats incompatible with most common game
| development styles? No shadows, no rigging/animation as main
| issues - or maybe I'm misunderstanding / behind on the research
| - please correct me.
| hasnain99 wrote:
| that awesome whats most programing language used
| hasnain99 wrote:
| is there have repo on git have please provide a link
___________________________________________________________________
(page generated 2024-11-06 23:00 UTC)