[HN Gopher] WorldGen: Open-source 3D scene generator for Game/VR/XR
___________________________________________________________________
WorldGen: Open-source 3D scene generator for Game/VR/XR
Author : ziyangxie
Score : 124 points
Date : 2025-04-28 18:39 UTC (2 days ago)
(HTM) web link (worldgen.github.io)
(TXT) w3m dump (worldgen.github.io)
| atrus wrote:
| We really need models that produce intermediate files. A .blend,
| a whole godot project, an actual krita/psd file.
|
| Things like this product such cool things....all the way up until
| you need to move a specific tree half a meter to the left, and
| then you're either stuck in some prompt/seed hell or have to open
| an abomination of a file in an editor.
| ziyangxie wrote:
| WorldGen supports exporting the generated scene as a vertices
| colored mesh in .ply format, which you can load into any 3D
| editor (like Blender). The output is also in metric scale, so
| you can easily align it with other assets
| ivape wrote:
| Looks like only nvidia cards. Can anyone guess how much vram you
| need for this?
| ziyangxie wrote:
| It is currently 26GB, but I will reduce it to under 24GB so
| that you can at least run it on a 4090.
| wordpad wrote:
| Is it available via API somewhere?
| Maxious wrote:
| Have you seen https://github.com/mit-han-lab/nunchaku for
| running flux on lower VRAM and thus faster performance?
|
| I have put up a PR https://github.com/ZiYang-
| xie/WorldGen/pull/7
| ziyangxie wrote:
| Optimized the webpage layout for mobile users.
| Ono-Sendai wrote:
| Not really proper 3D, more like a kind of 2.5D.
| pdntspa wrote:
| The splat outputs remind me of Cyberpunk 2077's braindances
| jimmySixDOF wrote:
| An opensource Blockade Labs Skybox.ai sounds good and thats a lot
| of tooling in your pipeline so congratulations putting it all
| together I would 100% clone this on a HuggingFace space !!
| socalgal2 wrote:
| Neat! But not quite what I was expecting. It doesn't really make
| a 3D "scene". It makes a 3D panorama (building a mesh based off
| the depth of the panorama). Move off the center and all the
| things that were occluded are missing.
|
| I guess, because we're in the time of generative content, that I
| was expecting it would recognize what's in the scene and fill in
| the missing parts. Maybe in version 2
| albumen wrote:
| From GitHub page: "We also support background inpainting for
| better scene generation, but it's currently an experimental
| feature, which may not work for all scenes. It can be enabled
| by setting WorldGen(inpaint_bg=True)"
| bengarney wrote:
| Is there an example of this?
|
| A skybox with depth is only marginally better than a skybox
| for any sort of 3d experience. Using the depth for occlusion
| would be kind of cool.
| tamat wrote:
| which for me doesnt make any sense as why to generate a 3D mesh
| that can only be seen from one point of view? why not map the
| panorama to an sphere?
| doshaa wrote:
| I really hope there is like an "AI" flair on projects like these,
| i love what they aim to do but man the saturation levels mixed
| with "enough to make it work in theory" effort makes me want to
| filter out these posts.
|
| I understand its all vibes and fun, but like at least some
| warnings that "this thing might lag your computer" and buttons
| that work.
| ziyangxie wrote:
| Thanks for your interest! the AI space is moving fast, and
| there is always something new grabbing attention. I believe it
| is better to release an MVP early and keep improving it, rather
| than holding everything back and risking being outdated or
| scooped. Just like in startups, I believe fast iteration is key
| for AI projects too.
| pedalpete wrote:
| Really nicely done, and I'm excited to see where this goes.
|
| I'm the founder of 3d real-world scene generator
| https://ayvri.com - the tech was acquired a while back, but I
| still have people reaching out to me to ask me to bring it back,
| and I've told people that the way to do it today would be to
| generate the scenes with splats and AI.
| firtoz wrote:
| I like that there are so many different approaches to generate
| worlds. One, few, or eventually all of them will stick.
|
| We have been playing around with something similar at Greybox
| too, where we ask Claude 3.7 to create Lua scripts to define the
| scenes with primitives (cubes, spheres, etc) that you can then
| move around. It's not perfect but did better than we expected!
|
| https://greybox.app/blog/articles/introducing-greybox-ai-cre...
| has a video at the top that shows what it looks like at the
| moment.
|
| We saw that Meta, Krea are working on "compose the scene from 3d
| model generations or imports as individual components", we'll
| give that a try too, soon.
| whywhywhywhy wrote:
| for what it does, its cool but claiming it's useful for
| games/vr/xr is a bit of a stretch, in it's current state it's
| only good for fixed position vr (which is a pretty rough
| experience) as the illusion falls apart when you move your head
| slightly. For games you could use it for environment maps or
| maybe a non-interactive background in menus or visual novel
| style.
|
| But kinda think claiming a panoramic generator as a "World
| Generator" is intentionally hoping people expect it to be more
| than it delivers.
___________________________________________________________________
(page generated 2025-04-30 23:02 UTC)