[HN Gopher] Project Genie: Experimenting with infinite, interact...
___________________________________________________________________
Project Genie: Experimenting with infinite, interactive worlds
Author : meetpateltech
Score : 362 points
Date : 2026-01-29 17:02 UTC (5 hours ago)
(HTM) web link (blog.google)
(TXT) w3m dump (blog.google)
| meetpateltech wrote:
| Google Deepmind Page: https://deepmind.google/models/genie/
|
| Try it in Google Labs: https://labs.google/projectgenie
|
| (Project Genie is available to Google AI Ultra subscribers in the
| US 18+.)
| nickandbro wrote:
| This could be the future of film. Instead of prompting where you
| don't know what the model will produce, you could use fine-
| grained motion controls to get the shot you are looking for. If
| you want to adjust the shot after, you could just checkpoint the
| model there, by taking a screenshot, and rerun. Crazy.
| JKCalhoun wrote:
| I feel like people are already currently doing this.
| Essentially storyboarding first.
|
| This guy a month ago for example: https://youtu.be/SGJC4Hnz3m0
| mosquitobiten wrote:
| Every character goes forward only, permanence is still out of
| reach apparently.
| mikelevins wrote:
| I've been experimenting with that from a slightly different
| angle: teaching Claude how to play and referee a pencil-and-
| paper RPG that I developed over about 20 years starting in the
| mid 1970s. Claude can't quite do it yet for reasons related to
| permanence and learning over time, but it can do surprisingly
| well up until it runs into those problems, and it's possible to
| help it past some obstacles.
|
| The game is called "Explorers' Guild", or "xg" for short. It's
| easier for Claude to act as a player than a director (xg's
| version of a dungeon master or game master), again mainly
| because of permance and learning issues, but to the extent that
| I can help it past those issues it's also fairly good at acting
| as a director. It does require some pretty specific stuff in
| the system prompt to, for example, avoid confabulating stuff
| that doesn't fit the world or the scenario.
|
| But to really build a version of xg on Claude it needs better
| ways to remember and improve what it has learned about playing
| the game, and what it has learned about a specific group of
| players in a specific scenario as it develops over time.
| matt_LLVW wrote:
| Looks good to me:
| https://youtu.be/15KtGNgpVnE?t=648&si=urWJGEFWuN5veh43
| montebicyclelo wrote:
| Reminds me of this [1] HN post from 9 months ago, where the
| author trained a neural network to do world emulation from video
| recordings of their local park -- you can walk around in their
| interactive demo [2].
|
| I don't have access to the DeepMind demo, but from the video it
| looks like it takes the idea up a notch.
|
| (I don't know the exact lineage of these ideas, but a general
| observation is that it's a shame that it's the norm for blog
| posts / indie demos to not get cited.)
|
| [1] https://news.ycombinator.com/item?id=43798757
|
| [2] https://madebyoll.in/posts/world_emulation_via_dnn/demo/
| ollin wrote:
| Yup, similar concepts! Just at two opposite extremes of the
| compute/scaling spectrum.
|
| - That forest trail world is ~5 million parameters, trained on
| 15 minutes of video, scoped to run on a five-year-old iPhone
| through a twenty-year old API (WebGL GPGPU, i.e OpenGL fragment
| shaders). It's the smallest '3D' world model I'm aware of.
|
| - Genie 3 is (most likely) ~100 billion parameters trained on
| millions of hours of video and running across multiple TPUs. I
| would be shocked if it's not the largest-scale world model
| available to the public.
|
| There are lots of neat intermediate-scale world models being
| developed as well (e.g. LingBot-World
| https://github.com/robbyant/lingbot-world, Waypoint 1
| https://huggingface.co/blog/waypoint-1) so I expect we'll be
| able to play something of Genie quality locally on gaming GPUs
| within a year or two.
| 0xcb0 wrote:
| I keep on repeating myself, but it feels like I'm living in the
| future. Can't wait to hook this up to my old Oculus glasses and
| let Genie create a fully realistic sailing simulator for me,
| where I can train sailing with realistic conditions. On boats I'd
| love to sail.
|
| If making games out of these simulations work, it't be the end
| for a lot of big studios, and might be the renaissance for small
| to one person game studios.
| neom wrote:
| ...and then, the pneumatics in your living room.
| jsheard wrote:
| Isn't this still essentially "vibe simulation" inferred from
| videos? Surface-level visual realism is one thing, but
| expecting it to figure out the exact physical mechanics of
| sailing just by watching boats, and usefully abstract that into
| a gamified form, is another thing entirely.
| falcor84 wrote:
| Why wouldn't it just hook it into something like physx?
| jsheard wrote:
| Google has made it clear that Genie doesn't maintain an
| explicit 3D scene representation, so I don't think hooking
| in "assists" like that is on the table. Even if it were,
| the AI layer would still have to infer things like object
| weight, density, friction and linkages correctly. Garbage
| in, garbage out.
| JeremyNT wrote:
| Yeah I have a whole lot of trouble imagining this replacing
| traditional video games any time soon; we have actually very
| good and performant representations of how physics work, and
| games are tuned for the player to have an enjoyable
| experience.
|
| There's obviously something insanely impressive about these
| google experiments, and it certainly feels like there's some
| kind of use case for them somewhere, but I'm not sure exactly
| where they fit in.
| nsilvestri wrote:
| The bottleneck for games of any size is always whether they are
| good. There are plenty of small indies which do not put out
| good games. I don't see world models improving game design or
| fun factors.
|
| If I am wrong, then the huge supply of fun games will
| completely saturate demand and be no easier for indie game devs
| to stand out.
| bdbdbdb wrote:
| It's very impressive tech but subject to the same limitations
| as other generative AI: Inconsistency, inaccurate physics,
| limited time, lag, massively expensive computation.
|
| You COULD create a sailing sim but after ten minutes you might
| be walking on water, or in the bath, and it would use more
| power than a small ferry.
|
| There's no way this tech can run on a PS5 or anything close to
| it.
| ziofill wrote:
| You raise good points, but I think the "it's not good enough"
| stance won't last for long.
| WarmWash wrote:
| Five years is nothing to wait for tech like this. I'm sure we
| will see the first crop of, however small, "terminally
| plugged in" humans on the back of this in the relatively near
| future.
| Avicebron wrote:
| Honestly getting a Sunfish is probably cheaper than the a VR
| headset if you want to "train sailing"
| avaer wrote:
| > If making games out of these simulations work, it't be the
| end for a lot of big studios, and might be the renaissance for
| small to one person game studios.
|
| I mean, if making a game eventually boils down to cooking a
| sufficient prompt (which to be clear, I'm not talking about
| text, these prompts are probably going to be more like video
| databases) then I'm not sure if it will be a renaissance for
| "one person game studios" any more than AI image generation has
| been a renaissance for "one person artists".
|
| I want to be optimistic but it's hard to deny the massive
| distribution stranglehold that media publishing landscape has,
| and that has nothing to do with technology.
| hagbarth wrote:
| > and might be the renaissance for small to one person game
| studios.
|
| Indie games are already bigger than ever as far as I know.
| srameshc wrote:
| What's the endgame here? For a small gaming studio, what are the
| actual implications?
| educasean wrote:
| I understand the ultimate end goal to be simulation of life. A
| near perfect replica of the real world we can use to simulate
| and test medicine, economy, and social impact.
| aurumque wrote:
| I would think that building a environment which can be managed
| by a game engine is the first pass. In a few years when we are
| able to render more than 60 seconds it could very well replace
| the game engine entirely by just rendering everything in
| realtime based on user interactions. The final phase is just
| prompts which turn directly into interactive games, maybe even
| multiplayer. When I see the progress we've made on things like
| DOOM, where it can infer the proper rendering of actions like
| firing weapons and even updating scores on hits and such it
| doesn't feel like we're very far off, a few years at most. For
| a game studio that could mean cutting out almost everything
| between keyboard and display, but for now just replacing the
| asset pipeline is huge.
| mikewittie wrote:
| We seem to think that Genie is good at the creative part, but
| bad at the consistency and performance part. How hard would
| it be to take 60 seconds of Genie output and pipe it into a
| model that generates a consistent and performant 3D
| environment?
| hiccuphippo wrote:
| It seems to be generating images in real time, not 3d scenes.
| It might still be useful for prototyping.
| saberience wrote:
| There are collisions though and physics seemingly, so it
| doesn't seen to be a huge stretch that this could be used for
| games.
| xyzsparetimexyz wrote:
| It means you should go the other way. Open world winning
| against smaller, handcrafted environments and stories was
| generally a mistake, and so is this.
| mediaman wrote:
| What does it mean, that open world winning was a mistake?
| That the market is wrong, and peoples' preferences were
| incorrect, and they should prefer small handcrafted
| environments instead of what they seem to actually buy?
| in-silico wrote:
| The endgame has nothing to do with gaming.
|
| The goal of world models like Genie is to be a way for AI and
| robots to "imagine" things. Then, they could practice tasks
| inside of the simulated world or reason about actions by
| simulating their outcome.
| rvz wrote:
| Screensavers for robots?
| sy26 wrote:
| I have been confused for a long time why FB is not motivated
| enough to invest in world models, it IS the key to unblock their
| "metaverse" vision. And instead they let go Yann LeCun.
| phailhaus wrote:
| Most people don't like putting on VR headsets, no matter what
| the content is. It just never broke out of the tech enthusiast
| niche.
| observationist wrote:
| LeCun wasn't producing results. He was obstinate and insistent
| on his own theories and ideas which weren't, and possibly
| aren't, going anywhere. He refused to engage with LLMs and
| compete in the market that exists, and spent all his effort and
| energy on unproven ideas and research, which split the
| company's mission and competitiveness. They lost their place as
| one of the top 4 AI companies, and are now a full generation
| behind, in part due to the split efforts and lack of
| enthusiastic participation by all the Meta AI team. If you look
| at the chaos and churn at the highest levels across the
| industry, there's not a lot of room for mission creep by
| leadership, and LeCun thoroughly demonstrated he wasn't suited
| for the mission desired by Meta.
|
| I think he's lucky he got out with his reputation relatively
| intact.
| halfmatthalfcat wrote:
| Were you there or just an attentive outsider?
| observationist wrote:
| Attentive outsider and acquaintance of a couple people who
| are or were employed there. Nothing I'm saying is
| particularly inside baseball, though, it's pretty well
| covered by all the blogs and podcasts.
| richard___ wrote:
| What podcast?
| observationist wrote:
| Machine Learning Street Talk and Dwarkesh are excellent.
| Various discord communities, forums, and blogs downstream
| of the big podcasts, and following researchers on X keeps
| you in the loop on a lot of these things, and then you
| can watch for random interviews and presentations on
| youtube when you know who the interesting people and
| subjects are.
| qwertyi0k wrote:
| Most serious researchers want to work on interesting
| problems like reinforcement learning or robotics or RNN or
| dozen other avant-garde subjects. None want to work on
| "boring" LLM technology, requiring significant engineering
| effort and huge dataset wrangling effort.
| observationist wrote:
| This is true - Ilya got an exit and is engaged in serious
| research, but research is by its nature unpredictable.
| Meta wanted a product and to compete in the AI market,
| and JEPA was incompatible with that. Now LeCun has a lab
| and resources to pursue his research, and Meta has
| refocused efforts on LLMs and the marketplace - it
| remains to be seen if they'll be able to regain their
| position. I hope they do - open models and relatively
| open research are important, and the more serious AI labs
| that do this, the more it incentivizes others to do the
| same, and keeps the ones that have committed to it
| honest.
| ezst wrote:
| Since a hot take is as good as the next one: LLMs are by the
| day more and more clearly understood as a "local maximum"
| with flawed capabilities, limited efficiency, a $trillion + a
| large chunk of the USA's GDP wasted, nobody even turning a
| profit from that nor able to build something that can't be
| reproduced for free within 6 months.
|
| When the right move (strategically, economically) is to not
| compete, the head of the AI division acknowledging the above
| and deciding to focus on the next breakthrough seems
| absolutely reasonable.
| throw310822 wrote:
| You really need to be obstinate in your convictions if you
| can dismiss LLMs at the time when everyone's job is being
| turned around by them. Everywhere I look, everyone I talk
| to, is using LLMs more and more to do their job and
| dramatically increase their productivity. It's one of the
| most successful technologies I've ever witnessed arriving
| on the market, and it's only just started- it's just three
| years old.
| qwertyi0k wrote:
| To be fair, this was his job description: Fundamental AI
| Research (FAIR) lab. Not AI products division. You can't
| expect marketable products from a fundamental AI research
| lab.
| YetAnotherNick wrote:
| It's "Facebook Artificial Intelligence Research", not
| fundamental. So basically involves both fundamental and
| applied research.
|
| [1]: https://engineering.fb.com/category/ai-research/
| qwertyi0k wrote:
| Ref: Yann lecun post on linkedin, 3years ago: FAIR now
| stand for "Fundamental AI Research"
| mapmeld wrote:
| In an industry of big bets, especially considering the
| company has poured resources and renamed itself to secure a
| place in the VR world... staking your reputation on
| everyone's LLMs having peaked and shifting focus to finding a
| new path to AI is a pretty interesting bet, no?
| anonnon wrote:
| This sounds similar to the arc of Carpathy, who also managed
| to preserve his reputation despite sending Tesla down a FSD
| deadend and missing the initial LLM boat.
| qwertox wrote:
| Isn't it more like this: JEPA looks at the video, "a dog walks
| out of the door, the mailman comes, dog is happy" and the next
| frame would need to look like "mailman must move to mailbox,
| dog will run happily towards him", which then an image/video
| generator would need to render.
|
| Genie looks at the video, "when this group of pixels looks like
| this and the user presses 'jump', I will render the group
| different in this way in the next frame."
|
| Genie is an artist drawing a flipbook. To tell you what happens
| next, it must draw the page. If it doesn't draw it, the story
| doesn't exist.
|
| JEPA is a novelist writing a summary. To tell you what happens
| next, it just writes "The car crashes." It doesn't need to
| describe what the twisted metal looks like to know the crash
| happened.
| general_reveal wrote:
| You are beyond correct. World models is what saves their
| Reality Labs investment. I would say if Reality Labs cannot
| productize World Models, then that entire project needs to be
| scrapped.
| slashdave wrote:
| Failures are not publicly reported, in general. Do you we know
| what they have invested in?
| anxtyinmgmt wrote:
| Demis stays cooking
| RivieraKid wrote:
| This would be really cool if polished and integrated with VR.
| cpeth wrote:
| Exactly this, it would essentially be a STTNG Holodeck
| ofrzeta wrote:
| I don't know ... it's impressive and all but the result always
| looks kind of dead.
| api wrote:
| It's super cool but I see it as a much more flexible open ended
| take on the idea of procedurally generated worlds where hard-
| coded deterministic math and rendering parameters are replaced
| by prompt-able models.
|
| The deadness you're talking about is there in procedural worlds
| too, and it stems from the fact that there's not actually much
| "there." Think of it as a kind of illusion or a magic trick
| with math. It replicates some of the macro structure of the
| world but the true information content is low.
|
| Search YouTube for procedural landscape examples. Some of them
| are actually a lot more visually impressive than this, but
| without the interactivity. It's a popular topic in the demo
| scene too where people have made tiny demos (e.g. under 1k in
| size) that generate impressive scenes.
|
| I expect to see generative AI techniques like this show up in
| games, though it might take a bit due to their high
| computational cost compared to traditional procedural
| generation.
| saberience wrote:
| This reminds me of the comments by programmers roughly two
| years ago:
|
| "Sure it can write a single function but the code is terrible
| when it tries to write a whole class..."
| sfn42 wrote:
| You say that as if those programmers aren't still right
| phailhaus wrote:
| I have no idea why Google is wasting their time with this. Trying
| to hallucinate an entire world is a dead-end. There will never be
| enough predictability in the output for it to be cohesive in any
| meaningful way, by design. Why are they not training models to
| help _write_ games instead? You wouldn 't have to worry about
| permanence and consistency at all, since they would be enforced
| by the code, like all games today.
|
| Look at how much prompting it takes to vibe code a prototype. And
| they want us to think we'll be able to prompt a whole world?
| asim wrote:
| Take the positive spin. What if you could put in all the inputs
| and it can simulate real world scenarios you can walk through
| to benefit mankind e.g disaster scenarios, events, plane
| crashes, traffic patterns. I mean there's a lot of useful
| applications for it. I don't like the framing at this time, but
| I also get where it's going. The engineer in me is drawn to it,
| but the Muslim in me is very scared to hear anyone talk about
| creating worlds.... But again I have to separate my view from
| the reality that this could have very positive real world
| benefits when you can simulate scenarios. So I could put in a 2
| pager or 10 page scenario that gets played out or simulated and
| allow me to walk through it. Not just predictive stuff but
| let's say things that have happened so I can map crime scenes
| or anything. In the end this performance art is because they
| are a product company being Benchmarked by wall street and
| they'll need customers for the technology but at the same time
| they probably already have uses for it internally.
| jsheard wrote:
| > What if you could put in all the inputs and it can simulate
| real world scenarios you can walk through to benefit mankind
| e.g disaster scenarios, events, plane crashes, traffic
| patterns.
|
| This is only a useful premise if it can do any of those
| things accurately, as opposed to dreaming up something kinda
| plausible based on an amalgamation of every vaguely related
| YouTube video.
| q3k wrote:
| > What if you could put in all the inputs and it can simulate
| real world scenarios you can walk through to benefit mankind
| e.g disaster scenarios, events, plane crashes, traffic
| patterns.
|
| What's the use? Current scientific models clearly showing
| natural disasters and how to prevent them are being ignored.
| Hell, ignoring scientific consensus is a fantastic political
| platform.
| seedie wrote:
| Imo they explain pretty well what they are trying to achieve
| with SIMA and Genie in the Google Deepmind Podcast[1]. They see
| it as _the_ way to get to AGI by letting AI agents learn for
| themselves in simulated worlds. Kind of like how they let
| AlphaGo train for Go in an enormous amount of simulated games.
|
| [1] https://youtu.be/n5x6yXDj0uo
| arionmiles wrote:
| If that's the goal, the technology for how these agents
| "learn" would be the most interesting one, even more than the
| demos in the link.
|
| LLMs can barely remember the coding style I keep asking it to
| stick to despite numerous prompts, stuffing that guideline
| into my (whatever is the newest flavour of product-specific
| markdown file). They keep expanding the context window to
| work around that problem.
|
| If they have something for long-term learning and growth that
| can help AI agents, they should be leveraging it for
| competitive advantage.
| phailhaus wrote:
| That makes even less sense, because an AI agent cannot learn
| effectively from a hallucinated world without internal
| consistency guarantees. It's an even stronger case for
| leveraging standard game engines instead.
| hmry wrote:
| "I need to go to the kitchen, but the door is closed. Easy.
| I'll turn around and wait for 60 seconds." -AI agent trained
| in this kind of world
| MillionOClock wrote:
| An hybrid approach could maybe work, have a more or less
| standard game engine for coherence and use this kind of
| generative AI more or less as a short term rendering and
| physics sim engine.
| elfly wrote:
| I've thought about this same idea but it probably gets very
| complicated.
|
| Let's say, you simulate a long museum hallway with some vases
| in it. Who holds what? The basic game engine has the
| geometry, but once the player pushes it and moves it, it
| needs to inform the engine it did, and then to draw the next
| frame, read from the engine first, update the position in the
| video feed, then again feed it back to the engine.
|
| What happens if the state diverges. Who wins? If the AI wins
| then...why have the engine at all?
|
| It is possible but then who controls physics. The engine? or
| the AI? The AI could have a different understanding of the
| details of the base. What happens if the vase has water
| inside? who simulates that? what happens if the AI decides to
| break the vase? who simulates the AI.
|
| I don't doubt that some sort of scratchpad to keep track of
| stuff in game would be useful, but I suspect the researchers
| are expecting the AI to keep track of everything in its own
| "head" cause that's the most flexible solution.
| MillionOClock wrote:
| Then maybe the engine should be less about really
| simulating the 3D world and just trying best to preserve
| consistency, more about providing memory and saving context
| for consistency than truly simulating a lot besides higher
| level concerns (at which point we might wonder if it
| couldn't be directly part of the model somehow), but
| writing those lines I realize there would probably still be
| many edge cases exactly like what you are describing...
| godelski wrote:
| > Why are they not training models to help write games instead?
|
| Genie isn't about making games... Granted, they for some reason
| they don't put this at the top. Classic Google, not
| communicating well... | It simulates physics
| and interactions for dynamic worlds, while its breakthrough
| consistency enables the simulation of any real-world scenario
| -- from robotics and modelling animation and fiction, to
| exploring locations and historical settings.
|
| The key part is simulation. That's what they are building this
| for. Ignore everything else.
|
| Same with Nvidia's Earth 2 and Cosmos (and a bit like Isaac).
| Games or VR environments are not the primary drive, the primary
| drive is training robots (including non-humanoids, such as
| Waymo) and just getting the data. It's exactly because of this
| that perfect physics (or let's be honest, realistic
| physics[0,1]). Getting 50% of the way there in simulation
| really does cut down the costs of development, even if we
| recognize that cost steepens as we approach "there". I really
| wish they didn't call them "world models" or more specifically
| didn't shove the word "physics" in there, but hey, is it really
| marketing if they don't claim a golden goose can not only lay
| actual gold eggs but also diamonds and that its honks cure
| cancer?
|
| [0] Looking right does not mean it is right. Maybe it'll match
| your intuition or undergrad general physics classes with
| calculus but talk to a real physicist if you doubt me here.
| Even one with just an undergrad will tell you this physics is
| unrealistic and any one worth their salt will tell you how
| unintuitive physics ends up being as you get realistic, even
| well before approaching quantum. Go talk to the HPC folks and
| ask them why they need superocmputers... Sorry, physics can't
| be done from observation alone.
|
| [1] Seriously, I mean look at their demo page. It really is
| impressive, don't get me wrong, but I can't find a single video
| that doesn't have major physics problems. That "A high-altitude
| open world featuring deformable snow terrain." looks like it is
| simulating Legolas[2], not a real person. The work is
| impressive, but it isn't anywhere near realistic
| https://deepmind.google/models/genie/
|
| [2] https://www.youtube.com/watch?v=O4ZYzbKaVyQ
| phailhaus wrote:
| But it's not simulating, is it? It's hallucinating videos
| with an input channel to guide what the video looks like. Why
| do that instead of just picking Unreal, Unity, etc and having
| it _actually_ simulated for a fraction of the effort?
| dyauspitr wrote:
| Why is it a dead end, you don't meaningfully explain that.
| These models look like you can interact with them and they seem
| to replicate physics models.
| phailhaus wrote:
| They don't though, they're hallucinated videos. They're
| feeding models tons and tons of 2D videos and hoping they
| figure out physics from them, instead of just using a game
| engine and having the LLM write something up that works 100%
| of the time.
| jimmar wrote:
| As a kid in the early 1980s, I spent a lot of time
| experimenting with computers by playing basic games and drawing
| with crude applications. And it was fun. I would have loved to
| have something like Google's Genie to play with. Even if it
| never evolved, the product in the demos looks good enough for
| people to get value from.
| anigbrowl wrote:
| It's been very profitable for drug dealers for centuries, who
| wouldn't want a piece of that market?
| phailhaus wrote:
| Because games already exist, and it would be easier for LLMs
| to write games rather than hallucinate videos.
| whalee wrote:
| This was a common argument against LLMs, that the space of
| possible next tokens is so vast that eventually a long enough
| sequence will necessarily decay into nonsense, or at least that
| compounding error will have the same effect.
|
| Problem is, that's not what we've observed to happen as these
| models get better. In reality there is some metaphysical
| coarse-grained substrate of physics/semantics/whatever[1] which
| these models can apparently construct for themselves in pursuit
| of ~whatever~ goal they're after.
|
| The initially stated position, and your position: "trying to
| hallucinate an entire world is a dead-end", is a sort of
| maximally-pessimistic 'the universe is maximally-irreducible'
| claim.
|
| The truth is much much more complicated.
|
| [1] https://www.arxiv.org/abs/2512.03750
| post-it wrote:
| And going back a little further, it was thought that
| backpropagation would be impractical, and trying to train
| neural networks was a dead end. Then people tried it and it
| worked just fine.
| phailhaus wrote:
| > Problem is, that's not what we've observed to happen as
| these models get better
|
| Eh? Context rot is extremely well known. The longer you let
| the context grow, the worse LLMs perform. Many coding agents
| will pre-emptively compact the context or force you to start
| a new session altogether because of this. For Genie to create
| a consistent world, it needs to maintain context of
| _everything_ , _forever_. No matter how good it gets, there
| will always be a limit. This is not a problem if you use a
| game engine and code it up instead.
| ollin wrote:
| Really great to see this released! Some interesting videos from
| early-access users:
|
| - https://youtu.be/15KtGNgpVnE?si=rgQ0PSRniRGcvN31&t=197 walking
| through various cities
|
| - https://x.com/fofrAI/status/2016936855607136506 helicopter /
| flight sim
|
| - https://x.com/venturetwins/status/2016919922727850333 space
| station, https://x.com/venturetwins/status/2016920340602278368
| Dunkin' Donuts
|
| - https://youtu.be/lALGud1Ynhc?si=10ERYyMFHiwL8rQ7&t=207
| simulating a laptop computer, moving the mouse
|
| - https://x.com/emollick/status/2016919989865840906 otter airline
| pilot with a duck on its head walking through a Rothko inspired
| airport
| RaftPeople wrote:
| I liked that first one and I hope someone creates one of going
| back to dinosaur age, i want to see that.
| echelon wrote:
| Tim is awesome.
|
| Ironically, he covered PixVerse's world model last week and
| it came close to your ask:
| https://youtu.be/SAjKSRRJstQ?si=dqybCnaPvMmhpOnV&t=371
|
| (Earlier in the video it shows him live prompting.)
|
| World models are popping up everywhere, from almost every
| frontier lab.
| post-it wrote:
| One step closer to the science-based dinosaur MMO we were
| promised.
| Valk3_ wrote:
| Any thoughts about Project Genie?
| ge96 wrote:
| Damn that was crazy the picture of the tabletop setup/cardboard
| robot and it becomes 3D interactive.
| WarmWash wrote:
| The actual breakthrough with Genie is being able to turn around
| and look back, and seeing the same scene that was there before. A
| few other labs have similar world simulators, but they all
| struggle badly with keeping coherence of things not in view.
| Hence why they always walk forwards and never look around.
| sfn42 wrote:
| And what if I go somewhere then go back there a week later?
| jsheard wrote:
| Best they can do is 60 seconds, for now at least.
| autonomousErwin wrote:
| Makes you wonder what the TTL caching for our universe is.
| dabbz wrote:
| Whatever the speed of light is I would imagine
| nozbufferHere wrote:
| Still amazed it took ML people so long to realize they needed
| and explicit representation to cache stuff.
| Legend2440 wrote:
| Genie does not use an explicit representation:
|
| >Genie 3's consistency is an emergent capability. Other
| methods such as NeRFs and Gaussian Splatting also allow
| consistent navigable 3D environments, but depend on the
| provision of an explicit 3D representation. By contrast,
| worlds generated by Genie 3 are far more dynamic and rich
| because they're created frame by frame based on the world
| description and actions by the user.
| emmettm wrote:
| The representation is learned. Also, see Sutter's "Bitter
| Lesson" essay
| abraxas wrote:
| What about Fei Fei Li's lab? I think they are generating true
| 3D worlds rather than frames of a video?
|
| Although that probably precludes her from having animations in
| those worlds...
| moohaad wrote:
| everyone will make his own game now
| JaiRathore wrote:
| I now believe we live in a simulation
| moomoo11 wrote:
| clearly
| cloudflare728 wrote:
| We will probably see Ready Player One in a few decades. Hoping to
| stay alive till then.
| HardCodedBias wrote:
| Decades?
|
| I mean, yes, the probability of having that level of tech in
| decades is quite high.
|
| But the technology is moving very fast right now. It sounds
| crazy, but I think that there is a 50% chance of having ready
| player one level technology before 2030.
|
| It's absolutely possible it will take more time to become
| economical.
| lexandstuff wrote:
| The mass-poverty and climate changed ravaged world parts, I
| could definitely see.
| krunck wrote:
| The more of this I see the more I want to spend time away from
| screens and doing those things I love to do in the real world.
| MillionOClock wrote:
| I love AI but I also hope it will paradoxically make people
| realize the value of real life experiences and human
| relationships.
| avaer wrote:
| Or maybe it will just make people realize the value of fake
| life experiences and human relationships.
| boogrpants wrote:
| The American public is no different than an American
| corporation; trying to extract as much allegiance and
| loyalty as possible for as little compensation as possible.
|
| Your neighbors in the street protesting for comprehensive
| single payer healthcare? Yeah they're perfectly fine
| leaving your existence up to "market forces".
|
| Copy-paste office workers everywhere reciting memorized
| platitudes and compliance demands.
|
| You're telling me I could interact even _less_ with such
| selfish (and often useless given their limited real
| skillset) people? Deal.
|
| America needs to rethink the compensation package if it
| wants to survive as a socio-political meme. Happy to call
| myself Canadian or Chinese if their offer is better. No
| bullets needed.
| jplusequalt wrote:
| >I think we'll eventually get to the point where these
| are real time and have consistent representations
|
| You have a dangerously low opinion of your fellow man,
| and while I sympathize with your frustration, I would
| humbly suggest you direct that anger at owners of
| companies/politicians, rather than aim it at your
| everyday citizen.
| boogrpants wrote:
| Your suggestion is meaningless semantic differentiation.
|
| Those owners and politicians are the result of exposure
| to American communities, schools, other institutions;
| they do not spontaneously exist.
|
| Americans prop up the system as such Americans will defer
| or their faith was misplaced to begin with. And that
| ain't right; they're America! So the awfulness will
| continue until moral improves!
|
| Atheist semantics while living theist like devotion to
| civil religion memes.
| switchbak wrote:
| Maybe some folks (ahem) disappearing into virtual worlds
| is a good thing for those left behind.
| koolala wrote:
| It is only detaching people from reality more. The internet
| used to be a window into the outside world and now AI is
| making it counterfeit manipulated fantasy.
| Sol- wrote:
| Dunno, I want to agree, but at the same time it's spoken like
| someone to whom these experiences and human relationship come
| easily. There are many people out there who, for some reason
| (anxiety, etc.), cannot easily access this part of the human
| condition, unfortunately.
|
| Perhaps better to roam a virtual reality than be starved in
| the real world.
| adventured wrote:
| Most people don't have access to anything particularly nice in
| the real world.
|
| It's reality privilege. Most of humanity will yearn for the
| worlds that AI will cook up for them, customized to their
| whims.
| jplusequalt wrote:
| >Most people don't have access to anything particularly nice
| in the real world.
|
| What data/metric are you drawing from to arrive at this
| conclusion? How could you even realistically make such a
| statement?
| echelon wrote:
| The more I see of this, the faster I want it to go!
|
| I'm developing filmmaking tools with World Labs' Marble world
| model:
|
| https://www.youtube.com/watch?v=wJCJYdGdpHg
|
| https://github.com/storytold/artcraft
|
| I think we'll eventually get to the point where these are real
| time and have consistent representations. I've been excited
| about world models since I saw the in-the-browser Pokemon demo:
|
| https://madebyoll.in/posts/game_emulation_via_dnn/demo/
|
| At some point, we'll have the creative Holodeck. If you've seen
| what single improv performers can do with AI, it's ridiculously
| cool. I can imagine watching entertainers in the future that
| summon and create entire worlds before us:
|
| https://www.youtube.com/watch?v=MYH3FIFH55s
|
| (If you haven't seen CodeMiko, she's an incredibly talented
| engineer and streamer. She develops mocap + AI streams.)
| jplusequalt wrote:
| >I think we'll eventually get to the point where these are
| real time and have consistent representations
|
| Just like how people in the 50s thought we would have flying
| cars and nuclear fusion by 2000.
| TacoCommander wrote:
| After a lifetime career in tech, I want to turn it all off.
| switchbak wrote:
| Seriously though. These cute distractions have turned into
| world and culture eating monsters.
| shimman wrote:
| Can't wait to see this being abused to show that "the
| prisoners were treated humanely" or that "there are no
| protests in the capital." At what point to we hold
| engineers responsible for creating these machines? How many
| must suffer because no one was willing to say no?
| moomoo11 wrote:
| I think this would be good for the people living in
| overcrowded, polluted and dirty cities in the world (let's be
| honest, they actually exist regardless of how that happened or
| why).
|
| Maybe they can unplug from 500+ AQI pollution and spend time
| with their loved ones and friends in a simulated clean world?
|
| Imagine working for 10-12 hours a day, and you come home to a
| pod (and a building could house thousands of pods, paid for by
| the company) where you plug in and relax for a few hours. Maybe
| a few more decades of breakthroughs can lead to simulated sleep
| as well so they get a full rest.
|
| Wake up, head to the factory to make whatever the developed
| world needs.
|
| (holy fuck that is a horrible existence but you know some
| people would LOVE for that to be real)
| jplusequalt wrote:
| There are posters in this very thread chain who want this
| reality to come to pass.
| moomoo11 wrote:
| I mean if I'm totally honest, it could be beneficial to me
| if something like this comes to be. Even in the developed
| world, we have a bunch of annoying people who complain/cry
| constantly about dumb things. They do that instead of doing
| something, whereas I can excuse the people trapped in
| hellholes overseas because it really isn't their fault they
| were oppressed and mistreated.
|
| They'd have their own economy and "life" and leave the rest
| of us alone. It would be completely transactional, so I'd
| have zero reason to feel bad if they do it voluntarily.
|
| If they can be happy in a simulated world, and others can
| be happy in the real world, then everyone wins!
| switchbak wrote:
| Hook it up to an always-on fle*light, and I'm sure you'd have
| millions of paying customers.
|
| Except you'll never have to leave your pod. Extract the $$
| from their attention all day, then sell them manufactured
| virtual happiness all night. It's just a more streamlined
| version of how many people live right now.
|
| I'll be running away from that hellscape, thanks.
| trenning wrote:
| Sounds similar to the black mirror ep
| https://en.wikipedia.org/wiki/Fifteen_Million_Merits
| slashdave wrote:
| Huh, yeah, the sky is blue outside and the sun is shining.
|
| Although, I am feeling a bit lazy so let me see if I can
| simulate a walk.
| alex_c wrote:
| Ironically, this brings us one step closer to believing the
| simulation hypothesis might be true... In which case, maybe
| there is no real world anyway ;)
| in-silico wrote:
| Everyone here seems too caught up in the idea that Genie is the
| product, and that its purpose is to be a video game, movie, or VR
| environment.
|
| That is not the goal.
|
| The purpose of world models like Genie is to be the "imagination"
| of next-generation AI and robotics systems: a way for them to
| simulate the outcomes of potential actions in order to inform
| decisions.
| avaer wrote:
| Soft disagree; if you wanted imagination you don't need to make
| a video model. You probably don't need to decode the latents at
| all. That seems pretty far from information-theoretic
| optimality, the kind that you want in a good+fast AI model
| making decisions.
|
| The whole _reason_ for LLMs inferencing human-processable text,
| and "world models" inferencing human-interactive video, is
| precisely so that humans can connect in and debug the thing.
|
| I think the purpose of Genie _is_ to be a video game, but it 's
| a video game for AI researchers developing AIs.
|
| I do agree that the entertainment implications are kind of the
| research exhaust of the end goal.
| in-silico wrote:
| Sufficiently informative latents _can_ be decoded into video.
|
| When you simulate a stream of those latents, you _can_ decode
| them into video.
|
| If you were trying to make an impressive demo for the public,
| you probably _would_ decode them into video, even if the real
| applications don 't require it.
|
| Converting the latents to pixel space also makes them
| compatible with existing image/video models and multimodal
| LLMs, which (without specialized training) can't interpret
| the latents directly.
| SequoiaHope wrote:
| Didn't the original world models paper do some training in
| latent space? (Edit: yes[1])
|
| I think robots imagining the next step (in latent space) will
| be useful. It's useful for people. A great way to validate
| that a robot is properly imagining the future is to make that
| latent space renderable in pixels.
|
| [1] "By using features extracted from the world model as
| inputs to an agent, we can train a very compact and simple
| policy that can solve the required task. We can even train
| our agent entirely inside of its own hallucinated dream
| generated by its world model, and transfer this policy back
| into the actual environment."
|
| https://arxiv.org/abs/1803.10122
| NitpickLawyer wrote:
| > I think the purpose of Genie is to be a video game, but
| it's a video game for AI researchers developing AIs.
|
| Yeah, I think this is what the person above was saying as
| well. This is what people at google have said already (a few
| podcasts on gdm's channel, hosted by Hannah Fry). They have
| their "agents" play in genie-powered environments. So one
| system "creates" the environment for the task. Say "place the
| ball in the basket". Genie creates an env with a ball and a
| basket, and the other agent learns to wasd its way around,
| pick up the ball and wasd to the basket, and so on. Pretty
| powerful combo if you have enough compute to throw at it.
| koolala wrote:
| What model do you need then? If you want 3D real-time
| understanding of how realities work? Are you focusing on
| "imagination" in a different abstract way?
| thegabriele wrote:
| Sure, but at some point you want humans in the loop i guess?
| thegabriele wrote:
| Sure, but at some point you want humans in the loop i guess?
| ACCount37 wrote:
| If you train a video model, you by necessity train a world
| model for 3D worlds. Which can then be reused in robotics,
| potentially.
|
| I do wonder if I can frankenstein together a passable VLA
| using pretrained LTX-2 as a base.
| empath75 wrote:
| I am not sure we are at the "efficiency" phase of this.
|
| Even if you just wire this output (or probably multiples
| running different counterfactuals) into a multimodal LLM that
| interprets the video and uses it to make decisions, you have
| something new.
| abraxas wrote:
| > if you wanted imagination you don't need to make a video
| model. You probably don't need to decode the latents at all.
|
| Soft disagree. What is the purpose of that imagination if not
| to map it to actual real world outfcomes. For this to compare
| them to the real world and possibly backpropagate through
| them you'll need video frames.
| sailingparrot wrote:
| > you don't need to make a video model. You probably don't
| need to decode the latents at all.
|
| If you don't decode, how do you judge quality in a world
| where generative metrics are famously very hard and
| imprecise? How do you go about integrating RLHF/RLAF in your
| pipeline if you don't decode, which is not something you can
| skip anymore to get SotA?
|
| Just look at the companies that are explicitly aiming for
| robotics/simulation, they *are* doing video models.
| dyauspitr wrote:
| That's part of it but if you could actually pull out 3D models
| from these worlds, it would massively speed up game
| development.
| avaer wrote:
| You already can, check out Marble/World Labs, Meshy, and
| others.
|
| It's not really as much of a boon as you'd think though,
| since throwing together a 3D model is not the bottleneck to
| making a sellable video game. You've had model marketplaces
| for a long time now.
| echelon wrote:
| > It's not really as much of a boon as you'd think though
|
| It is for filmmaking! They're perfect for constructing
| consistent sets and blocking out how your actors and props
| are positioned. You can freely position the camera, control
| the depth of field, and then storyboard your entire scene
| I2V.
|
| Example of doing this with Marble:
| https://www.youtube.com/watch?v=wJCJYdGdpHg
| avaer wrote:
| This I definitely agree with, before you had to massage
| the I2I and now you can just drag the camera.
|
| Marble definitely changes the game if the game is "move
| the camera", just most people would not consider that a
| game (but hey there's probably a good game idea in
| there!)
| echelon wrote:
| Whoa, whoa, whoa. That's just one angle. Please don't bin that
| as the only use case for "world models"!
|
| First of all, there are a variety of different types of world
| models. Simulation, video, static asset, etc. It's a loaded
| term, just as the use cases are widespread.
|
| There are world models you can play in your browser inferred
| entirely by your CPU:
|
| https://madebyoll.in/posts/game_emulation_via_dnn/ (my
| favorite, from 2022!)
|
| https://madebyoll.in/posts/world_emulation_via_dnn/ (updated,
| in 3D)
|
| There are static asset generating world models, like WorldLabs'
| Marble. These are useful for video games, previz, and
| filmmaking.
|
| https://marble.worldlabs.ai/
|
| I wrote open source software to leverage marble for filmmaking
| (I'm a filmmaker, and this tech is extremely useful for scene
| consistency):
|
| https://www.youtube.com/watch?v=wJCJYdGdpHg
|
| https://github.com/storytold/artcraft
|
| There are playable video-oriented models, many of which are
| open source and will run on your 3080 and above:
|
| https://diamond-wm.github.io/
|
| https://github.com/Robbyant/lingbot-world
|
| There are things termed "world models" that really shouldn't
| be:
|
| https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0
|
| There are robotics training oriented world models:
|
| https://github.com/leggedrobotics/robotic_world_model
|
| Genie is not strictly robotics-oriented.
| in-silico wrote:
| The entertainment industry, as big as it is, just doesn't
| have as much profit potential as robots and AI agents that
| can replace human labor. Just look at how Nvidia has pivoted
| from gaming and rendering to AI.
|
| The other examples you've given are neat, but for players
| like Google they are mostly an afterthought.
| echelon wrote:
| Robotics: $88B TAM
|
| Gaming: $350B TAM
|
| All media and entertainment: $3T TAM
|
| Manufacturing: $5T TAM
|
| Roughly the same story.
|
| This tech is going to revolutionize "films" and gaming. The
| entire entertainment industry is going to transform around
| it.
|
| When people aren't buying physical things, they're
| distracting themselves with media. Humans spend more time
| and money on that than anything else. Machines or
| otherwise.
|
| AI impact on manufacturing will be huge. AI impact on media
| and entertainment will be huge. And these world models can
| be developed in a way that you develop exposure and
| competency for both domains.
|
| edit: You can argue that manufacturing will boom when we
| have robotics that generalize. But you can also argue that
| entertainment will boom when we have holodecks people can
| step into.
| in-silico wrote:
| The _current_ robotics industry is $88B. You have to take
| into account the potential _future_ industry of general
| purpose robots that replace a big chunk of blue-collar
| work.
|
| Robots is also just one example. A hypothetically
| powerful AI agent (which might also use a world model)
| that controls a mouse and keyboard could replace a big
| chunk of white-collar work too.
|
| Those are worth 10's of trillions of dollars. You can
| argue about whether they are actually possible, but the
| people backing this tech think they are.
| benlivengood wrote:
| Agreed; everyone complained that LLMs have no world model, so
| here we go. Next logical step is to backfill the weights with
| encoded video from the real world at some reasonable frame rate
| to ground the imagination and then branch the inference on
| possible interventions (actions) in the near future of the
| simulation, throw the results into a goal evaluator and then
| send the winning action-predictions to motors. Getting timing
| right will probably require a bit more work than literally
| gluing them together, but probably not much more.
| reactordev wrote:
| Still cool though...
| hatmanstack wrote:
| this.
| slashdave wrote:
| This is a video model, not a world model. Start learning on
| this, and cascading errors will inevitably creep into all
| downstream products.
|
| You cannot invent data.
| 2bitencryption wrote:
| Given that the video is fully interactive and lets you move
| around (in a "world" if you will) I don't think it's a
| stretch to call it a world model. It must have at least some
| notion of physics, cause and effect, etc etc in order to
| achieve what it does.
| slashdave wrote:
| No, it actually needs none of that.
| whytaka wrote:
| They have a feature where you can take a photo and create a
| world from that.
|
| If instead of a photo you have a video feed, this is one step
| closer to implementing subjective experience.
| kingstnap wrote:
| Related: https://arxiv.org/abs/2601.03220
|
| This is a paper that recently got popular ish and discusses
| the counter to your viewpoint.
|
| > Paradox 1: Information cannot be increased by deterministic
| processes. For both Shannon entropy and Kolmogorov
| complexity, deterministic transformations cannot meaningfully
| increase the information content of an object. And yet, we
| use pseudorandom number generators to produce randomness,
| synthetic data improves model capabilities, mathematicians
| can derive new knowledge by reasoning from axioms without
| external information, dynamical systems produce emergent
| phenomena, and self-play loops like AlphaZero learn
| sophisticated strategies from games
|
| In theory yes, something like the rules of chess should be
| enough for these mythical perfect reasoners that show up in
| math riddles to deduce everything that *can* be known about
| the game. And similarly a math textbook is no more
| interesting than a book with the words true and false and a
| bunch of true => true statements in it.
|
| But I don't think this is the case in practice. There is
| something about rolling things out and leveraging the results
| you see that seems to have useful information in it even if
| the roll out is fully characterizable.
| slashdave wrote:
| Interesting paper, thanks! But, the authors escape the
| three paradoxes they present by introducing training limits
| (compute, factorization, distribution). Kind of a different
| problem here.
|
| What I object to are the "scaling maximalists" who believe
| that if enough training data were available, that
| complicated concepts like a world model will just
| spontaneously emerge during training. To then pile on
| synthetic data from a general-purpose generative model as a
| solution to the lack of training data becomes even more
| untenable.
| pizzafeelsright wrote:
| Environment mapping to AI generated alternative outcomes is the
| holodeck.
|
| I prefer real danger as living in the simulation is derivative.
| whytaka wrote:
| I think this is the key component of developing subjective
| experience.
| cyanydeez wrote:
| Like LLMs, though: Do you really think a simulation will get
| them to all the corner cases robots/AI needs to know about, or
| will it be largely the same problem -- they'll be just good
| enough to fool the engineers and make the business ops drool
| and they'll be put into production and suddenly we'll see in a
| year or two stories about robots crushing peoples hands,
| stepping in drains and falling over or falling off roofs cause
| of some bizarre miscommunication between training and reality.
|
| So, like, it's very important to understand the lineage of
| training and not just the "this is it"
| oceanplexian wrote:
| Yeah and the goal of Instagram was to share quirky pictures you
| took with your friends. Now it's a platform for influencers and
| brainrot; arguably it has done more damage than drugs to
| younger generations.
|
| As soon as this thing is hooked up to VR and reaches a tipping
| point with the general public we all know exactly what is going
| to happen. The creation of the most profitable, addictive and
| ultimately dystopian technology Big Tech has ever come up with.
| ceejayoz wrote:
| The good news is we'll finally have an answer for the Fermi
| Paradox.
| analog8374 wrote:
| If creating an infinite world is so trivially easy (relatively
| speaking) then occam suggests that this world is generated.
| avaer wrote:
| I don't know if that's the simplest explanation, considering
| how insanely complex the generation is (these world models
| might literally be the most complex things to ever be created).
|
| But I do think it's a partial existence proof.
| analog8374 wrote:
| One big simplifier is to only render what you're looking at.
| I wonder how one might demonstrate that.
| adventured wrote:
| This is as good of a place to mark it as any.
|
| Humanity goes into the box and it never comes back out. It's
| better _in there_ than it is _out there_ for 99% of the
| population.
| speak_on wrote:
| Compared to DeepMind's Genie 3 demo, this appears to have more
| morphing issues and less user interactivity with environmental
| consistency. Is this a stripped down version?
| binsquare wrote:
| It's ability to simulate physics intact is actually a huge
| breakthrough.
|
| I can't even fathom what it would be like for the future of
| simulation and physical world when it gets far more accurate and
| realistic.
| Bjorkbat wrote:
| Ironically the physics are kind of my biggest criticism. They
| call these "world models", but I think it's more accurate to
| call them "video game models" because they employ "video game
| physics" rather than real world physics, among other things
|
| This is most evident in the way things collide.
| binsquare wrote:
| It's getting better staggeringly fast, just a year ago I
| wouldn't expect it to be at even video game physics level so
| quickly.
|
| If there is a possibility where it continue to improve at a
| similar rate with llms. A way to simulate fluid dynamics or
| structural dynamics with reasonable accuracy and speed can
| unlock much faster pace of innovation in the physical world.
| (And validated with rigorous scientific methods)
| gambiting wrote:
| >>How we're building responsibly
|
| How are you justifying the enormous energy cost this toy is
| using, exactly?
|
| I don't find anything "responsible" about this. And it doesn't
| even seem like something that has any actual use - it's literally
| just a toy.
| bpiche wrote:
| This is the plot of The Peripheral, right? Love the way the
| second half of that book turned out. Never finished Agency..
| mupuff1234 wrote:
| If only Google had the technology for game streaming... Oh wait
|
| RIP Stadia.
| WhereIsTheTruth wrote:
| Stadia was lightyears ahead, but pro-Microsoft media
| assassinated it with FUD
|
| While "journalists" were busy bootlicking a laggy 720p Android
| only xCloud beta, Stadia was already delivering flawless
| 4K@60FPS in a web browser
|
| They killed the only platform that actually worked just to
| protect Microsoft
|
| This will be a textbook case study in how a legacy monopoly
| kills innovation to protect its own mediocrity
|
| Microsoft won't survive the century, they are a dinosaur on
| borrowed time that has already lost the war in mobile, AI, and
| robotics
|
| They don't create,, they just buy marrket share to suffocate
| the competition and ruin every product they touch
|
| Even their cloud dominance is about to end, as they are already
| losing their grip on the European market to antitrust and
| sovereign alternatives
| dominick-cc wrote:
| Finally all my anime figurines will come to life
| bigblind wrote:
| Anyone else going to try it and just keep getting a 404 page?
| artisin wrote:
| Best case, Google DeepMind cracks AGI by letting agents learn for
| themselves inside simulated worlds. Worst case, they've invented
| the greatest, most expensive screensaver generator in human
| history.
| pedalpete wrote:
| This is what we were building in 2018 with Ayvri, starting from
| 3d tiles with the aim of building a real-world view by using AI
| to essentailly re-paint and add detail to what was essentially a
| high-resolution and faster loading Google Earth (for outside
| cities, we didn't have building data).
|
| We saw a very diverse group of users, the common uses was
| paragliders, gliders, and pilots who wanted to view their or
| other peoples flights. Ultramarathons, mountain bike and some
| road-races where it provided an interactive way to visualize the
| course from any angle and distance. Transportation infrastructure
| to display train routes to be built. The list goes on.
| jlhawn wrote:
| Now I can't stop thinking about _The Experience Machine_ by Andy
| Clark. It theorizes that this is how humans navigate and
| experience the real world: Our brains generate what we think the
| world around is like and our senses don't so much directly
| process visual information but instead act like a kind of loss
| function for our internal simulations. Then we use that error to
| update our internal model of the world.
|
| In this view, we are essentially living inside a high-fidelity
| generative model. Our brains are constantly 'hallucinating' a
| predicted reality based on past experience and current goals. The
| data from our senses isn't the source of the image; it's the
| error signal used to calibrate that internal model. Much like
| Genie 3 uses latent actions and frames to predict the next state
| of a world, our brains use 'Active Inference' to minimize the gap
| between what we expect and what we experience.
|
| It suggests that our sense of 'reality' isn't a direct recording
| of the world, but a highly optimized, interactive simulation that
| is continuously 'regularized' by the photons hitting our retinas.
| shagie wrote:
| A kurzgesagt on this: Why Your Brain Blinds You For 2 Hours
| Every Day https://youtu.be/wo_e0EvEZn8 and the sources for that
| video - https://sites.google.com/view/sources-reality-is-not-
| real/
| sebasv_ wrote:
| I am stumped. Am I misreading, or are the folks at Google
| deliberately confounding two interpretations of "world model"?
| Dont get me wrong, this is really cool, and it will undoubtedly
| have its use. But what I am seeing is an LLM that can generate
| textures to be fed into a human-coded 3d engine (the "world
| model" that is demonstrated), and I fail to see how that brings
| us closer to AGI. For AGI we need "world models" as in "belief
| systems". The AI model must be able to reason about (learned)
| dynamics, which I dont see reflected in the text or video.
| littlekey wrote:
| >an LLM that can generate textures to be fed into a human-coded
| 3d engine
|
| I'm not certain but I think the LLM is also generating the
| physics itself. It's generating rules based on its training
| data, e.g. watch a cat walk enough and you can simulate how the
| cat moves in the generated "world".
| artur_makly wrote:
| let's reboot Leisure Suit Larry ;-)
| Havoc wrote:
| Are world models from the perspective of an observer in the world
| or zoomed out?
|
| Or in gaming terms do these models think FPS or RTS?
|
| Text models and pixel grid vision models is easy but struggling
| to wrap my head around what world model "sees" so to speak.
| almosthere wrote:
| So what is it doing in the real world, microwaving an elephant on
| high with 80kw every second and pouring out all the water in an
| sub-saharan African well every 4 minutes?
| user_hn_827 wrote:
| This is a fascinating project. The idea of infinite interactive
| worlds is a huge leap for gaming and simulation.
| spullara wrote:
| has the person who designed the movement control ever played a
| video game?
| throwaway314155 wrote:
| The "How we're building responsibly" section has nothing to do
| with acting responsibly. It should be called "Limitations"
| instead. Section reads LLM generated honestly.
___________________________________________________________________
(page generated 2026-01-29 23:00 UTC)