[HN Gopher] The AI magic behind Sphere's upcoming 'The Wizard of...
___________________________________________________________________
The AI magic behind Sphere's upcoming 'The Wizard of Oz' experience
Author : radeeyate
Score : 40 points
Date : 2025-04-09 13:38 UTC (3 days ago)
(HTM) web link (blog.google)
(TXT) w3m dump (blog.google)
| sema4hacker wrote:
| An actual video example would have helped, but nothing to see
| here.
| DoctorOW wrote:
| > On August 28, their work will debut at Sphere
|
| Probably isn't done yet
| CharlesW wrote:
| There are examples in this video: https://youtu.be/f01dsTigSmw
| AIPedant wrote:
| Man... I wrote a comment about how this project sounded
| tasteless and awful, but this is somehow much worse than I
| was expecting. The scene with the Wizard made me want to
| throw my tablet.
|
| Also, did they upscale the frame rate to 48fps??? Hard to
| tell from the video, but it looks like that's the case. If so
| these people are ridiculous hacks and should never be allowed
| to touch another classic film. ("24fps would be disorienting
| on such a large screen!" Maybe that's a sign this project
| should have never left the boardroom.)
| Ukv wrote:
| > Also, did they upscale the frame rate to 48fps??? Hard to
| tell from the video, but it looks like that's the case. If
| so these people are ridiculous hacks and should never be
| allowed to touch another classic film.
|
| Doesn't seem like it'd be an unreasonable choice to me if
| it's done well (I didn't notice any interpolation
| artifacts, checking through frames). People going to see a
| film at the Sphere probably aren't format purists expecting
| to see an entirely faithful and untouched version of the
| film, but rather a version adapted to take full advantage
| of the unusual medium.
| qingcharles wrote:
| I think it's hard to tell about the FPS. The video is
| recorded in 60fps, so it might be causing the recorded
| footage to look higher FPS.
| ipsum2 wrote:
| Wow, it looks better than I expected.
| mrandish wrote:
| Yikes, the parallax distortion is severe, especially when the
| camera pans. The metal wheel doesn't even look round.
| Unfortunately, the person shooting this video with their
| phone didn't zoom out to extreme wide angle to show what's
| going on in most of the field of view. The video looks like
| maybe a 60-70 degree slice out of the 165 degree Sphere
| screen. Of course, we already know the answer has to be:
| nothing. Nothing is going on in most of the screen.
| qingcharles wrote:
| This is absolutely incredible. I'm actually blown away by how
| good this looks and the technical challenges of making this
| happen. Fantastic work.
| leptons wrote:
| AI, huh? So will Dorothy have 5 fingers or 8 fingers on each
| hand?
| ohgr wrote:
| And there I was looking forward to 6 hotdogs for fingers.
| adzm wrote:
| Actually pretty neat how quickly this joke became anachronistic
| ge96 wrote:
| The NASCAR layout was pretty nuts and Anema (EDM) robot guy leaps
| off a cliff
| Xcelerate wrote:
| > "When the request came to us, I was almost jumping up and
| down," says Dr. Irfan Essa, a principal research scientist at
| Google DeepMind and director of its Atlanta lab.
|
| Wait, what? When did Google DeepMind open an Atlanta office? That
| seems like a news story in and of itself... maybe I've been under
| a rock the last few years.
| actuallyalys wrote:
| This article is kind of strange to me because it implies
| celluloid film is low quality, but it seems there's a rough
| consensus that you can can scan that size of film in to create
| good 4K video, if not higher. That's not enough for the Sphere's
| full 16K resolution, but that's more to do with it having
| extremely high resolution than the film being "tiny" or grainy.
| chungy wrote:
| There is a 4K Blu-ray of the film, I have a copy, it looks
| wonderful :)
| crazygringo wrote:
| Celluloid film from 1939 is good, but not _that_ good.
|
| I mean it's perfectly watchable on a 1080p display, but in 4K
| it's nowhere near the kind of 4K quality you get with modern
| digital cameras with modern lenses. Sure you can watch it its
| 4K scan (heck, they scanned it in 8K), but most of those pixels
| above 1080p are just showing you film grain and blur. Which is
| nice in terms of looking like it would with a real projector!
| But it's not "4K quality" in terms of resolvable detail.
| qingcharles wrote:
| I can answer this as I did some scanning work for Universal. It
| depends on a number of factors, including how the frame is
| rendered onto the celluloid (e.g. is it cropped). From my
| experience, 4K was the absolute edge you could pull from the
| best 35mm films. Most of the time it was really "3.5K" with a
| 4K sticker on it.
|
| I managed to pull 8K from some 70mm ride footage. I think it
| might have stretched to 10K if I'd had the equipment.
|
| Wizard of Oz is 3x35mm for the color scenes, though. So, in
| theory you could use some clever algorithm to almost certainly
| create 8K transfers from it by using the tiny differences
| between the three frames.
| invisible wrote:
| They showed a bit of this with a high production video of "how we
| did it" on Tuesday. The outpainting was amazing, details looked
| crisp, physics sim was pretty good, and the continuity for actors
| was good. The amount of tech they had to create and manage for
| the project is remarkable on its own.
|
| There were some perspective and lighting issues, and a few small
| glitchy artifact moments. They probably showed about 5 minutes of
| footage from various parts of the movie.
| AIPedant wrote:
| > Take the moment where the Cowardly Lion first pounces on his
| soon-to-be companions. The camera pans back and forth between the
| Scarecrow and Tin Man, with cuts to Dorothy hiding behind a tree
| in the distance. The experience at Sphere called for keeping all
| these elements together, in hyper-realistic detail.
|
| This sounds _awful,_ as if Google thinks direction and editing
| are technological limitations rather than artistic choices. This
| entire project seems fit for a culture that loves content and
| hates art. If these tasteless jokers ever do this to _Stalker_ I
| will probably riot. ("Carefully composed shots is for
| anachronistic dweebs, our society demands an Experience Like No
| Other.")
|
| > Every change, Hays notes, was made in close collaboration with
| Warner Bros., to ensure continuity with the spirit of the
| original.
|
| Considering Warner Bros. Discovery's track record is 3 years of
| selling out their own directors and writers, I would be more
| confident if Google were left to their own devices! DeepMind
| likely has more respect for the film than Warner Bros.
| management.
| skwirl wrote:
| Nobody going to the sphere is expecting the 1939 theater
| experience. And nobody has truly had that experience for 85
| years in any case. This movie is 10 years from being in the
| public domain, and would have been decades ago if not for the
| lobbying of moneyed interests. Perhaps it's time to stop
| clutching pearls and, if you don't like what they are doing at
| the sphere, just don't go see it.
| AIPedant wrote:
| I didn't say they shouldn't be allowed to do it, and the only
| thing I said about "the theater experience" was sneering at
| the idea. I am talking about the film itself, and saying that
| these edits to the film are tasteless trash. Claiming I'm
| "clutching at pearls" is a bad faith insult - my point is
| this shit fucking sucks!
|
| David Lynch used ML to remaster _Inland Empire_ , which was
| shot on a digital camcorder and was simply too dark and
| blurry. This was an excellent use of the technology. Blowing
| up _The Wizard of Oz_ for the sake of tech bros and tourists
| is a terrible use of the technology.
| happyopossum wrote:
| > as if Google thinks direction and editing are
| technological limitations
|
| "Google" isn't making the artistic decisions here - there's
| a full production staff from the studio doing that. Google
| is making what they ask for.
|
| >I am ... saying that these edits to the film are tasteless
| trash
|
| Making such a claim with zero knowledge or experience is a
| pretty bold move - how are you so confident here?
|
| While I didn't get to see the private preview shown at the
| sphere this week, I've spoken to about a dozen people who
| did and they were all very positive about it.
| bonoboTP wrote:
| > in hyper-realistic detail
|
| Last time I heard this kind of talk, they butchered Lion King
| with the CGI remake. Getting similar vibes here.
|
| Or when DVDs introduced that feature where you could switch
| camera angles. Well, turns out you don't want to do that
| because selecting the angle is part of the storytelling. So the
| feature basically died.
|
| The "bottleneck" for movies is not so much the realism. There
| are strongly diminishing returns going from filmstrips* to
| movies to color films to HD to 4K to 3D, at each step you gain
| less and less. The core is still the story, the characters, the
| worldbuilding etc.
|
| We see the same thing with video games. Until sometime in the
| 2000s, we would always ask about a new game "how is the
| graphics?", and we'd marvel at the new graphics capabilities.
| This is not really that big of a thing any more. Going from GTA
| 3 to 4 to 5 didn't increase the fun in proportion to the
| graphics quality. I mean, just look at the popularity of
| minecraft.
|
| The real problem is that the entertainment industry is bankrupt
| creativity-wise. They have no idea how to make really new
| stuff. Everything is a remake. Note that they aren't
| introducing this new medium with a new story, they are
| refurbishing an old movie.
|
| *filmstrips used to be kinda like a slideshow where you'd
| insert film into a projector and manually twist a knob to go to
| the next one, each slide showing a still frame and some text,
| telling a story. Fun times as a kid, not sure if it was as big
| in places other than the Eastern Bloc countries. Like here
| https://kultura.hu/uploads/media/default/0003/04/thumb_20396...
| kristopolous wrote:
| > The real problem is that the entertainment industry is
| bankrupt creativity-wise. They have no idea how to make
| really new stuff.
|
| It's about financial risk calculation. They aren't willing to
| take a bath on a flop. They're doing these things because
| it's financially less risky.
| mrandish wrote:
| > There are strongly diminishing returns going from
| filmstrips* to movies to color films to HD to 4K to 3D, at
| each step you gain less and less.
|
| I agree. For cinematic content viewed in a theater or in a
| living room couch context, going from analog SD to digital HD
| was huge. Going from HD (2K) to 4K can be good but the
| quality is mostly from more bits being devoted to the
| compression than from the extra pixels. Most theatrical
| digital presentation is in DCP format and still 2k and people
| think it looks great.
|
| Other video engineering things most people don't know:
|
| * Well done HDR10 (or Dolbyvision) can contribute hugely to
| image quality. For example, I'd choose a 2K movie in HDR10
| every time over the same movie in 4K without HDR.
|
| * Theatrical 3D presentation is generally pretty bad and
| should be avoided if you care about visual quality. It's
| often around half the brightness and half the resolution and
| yet costs more. Even if done ideally, the end effect doesn't
| deliver anything like how your eyes actually see a real
| environment. The details get technical but theatrical 3D
| projection is an unnatural artificially constructed effect
| that's just weird. People claiming "It looked _just like_
| reality! " just shows the power of the placebo effect and
| suggestion, because objectively measured, it's just not.
| Famous directors don't usually come out against theatrical 3D
| the way they do against some other things. The reason is 3D
| is a pricey up sell that's mostly all margin, so it generates
| serious money but, privately, they despise 3D from both a
| quality and aesthetic viewpoint.
| WalterBright wrote:
| I try to watch silent films, but I just don't like:
|
| 1. reading dialog cards, especially since I can lip read a bit
| and that's not what the characters are saying
|
| 2. the truly awful "period" music added
|
| 3. the grainy black and white
|
| I've seen some work on the 1929 film "Wings" that was done in
| high def, sound effects were added, and some small colorization
| of scenes.
|
| I'd like to take that further:
|
| 1. remove graininess and fuzziness with AI, and of course all
| scratches, etc.
|
| 2. add foley sound effects
|
| 3. replace dialog cards with dubbed dialog, lip read if possible
|
| 4. add a decent music sound track (Like what was done with
| "Metropolis")
|
| 5. colorize it!
|
| These are all very doable today, as I've seen each done in a
| small way.
|
| Why should we suffer all the limitations of silent movies? I just
| want to enjoy it, not suffer.
| cowmix wrote:
| OTOH, artistic choices were made at the time with these
| limitations in mind.
| WalterBright wrote:
| Nobody's taking anything away if you want to watch the
| original form.
|
| Besides, how many movies in the last 50 years are made in
| B+W, use title cards, have a soundtrack of someone plinking
| on a pipe organ, are fuzzy and grainy, etc?
|
| Most people won't even watch a B+W movie, let alone a silent
| one.
|
| P.S. Reddit has a subreddit for colorized photos. There is
| some amazing work done there. It really makes the photos more
| interesting. My wallpaper is showing one right now.
| mrandish wrote:
| As someone with a background in both film and video production
| technologies (as well as being a high-end home theater projection
| enthusiast) I think this project is doomed to be terrible - and
| not just on aesthetic grounds but also technically (I'll explore
| why it's technically doomed in the next paragraphs). But first,
| what _might_ be good about this? Well, it might be interesting
| seeing how well they get AI video to maintain consistency of
| elements between shots. The article is vague but, extrapolating,
| I assume they probably composited multiple establishing and
| master shots along with production stills and set schematics to
| stitch together extended master environment reference plans. Then
| 3D artists modeled, textured and lit photo-real versions of these
| extended environments to match the stitched reference material.
| Those were then used as fine-tuning to constrain the AI
| outpainting to that ground truth. It might be interesting to see
| these techniques used to extend the original WoZ 4:3 aspect to a
| 2.4:1 widescreen format. However, I think it 's going to be a
| disaster because they're extending the outpainting to encompass
| the Sphere's _extreme_ 165 degree horizontal field of view.
| Pushing that far will derail any aesthetic or technical value the
| project may have had.
|
| The biggest technical problem is a 165 degree FOV is not just
| ill-suited for theatrical storytelling, it's actively harmful
| because it significantly constrains the compositional and
| creative choices a director and cinematographer can make.
| Historically, Hollywood has experimented with a variety of wider
| (or taller) than typical cinematic formats including Omnivision,
| Circle-Vision 360, MagnaVision, Cinerama, IMAX and many others.
| Over many decades of experimentation, it became clear that, for
| cinematic storytelling, formats up to around 2.5:1 were mostly
| upside assuming the costs and space could be supported. Extra-
| wide formats like Cinerama and IMAX had creatively useful upsides
| but came with some significant downsides which could be minimized
| with careful handling. Ultra-wide formats like Omnivision,
| Circle-Vision and, now, Sphere, were primarily useful _only_ for
| theme parks and short "You Are There"-type features such as
| Disneyland's Circle-Vision Grand Canyon tour. It can be helpful
| to refer to the reference chart on this page showing the
| different FOVs in a typical theater overlaid with the SMPTE, THX
| and 20th Century Fox recommended FOVs.
| https://acousticfrontiers.com/blogs/articles/home-theater-vi...
|
| Experimentation showed that ultra-wide formats, which are more
| than double typical cinematic FOVs and originally developed for
| world fairs, are simply ill-suited to cinematic storytelling. In
| addition to audience fatigue during longer run-times, significant
| technical challenges of optical distortion, and high costs -
| perhaps the worst part was the director losing much of the
| ability to signal to the audience what's important through
| framing and composition. This signal channel between the director
| and audience usually goes unnoticed by most viewers but it's a
| profoundly important storytelling tool for directors and
| cinematographers. To be fair, I do think immersive/surround
| visual formats _can_ be useful in the context of a theme park
| attraction, amusement ride, VR headset or interactive gaming.
| They just don 't work well for cinematic storytelling - like
| Wizard of Oz. It's a good tool being used for the wrong job.
|
| Recently, I screened the "Postcards from Earth" movie at the
| Sphere. This movie was created specifically to launch the Sphere
| and is their featured demo. And they indeed struggled mightily
| with the issues I've outlined. Ultimately, they chose to mostly
| not use more than about a 60 degree slice from the center of
| their 165 degree canvas, at least for anything significant to the
| story. All that very expensive, compromise-causing extended FOV
| was relegated to ambient scenic support except for a few brief
| "stunts" where some large object would arrive from overhead or an
| edge. But even those would quickly move from being at the edges
| (and too close/big), to exist in the center 60 degrees like
| everything else. Also, Sphere content must strictly limit any
| camera panning, tilting or side dollying to avoid causing motion
| sickness.
|
| The issues I've outlined above are primarily "Production"
| problems with Ultra-Wide FOVs, however the extreme format also
| causes significant "Presentation" problems. These presentation
| problems come from the Sphere going all-in on creating such a
| large, extremely wide-angle, wrap-around presentation that fills
| the entire visual field for 17,000 seats. Unfortunately, choosing
| that "feature" as the top priority requires other important tech
| aspects of visual presentation like contrast, dynamic range and
| resolution to be significantly compromised. A key problem is that
| the wrap-around screen being 165 degrees causes it to illuminate
| itself nuking the contrast. The sides down near the horizon line
| are opposite and shining directly at each other. Another
| significant issue is that it's almost impossible to shoot or
| present real-world camera content able to fill the entire 165
| degree Sphere screen with a single natural image. As near as I
| could tell, the entire Postcards from Earth movie doesn't contain
| even a single full-screen frame that was shot with one camera.
| It's all CGI with occasional real-world camera shots composited
| into small frames within the wrap-around CGI field. This is
| because it's incredibly challenging (if not entirely impossible)
| to photograph a single image that wide and tall while keeping the
| perspective from being severely distorted. During the "Planet
| Earth"-type scenes, the director clearly had to go to great
| effort to keep any real-world object with straight lines from
| getting too big. On top of the significant cinematic,
| compositional and tech issues, the content of Postcards from the
| Earth is also weak. The story was trite, shop-worn and heavy-
| handed. The acting, music, cinematography, etc was overall weak -
| basically what I'd call "pretty good for a video game cut scene
| but certainly not AAA cinema grade."
|
| In terms of nice things to say... well, the audio presentation
| wasn't bad. By which I mean, it wasn't great but it was
| impressively good for that huge of a space dominated by a massive
| non-parabolic reflector. Basically, the massive size and unusual
| shape of the space make it extremely challenging to provide a
| decent audio field to the majority of seats. IMHO, the audio
| engineering team over-achieved in the degree to which they're
| able to address many of those challenges. There are a huge number
| of tuned speakers hidden behind the acoustically semi-transparent
| screen driven by a lot of DSP power. Very expensive and
| technically quite difficult. Unfortunately, the resulting audio -
| while technically impressive given the significant constraints,
| still isn't nearly as good as a well-tuned flagship Dolby Atmos,
| THX-certified theater in Hollywood or Manhattan. Another
| unfortunate aspect is the perforations in the screen required to
| enable the audio transparency further reduce the screen's ability
| to generate peak nits of brightness.
|
| If you want to experience today's highest fidelity theatrical
| imagery for cinematic storytelling, the Sphere isn't it. Sadly,
| it's not even close (which makes the >$100 price for a bad movie,
| presented poorly especially egregious). The best you can
| experience today is visiting one of the 31 real IMAX cinemas (out
| of 1700 IMAX branded screens (aka "LieMAX" screens)) but only
| when they are showing a movie A) Distributed in the full 15 perf
| / 70mm 2D IMAX format, and which was B) Specifically shot to
| utilize the full 1.43:1 aspect ratio of the full IMAX format.
| Unfortunately, that's a small minority of what's shown on those
| 31 screens. Most films shown on IMAX screens weren't really shot
| specifically for full 1.43 IMAX format (which is super tall
| compared to normal cinema aspect), so they're just open gate
| (unmasked) versions of films primarily framed and lit for typical
| wide format exhibition (like 2.39:1). Also, avoid anything shown
| in 3D because the 3D projection process (whether IMAX or not)
| always significantly reduces brightness, contrast and resolution
| vs 2D projection.
| imhoguy wrote:
| Maybe Jodorowsky could make his Dune adaptation finaly that way?
| They have enough painted material to feed AI.
___________________________________________________________________
(page generated 2025-04-12 23:01 UTC)