[HN Gopher] Games should have "optimise for streaming" option
___________________________________________________________________
Games should have "optimise for streaming" option
Author : mcraiha
Score : 51 points
Date : 2023-02-26 17:02 UTC (6 hours ago)
(HTM) web link (mcraiha.github.io)
(TXT) w3m dump (mcraiha.github.io)
| philippejara wrote:
| This is a good example of the title turning people off from the
| content. The content of the article is a nice intro to how some
| artifacts are created while streaming and how to deal with them,
| for sure it is interesting and useful for developers of games
| that wish to focus on streaming to build a name or community and
| lack that knowledge. There's hardly any argumentation as to why
| the general game should have this option contrary to what you'd
| expect with such a title.
| jmull wrote:
| There are always trade-offs.
|
| If you prioritize streamability you end up deprioritizing
| something else.
| AstixAndBelix wrote:
| I think this blog post only applies to simplistic time-sink
| videogames designed to seamlessly fit between the finely oiled
| cogs of capitalism.
|
| Anything story-driven and/or single-player with even a pretense
| of artistic integrity should stay as far away from wasting man-
| hours behind optimizing the graphics, UI and general aesthetic to
| please the h.264 overlords.
| vorpalhex wrote:
| Lots of streamers stream single player games and many fans
| consume those games purely via stream.
|
| Some of those viewers may have accessibility needs. All of them
| benefit from clear text on text heavy games.
| AstixAndBelix wrote:
| This sounds like PR speak. Games are meant to be played,
| optimizing for watching should be at the bottom of any
| developer's to-do list. Some minor quirks with noise or the
| UI are not enough to justify pouring God knows how many hours
| to create alternative viewing modes and rendering options.
|
| Streaming optimization is missing the forest for the trees,
| namely that in a world with better internet connections not
| only would streaming quality be solved but also every other
| problem with slow internet would be too
| joseda-hg wrote:
| There's plenty of games I've enjoyed exclusively via
| streaming
|
| Like TloU (Both) or Some Assasins Creed Games I did
| eventually buy most of those games, and maybe in a future I
| shall play them, but in the mean time they got a sale out
| of me, and to care more for future releases (Or even
| merchandise)
| AstixAndBelix wrote:
| And did you give a single thought about some occasional
| video compression artifacts?
| joseda-hg wrote:
| Not really, Maybe on Dark scenes or particle heavy
| enviroments, which being fair, are a bit more common on
| history driven games
|
| But, considering the lengths most streamers go to have
| everything on ultra because more quality really doesn't
| really hurt either, it doesn't seem like it'd be an
| unwelcome change
|
| I'd love some of the things sugested here, like a
| separate stream so text doesn't get lost to compression
| or changes like that
| vorpalhex wrote:
| It's easier to disable an effect (it's usually an engine
| boolean) than it is to run hundreds of thousands of miles
| of fiber optic cable. We can disable an effect in a patch
| cycle - and meaningfully improve accessibility for players.
|
| Literally millions of people watch streamers play games.
| There are, at the time of me writing this comment,
| 4,800,000 people watching twitch right now (according to
| TwitchTracker) - that excludes Youtube Live, Facebook Live,
| Steam, etc. Many games (Five nights at Freddy's, Minecraft,
| Fall Guys) owe their extreme popularity to streamers. The
| issues here apply even to recordings of your game.
|
| If you are ignoring streamers, you are ignoring a major
| part of your base, and an extremely public part of your
| base at that. You're also actively telling your players who
| need accessibility tools that you don't value them enough
| to invest some time over basic settings tweaks (that you
| should probably have anyways...).
| brucethemoose2 wrote:
| I dont think grain is central to the "artistic integrity" of a
| game. I turn it off in every Mass Effect run with zero regrets.
|
| And games look _better_ if other high frequency effects, like
| texture shimmering or shadow flicker, are denoised away.
| brucethemoose2 wrote:
| Running a denoising filter through ReShade or ffmpeg (or better
| yet with VapourSynth post processing for non live streaming) can
| help.
|
| In fact, hardware encoders have excellent built in denoising (and
| other postprocessing) functions. But they are so obscure that
| less than a dozen streamers on the planet are probably using
| them... I cant even remember the syntax now, I would have to go
| look it up in Staxrip and then see if it even works in OBS :/
| suzumer wrote:
| Don't the discrete cosine transforms that underlie most video
| codecs automatically remove noise, as noise is generally high
| frequency?
| psychphysic wrote:
| Don't think so, I read somewhere (like HN) that Netflix
| purposefully removes noise, transmits it and readds it
| locally saving some 30% bandwidth.
|
| And the HBO intro is a nightmare.
| [deleted]
| brucethemoose2 wrote:
| Not as well as more computationally expensive algorithms that
| take more frames/wider areas into account.
|
| And as the blog mentions, noise "tricks" the encoder into
| blowing its bitrate budget on preserving noise that looks
| like detail, instead of something more useful.
| bob1029 wrote:
| The DCT is lossless. It's the subsequent quantization step
| that chops certain frequency components.
| ec109685 wrote:
| How is it possible for those encoders to know that fog is less
| important than other elements of the game scene?
|
| It seems like removing the elements will ensure bits aren't
| wasted on those elements.
| brucethemoose2 wrote:
| It can't, but I have learned that you cant rely on devs to
| put needed visual features into games, especially old
| already-released games. And a mild denoising prefilter is a
| reasonably good universal way to help the hardware encoder,
| and they tend to be much better at preserving detail than the
| encoder itself.
| remexre wrote:
| I bet it'd be more work on the side of the game developer and
| streaming platform, but IMO Zoom, Google Meet, Jitsi, etc. aren't
| a ton better at streaming text than it looks like Twitch is from
| TFA. Even in the absence of effects (just small text on a monitor
| of differing resolution), things can be plenty hard to read.
|
| I suspect that it should be possible for at least the streaming
| platforms that have native applications to tie into the OS's
| screen-reading APIs to grab text and provide it out-of-band, e.g.
| by streaming rectangles of the background color and render text
| over it on the viewer's client. Then it'd be copy-pastable by the
| viewer too, and their own assistive technologies could work on
| it.
| bentcorner wrote:
| > _but IMO Zoom, Google Meet, Jitsi, etc. aren 't a ton better
| at streaming text than it looks like Twitch is from TFA_
|
| Come to think of it, it's a little bonkers that sharing a
| desktop over a meeting doesn't leverage any of the stuff that
| is in remote desktop and similar protocols - why send
| screenshots of text when viewer's PCs are capable of rendering
| the same text? I suppose the status quo is easier to implement
| but there'd be a measurable improvement in text quality and
| bandwidth usage if a different approach were taken.
| jeroenhd wrote:
| That's actually something I never really thought about but
| makes total sense.
|
| I think it's because applications are rendering their own
| text directly to the GPU these days. Back in the day, you
| used to use standard OS controls for everything which allow
| you to do exactly this without any direct attachment to the
| underlying rendering layers, but these days most applications
| are either Chromium with an icon or something rendered to a
| GPU canvas (Flutter style) to get easy animations for cheap,
| each with their own GPU acceleration system.
|
| RDP has several modes (block updates, video streams, you name
| it) and switches between them on the fly. Windows especially
| uses the win32 render hooks to significantly reduce bandwidth
| necessary to run native applications, integrating in the
| window event loop mechanism.
|
| I don't think browsers or streaming tools can leverage that
| tech as easily and even if they could I expect the benefit to
| be diminishing over the coming years. This understated use
| case for natice applications is often forgotten and at this
| point I don't expect the industry to move back to the old way
| of writing applications.
| brucethemoose2 wrote:
| Video codecs can be extremely good at encoding static
| content, with the right tuning, and trying to capture the
| text of _any_ arbitrary app on the screen would be quite a
| task.
|
| One of the bigger issues for text is 4:2:0 subsampling... not
| really sure how to work around that, as higher levels of
| subsampling are extremely expensive.
| dilyevsky wrote:
| Is it really chroma? I always thought poor text rendering
| was an artifact of dct+quantization
| brucethemoose2 wrote:
| Depends. The chroma subsampling still hurts text even at
| really high bitrates or in static scenes, and you can see
| a big difference in 4:4:4 or RGB videos.
| georgeecollins wrote:
| As a lifetime game developer publisher, I am going to disagree. I
| feel like half my career people have been trying to push streamed
| games. In the oldie days it was a company called OnLive. Then it
| was a bunch of other services including Stadia, Luna etc.
|
| Game developers don't want this. Gamers generally don't prefer it
| either, or they can be sold on the idea but consistently reject
| the experience. Why is this such a thing? It's because business
| people know that if they can convert game buyers into people who
| pay for streaming service they can play gate keeper and extract
| all the surplus value. That's not a priority for me!
| Hyption wrote:
| It's about watching others play.
| procinct wrote:
| Isn't the article talking about live streaming like on Twitch?
| As opposed to game streaming over a system like Stadia
| goosedragons wrote:
| I think it is. It still seems like a minority situation to
| me. How much of a player base is streamers that it makes
| sense to optimize games for Twitch? Wouldn't it make more
| sense for Twitch to optimize video codecs for games in mind
| instead?
| Brybry wrote:
| The viewership and exposure level is so great on
| Twitch/Youtube/Facebook/etc that games already have
| "streamer modes" that hide information to, for example,
| help prevent stream snipers (including big games like Call
| of Duty).
|
| Streaming is big. Making your game look good when streamed
| is probably worthwhile. Arguably some game genres and
| trends wouldn't even exist if not for streaming.
|
| Battle Royale games were based on a little known ARMA mod
| that was popularized by streamers like Lirik which lead to
| more people playing them which lead to more games like that
| being made.
| brucethemoose2 wrote:
| A small fraction of players, but its a critical fraction to
| market a game. Many games already have a "streamer mode"
| that disables licensed music among other things.
| brucethemoose2 wrote:
| This is confusing me as well. I thought it was about game
| "streaming," but after reading the comments I'm thinking
| maybe its about "game streaming."
|
| But the problem is fundamentally the same, as they are both
| using GPU hardware encoders to stream a game to a client.
| [deleted]
| nolok wrote:
| Then it doesn't need "optimize for streaming" it needs a
| straight to stream rather than render then re encode.
| bee_rider wrote:
| That seems like a pretty good idea generally. For a single-
| player game an extra straight-to-stream output could even
| show extra info; be third person, etc.
|
| Hell, why not include the ability to render locally, like a
| halo/StarCraft (and many other series of course, these are
| just the two I remember) replay.
| vorpalhex wrote:
| The player still has to see the game and "watching your own
| stream" introduces significant (inconsistent) latency
| making it inappropriate for any game with timing.
| [deleted]
| sovietmudkipz wrote:
| I might have misread something but I think the author means
| "streaming" to mean when you are streaming the game to an
| audience on a service like twitch. You mean it in the sense of
| playing the game on a game client that's actually rendered
| somewhere else (e.g. a data center)
| korroziya wrote:
| >Magic: The Gathering Arena is a free-to-play digital collectible
| card game developed and published by Wizards of the Coast
|
| More like "Free to play" if you're ok with getting creamed by
| people shoving cash into the game for packs and drafts. MTGA is
| as rigged as every other game is these days: power for money.
| b1ue64 wrote:
| If you actually look for good games and don't just play the
| latest AAA games made by big companies, you'll realize there's
| still good games out there
| toast0 wrote:
| Free to play strongly implies pay to win. If it were just free,
| they'd say that.
| procinct wrote:
| It often doesn't. Plenty of competitive games are free to
| play and make their money from cosmetics and battle passes.
| I'm not a huge fan of those things either but I don't think
| you could call LoL pay to win for example.
| mastax wrote:
| The noise thing is interesting. Modern games tend to have lots of
| noise. There's often a noise or "film grain" post processing
| filter to make the game look less artificial. Lighting effects
| often have noise injected at many stages to soften and liven them
| up. This was really noticeable in _Control_ for example, I think
| it may have been a surrealist stylistic choice. Modern video
| codecs (and maybe H.264 as well) have the ability to denoise to
| improve compression quality and then add noise /film grain back
| in after decoding. Theoretically the game engine could integrate
| with this by producing a noise map of sorts to add to the video
| stream.
| brucethemoose2 wrote:
| Only AV1 has this, and its a pretty niche feature at the
| moment.
|
| And a "noise buffer" is an interesting concept. I dunno if any
| other applications try to do this with AV1, or if thats even
| possible without major hacks (as its noise synthesis is
| basically designed for internal use).
| Animats wrote:
| This is for streaming low-end games on mobile, apparently. Are
| they thinking Amazon Luna?
|
| Game streaming as a service hasn't worked out well. It's too
| expensive a service to provide, since each user needs a rackmount
| gamer PC in the data center when on . Most of the streaming
| services of a few years ago went bust. Vortex closed in 2022 and
| Google Stadia closed recently. Shadow PC pivoted to B2B. NVidia
| GeForce Now doubled their prices.
|
| Technically, it works fine, but doing it profitably seems to
| require pricing around $30/month, and that's not selling.
|
| Should film grain should be removed from streaming movies, too?
| Some systems do that, and re-insert it in the player.
| brucethemoose2 wrote:
| > Should film grain should be removed from streaming movies,
| too?
|
| Yes! I hate film grain in Blu Rays, but it looks particularly
| awful in streams. And there are far better ways to fix banding
| if thats the goal of adding grain.
|
| I dont think many are leveraging AV1's grain synthesis yet.
| chitowneats wrote:
| This is not about streaming a game running on a remote machine
| for personal use. It's about streaming a recording of you
| playing and commentating to viewers. This is a big part of
| modern gaming culture that video game producers are not
| optimizing for. Some of the best free advertisement you can get
| for a game you develop is having a popular Twitch or YouTube
| streamer play it for their audience. It makes sense for all
| parties involved to support this workflow.
| galkk wrote:
| Honestly, this is not a secret that game companies are paying
| popular streamers to stream their games, especially on launch
| chitowneats wrote:
| Right. Paid advertisement is the next best thing :)
|
| Especially since they're already paying the streamers to
| play their games, it makes sense to provide an optimized
| game rendering mode for this.
| cultofmetatron wrote:
| given the similar requirements and higher margins, I'm
| surprised streaming CAD as a service never took hold. Imagine a
| service where you can model CAD parts and run simulations. the
| frames are rendered in high fidelity and streamed. latency
| isn't as much of a hard requirement and you can charge more per
| user in such a situation.
| Animats wrote:
| That's what Autodesk Fusion 360 is, sort of. It's partly
| local and partly remote.
| cultofmetatron wrote:
| rendering is still done remote. the dream of stadia or
| geforcenow is that the whole program runs on a remote
| machine and you'd justs stream the interface
| [deleted]
| ThrowawayTestr wrote:
| I think the article is about streaming games on Twitch and
| YouTube.
| jameshart wrote:
| I guess if game publishers care about how their content will
| appear in streaming channels they might adjust their graphical
| approaches to avoid things that won't compress well.
|
| Similar to how pop music production values changed as kids
| started listening to music over phone speakers.
|
| Or how _failing_ to account for how some images handle
| compression led to HBO shipping those unwatchably dark Game of
| Thrones episodes. I guess directors targeting streaming nowadays
| know not to do that?
|
| But I don't think this passes as normative advice for all game
| developers. _If_ as a game dev you value how your game looks on
| Twitch, you might want to think about some of the graphical
| flourishes you include.
|
| On the other hand if, as a game dev, you want people to
| experience your game personally and you want to discourage
| streaming consumption, maybe you crank these features up to the
| max so your game looks terrible streaming. Entirely up to you how
| you use this knowledge, really.
|
| Saying games "should" optimize for this is like saying back in
| the 80s/90s that movie directors "should" frame their shots so
| they can be cropped down to 4:3 because otherwise their movie
| will look bad on VHS and TV. Sure, some directors followed that
| approach; others went 'screw that' and framed shots wide so they
| were basically impossible to pan and scan without losing content.
| Macha wrote:
| I'm not sure streamers will opt for modes that prioritise the
| video encoder over their own experience though. Look at all the
| esports streamers that play the game on the lowest, ugliest
| graphics settings because the increased performance helps their
| reaction time or lowered effects help them focus. But if you
| watch e.g. tournaments organised by the game studio, they'll
| put the graphics settings on max to maximise the visual appeal
| over gameplay benefits.
| brucethemoose2 wrote:
| "Or how failing to account for how some images handle
| compression led to HBO shipping those unwatchably dark Game of
| Thrones episodes. I guess directors targeting streaming
| nowadays know not to do that?"
|
| That blame lies on the service encoders. They should be
| allocating more bitrate to dark scenes, and _gently_ debanding
| them, rather than relying on decades-old x264 defaults.
|
| But to be fair this wasn't well known back in early GoT days.
| aeturnum wrote:
| I think this is a SUPER astute observation and is looking to the
| future of what people will want from games engines.
|
| I also think we can take this further - games should be setting
| up their pipelines to offer a "compression friendly" version of
| the frame as an interstitial product before producing the final
| render. That way the game produces two native frame streams: one
| that's extremely compressible and one that's got all the bells
| and whistles. Obviously that takes more horsepower but the point
| is moving towards games engines as a broadcast-aware medium.
|
| This is going to be a huge part of what makes a game accessible
| and successful going forward and the people who make tools for
| games will benefit from taking that seriously.
| vsareto wrote:
| It would be nice to have, but probably too much dev/design/QA
| effort for the payoff. Then you have to consider the confusion of
| presenting two different appearances of the game to the audience.
|
| This seems like it should be solved by streaming services, not
| game developers.
| vorpalhex wrote:
| More games are already building a "streaming mode" where the UI
| is shifted, any passwords (eg multiplayer lobbies) are
| redacted, etc.
|
| Many games rely on streamers as a kind of marketing.
|
| Adding effect tweaks into streamer mode isn't a bad idea.
| jayd16 wrote:
| It's probably not a big deal if it's just another visual
| quality setting. You just make another set of LODs that remove
| some noisy effects.
| CyanBird wrote:
| Adding a checkbox which defines if the game is on "streaming
| mode" or not could be cool
|
| What would not be cool would be the huge expense of needing
| to do two ui/Ux passes on user interface and qa that entire
| process and bugs thereof. The piece is also talking of vfx
| creating issues and these are wholly unfixable unless at
| enormous expense. At that point they would instead need to be
| disabled which would make the game look overall worse and as
| a less appealing product and as an art director I would *not*
| want my game to be streamed under said circumstances
|
| That said there are some good takeaways from it, a setting to
| disable film grain or improve font/ui readability
| avtolik wrote:
| Another important issue for the streamers/youtubers is the
| soundtrack of the games. Some tracks can cause DMCA takedown
| requests. Sometimes even automatic. Game devs have taken notice
| though. Some more recent and popular games have streamer mode
| which excludes tracks like these.
| kevincox wrote:
| To be clear the article is mostly talking about optimizing for
| video compression. However it does also have some points about
| tooltips that maybe should be always visible as a viewer can't
| choose when to activate them.
| Brybry wrote:
| So one neat thing about Twitch is that it has channel
| extensions which are basically interactive overlays.
|
| So, for example, with some card games viewers can mouse over a
| card in the stream video and get tooltips/card
| text/information.
|
| It only works for some live content and not vods though. I
| imagine storing all the required information for vods would be
| challenging.
| pr0zac wrote:
| I completely agree with the article but its a little weird that
| the example game used is MTG Arena when thanks to streaming
| extensions I find it's one of the best games to watch on Twitch.
|
| Almost every MTGA streamer I've seen uses Untapped
| (https://articles.mtga.untapped.gg/set-up-twitch-extension-fo...)
| which solves issues 2 and 3 since you can in fact just mouse over
| cards to see what they are as well as get detailed info on the
| deck being run and game state.
| thom wrote:
| This is true, but it would be nice if Arena got configurable
| backgrounds at some point even without the streaming case.
___________________________________________________________________
(page generated 2023-02-26 23:02 UTC)