[HN Gopher] Video Live Streaming: Notes on RTMP, HLS, and WebRTC
___________________________________________________________________
Video Live Streaming: Notes on RTMP, HLS, and WebRTC
Author : vr000m
Score : 115 points
Date : 2022-06-06 16:11 UTC (6 hours ago)
(HTM) web link (www.daily.co)
(TXT) w3m dump (www.daily.co)
| fiestajetsam wrote:
| WebRTC is not the future of low latency live streaming... at
| least not outside of video conferencing. It's incredibly complex
| as a specification, has limitations and numerous issues that set
| limits in how scalable it can be. Conversely, for HTTP segment
| based formats like HLS and DASH have limits due to their design
| (and that of HTTP) which set how low latency can actually go.
|
| Where's the future? A likely candidate may come out of the "Media
| over QUIC" work in the IETF, which already has several straw man
| protocols in real world use (Meta's RUSH, Twitch's WARP). It'll
| be a few more years before we see a real successor, but whatever
| it is will likely be able to supersede both WebRTC and HLS/DASH
| where QUIC and/or WebTransport is available.
| themerone wrote:
| Corporate firewalls will be blocking QUIC until the end of
| time. Anyone implementing streaming over QUIC will have to have
| to implement an HTTP/2 fallback, probably WebRTC, but maybe we
| will get something new.
| vr000m wrote:
| In the case of QUIC, it is likely that the streaming would be
| over H/3 (HTTP3) or HTTP over QUIC. They may fallback to H1
| or H2 but typically over a long enough time, firewall rules
| become more relaxed.
| csmpltn wrote:
| > "Where's the future? A likely candidate may come out of the
| "Media over QUIC" work in the IETF"
|
| The "future" is going to be a goddamned UDP socket sending
| compressed media streams across the web. We've reached peak
| abstraction. We need to come back to first principles, instead
| of piling on more crap on-top of the browser.
| Sean-Der wrote:
| I am heavily biased toward WebRTC. Here is my take on it
| though!
|
| > It's incredibly complex as a specification
|
| What is complex about it? I can go and read the IETF drafts,
| webrtcforthecurious.com, https://github.com/adalkiran/webrtc-
| nuts-and-bolts and multiple implementations.
|
| QUIC/WebTransport _seems_ simple because it doesn 't address
| all the things WebRTC does.
|
| > has limitations and numerous issues that set limits in how
| scalable it can be
|
| https://phenixrts.com/en-us/ does 500k viewers. I don't think
| anything about WebRTC makes it unscalable.
|
| -----
|
| IMO the future is WebRTC.
|
| * Diverse users makes the ecosystem rich. WebRTC supports
| Conferencing, embedded, P2P/NAT Traversal, remote control...
| Every group of users has the made the ecosystem a little
| better.
|
| * Client code is minimal. For most users they just need to
| exchange Session Descriptions and they are done. You then have
| additional APIs if you need to change behaviors. Other
| streaming protocols expect you to put lots of code client side.
| If you want to target lots of platforms that is a pretty big
| burden.
|
| * Lots of implementations. C, C++, Python, Go, Typescript
|
| * The new thing needs to be substantially better. I don't know
| what _n_ is, but it isn 't enough to just be a little better
| then WebRTC to replace it.
| sumy23 wrote:
| Having attempted to WebRTC as a generic video transport, I
| can say that WebRTC has insurmountable problems. The two
| biggest issues are:
|
| 1) Lack of client-side buffering. This is a benefit in real-
| time communication, but it limits your maximum bitrate to
| your maximum download speed. It's also incredibly insensitive
| to network blips.
|
| 2) Extremely expensive. To keep bitrate down, video codecs
| only send key frames every so often. When a new client starts
| consuming a video stream they need to notify the sender that
| a new key frame is needed. For a video call, this is fine
| because the sender is already transcoding their stream so
| inserting a key frame isn't a big deal. For a static video,
| needing to transcode the entire thing in real time with
| dynamic key frames is expensive and unnecessary.
| vr000m wrote:
| Webrtc protocol doesn't dictate 1 or 2. Although browsers
| do implement some of their own assumptions for this. By
| default the client side buffer can be orders of 100s of
| milliseconds. this is as you pointed out tuned for real-
| time or live applications.
|
| If you're doing something like YouTube/Netflix and want to
| avoid going to a lower definition of the stream, that too
| can be tuned, albeit you'd want to use simulcast and
| implement your own player (to feed the video and audio
| frames for decoding at the pace you dictate).
| Karrot_Kream wrote:
| None of these problems are specific to WebRTC. You'll run
| into them in a WebRTC implementation, you'll run into them
| with QUIC, even with ffmpeg on the CLI you'll need to
| specify buffer sizes. As you mention these are both
| problems with livestreaming and the more you buffer, the
| less "live" your stream becomes. If you're interested in
| transmitting static videos, then why not go with HLS or
| even just making the static file available for direct
| download through HTTP instead of a live technology?
| sumy23 wrote:
| The buffer sizes in ffmpeg are more about ensuring that
| the calculated bitrate is accurate iirc than ensuring
| smooth streaming (although you need your bitrate enforced
| to guarantee smooth streaming).
| Karrot_Kream wrote:
| IIRC (it's been a bit since I've configured this), you
| can specify both codec buffers and buffers for streaming
| to smooth out issues reading from the codec output. I
| could be wrong though.
| Sean-Der wrote:
| 1.) Why can't you buffer on the client side for WebRTC?
| That sounds like a client issue (what library were you
| using?) not the protocol.
|
| 2.) I use the same tactic as HLS. Generate your video with
| a reasonable (~2 seconds) keyframe interval. When a new
| client connects start sending at the keyframe.
| sumy23 wrote:
| 1) The point of WebRTC is that it's real-time. If you
| buffer then it's not real-time.
|
| 2) Adding key frames increases the bitrate greatly which
| exacerbates problem 1.
| Sean-Der wrote:
| 1) I don't think WebRTC has a specific point. Lots of
| users came together with their use cases and was designed
| by consensus. WebRTC can (and does) have toggles around
| latency/buffering.
|
| 2.) I am not aware of a way you can no keyframes, but be
| decodable at anytime. I just have done it 'HLS Style' or
| WebRTC 1:1. Curious if anyone else has different
| solutions.
| vr000m wrote:
| Most common browser implementations use an Open GOP. That
| means an IFrame is implemented when needed. On scene
| change or when there's high motion.
|
| Only naive implementations would burst an IFrame on to
| the network, most pace them. And if needed, you could
| split your iframe into several frame intervals and decode
| them without creating a burst by bit rate.
|
| Actually a lot of webrtc implementations use 1s or 2s GOP
| length. Again depends on how much control you've on your
| pipeline. Browsers implementations do make some
| assumptions on usecase.
| solar-ice wrote:
| That is not what open GOP means. Open GOP means pictures
| can reference IDR frames other than the most recent one
| in decode order, and is a pain in the ass for various
| reasons, but is technically more efficient. You're
| referring to a dynamic GOP.
| sumy23 wrote:
| 1) WebRTC and RTP both have RT in their name. RT stands
| for real-time. If I recall correctly, the only buffer
| WebRTC has is the jitter buffer, which is used for packet
| ordering, not for ensuring that enough has buffered to
| handle bitrate spikes.
|
| 2) Yes, you either need a high keyframe interval or some
| type of out-of-band signaling framework to generate
| keyframes. WebRTC uses RTCP. A good question is why does
| WebRTC feel RTCP is necessary at all? Why not generate a
| keyframe every N seconds like you do with HLS and remove
| the complexity of RTCP entirely? The answer is that many
| clients cannot handle the bitrate at real-time speeds.
| saurik wrote:
| 1) That is a specific implementation, and has nothing to
| do with the protocol, which certainly doesn't define a
| "jitter buffer". People routinely use RTMP--which also
| has RT in the name--to transfer content to streaming
| services with _massive_ buffers at every step in the
| pipeline.
| Karrot_Kream wrote:
| > QUIC/WebTransport seems simple because it doesn't address
| all the things WebRTC does.
|
| Partially agree here, but the design of
| QUIC(/WebTransport/TCPLS) make some of the features in WebRTC
| unnecessary:
|
| 1. No need for STUN/TURN/ICE. With QUIC you can have the
| NATed party make an outbound request to a non-NATed party,
| then use QUIC channels to send/receive RDP from the sender
| and receiver.
|
| 2. QUIC comes with encryption so you don't need to mess with
| DTLS/SRTP
|
| 3. Scaling QUIC channels is much more similar to scaling a
| stateless service than scaling something heavily stateful
| like a videobridge and should be easier to manage with modern
| orchestration tools.
|
| 4. For simple, 1:1 cases, QUIC needs a lot less signaling
| overhead than a WebRTC implementation. For other VC
| configurations, a streaming layer on QUIC will probably need
| to implement some form of signaling and will end up looking
| just like WebRTC signaling.
|
| ---
|
| I just wish WebRTC wasn't so prescriptive of DTLS/SRTP. I'm
| often fiddling around with VC and video feeds on private
| networks (for example IPSec or an encrypted VPN like
| Zerotier), and having to opt into the whole CA system there
| makes it a bit of a pain. There's also the background that
| having the browser read from a video or voice source isn't
| always very low-latency even if the DTLS/SRTP comms is going
| as fast as the network can, which leads to slower glass-to-
| glass latency, though there are non-browser ways to use
| WebRTC and many language frameworks as you indicated.
|
| All-in-all small complaints for a good technology stack
| though.
| vr000m wrote:
| ICE is needed when both parties are NATes, if one party was
| not mated, we'd not need ICE in webrtc either.
|
| Agree on 2.
|
| On 3. The videobridge needs state on who is on the session
| and who to forward to. that requirement doesn't go away
| with QUIC. Unless you're thinking that the video streams
| are some kind of a named resource or object.
|
| I think the most people gripe about is SDP and it's
| prescription of negotiation and encoding. I agree that
| capability negotiation can be vastly simplified given some
| of the capabilities can be inferred later in the session.
| Karrot_Kream wrote:
| 1. Fair on ICE. But if only one party is NATed, then you
| don't need STUN (or TURN if there's CGNAT involved.)
|
| 3. Yeah I was thinking about named resources/objects. If
| you could generate them predictably, QUIC+RTP can
| simplify a lot of things.
| parineum wrote:
| I don't know much about webrtc but I do have some security
| cameras, frigate and home assistant all working together with
| rtmp streams.
|
| There are some webrtc solution for getting those streams into
| home assistant with low latency but they are... I don't know
| the word. They aren't difficult to set up because the
| instructions are very simple, however, they don't work when I
| follow them and, from reading forums, that's not uncommon. I
| have _no_ idea why it doesn't work.
|
| I don't really understand why I can't spin up a docker
| container that will take my rtmp streams and convert them to
| webrtc then hook that into home assistant.
|
| I've gathered that webrtc just doesn't work that way but why
| can't it?
| Karrot_Kream wrote:
| Heh, welcome to the world of livestreaming media. The
| reason why it's hard to create this kind of simple "stream
| in, stream out" abstraction is because most IP Voice/Video
| stacks are architected very differently than stateless net
| protocols that are popular today. IP streaming generally
| works by:
|
| 1. A signaling layer that helps setup the connection
| metadata (a layer where the sender can say they're the
| sender, that they'll be sending data to port n, that the
| data will be encoded using codec foo, etc)
|
| 2. Media streams that are opened based on the metadata
| transferred over the signaling layer that are usually just
| streams of encoded packets being pushed over the wire as
| fast as the media source and the network allows.
|
| Most IP Media stacks (RTSP, RTMP, WebRTC, SIP, XMPP,
| Matrix, etc) follow this same pattern. This is different
| than "modern" protocols like HTTP where signaling is bound
| together with data using framing (e.g. HTTP headers for
| signaling vs the HTTP request/response body for data.) This
| design makes IP media stacks especially fragile to NAT
| connectivity issues and especially hard to proxy. There are
| typically good reasons this is done (due to latency, non-
| blocking reads, head-of-line blocking, etc) but these "good
| reasons" are becoming less good as innovations in lower
| networking layers (like QUIC or TCPLS) create conditions
| that make it much easier to organize IP Media in a manner
| more similar to HTTP. Hopefully one day you'll just be able
| to take IP Media streams and "convert" or "proxy" them from
| one format to another.
| vr000m wrote:
| All the listed protocols came after HTTP. RTSP, SIP
| borrowed heavily (albeit badly in retrospect) from HTTP.
|
| I do not have all the historical context (early 90s), but
| for WebRTC, the idea was to not define any new
| protocol(s) or do a clean slate design. but rather to
| just agree on the flavors of the various protocols, and
| then to universally implement those. We already had SDP,
| RTSP, RTP, SAP, etc. And the idea was to cobble together
| the existing protocols into something everyone could
| agree on (the young companies, the old companies, etc)
|
| We ended up defining variations to the flavors that we
| already had and for the most part everything turned out
| okay (maybe the SDP plan wars did not end up where we
| wanted it, but... it was a good enough compromise).
|
| For realtime media, if we are able combine
| "locator:identifier" issue, we will be able to make media
| and signaling work inband.
| irq-1 wrote:
| WebRTC is used by phenixrts as the delivery from server to
| client. The promise of WebRTC was P2P direct connections for
| video/data transport, and server/client for coordination and
| fallback.
|
| https://phenixrts.com/en-us/faqs.html
|
| > The scalability of Phenix's platform does not come from the
| protocol itself, but from the systems built and deployed to
| accept WebRTC connections and deliver content through them.
| Our platform is built to scale out horizontally. In order to
| serve millions of concurrent users subscribing to the same
| stream in a short period of time, resources need to be
| provisioned timely or be available upfront.
|
| https://webrtc.org/
|
| > With WebRTC, you can add real-time communication
| capabilities to your application that works on top of an open
| standard. It supports video, voice, and generic data to be
| sent between peers...
| pthatcherg wrote:
| It's been almost 3 years since I first presented on
| WebTransport + WebCodecs: https://youtu.be/VD5GBLBiSxo
|
| Live streaming was a motivating example for both of those, as
| you can tell from the video. And both of them grew out of our
| efforts to make WebRTC better for live streaming.
| vr000m wrote:
| Media over QUIC is interesting. For RTP or peer to peer QUIC,
| there is more work to be done. But you will end up engineering
| many of the same things as webrtc suit of protocols (ICE --
| STUN, TURN, MULTIPLEXING, etc).
|
| QUIC and webtransport can definitely already do DASH/HLS
| without some of the protocol complexity by using the
| QuicStreams (but to use QUICs underlying features, DASH/HLS
| need to change as well).
|
| Some of us wrote a position statement in 2017, see
| https://datatracker.ietf.org/doc/html/draft-rtpfolks-quic-
| rt.... There are new documents around media ingest being
| proposed currently.
| kwindla wrote:
| I agree that RTP over QUIC [1] is closer to what we'd build
| today if we were starting from scratch than WebRTC is. (Partly
| benefiting from the lessons learned getting to WebRTC 1.0, of
| course.)
|
| It's worth noting that QUIC is also a very complex
| specification and is only going to get more complex as it
| continues through the standardization process. In parallel,
| there's ongoing work on the next generation of the WebRTC spec.
| [2] (WebRTC-NV also adds complexity. Nothing ever gets
| simpler.)
|
| My guess is that we're at least three years away from being
| able to use anything other than HLS and WebRTC in production.
| And -- pessimistically because I've worked on video for a long
| time and seen over and over that new stuff always take
| _forever_ to bake and get adoption, maybe that's going to be
| more like 10 years.
|
| [1] https://github.com/mengelbart/rtp-over-quic-draft [2]
| https://www.w3.org/TR/webrtc-nv-use-cases/
| jzer0cool wrote:
| In each of these approaches will any of these (or something else)
| also provide approaches to getting the stream saved for viewing
| again later on a remote server? I'm guessing I need some client
| which also connects to the stream and save which can then be
| served again. Or is still a solution already baked in available?
| dbrueck wrote:
| From a technical perspective, that's more or less baked into
| HLS: these are just discrete files being served over HTTP, so
| you can save the files to disk along with the manifest file and
| you have the content.
|
| In practice, there is a little more to it. For example, when
| DRM is enabled, you need some way to preserve the decryption
| keys. And for live content, the manifest file usually just
| tells the client about a sliding window of files, so you need a
| tiny bit of additional client side logic to pay attention to
| this fact.
|
| One cool thing about DASH/HLS is that you can do some pretty
| complex mixing of content - you can build a traditional TV-
| channel like experience that mixes live and prerecorded
| content, you can replace and inject ads, you can make live
| content immediately available for on-demand playback, etc.
| pavlov wrote:
| Daily's API will let you record the stream as a MP4 video file
| stored on Amazon S3 where it's immediately available after the
| live stream ends. Everything happens automatically on the same
| server that encodes the live stream.
|
| (The original post is written by Daily's CEO.)
| vr000m wrote:
| Pavlov's comment is correct. I came to add that soon the
| stream can be stored on customer's own S3. Ergo, you'd be
| able to do a call in real-time, store it on your S3 account
| and make it available for streaming.
|
| On your own S3, this would be a multipart upload.
| rexreed wrote:
| If we use Vimeo for video hosting would there be some way
| to automatically post to Vimeo or some other way to get the
| file up there and available for immediate viewing? Other
| approaches besides S3?
| pavlov wrote:
| If Vimeo offers an ingest API using either RTMP(S) or
| HLS, that would be one way to get the stream from Daily
| directly to them without any extra processing step in
| between.
| at_a_remove wrote:
| This all looks like a lot of Very New Design Decisions. I tend
| not to be an early adopter on these, although it is mostly moot
| because I am no longer in the video delivery business. I _have_
| been considering streaming sets of short films, commercials,
| trailers, and movies to friends in a live stream via a pipeline
| of comparatively hoary old tech:
|
| 1) A playlist of multiple different video files in VLC as a
| source for ...
|
| 2) OBS Studio, which produces RTMP to be consumed by ...
|
| 3) nginx, which calls ffmpeg to produce multiple birates and
| resolutions to be rebundled as HLS to be sent to ...
|
| 4) a "live TV" channel of my own specification as an input to
| JellyFin, which can be read by ...
|
| 5) various clients on Roku, Apple TV, Firestick, Chromecast
| "apps"
|
| At this point, I don't think the industry will ever really settle
| down to something manageable.
| Karrot_Kream wrote:
| If your friends have access to VLC on all of their devices, and
| have some appetite for tech, why not use an RTSP server and
| just hand them RTSP URLs as "channels"? You don't get browser
| playback, but as long as everyone has VLC or an RTSP capable
| media player they can just open the stream on whatever device
| they have.
| at_a_remove wrote:
| Said friends are not technical. They want something on their
| Roku, their Firestick, etc.
| Karrot_Kream wrote:
| Oh gotcha. Ya, then your setup sounds great, and it looks
| like ffplayout will simplify a lot of the steps!
|
| I just proposed that setup because that's what my partner
| and friends do to view livestreams of movies together or,
| specifically with my partner, our home security camera
| setup. We're all technical though and I don't offer the
| rest of my family this setup.
| at_a_remove wrote:
| I somehow ended up as Technical Guy out of the people I
| know. It can be a burden.
|
| Livestreaming is, well, "the space" exists but it largely
| occupied by solutions where the Hollywood types figured
| this time they won't be caught on the back foot, so
| they're busy policing what is sent around, so the
| homegrown stuff is largely where it is at if you want
| shove movies around, but then you're veering into "how
| technical are the recipients?" and it gets interesting.
|
| I used to run a RealServer decades ago. Video is always
| ... interesting.
| j1elo wrote:
| Why steps 2 and 3? I mean, I don't totally see the reasons for
| not consuming the video files directly with ffmpeg, instead of
| going in such roundabout way to bring the media into it.
| at_a_remove wrote:
| Well, and understand I haven't yet attempted this, but
| apparently the discontinuities between switching files has
| caused a lot of problems downstream for Jellyfin, so OBS
| Studio would exist as kind of a way to smooth out the
| transitions.
| [deleted]
| wuyishan wrote:
| I do something like this by using
| https://github.com/ffplayout/ffplayout-engine
| at_a_remove wrote:
| Thank you, that looks like that could replace steps one, two,
| and three with a single step!
| difosfor wrote:
| We're using DASH and experimenting with LL-HLS to get reliable
| and affordable (using CDNS) end to end latency of 3 seconds with
| which you can already interact live with your audience without
| too much trouble. Latencies down to about 1 second are also
| feasible already at the cost of some client side buffering. So I
| would have expected to see more information about DASH than HLS
| here. It's usable on all platforms except iPhones and Apple TV
| where we're forced to use HLS at higher Latencies for now and
| hopefully later also DASH or LL-HLS once that's reliably usable.
| See: https://liveryvideo.com
| vdnkh wrote:
| Twitch latency in low-latency mode is ~2s (you can check for
| yourself in the settings cog menu -> advanced -> video stats)
| because of "prefetch segments", which are delivered via HTTP 1.1
| streaming body responses. The client requests this segment and
| long polls while frames are delivered as they're being encoded.
| It's simpler and cheaper than WebRTC or Apple's low-latency HLS
| parts, and scales to hundreds of thousands of viewers. The
| downside is that it's not part of the HLS specification so
| support for it needs to be bespoke, but it is a proven technology
| that wasn't covered here (and to my knowledge, the most widely
| used low latency HTTP solution by # of users). Twitch has been
| using this for years.
| aidenn0 wrote:
| I'm a bit surprised to not see RTP mentioned anywhere. It was a
| major standard for real-time voice last time I was looking at
| these things, and it was starting to get some use for
| videoconferencing.
| vr000m wrote:
| A future blogpost will talk about all the networking work that
| went into optimizing for these large size participation.
|
| Webrtc uses a few protocols. RTP is very central to it, ICE and
| SDP are also very important protocol for NATs/firewalls and
| capability selections. While we use webrtc as a shortcut to
| refer to the suite of protocols and didn't call out RTP
| specifically for that reason.
| dbrueck wrote:
| Great article, thank you! Minor nit about footnote #10 (the very
| bad network failure mode): it really depends on where the
| bottleneck is and how the client has been implemented to react -
| there's a huge spectrum of client implementations out there,
| ranging from nearly dumb to those having wizard level heuristics.
|
| If it's the user's last mile connection (between e.g. their home
| and their ISP), then the big HLS/DASH/etc buffer translates into
| a lot of time to react. So clients have the option to shift quite
| low - and do so quickly - if there are some very low bandwidth
| options, in theory even switching down to an audio-only or
| nearly-audio-only stream if one is provided, and can also choose
| to be optimistic/aggressive to resume playback as soon as one
| full chunk is downloaded - and some implementations will even
| resume playback when less than a full chunk is downloaded. The
| client side logic has a lot of latitude here to balance fast
| start/resume times vs sustaining playback.
|
| When the bottleneck or failure is elsewhere, HLS can be
| incredibly durable. For extremely high profile events, for
| example, there are typically multiple CDNs involved, multiple
| sources going to independent encoders, etc. So an HLS/DASH client
| might talk to many different servers on a given CDN, as well as
| servers on alternate CDNs, and even grab what amount to being
| different copies of the stream spit out by different encoders.
| It's not uncommon for a client to be testing different CDN
| endpoints throughout playback to migrate away from congestion
| automatically.
| liuliu wrote:
| There is a new Low Latency HLS protocol:
| https://developer.apple.com/documentation/http_live_streamin...
| that claims can get down to 300ms latency.
| vr000m wrote:
| Low latency HLS is creating partial segments by bucketing 200ms
| of frames instead of 6s segments in standard HLS. Whereas in
| webrtc, the endpoint is sending the frame as soon as it is
| ready.
|
| The apples apples comparison here is 0ms (in webrtc, no send
| side buffering) vs 200ms (in low latency HLS) or 6s (in
| standard HLS). This is independent of latency of the endpoint
| from CDN or source.
|
| Another distinction is playback wait time, i.e., how quickly
| upon joining can it start rendering video.
|
| I'm assuming the full reference picture (typically, an IFrame
| or a golden frame depending on the codec) in low latency HLS is
| only available at the start of each 6s segment and not in
| partial segments. So joining a live stream, the receiving
| endpoint would have to wait at most 6s before rendering.
|
| Similarly in webrtc, it's up to the system to generate a
| reference frame at regular intervals, as low as every second.
| Or to do it reactively, a receiving endpoint can ask the sender
| to send a new reference picture. This is done via a Full intra
| request, the wait time can be as quick as 1.5 times of the
| round trip time (as new codecs can generate a new iframe
| instantaneously upon receiving a request). There's a slight cpu
| penalty for this which means that the sender getting too many
| full intra requests may typically throttle the response to 1s.
|
| So Apples to Apples comparison for wait time would be up to 1s
| for webrtc vs 6s for HLS.
| jalino23 wrote:
| real world around: HLS: 10-30s LL-HLS: 2-6s WebRTC: 0-2s. add
| couple more startup seconds depending slow is your signaling
| vr000m wrote:
| Agreed. My calculations are totally skipping
| finding/updating the manifests or exchanging the SDP
| Offer/Answers.
| liuliu wrote:
| You don't necessarily need to wait for a reference picture to
| start playback. Modern codecs all support "intra-refresh",
| which allows you to reconstruct a reference frame from a set
| of existing frames. With that, you can set periodic intra
| refresh much lower than 6s keyframe intervals.
| giantrobot wrote:
| An HLS segment can carry any number of GOPs. A GOP length of
| FPS/2 or FPS/4 will get you an I-frame pretty quickly
| allowing the GOP to be decoded. MPEG-DASH can do the same
| IIRC. So there doesn't _need_ to be a segment length delay in
| playback and typically is not.
| phlhr wrote:
| this will be the winner purely because Apple devices are the
| lowest common denominator. That is, one must support iOS and if
| iOS only supports LL-HLS but other devices optionally support
| LL-HLS then LL-HLS is the winner.
| kwindla wrote:
| [post author here]
|
| In addition to what vr000m said above, I'll just add that when
| you make HLS chunks smaller, you're reducing the leverage you
| get from HLS's core design decisions. I tried to cover some of
| this in the post.
|
| One way to think about this intuitively is that HLS and WebRTC
| are opposite ends of one important trade-offs axis.
|
| HLS is about delivering media streams in a way that scales as
| cost-effectively as possible.
|
| WebRTC is about delivering media frames at the lowest possible
| latency.
|
| These are very different goals, and given current
| infrastructure and standards it's not possible to have your
| cake and eat it too, here. That may change in the future as
| low-latency video becomes more and more important. QUIC, for
| example, is a new approaches to building out a full stack that
| works around some of the fundamental tradeoffs that exist
| today.
|
| The result is that pushing HLS segments down to 200ms is not at
| all a clear win. We'll see what happens as HLS implementations
| improve. And I should say that my brain has been warped by
| working on UDP/RTP stuff for a long time. But my bet is that
| using 200ms HLS segments is, for most real-world users, going
| to make HLS worse in every way than WebRTC would be, for the
| same use cases. (That's definitely true today with the early
| implementations of LLHLS.)
| mchusma wrote:
| I appreciated your post so thank you. I would be more
| interested in understanding why you don't see <1s HLS chunk
| sizes as working in most cases? I feel like p99 real world
| latency stuff would show some natural buffer sizes.
___________________________________________________________________
(page generated 2022-06-06 23:01 UTC)