[HN Gopher] Show HN: AV1 and WebRTC
___________________________________________________________________
Show HN: AV1 and WebRTC
Author : Sean-Der
Score : 62 points
Date : 2022-04-12 14:20 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Sean-Der wrote:
| I just finished adding AV1 support to Pion WebRTC. What AV1 is
| able to do is pretty amazing! I spent a lot of my time with H264
| because of legacy devices/hardware acceleration. Seeing the jump
| in quality + reduction in bandwidth usage it is going to be hard
| to go back.
| out_of_protocol wrote:
| Did you try VP9? Huge jump in quality as well. Slightly worser
| than AV1, available for ages (not apple though?)
| alphabettsy wrote:
| VP9 isn't better than HEVC though is it? And HEVC has
| hardware encoding and decoding support on most modern
| devices.
| ww520 wrote:
| HEVC is loaded with patents. VP9 is Google's response to
| avoid the minefield.
| giantrobot wrote:
| The device's hardware encoders/decoders have licenses
| already. Unless you're shipping your own you're not
| likely have to pay any royalties. If you're so successful
| the MPEG-LA comes a' knockin' you should have the money
| to pay the royalties.
|
| The whole purpose of the MPEG-LA is to be a single entity
| to approach about licenses held by a bunch of companies.
| Avoiding paying licenses means avoiding codecs well
| supported by mobile hardware.
| cma wrote:
| You end up with artificially limited numbers of encode
| streams and stuff with that, at least with nvenc (I think
| they doubled the limit at the beginning of the pandemic
| though).
| out_of_protocol wrote:
| 1. For HEVC it's not a single patent pool but many, most
| of them don't have clearly defined pricing
|
| 2. Firefox don't support hevc nor it will be in the
| future
|
| P.S. https://developer.mozilla.org/en-
| US/docs/Web/Media/Formats/V... VP9 is supported by all
| major browsers
| alphabettsy wrote:
| Sure, but few devices have VP9 hardware encoding which
| means re-encoding to view or less efficient software
| decoding.
| judge2020 wrote:
| As far as I can tell, vp9 hardware decoding _should_ be
| available[0][1], but safari will fallback to software
| decoding[2]. I can 't find much of a definitive source for
| Apple's supported codecs.
|
| 0: https://9to5mac.com/2020/06/24/apple-adds-webp-hdr-
| support-a...
|
| 1: https://en.wikipedia.org/wiki/Apple_M1#Other_features
|
| 2: https://trac.webkit.org/changeset/284523/webkit/
| simulate-me wrote:
| What is the target use-case of pion? Until AV1 support is added
| to libwebrtc, how can this be used that wouldn't be better served
| by using RTP directly?
| Sean-Der wrote:
| AV1 support is already available in libwebrtc!
|
| So when I started building Pion the target use case was to make
| it easier to build scalable servers. Instead of interacting
| with a WebRTC servers REST API to query information/load
| balance I wanted to have it all in one code base. It also is
| really useful to have Media+Transport decoupled. Lots of use
| cases I didn't realize grew out of that.
|
| * Teleoperation/robotics (https://github.com/Ragnar-H/TelloGo)
|
| * Control remote software (https://github.com/m1k1o/neko)
|
| * Cross platform file sharing
| (https://github.com/saljam/webwormhole)
|
| * Sending pre-recorded media (RTMP/HLS/RTSP -> WebRTC)
|
| * Custom DataChannel servers/bridges
| (https://snowflake.torproject.org/)
|
| Lots of other cool ones in https://github.com/pion/awesome-pion
| I need to update it. It has been a bit since I have looked
| through
| https://pkg.go.dev/github.com/pion/webrtc/v3?tab=importedby
| simulate-me wrote:
| Thanks that's helpful. Does pion support RTCP? I've tried
| sending pre-recorded media over WebRTC before. The issue I
| ran into is that I needed to send keyframes too often leading
| to a bitrate that was too high for users with slower internet
| connections. For whatever reason, not all browsers seemed to
| support keyframe intra refresh, so there were large bitrate
| spikes whenever a keyframe was sent. RTCP would have solved
| this problem by allowing clients to request additional
| keyframes when needed rather than using a high keyframe
| interval.
| Sean-Der wrote:
| Yes it does support RTCP. https://github.com/pion/rtcp
|
| We also provide https://github.com/pion/interceptor.
| Interceptor is implementations of common RTCP workflows.
| Things like Congestion Feedback, NACK generation and
| Congestion Controllers. The idea is that you can use the
| ones we provide, or use your own!
| simulate-me wrote:
| That's cool. Do you have any examples of doing real-time
| media streaming? I guess pion is just the transport
| layer; so you need a separate media library?
| Sean-Der wrote:
| What in particular are you looking for? If you want the
| 'conference use case' LiveKit, Galene and ion-SFU are all
| popular
| simulate-me wrote:
| I'm looking to take a pre-recorded video file and stream
| it to a WebRTC client. I use an SFU, but I don't think
| that's the important part. The main issue is transcoding
| the video in a format that the browser supports and in a
| way that can respond to RTCP feedback signals. For
| instance, you can generate an RTP SDP file to use as the
| output for an FFmpeg CLI command, but the FFmpeg CLI has
| no support for RTCP. You had mentioned that pion was used
| to stream pre-recorded media over WebRTC, so I was hoping
| you had an example of doing so that properly handled
| RTCP.
| Sean-Der wrote:
| For getting media out of ffmpeg you have
| https://github.com/pion/webrtc/tree/master/examples/rtp-
| forw...
|
| For playing media from disk you have
| https://github.com/pion/webrtc/tree/master/examples/play-
| fro...
|
| To see how to accept and process RTCP see
| https://github.com/pion/webrtc/tree/master/examples/rtcp-
| pro...
|
| The example-webrtc-applications repo also shows how you
| could get media from GStreamer.
| Sean-Der wrote:
| I can't seem to respond simulate-me because our comments
| are too deeply nested.
|
| Pion doesn't have a way to exactly implement your
| business logic. You have two choices (and they both can
| be implemented with Pion)
|
| * Store everything since the last keyframe and RTX to the
| user. You have access to PLIs and NACKs so you can do
| that.
|
| * Request a new keyframe from the encoder.
|
| I would love to ship something that exactly meets your
| need, but I don't think I can.
| simulate-me wrote:
| You can respond if you click the timestamp of the comment
| :).
|
| I understand that pion is a transport library, I was
| mostly wondering if you've seen anyone solve this issue.
| To give you a little background, we built an app that
| allowed a movie containing a children's story to be read
| aloud with the participants.The movie streamed from our
| servers of WebRTC. It worked well, but we found that
| users in Europe often have DSL and using a keyframe
| interval of 3 seconds was too much for their connections
| to handle. Increasing the keyframe interval lead to
| situations where users saw nothing when they first
| connected. We eventually switched to synchronized local
| playback for media, but it's much more difficult to time
| perfectly and people have noticed it's more out-of-sync
| than before.
| Sean-Der wrote:
| I am happy to help you implement either of the solutions
| I have seen (from my other comment)! Email me sean @
| pion.ly
|
| But people are successfully streaming pre-recorded media.
| You have to do out of band communication, and that will
| just have to be custom afaik :/
| Uehreka wrote:
| Just a heads up, in future conversations you may want to
| mention the DSL part at the beginning. WebRTC over DSL is
| definitely not impossible, but you're going to be dealing
| with whole classes of issues most WebRTC devs aren't
| running into on a regular basis. It's also unfortunately
| true that many modern use cases for WebRTC are only
| possible because cable, fiber, and 3G/4G/5G have
| increased the bandwidth available to most users. So it
| may be the case that what you're looking for does not in
| fact exist.
|
| But keep looking! Just make sure to mention that you're
| targeting DSL so people know not to recommend things
| you've already ruled out.
| simulate-me wrote:
| I see, so these examples still suffer from the keyframe
| issue. As far as I know, there is no good solution to
| this issue, but I'd be interested in hearing how pion's
| consumers handle this issue. Streaming a file from disk
| is ideal because it's really cheap and doesn't require
| any video transcoding, but as soon as you need to insert
| additional keyframes, then at least some portion of the
| source video needs to be transcoded again.
| HenrikB wrote:
| "AOMedia Video 1 (AV1) is an open, royalty-free video coding
| format initially designed for video transmissions over the
| Internet." [0]
|
| [0] https://en.wikipedia.org/wiki/AV1
___________________________________________________________________
(page generated 2022-04-12 23:01 UTC)