[HN Gopher] OBS Studio Lands AV1 and HEVC RTMP Streaming Support
___________________________________________________________________
OBS Studio Lands AV1 and HEVC RTMP Streaming Support
Author : mikece
Score : 161 points
Date : 2023-03-27 11:11 UTC (11 hours ago)
(HTM) web link (www.phoronix.com)
(TXT) w3m dump (www.phoronix.com)
| Torkel wrote:
| A "fun" thing here is that it isn't possible to create or open
| these streams in ffmpeg due to neither hevc or AV1 being part of
| the Adobe RTMP standard. At least as per this thread:
|
| https://trac.ffmpeg.org/ticket/6389
| ZeroGravitas wrote:
| As part of this work they've proposed updates to the spec:
|
| https://github.com/veovera/enhanced-rtmp
| Torkel wrote:
| Nice! That looks great!, didn't know about it.
|
| The hardline opposition to adding hevc etc support to rtmp
| came from ffmpeg+gstreamer afaik, and from the document it
| looks like at least ffmpeg is on board.
|
| ...Might have to brush off our broadcast software and add
| this :)
| rektide wrote:
| That veovera repo is a week old. ;)
| [deleted]
| Sean-Der wrote:
| I am heavily conflicted on this. I am happy that people get
| access to better codecs. Better compression and those that aren't
| encumbered with patents/licensing (AV1)
|
| I am sad that we are extending RTMP instead of adopting something
| better. I want things like
|
| * Codec negotiation happening at connect time, so we don't
| hardcode things
|
| * Multi-track. So users can upload multiple audio tracks, or do
| Simulcast. I would love a world where users upload the multiple
| quality levels. Then anyone could run a stream service, not just
| those that can afford transcodes.
|
| * P2P. Stream your OBS output directly to another user. It is
| frustrating to see users struggle with setting up RTMP
| servers/HLS when they want to share their video with just one or
| two other people. The other use case I really care about is co-
| streaming.
|
| * Standardized. I would like to see whatever protocol we use be
| discussed in the IETF. It frustrates me that it is just companies
| behind closed doors deciding things. If an individual/open
| source/startup has an idea or use case it is never going to make
| it into RTMP.
|
| I have been trying to add WebRTC support
| https://github.com/obsproject/obs-studio/pull/7926 so I am
| hopeful for the future.
| rcme wrote:
| Does codec negotiation really make sense? Encoding video is so
| expensive that you can only realistically support one or two
| codecs at a given time. Anything else is wasteful.
|
| > * P2P. Stream your OBS output directly to another user. It is
| frustrating to see users struggle with setting up RTMP
| servers/HLS when they want to share their video with just one
| or two other people. The other use case I really care about is
| co-streaming.
|
| The nginx RTMP plugin supports converting an RTMP stream into
| an HLS stream. It's pretty easy to set up. Even if OBS used
| WebRTC or some other p2p solution, you would still need some
| type of signaling / connection negotiation setup.
| formerly_proven wrote:
| > Encoding video is so expensive that you can only
| realistically support one or two codecs at a given time.
| Anything else is wasteful.
|
| Even consumer nVidia cards support at least three
| simultaneous encode streams, which would be perfectly
| cromulent for a relay-only streaming service (high / medium /
| phone).
| kevincox wrote:
| Although in practice you often don't want to do these
| completely separately. What you want to do is encode the
| lowest quality as a "base" then medium as extra data that
| you can add on top, then high as extra data on top of that.
| This does make the higher qualities slightly less efficient
| but requires only slightly more data to broadcast different
| qualities. It would be extra cool in P2P use cases where
| everyone will be re-sharing the low quality instead of the
| users with the best connections using a completely separate
| stream and leaving no good connections to re-share the low
| quality stream.
|
| Of course at some number of viewers in a centralized setup
| it probably does make sense to send just upload the top
| quality and make separate lower quality streams for small
| efficiency gains that add up.
| Sean-Der wrote:
| > Does codec negotiation really make sense
|
| On startup you could be connecting two clients that have
| completely different sets of codecs. What if you have a low
| powered device that only has a H264 hardware decoder? What if
| you have two desktops that prefer AV1. WebRTC lets each side
| publish what codecs they support, and their priority
|
| > It's pretty easy to set up
|
| Yea it is not bad if you are comfortable with those things. I
| want to empower users who have no idea what Linux, codecs or
| systemd is. I want to create something like 12 year old me
| would have used. I could have had a lot of fun with my
| friends if we could have done P2P.
|
| > you would still need some type of signaling / connection
| negotiation setup.
|
| I use QR codes for this.
|
| For public signaling is also cheap enough that it could be
| provided for free. I think the project name was simplewebrtc?
| Hosting RTMP/HLS for free is prohibitively expensive though.
| favsq wrote:
| >What if you have a low powered device that only has a H264
| hardware decoder? What if you have two desktops that prefer
| AV1.
|
| You recode on the server.
| Sean-Der wrote:
| You don't have a server (and in my cases you don't want)
|
| * I don't want to upload my video from my security camera
| to a remote host. It has security and privacy
| implications
|
| * I don't have the bandwidth available
|
| * I don't want to pay the bandwidth and compute costs.
|
| * I don't want to incur the extra latency
| favsq wrote:
| That's not a good solution for... almost anybody. Most
| people would rather have the remote server recode, they
| would rather not have to open ports (with some
| connections you just can't), they would rather not expose
| their IP addresses, etc.
|
| I can see your point but it has nothing to do with what
| platforms like Twitch are for.
| Sean-Der wrote:
| You wouldn't open ports, you are in the same network.
|
| I don't think P2P is the answer to everything, in the
| same manner servers aren't the answer to everything.
| MawKKe wrote:
| I've been wondering about that multi track thing w.r.t video
| live streams
|
| Wouldn't it make sense to have separate channels for distinct
| content streams (e.g game capture, face camera, overlays..) so
| that you could independently choose/optimize each channel's
| compression and thus make the whole stream more efficient?
|
| I don't know anything about compression algorithms, but I
| hypothesize that maybe now the algos would also be able to do a
| better job since the frames have less abrupt discontinuities
| (in the viewport perspective).
|
| This would also enable stream viewers's to activate/deactivate
| channels when needed. For example, when the overlay/webcam/etc
| occludes the view on layer (channel) below.
| kevincox wrote:
| > so that you could independently choose/optimize each
| channel's compression and thus make the whole stream more
| efficient?
|
| In general the compression algorithm will make an intelligent
| decision on where to spend bits. However you may gain a bit
| more accuracy here. The better win may be to set priorities
| (maybe make the scrolling text crisp right away because it
| doesn't move often anyways) but I am still skeptical that you
| would see much benefit.
|
| > enable stream viewers's to activate/deactivate channels
| when needed
|
| On the other hand this would be very cool. I can also see
| moving away from fixed aspect ratios to moving the bits
| around the screen. Maybe the webcam can be above the (non-
| video) chat rather than over top of the video. Or even just
| mapping the different streams (that I have selected) into my
| aspect ratio better (like Google Meet and other video
| conferencing tools do).
| der_rod wrote:
| Well there just isn't "something better". Yet.
|
| MoQ (Media over QUIC) development is underway at the IETF -
| with involvement from Twitch/YouTube/Facebook/Cisco/etc. - but
| will probably still take quite some time before its finalised
| and can be adopted.
| Sesse__ wrote:
| SRT is definitely "something better". It doesn't support
| codec negotiation, but it's more generally much more
| reasonable than RTMP for video ingest.
| der_rod wrote:
| Doesn't support AV1.
| Sesse__ wrote:
| SRT is transport-agnostic. I've streamed AV1 within MKV
| within SRT.
| Dylan16807 wrote:
| 1. It looks like RTMP didn't support it, and arguably
| still doesn't, since they link to an "enhanced RTMP"
| spec.
|
| 2. _" Unlike some other protocols that only support
| specific video and audio formats, SRT does not limit you
| to a specific container or codec, since it is media or
| content agnostic. SRT operates at the network transport
| level, acting as a wrapper around your content. This
| means it can transport any type of codec, resolution or
| frame rate. This is important because it can future proof
| workflows by working transparently with MPEG-2, H.264,
| and HEVC for example."_
| der_rod wrote:
| 1. Yes, it's an extension to the spec published by an
| organisation Adobe and Google are a part of. It extends
| the protocol in a backwards-compatible way.
|
| 2. SRT, in practice, is only ever used with MPEG-TS as
| the container for audio/video data, which does not
| support AV1.
| Dylan16807 wrote:
| The point is, you don't even need to extend the protocol
| for SRT. You just need to set up servers that are ready,
| just like companies are in the middle of doing for other
| techs.
|
| Which I'm sure is still significant work, but is it much
| harder to do with SRT?
| der_rod wrote:
| > You just need to set up servers that are ready
|
| The "just" is doing a lot of work here. Extending
| existing RTMP infrastructure is obviously much easier
| than spinning up an entirely different one. And you'd
| still have specify a way to use anything but MPEG-TS to
| send audio/video over SRT, which de facto does not exist
| (at least not that I can find).
|
| I agree that extending RTMP isn't ideal, but other
| solutions have their own set of limitations, problems,
| and challenges. Whether that's SRT, RIST, or WebRTC.
|
| For better and worse, RTMP and dealing with it at scale
| is well understood.
| Sean-Der wrote:
| I use WebRTC today and am happy with it! Works great for
| broadcasting from Web, Mobile and tools that support it
| (GStreamer)
|
| I have a PR open to OBS for it right now
| https://github.com/obsproject/obs-studio/pull/7926
| londons_explore wrote:
| When a PR sits for many months and has hundreds of
| comments, it probably means it's getting near to bikeshed
| territory and it would have been better to get small PR's
| in adding the groundwork and a project plan agreed with
| leads beforehand...
| Sean-Der wrote:
| Project leads created the original PR + RFC
|
| * PR https://github.com/obsproject/obs-studio/pull/7192
|
| * RFC https://github.com/obsproject/rfcs/pull/43
|
| I tried to keep the size as small as I could. It is just
| basic audio+video (no Simulcast/ICE Renegotiation etc...)
| It has been a tough process, but I am going to keep
| working on it until I can get consensus.
| tracker1 wrote:
| How many of those users are direct vs. relaying through a
| TURN server? I'm genuinely curious, as I kind of anticipate
| that the majority of users are relaying, but I guess if p2p
| requests can open router ports in the common config, that
| could work.
| londons_explore wrote:
| IPv6 users usually manage a direct connection, and that
| is ~half the world now.
|
| IPv4 users seem to manage a connection ~70% of the time
| in my experience. Remember only one peer needs to have a
| router with a suitable config for everything to work.
| toast0 wrote:
| > I would love a world where users upload the multiple quality
| levels. Then anyone could run a stream service, not just those
| that can afford transcodes.
|
| I believe both AV1 and HEVC/h.265 specify optional Scalable
| Video Coding, although I don't know if that's a common feature
| of encoders or if this integration supports it. If the stars
| align, it's straightforward to choose parts of the stream to
| throw away to fit into available bandwidth or decode capacity
| without transcoding. Of course, decoder support is also needed.
| ofchnofc wrote:
| I don't normally write this type of comment, but you are an
| absolute powerhouse and I'm so glad to see you pushing WebRTC,
| WebRTC+OBS, and playing the social game a bit to drum up
| support.
|
| WebRTC has so much potential, I have big ideas for using it in
| a number of projects, if I can ever just clone myself a couple
| of times. Thanks for your work and enthusiasm!
|
| I constantly dream of a much more simple, but also powerful OBS
| replacement, with separated components for sending inputs to a
| "compositing" server, that can then rebroadcast via WebRTC.
| Maybe with customizable templates that could be more powerful
| than OBS's scenes. With the ability to let viewers
| customize/change their views further... It's a bit much to type
| out all here, but the WebRTC-powered future is very exciting.
| zamadatix wrote:
| Even if WebRTC or something that supports all of these is added
| I still wouldn't be sad we are extending RTMP. If the next hot
| new thing requires artificially limiting the old thing then
| it's not really the next hot new thing.
| Sean-Der wrote:
| The lack of collective 'internet user' ownership makes me
| sad. Many people want to improve RTMP, but an open discussion
| doesn't exist.
|
| If RTMP was in the IETF my comment wouldn't apply anymore.
| syntaxfree wrote:
| I bought the Loom yearly subscription because it was inexpensive
| and features unlimited videos (so I can just record myself
| working for hours and trim). But I'd much prefer something that
| cleanly works in Linux.
| freedomben wrote:
| Does Loom work in linux? I'd likely pay for it if it did, but
| as a linux user I don't even bother checking that kind of stuff
| anymore :-(
| rozhok wrote:
| It does work as a browser plugin, but I don't like the UX.
| phkahler wrote:
| For the OBS fans who have been wanting a builtin equalizer for
| years, you get the 3-band one and that's it. I submitted a decent
| 8-band, asked for feedback, and after a few months it was
| rejected. They want you to use plug-ins.
|
| I'd be happy to improve the one I submitted (more bands, steeper
| rolloff,etc) but the project does not want such a feature.
|
| I'm not butthurt, I just find it odd but half expected the
| rejection going in based on what I read.
| moritonal wrote:
| Could you expand on the context? Why would an open source
| project not want what sounds like an obvious improvement?
| onion2k wrote:
| _Why would an open source project not want what sounds like
| an obvious improvement?_
|
| Every additional feature is another thing to maintain and
| test. Saying no leaves more time for _everything else_ so
| there needs to be an exceptional reason to say yes. This is
| true for all projects, not just open source.
| andreareina wrote:
| Not the GP or associated with OBS but they could conceivably
| want to not handle anything that can be (and are already
| being) done via plugins. What's an obvious improvement to you
| is new code (always a risk) doing something that's already
| solved.
| phkahler wrote:
| I suspect it's because the project is primarily focused on
| video. Another reason could be that audio effects can be an
| endless rabbit hole of constantly striving for some elusive
| perfection. Best to leave it to plugings. On the other hand,
| I think basic functionality out of the box is still a good
| idea and the 3-band doesn't quite cut it. But thats just me -
| I just used it once and EQ was the one thing I found wanting.
|
| The Cynic in me wonders if the project is funded by plug-in
| vendors and this might be a conflict! I lean towards the
| rabbit hole theory though.
| corobo wrote:
| My quick assumption is that they don't want to maintain
| code you wrote forever. If it's done as a plugin they don't
| need to worry about it.
|
| I don't know how complex the code is though, of course.
|
| E: Their response seems to be that users who would find
| this useful already use a plugin.
|
| https://github.com/obsproject/obs-studio/pull/7961
| phkahler wrote:
| >> Their response seems to be that users who would find
| this useful already use a plugin.
|
| But nee users will have to figure that out. That is one
| of the worst excuses they could have offered.
|
| BTW the code is completely self contained. Nothing to
| maintain. That's due to a clean API on their part.
| wsgeorge wrote:
| > Nothing to maintain.
|
| Never say never...
| throw0101b wrote:
| > _I suspect it 's because the project is primarily focused
| on video._
|
| Audio is 80% of video.
|
| Most people will watch to a potato-quality video if the
| person talking can be understood fairly well, but will
| probably not bother with something where the audio is
| echo-y, with wind, and sounds like it was recorded over a
| wet string, even if the image is 4K/HDR.
|
| How advanced the included audio controls is certainly up
| for debate, but it's not possible to focus on 'just' video.
| bigtex wrote:
| This is very true, as someone helping a church do there
| weekly services, audio is king. Even the best soundbar at
| maxxed volume can't overcome a service broadcast using
| the built-in microphone on an iPhone. But get the
| microphone audio plugged directly into the stream and
| that changes everything.
| someguydave wrote:
| why not let streaming software just stream? equalizing
| audio should have happened before it got to the streaming
| software
| VanTheBrand wrote:
| Have you used OBS? It's not just "streaming software"
| it's a software audio video mixer.
| ilyt wrote:
| Audio-wise it should probably support LADSPA/VST plugins
| and that's all, there is EVERYTHING audio wise available
| between those 2 formats. I think it supports VST2 currently
| ?
| PaulDavisThe1st wrote:
| OBS is not a DAW. On every (?) platform where it runs, there
| are ways to connect capable audio processing software to OBS,
| and that software already has all the capabilities you might
| want (and more).
|
| Yes, it's a bit more convenient to do it all in the same
| program, but the complexity and maintainance expansion has
| its own costs.
| albertzeyer wrote:
| Can you link the PR?
|
| But also, what's the problem with having this as a plugin?
| Isn't it better to keep the core OBS small, and provide extra
| functionality via plugins?
| phkahler wrote:
| >> Can you link the PR?
|
| https://github.com/obsproject/obs-studio/pull/7961
|
| I guess most frustrating to me was the request to format the
| code and fix the commit message as if it was on track, only
| to be rejected for "we don't want this in the first place".
|
| Plugins suck. There are a lot of people who will not bother
| to go that route. It immediately raises questions about what
| OS do they work on, do they cost money, how do I install
| them?
|
| They include compression and noise gate among others. Enough
| to clean up my microphone input. But not much in the way of
| tone control.
| capableweb wrote:
| > I guess most frustrating to me was the request to format
| the code and fix the commit message as if it was on track,
| only to be rejected for "we don't want this in the first
| place".
|
| Usually, those things are easy to check, almost a part of
| just triage where you try to cover lots of ground but just
| the surface. Takes small amount of time for the maintainer
| unless there is a lot of ground to cover obviously.
|
| But unless they explicitly signal the change is wanted, I
| wouldn't (in the future) take that as a signal that the
| change in general is wanted.
|
| As a tip, before trying to contribute any larger patches to
| FOSS, create a issue to ask if this is something they'd
| actually be willing to merge, if you chose to build it.
| Then you know for sure up front if it's worth spending time
| on.
|
| > Plugins suck. There are a lot of people who will not
| bother to go that route. It immediately raises questions
| about what OS do they work on, do they cost money, how do I
| install them?
|
| Personally, I appreciate the architecture of "small core -
| allow extensions" so I'm happy they don't expand OBS in
| that way.
|
| As well, I'd appreciate it a lot if you made what you did
| into a plugin, better audio control is wanted by a large
| set of OBS and I know many do the piping outside of OBS to
| get a proper equalizer. So if you spend time on making it
| into an extension, I'm sure you'd find lots of people
| grateful for that.
| phkahler wrote:
| >> As a tip, before trying to contribute any larger
| patches to FOSS, create a issue to ask if this is
| something they'd actually be willing to merge,
|
| Oh I completely agree. This was scratching an itch on my
| part, plus I had been under the impression they didn't
| want it. But it wasn't clear, did they not want to
| develop it themselves? Why the 3-band? So I finally sat
| down and did something and fit the submission guidelines
| to find out.
|
| Now I kinda have the bug. It might be best to make some
| kind of pipewire effects that can be used in anything (on
| linux).
| capableweb wrote:
| I'm guessing the 3-band was added way earlier when they
| accepted more code or as a test for the extensions in
| general, but I'm no OBS developer, I cannot really answer
| to that.
|
| Pipewire effects for equalizer and so much more already
| exists (personally I use Easy Effects for that), and the
| people I know using OBS are kind of stuck with Windows
| because of other reasons. Also, it's nice to have
| everything you need in one package.
|
| But, you're the author after all, you decide what you
| want to do :)
| ehutch79 wrote:
| The entire recording industry uses plugins.
|
| The majority of the time a DAW ships with things like EQs,
| compressors, etc, they are actually some kind of plugin.
|
| Plugins don't suck, your opinion is not everyone else's
| fact.
| PaulDavisThe1st wrote:
| Time to embrace contradiction.
|
| Plugins _do_ suck. But the entire recording industry uses
| plugins.
|
| Both are true. Loading arbitrary blobs of 3rd party code
| into your plugin host is a nightmare; adding arbitrary
| blobs of amazing new functionality into your plugin host
| is amazing.
| freedomben wrote:
| That's super gross, and that's the reason I so rarely
| contribute nowadays to medium to big FOSS projects. My time
| is very valuable and they treat it like it's nothing. Of
| course they need to reject some stuff and it's important to
| keep the project maintainable, but that could have been
| done a lot earlier and a lot more politely.
| gorjusborg wrote:
| > That's super gross, and that's the reason I so rarely
| contribute nowadays to medium to big FOSS projects.
|
| Reading that PR discussion, I don't see any sort of
| 'gross'-ness.
|
| The submitter submits a (presumably unrequested) PR. One
| project maintainer adds comments to bring it up to
| project standards. Later, _another_ dev clarifies that
| after discussion, they agree that it doesn 't belong in
| the core project.
|
| For anyone else out there who doesn't like 'their time
| wasted', discuss your idea with project maintainers
| before submitting a PR. Just because you think it is a
| good idea doesn't mean it fits within their vision for
| the project.
| freedomben wrote:
| I agree it's not gross from the perspective of the _team_
| , who are familiar with who can and who can't
| approve/merge PRs, but from the contributor's perspective
| (who has no idea who is and isn't authoritative) it is.
| The person requesting changes seems authoritative and it
| does seem like it's going to be merged. But there are
| multiple perspectives here, and one who invested a lot
| more effort than everyone else, and from that perspective
| it's gross.
|
| > _For anyone else out there who doesn 't like 'their
| time wasted', discuss your idea with project maintainers
| before submitting a PR. Just because you think it is a
| good idea doesn't mean it fits within their vision for
| the project._
|
| Of course. I give the same advice. But that's not a valid
| excuse for treating would-be contributors like this. You
| can reject PRs but in a more respectful and polite
| manner, without the misleading additional time-wasting
| requests. Rejection and professionalism/politeness are
| not mutually exclusive.
|
| clarification request: with scare qutoes around "their
| time wasted," does that imply that you _do_ like your
| time wasted? or that you don 't think all the effort
| spent making the PR was wasted?
| gorjusborg wrote:
| > clarification request: with scare qutoes around "their
| time wasted," does that imply that you do like your time
| wasted?
|
| You know the answer, so forgive me for not responding to
| your not-in-good-faith question.
|
| To clarify my point: If you think about this from the
| project maintainer's standpoint, the submitter is the one
| doing something wrong if they are getting angry at this
| outcome. They are speculatively doing work _assuming_
| that it is going to eventually get integrated, without
| seeking any information about its value beforehand.
| freedomben wrote:
| > _You know the answer, so forgive me for not responding
| to your not-in-good-faith question._
|
| No I don't, otherwise I wouldn't have asked. And if
| you're going to assume bad faith on my end then there's
| probably not much more to discuss.
|
| Curious, what if the maintainer had just replied with a
| "finger" emoji instead? Would that change your opinion on
| whether this was gross, or is the manner of the reply
| completely irrelevant to you? (Since you seem to be
| presuming bad faith on my part, I feel I have to add this
| disclaimer to my thought experiment. Don't assume I'm
| implying there is some sort of equivalency here with the
| response that was given and a hypothetical "finger"
| response. I am not. This is a thought experiment, not an
| implication that the maintainer acted in such a way).
|
| If so, then I would suggest that our disagreement here is
| just about what is polite and what's not, and what a
| person's social responsibility is regarding being polite.
| In my opinion there's a responsiblity on the maintainer
| to be polite in this case. If you disagree with that,
| that's just an "agree to disagree" sort of thing.
|
| If your opinion wouldn't change if the response had been
| a "finger" emoji, then I would suggest that our
| disagreement here is about whether politeness is
| something that matters.
| cs02rm0 wrote:
| _I guess most frustrating to me was the request to format
| the code and fix the commit message as if it was on track,
| only to be rejected..._
|
| That does suck, from multiple people too. Code review
| processes that put off developers from contributing and
| don't value their time, rather than helping them to
| contribute are a pet hate of mine.
| mikece wrote:
| > Plugins suck. There are a lot of people who will not
| bother to go that route.
|
| Serious people _WILL_ explore the plugins route, but one
| has to go from "What is this thing I just downloaded and
| how do I use it?" to "Okay, I've got the hang of this, I'm
| pretty good at it, but the stock tools are holing me back
| from making something better." I would venture to say that
| the majority using OBS will never make it to that second
| phase. For those who do, have a well SEOed blog post and
| YouTube/TikTok video ready to show them how to take it to
| 11.
___________________________________________________________________
(page generated 2023-03-27 23:02 UTC)