https://github.com/obsproject/obs-studio/pull/7926 Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + For + Enterprise + Teams + Startups + Education + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} obsproject / obs-studio Public * * Notifications * Fork 6.5k * Star 43.6k * Code * Issues 431 * Pull requests 196 * Discussions * Actions * Projects 3 * Wiki * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Wiki * Security * Insights New issue Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community. Pick a username [ ] Email Address [ ] Password [ ] [ ] Sign up for GitHub By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails. Already on GitHub? Sign in to your account Jump to bottom Add WebRTC (WHIP) output support #7926 Draft kc5nra wants to merge 20 commits into obsproject:master base: master Choose a base branch [ ] Branches Tags Could not load branches Branch not found: {{ refName }} {{ refName }} default Could not load tags Nothing to show {{ refName }} default Are you sure you want to change the base? Some commits from the old base branch may be removed from the timeline, and old review comments may become outdated. Change base from Sean-Der:webrtc-output-plugin Draft Add WebRTC (WHIP) output support #7926 kc5nra wants to merge 20 commits into obsproject:master from Sean-Der :webrtc-output-plugin +3,685 -13 Conversation 33 Commits 20 Checks 10 Files changed 25 Conversation This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters kc5nra Copy link Collaborator @kc5nra kc5nra commented Dec 14, 2022 * edited by DDRBoxman Co-authored-by: Sean DuBois sean@siobud.com Co-authored-by: DDRBoxman colin@recursivepenguin.com First and foremost, thanks to the original author of the draft PR for webrtc Colin Edwards (DDRBoxman). Description This PR adds WebRTC support to OBS. WebRTC is an alternative to RTMP that is being adopted by more services. Adding WebRTC support to OBS would benefit OBS and its users greatly. These are some of the highlights. Following this I also have a FAQ for questions we have already received regarding this PR. This initial version is intentionally simplified to only support output. Later PRs will add source support. Specifically, this adds a plugin that extends support for: * WebRTC Service type with recommended settings for maximum compatibility with webrtc peers * WebRTC Output * Rust implementation of the WebRTC stack with FFI bindings Remaining tasks: * [ ] Fix-up remaining CI build environments to correctly install and build rust * [ ] Ensure that the rust static libraries are built against the correct OSX compatability version * [ ] Emit the correct dropped frames amount from peer stats * [ ] ... Motivation and Context Subsecond Latency With WebRTC we are able to achieve sub second latency. With our initial measurements we see ~120 Milliseconds from Broadcaster to Playback. We believe we can continue to bring down this number as well. image Interactive latency lets broadcasters create new experiences they couldn't before. OBS users can now create interactive experiences with their viewers. This would allow talk shows and other productions that require conversational latency to use OBS. Remove FTL from OBS FTL is a protocol that was created for Mixer. It provided sub-second latency and was used by a few Open Source projects and streaming services. The protocol has been abandoned and it would be nice to remove it from OBS. Adding WebRTC would make that possible since it provides the latency required in addition to the many benefits inherent in the WebRTC stack like encryption, network topology strategies, robust congestion control, etc. Latest in Video Compression WebRTC will let OBS users experiment with newer codecs. WebRTC is flexible in what codecs it supports so users could use H265 and AV1. AV1 users are reporting a 50% reduction in bandwidth required. Broadcast all angles WebRTC supports multiple video streams in the same session. OBS could experiment with broadcasting multiple scenes at once! Viewers could then switch between scenes or different views in the same game. Simulcast WebRTC allows broadcasters to upload multiple streams of different quality. OBS users could upload 'high', 'med', and 'low' streams themselves. This would reduce the server costs greatly, allowing more Peer-to-Peer (if you need it) With WebRTC OBS users could share their video without needing a video server at all! WebRTC can establish a P2P connection so users could send video from OBS directly to their users. No more worrying about setting up special servers to ingest and transcode. In the future we could add a WebRTC source. Then broadcasters could add WebRTC sources to their scenes. This means low latency collaboration with no backend required! How Has This Been Tested? This has currently been tested on Linux and Mac with locally run broadcast-box as well as remotely hosted WHIP endpoints. Configurations tested: * Mac OSX M1 Monterey * Debian Sid Original draft PR: #7192 Closes #7192 Types of changes * New feature (non-breaking change which adds functionality) Checklist: * [*] My code has been run through clang-format. * [*] I have read the contributing document. * [*] My code is not on the master branch. * [ ] The code has been tested. * [ ] All commit messages are properly formatted and commits squashed where appropriate. * [ ] I have included updates to all appropriate documentation. Sorry, something went wrong. 29 LGCW, zjp-CN, lapppy-git, pbnj, MGlolenstine, ZaneH, jwoertink, ludovicgueth, MrDOS, thedanfernandez, and 19 more reacted with thumbs up emoji 18 DDRBoxman, kevmo314, MGlolenstine, runiq, ZaneH, neopunisher, jwoertink, Macil, MarkBennett, pritkr, and 8 more reacted with hooray emoji [?] 28 steveseguin, null-dev, ggarber, zjp-CN, irvinebroque, kevmo314, pbnj, MGlolenstine, erlend-sh, NiklasEi, and 18 more reacted with heart emoji 19 maxstoller, steveseguin, Be-ing, zjp-CN, DDRBoxman, MGlolenstine, fasterthanlime, ZaneH, juandelacruz-calvo, jwoertink, and 9 more reacted with rocket emoji 8 Be-ing, MGlolenstine, ZaneH, GageBachik, rveitch, Sean-Der, kowalski7cc, and Congee reacted with eyes emoji All reactions * 29 reactions * 18 reactions * [?] 28 reactions * 19 reactions * 8 reactions @kc5nra Copy link Collaborator Author kc5nra commented Dec 14, 2022 FAQ How can I use this today? Multiple companies provide WebRTC Broadcast/Playback today. Cloudflare and Millicast are two popular options. Gilmesh has created and is hosting a public WebRTC server at https:// b.siobud.com/. You can run this locally for developing and testing as well https://github.com/glimesh/broadcast-box Why webrtc-rs as the WebRTC library? WebRTC has many libraries that we could have used. The first one we evaluated was Google's implementation known as libwebrtc. The time required to fetch+download would have been too much of a burden for the OBS code base. These metrics come from a Macbook M1 with a 400 Mb /s connection. Fetching the source of libwebrtc: 11 minutes 20 seconds Installing the dependencies: 3 minutes and 30 seconds Building libwebrtc: 7 minutes Size added by libwebrtc: 35 megabytes This caused us to evaluate other options. We arrived at https:// webrtc.rs/ and it had everything we needed! Additional Build time: 40 seconds Size Added: 25 Megabytes We believe we can bring the size down as well. We haven't performed any size optimizations yet. What is WHIP? WHIP is the process that WebRTC uses to establish a session between two WebRTC Agents (Client/Server in our case). The actual definition of it can be found at https://www.ietf.org/archive/id/ draft-ietf-wish-whip-05.txt All reactions Sorry, something went wrong. @kc5nra @Sean-Der @DDRBoxman obs-webrtc:: Add WebRTC output support ... 18dcd1d Co-authored-by: Sean DuBois Co-authored-by: DDRBoxman @kc5nra kc5nra force-pushed the webrtc-output-plugin branch from 06925eb to 18dcd1d Compare Dec 14, 2022 @kc5nra fixup: Use OBS alias in cmake and remove unused packages 7a6c697 tt2468 tt2468 reviewed Dec 14, 2022 View changes UI/window-basic-settings-stream.cpp Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-output.c Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-service.c Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-services-main.c Outdated Show resolved Hide resolved plugins/obs-webrtc/CMakeLists.txt Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-services-main.c Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-output.c Outdated Show resolved Hide resolved plugins/obs-webrtc/webrtc-output.c Outdated Show resolved Hide resolved @tt2468 tt2468 added Work In Progress Seeking Testers Build artifacts on CI Request for Comments More feedback & discussion is requested New Feature New feature or plugin UI/UX Anything to do with changes or additions to UI/UX elements. labels Dec 14, 2022 Sean-Der added 2 commits Dec 14, 2022 @Sean-Der Update audio/video logic to else if ... edf39a8 Instead of having two independent if statements @Sean-Der Remove webrtc-services-main ... 218663b Not needed @Sean-Der Sean-Der force-pushed the webrtc-output-plugin branch from e8a10c8 to 218663b Compare Dec 14, 2022 @davidzhao davidzhao mentioned this pull request Dec 14, 2022 WHIP Support livekit/livekit#1083 Open Be-ing Be-ing reviewed Dec 14, 2022 View changes plugins/obs-webrtc/webrtc/build.rs Outdated Show resolved Hide resolved @llspalex llspalex mentioned this pull request Dec 14, 2022 WebRTC-HTTP ingestion protocol (WHIP) AirenSoft/OvenMediaEngine#383 Open Be-ing Be-ing reviewed Dec 14, 2022 View changes plugins/obs-webrtc/CMakeLists.txt Outdated Show resolved Hide resolved kc5nra added 2 commits Dec 15, 2022 @kc5nra Fixate corrosion dependency to v0.3.1 cf6354d @kc5nra Move the bindings.h into cmake binary dir 7beb93e phillipmiller0044 phillipmiller0044 reviewed Dec 15, 2022 View changes Copy link @phillipmiller0044 phillipmiller0044 left a comment * edited by RytoEX Admin edit: this was spam. Sorry, something went wrong. All reactions Be-ing Be-ing reviewed Dec 15, 2022 View changes plugins/obs-webrtc/CMakeLists.txt Outdated Show resolved Hide resolved @phillipmiller0044 This comment was marked as spam. Sign in to view tt2468 and others added 2 commits Dec 15, 2022 @tt2468 Remove unused variable declarations ... c92ff32 Should fix Windows build. @kc5nra Use scoped env var for generated header dir d909d30 tt2468 tt2468 reviewed Dec 15, 2022 View changes plugins/obs-webrtc/webrtc/src/ffi/output.rs Show resolved Hide resolved @tt2468 tt2468 changed the title [DEL:Add WebRTC output support:DEL] [INS:Add WebRTC (WHIP) output support:INS] Dec 15, 2022 tt2468 and others added 4 commits Dec 15, 2022 @tt2468 Minor code cleanup and nitpick 7380d9e @tt2468 Minimize impact of apply_video_settings f90b2c7 @kc5nra Rename most usages of webrtc to whip 3c714f8 @kc5nra Fixup some naming conventions in logs and help 11d9f9c tt2468 and others added 6 commits Dec 15, 2022 @tt2468 Set Content-Type header for WHIP request ... ac412b1 Required by spec, enforced by Cloudflare @kc5nra Add DELETE support and swap order of stop output and close peer 0cbede5 @tt2468 Rename references to stream key to bearer token ... 468fa1c - Does the above - Also makes bearer token optional - Stores bearer token as `bearer_token` in settings - Removes the `.get_key` callback method and such, instead uses info from obs_data_t directly. @tt2468 Updates to rust lib ... ca0a724 - Only pass `Authorization` header if bearer token is not empty - Partial fix for DELETE using the wrong URL. I honestly couldn't figure out for the life of me how to appropriately return a string from the `offer()` function of the WHIP resource URL, so I'm giving up there. I dunno. I guess returning strings in rust is apparently incredibly complicated. @kc5nra Use location header from offer for delete 19710d1 @kc5nra Grab location from offer response for use with delete 75c1acf @kc5nra kc5nra force-pushed the webrtc-output-plugin branch from 6994c0d to 75c1acf Compare Dec 15, 2022 @kc5nra Use built-in url construction and use Url 9162377 Be-ing Be-ing reviewed Dec 15, 2022 View changes .gitignore @@ -43,6 +43,8 @@ GeneratedFiles/ .vscode/ /CI/include/*.lock.json install_temp/ /plugins/obs-webrtc/webrtc/target/ /plugins/obs-webrtc/webrtc/bindings.h Copy link @Be-ing Be-ing Dec 15, 2022 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment This line can be removed now. Sorry, something went wrong. All reactions PatTheMav PatTheMav reviewed Dec 15, 2022 View changes plugins/obs-webrtc/CMakeLists.txt list(APPEND obs-webrtc_SOURCES obs-webrtc.rc) endif() add_library(obs-webrtc MODULE ${obs-webrtc_SOURCES} $ {obs-webrtc_HEADERS}) Copy link Member @PatTheMav PatTheMav Dec 15, 2022 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Always define the target at the top of the file, then use target_* functions to add sources after. Avoid using variables with file lists (that's CMake 2.x style, we require CMake 3.16 style at least). Sorry, something went wrong. 1 Be-ing reacted with thumbs up emoji All reactions * 1 reaction PatTheMav PatTheMav reviewed Dec 15, 2022 View changes plugins/obs-webrtc/CMakeLists.txt set(obs-webrtc_HEADERS whip-output.h) set(obs-webrtc_SOURCES whip-service.c whip-output.c webrtc.c) if(WIN32) Copy link Member @PatTheMav PatTheMav Dec 15, 2022 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Use 'if(OS_WINDOWS)` - support for the old OS-specific variables is soft-deprecated by CMake. Sorry, something went wrong. All reactions PatTheMav PatTheMav reviewed Dec 15, 2022 View changes plugins/obs-webrtc/CMakeLists.txt if(WIN32) set(MODULE_DESCRIPTION "OBS webrtc module") configure_file(${CMAKE_SOURCE_DIR}/cmake/bundle/windows/ obs-module.rc.in Copy link Member @PatTheMav PatTheMav Dec 15, 2022 There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Always e close directory/file specifications that are composed with user-defined variables in double quotes. Sorry, something went wrong. 1 Be-ing reacted with thumbs up emoji All reactions * 1 reaction @kc5nra Fix delete resource uri construction cbe1164 Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Reviewers @Sean-Der Sean-Der @PatTheMav PatTheMav @Be-ing Be-ing @tytan652 tytan652 @tt2468 tt2468 @phillipmiller0044 phillipmiller0044 Assignees No one assigned Labels New Feature New feature or plugin Request for Comments More feedback & discussion is requested Seeking Testers Build artifacts on CI UI/UX Anything to do with changes or additions to UI/UX elements. Work In Progress Projects None yet Milestone No milestone Development Successfully merging this pull request may close these issues. None yet 7 participants @kc5nra @phillipmiller0044 @Sean-Der @PatTheMav @Be-ing @tytan652 @tt2468 Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.