Working on a frontend to youtube-dl which doubles as a fully- functional video-hosting website reddit / selfhosted ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 20:18 not done yet, just figured I'd share my progress. demo site is [https://prometheus.tube](https://prometheus.tube) and discord is [https://discord.gg/H4uxCGPAcp](https://discord.gg/H4uxCGPAcp) Some features: 1. In-app video archival controls for any site and url supported by yt-dlp (channel, search criteria, playlist, etc) 2. personalized ML-based video recommendations 3. video search, and various sorting options (paginated) 4. user channels 5. video comments 6. video transcoding to h264 DASH 7. user authentication (registration, password reset, login) ---------------------------------------------------------------------- Woke_killa · 2023-11-23 20:28 Well done, there is a need for this type of service. ---------------------------------------------------------------------- syricium · 2023-11-23 21:55 Like TubeArchivist? ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 21:57 it's similar in nature ---------------------------------------------------------------------- rglullis · 2023-11-23 22:17 If you are doing this just to scratch your own itch, fine. But given that something similar already exists, how about trying to find a different angle and/or key feature to make it actually useful? For example: I am working on a tool to [mirror posts and comments from Reddit to Lemmy](https://github.com/mushroomlabs/fediverser) and it would be really cool to have something similar to the Peertube (the federated and self-hosted version of Youtube). Another idea would be to take the videos from youtube and make it available on [IPFS](https://ipfs.io). **This response is from an [original comment on the Fediverse](https://selfhosted.forum/comment/22569) and bridged via [Fediverser](https://github.com/mushroomlabs/fediverser).** ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 22:27 This is a different tool with a diverging featureset (see above for a small list of features). My purpose in life is not to sell you on the use of my open-source software, but you can use it if you want. ---------------------------------------------------------------------- Windows_XP2 · 2023-11-23 22:47 Neat project. Would've been nice if this had existed before I decided a while back just to have Jellyfin index all of my downloaded videos. I tried YoutubeDL-material, but I had all kinds of issues with it, so I just added all of them to Jellyfin. ---------------------------------------------------------------------- JVAV00 · 2023-11-23 22:50 What the hell do you watch ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 22:58 see the demo site ---------------------------------------------------------------------- greenlightison · 2023-11-23 23:07 Genius ---------------------------------------------------------------------- Factual_Signage · 2023-11-23 23:15 weeb trash apparently. ---------------------------------------------------------------------- privacyplsreddit · 2023-11-23 23:22 Looks good. How far away from a 1.0 or minimum viable product are you? You say demo but it looks pretty feature rich. ---------------------------------------------------------------------- rglullis · 2023-11-23 23:26 > It's designed primarily for multiuser collaborative archival. So is fediverser - along with its goal of helping people migrating away from reddit. What I'm trying to say is that if we take a look at "your" featureset: 1. In-app video archival controls for any site and url supported by yt-dlp (channel, search criteria, playlist, etc) 2. personalized ML-based video recommendations 3. video search, and various sorting options (paginated) 4. user channels 5. video comments 6. video transcoding 7. user authentication Only the first two are really interesting. You can get all others for free if you build your solution on top of the other many existing video platforms out there, Peertube especially. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 23:28 then do it ---------------------------------------------------------------------- LavaCreeperBOSSB · 2023-11-23 23:32 What exactly does this do? Youtube-DL's a video and then lets you watch it? ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-23 23:34 ehh, hard to define. ideally i'd want rate-limiting, individual user uploads, and easier setup for self-hosting. ---------------------------------------------------------------------- rglullis · 2023-11-24 00:17 Are you familiar with Peertube? ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 00:28 "maybe if I insult his project, he'll do shit for me" -you ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 01:56 Wow you have amazing timing. I was just thinking about what a good method would be to emulate the YouTube experience with local videos. I have a jellyfin container, but wanted something different for essentially bite sized videos I collect. Can’t wait to try it OP! ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 01:59 OP, I seriously have no idea why gate keepers rush into tell you how to pilot your development of your own app because *they* find another app similar. I have no idea why this is the hill these kinds of folk repeatedly rush into. No idea at all, none. That said, they can get stuffed, for the rest of us non coders we appreciate deeply your project you’re willing to share with the community. I just want to say loudly those annoying people do not represent what a community like this is. Please don’t end up discouraged. ---------------------------------------------------------------------- rglullis · 2023-11-24 02:00 You are getting overly defensive for no reason. All I am suggesting is to consider building on top of an existing project instead of developing yet-another variant of something that already exists and is mature. Also, Peertube is not "my" project so I don't understand where you think you'd "be doing shit for me". I am suggesting you taking a look at it because it would save you from doing a bunch of code to let you focus on the use-cases that *actually add value*. ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 02:00 👏 ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 02:00 Please get lost. ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 02:01 No they are getting defensive because you came off like an ass. You could disagree but either way no one is asking you to return. ---------------------------------------------------------------------- randonamous · 2023-11-24 03:09 drop any anime related imagery from this. stick with the product and showcasing that, do not personify yourself into this. The topic sounded good, the pics when I clicked it ... no longer interested. Good luck. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 03:11 LMAO ---------------------------------------------------------------------- crysisnotaverted · 2023-11-24 03:15 You are an elitist asshole, and you seem to be devoid of reading comprehension. Why don't you let them finish the framework of their fucking project and get to the part where they add features before going: **"Wah, wah, why aren't you making exactly what I want???"** Like a mindless sheep expecting others to just spoonfed you shit and bend to your will. You then proceed to beat your own dick about the projects you are working on, in an act of unrepentant self- aggrandizing about shit unrelated. ---------------------------------------------------------------------- QuackdocTech · 2023-11-24 03:30 >The topic sounded good, the pics when I clicked it ... no longer interested It must really suck being you lmao. I couldn't imagine dropping something because of anime imagery, all of the great software you have to avoid just because drawings make you cringe too much... that's pitiful, it really is. ---------------------------------------------------------------------- Trolann · 2023-11-24 03:32 They are being appropriately defensive for absolutely the right reasons ---------------------------------------------------------------------- BackgroundAmoebaNine · 2023-11-24 04:08 " Make it corporation friendly, sterile, and devoid of personality. This is how you make it appeal to me and many others" - said the 1 out of the only 2 people to feel this way about this project. Wow OP I seldomly return to a thread this often but I'm annoyed with these types of people. ---------------------------------------------------------------------- NSA-SURVEILLANCE · 2023-11-24 04:21 You should totally do the things you've listed. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 04:26 true ---------------------------------------------------------------------- rglullis · 2023-11-24 05:05 More like "why are you planning or reinventing a bunch of wheels when your energy and talent could be used elsewhere", but sure... Let's interpret everything through the worst possible lens and live in the world where asking anything slightly challenging of one's plans is " elitist". ---------------------------------------------------------------------- BandOfBroskis · 2023-11-24 05:06 Cool, I'll definitely try it out! I'm currently running a similar ydl front-end but recently my database was corrupted and so it's not showing my older videos anymore. They still exist on disk but not showing up and I can't figure out how to re-add them. So, that would be a good feature: being able to easily add local files to the hosting side. 👍 ---------------------------------------------------------------------- crysisnotaverted · 2023-11-24 05:25 >how about trying to find a different angle and/or key feature to make it actually useful? > >if we take a look at **"your"** featureset > >Only the first two are really interesting If you can't see how you're talking down, I really don't know what to say. You pick two ideas and say they're interesting, and then hand them a shit sandwich and say that they should throw the rest of the project away and just fork something and give up. Seemingly because you think going "Dance monkey, dance!" will make somebody alter their pet project to make something more amusing to you. You then go on to say that them building it the way they want doesn't >focus on the use-cases that actually add value Who the hell are you that makes you the arbiter of value? Get over yourself. Just because somebody doesn't want to fork an existing project doesn't make their project lesser, and I don't understand that mindset. ---------------------------------------------------------------------- rglullis · 2023-11-24 05:52 > just fork something and give up. The suggestion was to build on top of something, not to *fork* anything. > makes you the arbiter of value? I *asked* why OP was working on something when there are mature/established projects that already provide the functionality that he claims to be working on. I have no idea why you are taking that as "being the arbiter of value*. > I don't understand that mindset. Like I said in the very first comment: if OP wants to scratch an itch and work on this for fun/learning/challenge, go on and be merry. But If the idea is to create something for the specific use case, then it would be a lot *easier* and *productive* to look at the existing solutions and re-use what is already available. ---------------------------------------------------------------------- carval444 · 2023-11-24 06:04 Realy cool project! Are you able to make categories of your own, and sort the videos? ---------------------------------------------------------------------- Chimerization · 2023-11-24 06:08 momoi and midori? instant trust ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 06:12 sorting: yes, see demo site categorization for archived videos isn't fleshed out yet, they're currently all categorized as "otomad". I have a tag -> category inference system that I could use here, as in: if it's tagged with "x", then it's a "y". I'll probably work on that tomorrow. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 06:35 importing videos from disk would be awkward unless i had all of the metadata from youtube-dl. if i had the full metadata dump for each video i could probably make it work ---------------------------------------------------------------------- BandOfBroskis · 2023-11-24 07:01 In my case I actually still do have all the ydl files: json, thumbnail image and video file but I can see how it would be a problem w/o the data. If you did manage to figure out how to import local files, that'd be sweet though. 😉 ---------------------------------------------------------------------- carval444 · 2023-11-24 07:06 Realy cool! Gonna try it out when it's released! ---------------------------------------------------------------------- CommanderMatrixHere · 2023-11-24 07:32 Hi. Please save my time and sanity from looking into your repo and if possible for you to answer these questions: How are you processing videos? Downloading and streaming or streaming directly? From my VERY VERY limited experience with ytdl, you can stream directly from YouTube. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 08:02 not streaming directly from any origin video site ---------------------------------------------------------------------- alex2003super · 2023-11-24 08:21 Why even bother making Linux, when BSD and Windows already exist smh ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-11-24 08:51 !REMINDME 60 DAYS ---------------------------------------------------------------------- RemindMeBot · 2023-11-24 08:51 I will be messaging you in 2 months on [**2024-01-23 08:51:12 UTC**](h ttp://www.wolframalpha.com/input/?i=2024-01- 23%2008:51:12%20UTC%20To%20Local%20Time) to remind you of [**this link **](https://www.reddit.com/r/selfhosted/comments/1829vzd/working_on_a_ frontend_to_youtubedl_which_doubles/kajnjz8/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=Remin dMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2F selfhosted%2Fcomments%2F1829vzd%2Fworking_on_a_frontend_to_youtubedl_w hich_doubles%2Fkajnjz8%2F%5D%0A%0ARemindMe%21%202024-01- 23%2008%3A51%3A12%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)] (https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete %20Comment&message=Delete%21%201829vzd) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remind mebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?t o=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside %20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Yo ur Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot& subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](ht tps://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot %20Feedback)| |-|-|-|-| ---------------------------------------------------------------------- somebodyknows_ · 2023-11-24 09:01 What I miss is added metadata ---------------------------------------------------------------------- Aggravating-Vehicle9 · 2023-11-24 09:02 This is amazing! Could you support h265? Perhaps provide an ffmoeg command template? That would allow us to save vast amounts of storage space. Notification (e. g. pushbullet) on a per -channel basis ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 09:07 h265 probably yeah, already on my todo list you want notifications if a channel you're following has a new video, or something? ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 09:09 as in? ---------------------------------------------------------------------- rglullis · 2023-11-24 09:29 Bad analogy, my friend. BSD (the open source variant) did not exist when Torvalds first started to work on the kernel. ---------------------------------------------------------------------- Katzoconnor · 2023-11-24 09:41 Wow. What a dickbag. How about you encourage the spirit of competition instead of trying to discourage new projects? And Lemmy needs all the help it can get in drawing over excited new users—if half the users are like you, then you folks can turn off your bot and stay there. ---------------------------------------------------------------------- PVTD · 2023-11-24 11:35 And then give the option to interconnect every server so we recreate YouTube in a type of decentralized way. (Not sure about the correct terms the use here) but this project looks dope! ---------------------------------------------------------------------- rglullis · 2023-11-24 11:55 > How about you encourage the spirit of competition Wait, really? Do you think that encouraging competition over collaboration is a *good thing*? ---------------------------------------------------------------------- Ok_Contract_8209 · 2023-11-24 12:47 PeerTube already has federation implemented. ---------------------------------------------------------------------- PVTD · 2023-11-24 13:07 FEDERETAION!! Thanks :) ---------------------------------------------------------------------- TheTwelveYearOld · 2023-11-24 13:59 No one here mentioned Invidious but how does it compare to it? ---------------------------------------------------------------------- pinkSh4d0w · 2023-11-24 15:08 You may want to check AV1 encoding. I'm not sure which one is better. ---------------------------------------------------------------------- AlteRedditor · 2023-11-24 17:32 The description of the video and all the metadata you can grab from YouTube. ---------------------------------------------------------------------- AlteRedditor · 2023-11-24 17:33 * `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-info-json`. Use `--no- embed-info-json` or `--compat-options no-attach-info-json` to revert this Check [yt-dlp](https://github.com/yt-dlp/yt-dlp) ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 17:39 I already grab the description, title, author, and tags (stored in Postgres) and write the metadata dump to disk alongside the video ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 17:57 i don't really love comparisons to other similar software, but i can probably add a table in the README if that's what people want. ---------------------------------------------------------------------- ok_this_is_epic569 · 2023-11-24 17:59 transcoding with av1 is too slow. h265 i haven't tested yet, hoping it'll be more reasonable. h264 atm is a sweet spot ---------------------------------------------------------------------- jaketeater · 2023-11-25 05:04 I did something similar years ago with XBMC. I had a script that regularly checked my YT watch list and then downloaded the video and added a file with info for the scraper along side the video for XBMC