[HN Gopher] Browser extension and local backend that automatical...
___________________________________________________________________
Browser extension and local backend that automatically archives
YouTube videos
Author : fcpguru
Score : 109 points
Date : 2025-08-02 16:03 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| fcpguru wrote:
| ~/os/starchive (main)[56daf7] $ ls -lh data
|
| total 3207312
|
| -rw-r--r-- 1 aa staff 525M Aug 2 09:11 2PMzaym-StM.mov
|
| -rw-r--r-- 1 aa staff 362M Aug 2 09:10 CHbawkGc_os.mov
|
| -rw-r--r-- 1 aa staff 658M Aug 2 09:11 lqR7VV8ftys.mov
|
| ~/os/starchive (main)[56daf7] $ ./starachive
|
| Server starting on port 3009...
|
| JSON received: map[videoId:CHbawkGc_os]
|
| Added video CHbawkGc_os to queue. Queue length: 1
|
| Processing video CHbawkGc_os. Remaining in queue: 0
| computegabe wrote:
| Interesting. I was looking into creating an extension that
| manually manipulates and intercepts the vnd.yt-ump [1] requests,
| then use webcodecs to process everything in the browser.
|
| [1]:
| https://github.com/gsuberland/UMP_Format/blob/main/UMP_Forma...
| fcpguru wrote:
| oh wow, yeah https://github.com/yt-dlp/yt-dlp sounds like the
| easier path.
| mikae1 wrote:
| _> Videos are saved to the . /data/ directory and converted to
| MOV format using ffmpeg with hardware acceleration_
|
| Transcoded (ouch) or just remuxed to a mov container? Have to
| investigate.
| atahanacar wrote:
| Relevant lines:
| https://github.com/andrewarrow/starchive/blob/136030c6ef11a5...
| fcpguru wrote:
| the video has to be re-encoded because apple quicktime doesn't
| like the youtube video format. But the audio can just be
| copied. My mac's fan never spins with the hardware acceleration
| so it runs in the background and I just forget about it.
| ahoef wrote:
| I detest QuickTime more than any other piece of software
| 1718627440 wrote:
| Why does Apple take the effort to maintain and ship different
| encoding libraries? I would've expected to both the Safari
| engine and Quicktime to simply depend on
| libappleavsmth.dylib?
| fcpguru wrote:
| wow i went down an AI rabbit whole learning the answer to
| this: https://chatgpt.com/share/688e818d-67ac-8010-913d-618
| f3534f1...
| WithinReason wrote:
| Now add DHT so clients can download videos from each other as a
| torrent and you solved global video distribution.
| rwmj wrote:
| That's basically PeerTube?
| WithinReason wrote:
| PeerTube doesn't have all of youtube's videos on it
| erinnh wrote:
| Ive been using Tubearchivist with the extension for this.
|
| https://github.com/tubearchivist/browser-extension
|
| I really like the WebUI of Tubearchivist itself.
| fcpguru wrote:
| the main feature I want is to just browse youtube like normal
| in firefox like I always do. And completely forget starchive is
| running. Then later in the day I'm pleasntly suprised that any
| video I want to clip is already downloaded and ready. I never
| know which one I'll want to download and I don't want to have
| to click any button.
| nemomarx wrote:
| What are you clipping them for?
| fcpguru wrote:
| Usually thoughout the day I'll be watching many different
| videos and then one will stick with me. Someone will have
| made a really good point at like time code 3 mins and 17
| seconds or something. If I have to right then and there
| pause the video and start a download it takes me out of the
| moment. I like it so much better to just at the end of the
| day go back and find good moments and place them in a their
| own videos. Examples:
|
| https://www.youtube.com/watch?v=ksHaSnEs4WM
|
| https://www.youtube.com/watch?v=KRfsAufKrzk
|
| https://www.youtube.com/watch?v=6EoH-Qy_xw8
| socalgal2 wrote:
| Cool but ... this also sounds like hording behavior. The number
| of things I've saved over the years only to throw them away years
| later and realize that saving them in the first place was a waste
| of time.
|
| In the 90s my friend's mom would video tape AMC movies. She had
| 300+ tapes. Maybe she had a few rare ones but now all those
| movies are available on demand either legally or illegally and in
| much better quality. Another friend kept all of his 1980s
| computer magazines (Byte, etc...) and moved these extremely heavy
| boxes through 30+ years of moves. I doubt he ever opened a single
| magazine since the moment he saved them. Then they all appeared
| on The Archive and he finally got rid of them.
|
| To be clear, I have a few youtube videos saved on my local
| storage. I'm just thinking that saving every video I watch
| reminds me of the things I've personally over-saved.
|
| Actually that reminds me. I met up with the magazine saving
| friend recently which is when I verified that he finally got rid
| of his stash. It made me think about things I'm still saving that
| if I reflect on I know I will never actually look at. For example
| I have box of about eight 3.5 inch floppy disks from my Amiga
| days. The odds that I'm going to get an Amiga or download an
| Amiga emu and get a drive to read those are close enough to zero
| that I should throw them away. Similarly I have a book of CD-ROMs
| of backed up data from the 90s. There's a close to 0% chance that
| I'm never going to bother look at their contents.
| fcpguru wrote:
| oh that's not why I want them local. I want to open them in
| final cut pro and edit them and use parts in other videos. I
| delete the data folder at the end of each day.
| nemomarx wrote:
| Getting them on a public shared archive is probably a good
| outcome though. There was that lady who taped hundreds of hours
| of daytime TV and archiving that has some interesting
| historical uses?
|
| But a personal copy I'm not sure has much point yeah.
| mananaysiempre wrote:
| > Another friend kept all of his 1980s computer magazines
| (Byte, etc...) and moved these extremely heavy boxes through
| 30+ years of moves.
|
| I don't think IA has all early issues of the Microsoft Systems
| Journal (later MSDN Magazine), among others. So this can be
| useful. (Also, what kind of person do you think put the
| magazines up on IA in the first place?..)
| asdefghyk wrote:
| Lots magazines never made it to the archives and have been
| lost.
| bombcar wrote:
| Digital hoarding takes nearly no practical space.
|
| And there's a number of YouTube videos o wish I could still
| access.
| ThrowawayTestr wrote:
| Hard drives are cheap and compact. The real issue is archiving
| with no organization or indexing.
| jh00ker wrote:
| > I have a book of CD-ROMs of backed up data
|
| >There's a close to 0% chance that I'm never going to bother
| look at their contents.
|
| More likely scenario, your children, grandchildren or other
| family members go through your shit after you pass away and
| discover stuff about you that perhaps you never wanted to
| share.
|
| This is something I think about a lot because I don't have a
| "digital legacy plan."
| npteljes wrote:
| Store your archive encrypted, and then later you can decide
| if you share the password or not :)
| socalgal2 wrote:
| > More likely scenario, your children, grandchildren or other
| family members go through your shit after you pass away
|
| I think that's not really likely. I'm pretty sure if you poll
| you'll find that few children care about their parent's
| "stuff". You can find plenty of people who've lost parents
| who found that they didn't have any interest in going through
| their parents stuff and then from that realized their
| children would be the same to them.
|
| Most children aren't going to dig through anything more than
| a physical photo album, and when they do, the only pictures
| that are relevant to them are those with people they know.
| The rest only have meaning to the dead parent. They aren't
| going to dig through hard drives or CDs unless they are
| searching for financial documents so they can finish up their
| parent's financial affairs.
|
| > discover stuff about you that perhaps you never wanted to
| share
|
| I do worry about that. I just tell myself I'll be dead so it
| doesn't really matter.
| globalise83 wrote:
| On your deathbed, you say: "My only regret is forgetting
| where I saved my Bitcoin keys".
| hkon wrote:
| With enough space available hoarding is just thinking ahead.
| SilverElfin wrote:
| I would like to be able to search old videos I've seen
| sometimes. Like to find that one recipe I saw or to pull out
| that one fact I thought I heard. Or sometimes just to listen to
| a song that later got made private or deleted outright. When
| YouTube deletes a video it doesn't even leave the title in your
| playlist so it can be frustrating to try and find the same
| thing again.
| danieldk wrote:
| I am the exact opposite and sell or throw away pretty much
| everything that I don't use. I find that doing so not only
| clutters the house less, but also gives you less to worry
| about.
|
| My general rule is - if I didn't use it for a year, I don't
| need it. There are obviously some exceptions like a fire
| extinguisher (which I hope to never use) and digitized photos,
| which only go through a careful selection.
|
| I think the thing I kept the longest was a Libranet Linux 3.0
| CD set because I worked for Libra Computer Systems for a while
| and this was the release that I helped building. A few years
| ago I threw it away, I think after I saw someone uploaded it to
| archive.org. When I'm 60 and want to install it again for good
| old time's sake I can.
|
| tl;de: if you don't use something for a year, you probably
| don't need it.
| pessimizer wrote:
| I don't get around to using plenty of things for the _first
| time_ a year after I 've purchased them. That policy in my
| life would be a nightmare of constantly rebuying stuff, or
| failing to rebuy stuff that is now gone forever.
|
| Almost everything that has become indispensable in my life
| took years to integrate into my life to any significant
| degree.
|
| "Need" is a weasel word. You don't need anything.
| redserk wrote:
| I'd own a lot less stuff if there were more opportunities
| to rent infrequently used items.
|
| As it stands, I have a workshop and electronics bench with
| many tools that will go unused for years but are critical
| when I need them and too expensive to buy and throw away.
| graemep wrote:
| Physical media had a much higher cost in terms of both the cost
| of the media and the space it uses so you can horde a lot more.
|
| Maybe something a bit more selective than this though!
| lukebechtel wrote:
| Yes!
|
| Hoarding is bad when it's costly, due to space, time, or
| money.
|
| Digital media hoarding is thus not bad at all!
| socalgal2 wrote:
| You have to define "cost". I have a "server" with 3
| external drives connected. One is "media" and 2 are for
| backups. I have a drawer with 11 external HD drives which I
| haven't used in years that used to be my "media" and backup
| drives. Each of those represent money (buying the drive)
| and time (copying stuff from old 1TB drives to 2TB drives
| to 4TB drives) etc....
|
| So there is a cost to digital media hording.
|
| I wanted to save the videos I'd captured from my car's
| cameras but there's ~250gb every 3-4 months or so which is
| a more money needed. Plus, if I wanted them actually
| available to access I'd need a way to plug in more drives
| live into my server so more $$$$ and I'd need to back them
| up for when the drives fail so more $$$$.
|
| So yea, there is a cost to digital media hording.
| toomuchtodo wrote:
| PSA: if you have a collection or other artifacts for ingest by
| IA, I'll cover reasonable shipping costs to get them there.
| Above a certain size, they'll handle logistics of packing and
| shipping for ingest.
|
| https://help.archive.org/help/how-do-i-make-a-physical-donat...
|
| Tools to make this easy exist if you already have digital
| versions.
|
| https://github.com/jjjake/internetarchive
|
| And don't forget to send a few dollars if and when you can.
|
| https://archive.org/donate
|
| (no affiliation, I just like the public good)
| pessimizer wrote:
| > Then they all appeared on The Archive and he finally got rid
| of them.
|
| Sometimes you're the person who is uploading them to public
| archives. Because everybody else threw them all away, and you
| saved them until the technology made archiving practical
| enough.
|
| I've been replacing all of my physical media for years, but the
| reason I can do that now is because other people scan/rip and
| archive/share the stuff. You also have unique stuff that you
| may not even know is unique. When you find something in your
| house that you can't find online, scan it and you're paying
| everybody back for all of the scanning they did for you.
|
| With the CD-ROMs, you should just glide through them one by one
| and check if you can find the stuff online. If you can, throw
| them in the trash. If you can't, copy their contents to a
| folder, and throw them in the trash. Go through the folder over
| the next hour or next 20 years (however long it takes to get
| around to it) and take the things you can't find online that
| you think somebody might want, and get those things to that
| somebody (uploading to archive.org is always a good place to
| start.)
|
| edit: I know for a fact that for a lot of people, uploading
| somewhere on the internet is their standard pre-deletion
| ritual.
| attila-lendvai wrote:
| hoarding, or maybe just anti-censorship measure.
| ivanjermakov wrote:
| I'm achieving this with a single yt-dlp script reading url from a
| clipboard.
| fcpguru wrote:
| oh but there's still the thought of having to press copy. My
| favorite thing about this is I just forget I even have it
| running and browse youtube like normal. Then later anything
| I've watched that day is already downloaded.
| Szpadel wrote:
| I creates something similar in concept but with different goal. I
| wanted to be able to watch videos with sponsor block on iPad
| ideally using Plex.
|
| I found self hosted solution like this but I was very
| dissatisfied with how that worked
|
| on other hand I wanted to check out loco.rs framework, so I
| decided to implement my own solution.
|
| basically you are able to add channels/playlists on many many
| platforms that yt-dlp supports, you can select what should be cut
| out using sponsor block and you choice how many days you want it
| (videos older that that are automatically deleted)
|
| if you are interested, you can check it out:
| https://github.com/Szpadel/LocalTube
| amelius wrote:
| It would be nice if the extension wrote them to some shared
| repository. That way, the videos could be preserved for humanity
| without Google having a say in it.
|
| Added benefit: every video would have to be archived only once.
| Alive-in-2025 wrote:
| But then companies could sue to wipe out the centralized repo.
| So to be safe, you'd copy things to the central repo and also
| have a local copy. ;-)
|
| Next, you try to centralize all the private copies so only one
| person has to keep theirs. Solution is end copyright for things
| over x years in age. Instead in the us we keep pushing back the
| date.
| amelius wrote:
| Depends where the central server is. Nobody is wiping annas
| archive, for example.
| syntaxing wrote:
| Whoa! I asked about something like this 2 years ago but never got
| to making anything [1]! Super exciting something like this
| exists!
|
| [1] https://news.ycombinator.com/item?id=37885584
| myself248 wrote:
| Oh, this is huge and important. The number of things I watch
| that're just gone when I go back to look again!
|
| Youtube is an archive like a grocery store is a food archive. [1]
|
| If it was worth watching in the first place, it's worth saving.
| Reducing the friction of doing so is going to help a lot of
| people.
|
| (1: I'm getting this quote wrong, what's the actual and
| attribution??)
| fcpguru wrote:
| ha, I'm not sure who said Youtube is a video archive like a
| grocery store is a food archive but that's excellent.
| untech wrote:
| See also ArchiveBox, which supports YT saving as well, but can
| save other content too
|
| https://github.com/ArchiveBox/ArchiveBox
| busymom0 wrote:
| My archiving app called HEAP can be configured using a simple
| apple script and yt-dlp to do this too. And since it's a native
| macOS app instead of a browser extension, it works via all
| browsers:
|
| https://apps.apple.com/ca/app/heap-website-full-page-image/i...
| globular-toast wrote:
| I've had this idea myself so cool to see it implemented.
|
| What I'd really like is a kind of universal web caching backend.
| So everything I access goes through a cache and I have the option
| of viewing from cache if something goes offline or changes. I
| could also mark things as "favourite" so they don't ever expire
| from the cache. Does such a thing exist?
| fcpguru wrote:
| trying to just grab from the actual browser cache is very hard
| for video. If you look at the complexity of yt-dlp you'll see
| why that's so much easier than trying to grab various formats
| from cache.
| ProofHouse wrote:
| I don't really get the purpose of this broadly, because doesn't
| YouTube keep videos online unless the creator took them down
| which is probably not the case 95% of the time? That said for a
| niche or a high likelihood of a video being removed, or if you
| really want to be 100% certain it makes sense, but would I be
| accurate in that statement or am I missing something?
| fcpguru wrote:
| I'm not trying to save them forever. I just want them local so
| I can take clips from them for other videos. I use them as
| source input to final cut pro.
___________________________________________________________________
(page generated 2025-08-02 23:00 UTC)