Post ASLyc3tbE7b1Fj88Qa by peepstein@mstdn.social
(DIR) More posts by peepstein@mstdn.social
(DIR) Post #ASLxL10Ybu4bQ3gzLM by danluu@mastodon.social
2023-02-05T00:07:02Z
0 likes, 0 repeats
How long before we have good automated information extraction from videos?About five years ago, I felt sad that the best / most informative sources of information on so many things had moved to video because video monetizes much better than text, but it now seems plausible that, in the next N years, automated systems will be able to take a 30 minute video and turn it into text that takes 5 minutes to read.
(DIR) Post #ASLxL1eyBeOpRObGKm by simon@fedi.simonwillison.net
2023-02-05T00:53:29Z
0 likes, 0 repeats
@danluu I honestly think we are there today https://fedi.simonwillison.net/@simon/109691639446505703
(DIR) Post #ASLxUzl278SERABpaK by simon@fedi.simonwillison.net
2023-02-05T00:55:20Z
0 likes, 0 repeats
Here's the full process I went through to summarize a 35 minute YouTube video using Whisper plus GPT3It had some manual steps, but automating it doesn't feel like it would be a particularly hard project using the tools we have right nowhttps://gist.github.com/simonw/9932c6f10e241cfa6b19a4e08b283ca9
(DIR) Post #ASLyc3tbE7b1Fj88Qa by peepstein@mstdn.social
2023-02-05T01:07:35Z
0 likes, 0 repeats
@simon wow. I skimmed the transcript and it was a bit of a mess. Yet GPT got it. LLMs of a certain size and with the right optimizations are just going to bring about tremendous change to our world. It’ll take a long time to see the full impact but it’s kind of crazy.
(DIR) Post #ASM5X2ZhPU9LO3NIUy by danluu@mastodon.social
2023-02-05T02:25:33Z
0 likes, 0 repeats
@simon I tried this and it worked better than the other suggestion I got, summarize.tech, which produced complete garbage.This at least produced something that would look reasonable to someone who hadn't watched the video, but it has no concept of what information is important and both fails to present important information and spends a lot of text on unimportant details, so I'd say that we're not really there today in that I could already get bad information in text without the video.
(DIR) Post #ASM8ZdII9x09iAA0GG by simon@fedi.simonwillison.net
2023-02-05T02:59:29Z
0 likes, 0 repeats
@danluu there's definitely tons of room for improvement- - I want something with output looking more like this (which I constructed by hand), plus links to jump to relevant points in the video: https://simonwillison.net/2022/Nov/26/productivity/But... I could imagine how such a thing could be built today - I don't think it's even a year off
(DIR) Post #ASMDUfAOGlrPqFujaa by macbraughton@infosec.exchange
2023-02-05T03:54:36Z
0 likes, 0 repeats
@simon this is great, Simon. I find it’s so easy to spend a day working on something that you end up changing later and then loose track of all the effort and discovery that lead to that breakthrough. This is a great way to document that process and leave a trail for yourself and others to pick up on. Thanks for sharing!