youtube-feed.md - www.codemadness.org - www.codemadness.org saait content files
(HTM) git clone git://git.codemadness.org/www.codemadness.org
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) README
(DIR) LICENSE
---
youtube-feed.md (4515B)
---
1 ... improved at least for my preferences ;)
2
3 It scrapes the channel data from Youtube and combines it with the parsed Atom
4 feed from the channel on Youtube.
5
6 The Atom parser is based on sfeed, with some of the code removed because it is
7 not needed by this program. It scrapes the metadata of the videos from the
8 channel its HTML page and uses my custom JSON parser to convert the
9 Javascript/JSON structure.
10
11 This parser is also used by the [json2tsv](https://codemadness.org/json2tsv.html) tool. It has few dependencies.
12
13
14 ## Features
15
16 * Add the video duration to the title to quickly see how long the video is.
17 * Filter away Youtube shorts and upcoming videos / announcements: only videos are shown.
18 * Supports more output formats: Atom, [JSON Feed](https://www.jsonfeed.org/version/1.1/) or
19 [sfeed](https://codemadness.org/sfeed.1.txt) Tab-Separated-Value format.
20 * Easy to build and deploy: can be run as a CGI program as a static-linked
21 binary in a chroot.
22 * Secure: additionally to running in a chroot it can use pledge(2) and unveil(2)
23 on OpenBSD to restrict system calls and access to the filesystem.
24
25
26 ## How to use
27
28 There is an option to run directly from the command-line or in CGI-mode. When
29 the environment variable $REQUEST_URI is set then it is automatically run in
30 CGI mode.
31
32
33 Command-line usage:
34
35 youtube_feed channelid atom
36 youtube_feed channelid gph
37 youtube_feed channelid html
38 youtube_feed channelid json
39 youtube_feed channelid tsv
40 youtube_feed channelid txt
41
42
43 CGI program usage:
44
45 The last basename part of the URL should be the channelid + the output format
46 extension. It defaults to TSV if there is no extension.
47 The CGI program can be used with a HTTPd or a Gopher daemon such as geomyidae.
48
49 For example:
50
51 Atom XML: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml
52 HTML: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.html
53 JSON: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.json
54 TSV: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.tsv
55 twtxt: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.txt
56 TSV, default: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw
57
58 Gopher dir: gopher://codemadness.org/1/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw.gph
59 Gopher TSV: gopher://codemadness.org/0/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw
60
61
62 An OpenBSD httpd.conf using slowcgi as an example:
63
64 server "codemadness.org" {
65 location "/yt-chan/*" {
66 request strip 1
67 root "/cgi-bin/yt-chan"
68 fastcgi socket "/run/slowcgi.sock"
69 }
70 }
71
72
73 ## Using it with [sfeed](https://codemadness.org/sfeed.html)
74
75 sfeedrc example of an existing Youtube RSS/Atom feed:
76
77 # list of feeds to fetch:
78 feeds() {
79 # feed <name> <feedurl> [basesiteurl] [encoding]
80 # normal Youtube Atom feed.
81 feed "yt IM" "https://www.youtube.com/feeds/videos.xml?channel_id=UCrbvoMC0zUvPL8vjswhLOSw"
82 }
83
84
85 Use the new Atom feed directly using the CGI-mode and Atom output format:
86
87 # list of feeds to fetch:
88 feeds() {
89 # feed <name> <feedurl> [basesiteurl] [encoding]
90 # new Youtube Atom feed.
91 feed "idiotbox IM" "https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml"
92 }
93
94 ... or convert directly using a custom connector program on the local system via the command-line:
95
96 # fetch(name, url, feedfile)
97 fetch() {
98 case "$1" in
99 "connector example")
100 youtube_feed "$2";;
101 *)
102 curl -L --max-redirs 0 -H "User-Agent:" -f -s -m 15 \
103 "$2" 2>/dev/null;;
104 esac
105 }
106
107 # parse and convert input, by default XML to the sfeed(5) TSV format.
108 # parse(name, feedurl, basesiteurl)
109 parse() {
110 case "$1" in
111 "connector example")
112 cat;;
113 *)
114 sfeed "$3";;
115 esac
116 }
117
118 # list of feeds to fetch:
119 feeds() {
120 # feed <name> <feedurl> [basesiteurl] [encoding]
121 feed "connector example" "UCrbvoMC0zUvPL8vjswhLOSw"
122 }
123
124
125 ## Screenshot using sfeed_curses
126
127 [](https://codemadness.org/downloads/screenshots/sfeed_curses_youtube.png)
128
129
130 ## Clone
131
132 git clone git://git.codemadness.org/frontends
133
134
135 ## Browse
136
137 You can browse the source-code at:
138
139 * <https://git.codemadness.org/frontends/file/youtube/feed.c.html>
140 * <gopher://codemadness.org/1/git/frontends/file/youtube/feed.c.gph>
141
142 The program is: youtube/feed
143
144
145 ## Dependencies
146
147 * C compiler.
148 * LibreSSL + libtls.
149
150
151 ## Build and install
152
153 $ make
154 # make install
155
156
157 ## That's all
158
159 I hope by sharing this it is useful to someone other than me as well.