[jay.scot]
[029]


--[ sfeed, fdm, rdrview and mutt


A few years ago I briefly mentioned my RSS setup. Since then I have
improved on it and I thought it might be worth sharing how I consume
these feeds. It may seem like a complicated setup and you are wondering
why go to this effort when you can simply use something like newsraft or
newsboat. Well, I don't know if it's just the feeds I view, but
frequently they don't include the full body of the article and I find
this quite annoying. On top of that, I like to have everything under one
application; having that sense of familiarity is nice. It's also good to
have a local copy of the article, more than once I have seen articles
just disappear into the ether.


	sfeed ---> fdm ---> rdrview ---> mutt


Above is the basic flow of the setup and you can watch [0] a screen
recording of my normal usage, link below. This shows me running a
wrapper script that updates the feeds with sfeed, converts this to mbox
and then calls fdm to do it's thing. I also open the feeds in mutt,
showing the article content is now within the email body, I then open a
few of the links in lynx/rdrview directly to show you can still view
them externally, if needed.


To achieve this setup, we need to do the following:


	1. sfeed_update downloads the latest feeds.
	2. sfeed_mbox creates an mbox file.
	3. fdm converts this to Maildirs based on the feed name.
	4. fdm rewrites the body, using rdrview.
	5. mutt then becomes the reader for these feeds.


I won't cover the first 2 steps in much detail as these are covered very
well in the sfeed documentation. In a nullshell, once you have created
your feeds file, you run these two commands.


	sfeedroot="$HOME/.config/sfeed"
	feedsdir="${sfeedroot}/feeds"

	sfeed_update "${sfeedroot}/sfeedrc"
	sfeed_mbox "${feedsdir}"/* >${sfeedroot}/mbox


Step 3 is also partly covered in the sfeed documentation, but I have
stripped down my own fdm config below just to show at what point I call
the script to rewrite the body. You can run fdm with this config, because
the feed is disabled by default we must include the account name when
running it. I only do this because I use fdm to fetch my normal mail,
which also has a bunch of other filters, such as dumping mailing lists
into other locations. The main workhorse of this script is match account
on feeds, where it calls the action feedget:


	fdm -f ${fdmconfig} -afeeds fetch


This is the minimal fdm config you would need ~/.config/fdm/config


   # macros, paths to cache file, where we want the Maildirs to live.
   $path           = "%h/.mail"
   $feedcache      = "%h/.mail/fdm.cache"
   $feeddir        = "%h/.mail/feeds/"
   $strurl         = "(http[s]?://.*|gemini://.*)"

   # options, setting up the cache so we know what's a new feed.
   set unmatched-mail keep
   set no-received
   cache "${feedcache}"

   # accounts, disabled by default so we have to explicitly call it.
   account "feeds" disabled mbox "%[home]/.config/sfeed/mbox"

   # feed actions
   action "feedtag" tag "msgid" value "%1"
   action "feedget" rewrite "%h/bin/fdm_parse_feeds.sh"
   action "feedsave" {
		   maildir "${feeddir}%1"
		   add-to-cache "${feedcache}" key "%[msgid]"
		   keep
   }

   # match on any feed from sfeed mbox and call the actions.
   match account "feeds" {
		   match case "^Message-ID: (.*)" in headers action "feedtag" continue
		   match matched and in-cache "${feedcache}" key "%[msgid]" action keep
		   match $strurl action "feedget" continue
		   match case "^X-Feedname: (.*)" in headers action "feedsave"
   }


The heavy lifting of this script is when a feed is matched, it then
calls the action feedget. This action calls an external script which
either runs gemget to pull the page if its a gemini link, or if its a
normal web page, run rdrview. The email is then returned back to fdm
with the new content body.


Here is the fdm_parse_feeds.sh script.


   #!/bin/sh

   data=$(cat)
   url=$(echo "$data" | grep -o -E 'https?://[^"]+|gemini://[^"]+')
   uri_lower="$(printf "%s" "$url" | tr '[:upper:]' '[:lower:]')"

   case "$uri_lower" in
		   content=$(gemget -o - "$url")
		   ;;
		   content=$(rdrview -H "$url" | lynx -stdin --dump -force_html)
		   ;;
   esac

   printf "%s0s" "$data" "$content"


I hope this helps anyone wanting to experiment with sfeed and fdm, or
looking at different ways of going a bit more offline. As always, you
can find all of these config my dotfiles repo, link on the gopher
homepage if needed.


0. gopher://jay.scot/9/files/misc/rss-mutt-demo.mkv


.EOF