itNew article about software packaging - monochromatic - monochromatic blog: http://blog.z3bra.org	Err	z3bra.org	70
hgit clone git://z3bra.org/monochromatic	URL:git://z3bra.org/monochromatic	z3bra.org	70
1Log	/scm/monochromatic/log.gph	z3bra.org	70
1Files	/scm/monochromatic/files.gph	z3bra.org	70
1Refs	/scm/monochromatic/refs.gph	z3bra.org	70
i---	Err	z3bra.org	70
1commit 251e9d348d825c5b45f0ecef5d19692bdc25ef01	/scm/monochromatic/commit/251e9d348d825c5b45f0ecef5d19692bdc25ef01.gph	z3bra.org	70
1parent c324148a7e8621ec00c94808b1e52ff8fd0f2f1b	/scm/monochromatic/commit/c324148a7e8621ec00c94808b1e52ff8fd0f2f1b.gph	z3bra.org	70
hAuthor: z3bra <willyatmailoodotorg>	URL:mailto:willyatmailoodotorg	z3bra.org	70
iDate:   Fri, 16 Sep 2016 00:57:56 +0200	Err	z3bra.org	70
i	Err	z3bra.org	70
iNew article about software packaging	Err	z3bra.org	70
i	Err	z3bra.org	70
iDiffstat:	Err	z3bra.org	70
i  A 2016/09/pack-it-up.txt              |     220 +++++++++++++++++++++++++++++++	Err	z3bra.org	70
i  M config.mk                           |       3 ++-	Err	z3bra.org	70
i  M index.txt                           |       1 +	Err	z3bra.org	70
i	Err	z3bra.org	70
i3 files changed, 223 insertions(+), 1 deletion(-)	Err	z3bra.org	70
i---	Err	z3bra.org	70
1diff --git a/2016/09/pack-it-up.txt b/2016/09/pack-it-up.txt	/scm/monochromatic/file/2016/09/pack-it-up.txt.gph	z3bra.org	70
it@@ -0,0 +1,220 @@	Err	z3bra.org	70
i+# [Pack it up!](#)	Err	z3bra.org	70
i+## &mdash; 16 September, 2016	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Today was a big day for me. It was the day all my software could play	Err	z3bra.org	70
i+together to reach a single goal:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+*Maintaining and hosting software packages!*	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Here are the tools:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+* [pm](http://z3bra.org/pm) - manage local packs	Err	z3bra.org	70
i+* [sick](http://z3bra.org/sick) - sign and check files	Err	z3bra.org	70
i+* [synk](http://z3bra.org/synk) - synchronize files between hosts	Err	z3bra.org	70
i+* [wendy](http://z3bra.org/wendy) - run a command on filesystem change	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Tied together with a fistful of scripts, and some easy preparation,	Err	z3bra.org	70
i+they help medeploy and manage my own software accross multiple operating	Err	z3bra.org	70
i+systems without efforts.	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+This process includes three main tasks: packaging, deploying and	Err	z3bra.org	70
i+installing. This whole process is still at and early stage, and will	Err	z3bra.org	70
i+get stronger and sharper with time (and shell script will most likely	Err	z3bra.org	70
i+turn into actual programs).  	Err	z3bra.org	70
i+Here is an explanation of how it works:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+0. pack an application and sign the tarball	Err	z3bra.org	70
i+1. upload the tarball to an online repository	Err	z3bra.org	70
i+2. get the latest tarball for a utility	Err	z3bra.org	70
i+3. install/update it locally	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+The process is straigh-forward, and multiple parts can be automated.	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+### Packaging	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+In order to build my softwares, I need either `make` or	Err	z3bra.org	70
i+[`mk`](http://doc.cat-v.org/plan_9/4th_edition/papers/mk) (it's tending	Err	z3bra.org	70
i+to be `mk` only, but for now I still need both). Then process is then	Err	z3bra.org	70
i+fairly simple:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        $ cd $utility	Err	z3bra.org	70
i+        $ mk	Err	z3bra.org	70
i+        # mk install	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+I wrote a quick script to "pack" these utilities for me. It will build	Err	z3bra.org	70
i+the software, install it to a temporary location, create a tarball out	Err	z3bra.org	70
i+of it, and sign this tarball with my private key:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        #!/bin/sh	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        # user specific variables	Err	z3bra.org	70
i+        SICKKEY=$HOME/.sick.d/$USER.key	Err	z3bra.org	70
i+        REPO=/var/www/htdocs/dl.z3bra.org/pack	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        # guess the pack name from current directory, or use given name	Err	z3bra.org	70
i+        DIR=$(basename `pwd`)	Err	z3bra.org	70
i+        PKG=${1:-$DIR}	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        # set version from latest git tag, if applicable	Err	z3bra.org	70
i+        TAG=$(git tag | sed -n '$p' | tr -d a-z)	Err	z3bra.org	70
i+        VER=${TAG:-0.0}	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        # this part should die...	Err	z3bra.org	70
i+        test -f mkfile && MK=mk || MK=make	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        # build pack and install to ./rootfs/usr	Err	z3bra.org	70
i+        $MK	Err	z3bra.org	70
i+        $MK DESTDIR=$(pwd)/rootfs PREFIX=/usr MANDIR=/usr/share/man install	Err	z3bra.org	70
i+        (	Err	z3bra.org	70
i+                cd rootfs	Err	z3bra.org	70
i+                mkdir -p $REPO	Err	z3bra.org	70
i+                # pack and sign the installed utility	Err	z3bra.org	70
i+                tar cvj * | sick -s -f ${SICKKEY} > $REPO/${PKG}#${VER}.tar.bz2	Err	z3bra.org	70
i+        )	Err	z3bra.org	70
i+        rm -rf rootfs	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        # simply ensure that the file has been created correctly	Err	z3bra.org	70
i+        echo	Err	z3bra.org	70
i+        ls $REPO/${PKG}#${VER}.tar.bz2	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+At this point, to pack one of my utilities, all I need is:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        $ git clone git://z3bra.org/skroll	Err	z3bra.org	70
i+        $ cd skroll	Err	z3bra.org	70
i+        $ pack	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+And I'm done :)	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+### Deploying	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+This part require a bit of setup. My current repository is at	Err	z3bra.org	70
i+http://dl.z3bra.org/pack, which is, locally in	Err	z3bra.org	70
i+`/var/www/htdocs/dl.z3bra.org/pack`. My tool `synk` can get a file	Err	z3bra.org	70
i+synchronized  between two peers, but they will have the same path,	Err	z3bra.org	70
i+which is why I also created this directory on my local machine.	Err	z3bra.org	70
i+I also need to upload my public key (for `sick` checks) and a list	Err	z3bra.org	70
i+of what's currently in the repo.	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+First, here is the `repogen` script, which will list the content of the	Err	z3bra.org	70
i+local repo, and write the pack names and version available to a file:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        #!/bin/sh	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        REPO=/var/www/htdocs/dl.z3bra.org/pack	Err	z3bra.org	70
i+        for tarball in $(find $REPO -name '*.tar.bz2'); do	Err	z3bra.org	70
i+                pkg=$(basename $tarball | cut -d# -f1)	Err	z3bra.org	70
i+                ver=$(basename $tarball | cut -d# -f2 | sed 's/.tar.bz2//')	Err	z3bra.org	70
i+                printf '%s\t%s\n' "$pkg" "$ver"	Err	z3bra.org	70
i+        done | sort | tee $REPO/.list	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+I also copied the public key as `.key` in the directory.	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Now everything is ready for `synk`ronisation (over a VPN, in this case):	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        find /var/www/htdocs/dl.z3bra.org/pack -type f | synk -h apophis.2f30	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+It can be automated using `wendy`, so that everytime the `.list` file	Err	z3bra.org	70
i+is modified by `repogen`, everything is replicated on the remote repo:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        wendy -m 8 -f $REPO/.list sh -c "find $REPO -type f | synk -h apophis.2f30"	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+### Installing	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Now that we can create packs and upload them quickly to the repository,	Err	z3bra.org	70
i+it's time to install them!	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Using the `.list` file, we can check what's available. With the `.key`	Err	z3bra.org	70
i+file, we can ensure that no-one tampered with our pack during the	Err	z3bra.org	70
i+retrieval process. Using `pm`, we can install and update our packs for	Err	z3bra.org	70
i+daily use.	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+All we need now, is a utility to fetch packs from the repo. I named this	Err	z3bra.org	70
i+script "`repo`"!	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        #!/bin/sh	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        url="http://dl.z3bra.org/pack"	Err	z3bra.org	70
i+        cache="$HOME/.cache/repo"	Err	z3bra.org	70
i+        keyring="$HOME/.sick.d"	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        usage() {	Err	z3bra.org	70
i+                echo "usage: $(basename $0) [-s] [-t reponame] [PKG..]" >&2	Err	z3bra.org	70
i+                exit 1	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        reposync() {	Err	z3bra.org	70
i+                mkdir -p ${cache}	Err	z3bra.org	70
i+                curl -Ls ${url}/.list | tee ${cache}/.list	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        repocheck() {	Err	z3bra.org	70
i+                sick -f $HOME/.sick.d/egull.pub	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        repotrust() {	Err	z3bra.org	70
i+                curl -sL ${url}/.key > ${keyring}/${name}.pub	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        repoget() {	Err	z3bra.org	70
i+                pkg="$1"	Err	z3bra.org	70
i+                ver=$(grep -E "^${pkg}        " ${cache}/.list | tac | sed 1q | cut -f2)	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+                file="${pkg}#${ver}.tar.bz2"	Err	z3bra.org	70
i+                html="${pkg}%23${ver}.tar.bz2"	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+                curl -sL ${url}/${html} | repocheck | ifne sponge ${cache}/${file}	Err	z3bra.org	70
i+                test -f ${cache}/${file} \	Err	z3bra.org	70
i+                        && readlink -f ${cache}/${file} \	Err	z3bra.org	70
i+                        || echo "$pkg: signature check failed" >&2	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        repolist() {	Err	z3bra.org	70
i+                pg -e ${cache}/.list	Err	z3bra.org	70
i+        }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        test $# -eq 0 && { repolist && exit 0; }	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        case $1 in	Err	z3bra.org	70
i+                -s) reposync; exit 0 ;;	Err	z3bra.org	70
i+                -l) repolist; exit 0 ;;	Err	z3bra.org	70
i+                -t) test -z "$2" && usage || { repotrust $2; exit 0; } ;;	Err	z3bra.org	70
i+        esac	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        for n in $@; do	Err	z3bra.org	70
i+                repoget $n	Err	z3bra.org	70
i+        done	Err	z3bra.org	70
i+        	Err	z3bra.org	70
i+        exit 0	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+First, we retrieve the ed25519 public key from the repo:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        $ repo -t z3bra	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Then, the package list:	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        $ repo -s	Err	z3bra.org	70
i+        libwm   1.1	Err	z3bra.org	70
i+        skroll  0.6	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+And finally, install it!	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+        $ ROOT=$HOME/.local	Err	z3bra.org	70
i+        $ export ROOT	Err	z3bra.org	70
i+        $ pm -a $(repo skroll)	Err	z3bra.org	70
i+        $ pm -i	Err	z3bra.org	70
i+        skroll        0.6	Err	z3bra.org	70
i+        $ echo amazing! | skroll	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+### Conclusion	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+Ok, so this whole post was rather long, and not especially good at	Err	z3bra.org	70
i+describing the actual workflow. So as usual. Here is a quick video to	Err	z3bra.org	70
i+show off the whole process!	Err	z3bra.org	70
i+	Err	z3bra.org	70
i+<video controls>	Err	z3bra.org	70
i+        <source src='/vid/20160916-skroll-install.webm' type='video/webm'>	Err	z3bra.org	70
i+</video>	Err	z3bra.org	70
i+<span class='caption'>Packagin, deploying and installing	Err	z3bra.org	70
i+[`skroll`](http://z3bra.org/skroll) on my system</span>	Err	z3bra.org	70
i+	Err	z3bra.org	70
1diff --git a/config.mk b/config.mk	/scm/monochromatic/file/config.mk.gph	z3bra.org	70
it@@ -32,7 +32,8 @@ PAGES   =   index.html \	Err	z3bra.org	70
i             2015/08/install-alpine-at-onlinenet.html \	Err	z3bra.org	70
i             2016/01/make-your-own-distro.html \	Err	z3bra.org	70
i             2016/03/hand-crafted-containers.html \	Err	z3bra.org	70
i-            2016/08/desktop-streaming.html	Err	z3bra.org	70
i+            2016/08/desktop-streaming.html \	Err	z3bra.org	70
i+            2016/09/pack-it-up.html	Err	z3bra.org	70
i 	Err	z3bra.org	70
i FEEDS = rss/feed.xml	Err	z3bra.org	70
i EXTRA = css img vid data errors favicon.ico	Err	z3bra.org	70
1diff --git a/index.txt b/index.txt	/scm/monochromatic/file/index.txt.gph	z3bra.org	70
it@@ -1,3 +1,4 @@	Err	z3bra.org	70
i+* 0x001d - [Pack it up!](/2016/09/pack-it-up.html)	Err	z3bra.org	70
i * 0x001c - [Desktop streaming](/2016/08/desktop-streaming.html)	Err	z3bra.org	70
i * 0x001b - [Hand-crafted containers](/2016/03/hand-crafted-containers.html)	Err	z3bra.org	70
i * 0x001a - [Make your own distro](/2016/01/make-your-own-distro.html)	Err	z3bra.org	70
.