https://github.com/PuellaeMagicae/unix-in-lisp/blob/master/TUTORIAL.org

Skip to content

Navigation Menu

Toggle navigation
 
Sign in

  * Product
      +  
        Actions
        Automate any workflow
      +  
        Packages
        Host and manage packages
      +  
        Security
        Find and fix vulnerabilities
      +  
        Codespaces
        Instant dev environments
      +  
        GitHub Copilot
        Write better code with AI
      +  
        Code review
        Manage code changes
      +  
        Issues
        Plan and track work
      +  
        Discussions
        Collaborate outside of code
    Explore
      + All features
      + Documentation
      + GitHub Skills
      + Blog
  * Solutions
    By size
      + Enterprise
      + Teams
      + Startups
    By industry
      + Healthcare
      + Financial services
      + Manufacturing
    By use case
      + CI/CD & Automation
      + DevOps
      + DevSecOps
  * Resources
    Topics
      + AI
      + DevOps
      + Security
      + Software Development
      + View all
    Explore
      + Learning Pathways
      + White papers, Ebooks, Webinars
      + Customer Stories
      + Partners
  * Open Source
      +  
        GitHub Sponsors
        Fund open source developers
      +  
        The ReadME Project
        GitHub community articles
    Repositories
      + Topics
      + Trending
      + Collections
  * Enterprise
      +  
        Enterprise platform
        AI-powered developer platform
    Available add-ons
      +  
        Advanced Security
        Enterprise-grade security features
      +  
        GitHub Copilot
        Enterprise-grade AI features
      +  
        Premium Support
        Enterprise-grade 24/7 support
  * Pricing

Search or jump to...

Search code, repositories, users, issues, pull requests...

Search
[                    ]
Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

[                    ] [ ] Include my email address so I can be
contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Name [                    ] 
Query [                    ]

To see all available qualifiers, see our documentation.

Cancel Create saved search
Sign in
Sign up Reseting focus
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session. You switched accounts on another tab or window. Reload
to refresh your session. Dismiss alert
{{ message }}
PuellaeMagicae / unix-in-lisp Public

  * Notifications You must be signed in to change notification
    settings
  * Fork 6
  * Star 121

  * Code
  * Issues 8
  * Pull requests 0
  * Actions
  * Projects 0
  * Security
  * Insights

Additional navigation options

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Security
  * Insights

Files

 master

Breadcrumbs

 1. unix-in-lisp

/

TUTORIAL.org

[                    ]
Blame
Blame

Latest commit

 

History

History
 
162 lines (152 loc) * 5.14 KB
 master

Breadcrumbs

 1. unix-in-lisp

/

TUTORIAL.org

Top

File metadata and controls

  * Preview
  * Code
  * Blame

162 lines (152 loc) * 5.14 KB
Raw

Unix in Lisp shell tutorial

 

The missing semester

 

Shell Tools and Scripting

 

mcd () {
    mkdir -p "$1"
    cd "$1"
}

(defmacro mcd (dir)
  `(fg
     (mkdir -p ,dir)
     (cd ,dir)))

Data Wrangling

 

To get the data, in POSIX shell they write:

ssh myserver journalctl

In Unix in Lisp, we write something very similar:

(def journal (ssh myserver journalctl))

Note that we conveniently store the result into journal so we can use
it later. In POSIX shell they may use a temporary file, and they may
be relunctant to do so to avoid littering in the file system. In any
case, if we also want to store it in a file, just

(defile journal.log (ssh myserver journalctl))

In POSIX shell, they do the following to narrow down the lines.

ssh myserver journalctl | grep sshd

In Unix in Lisp, we may do something similar

(pipe journal (grep sshd))

We may do the same thing in a way that looks like more "common" Lisp:

(filter (lambda (line) (ppcre:scan "sshd" line)) journal)

In POSIX shell, they do the following to narrow down more:

ssh myserver journalctl
 | grep sshd
 | grep "Disconnected from"

We can translate it "literally" into Unix in Lisp:

(pipe journal (grep sshd) (grep "Disconnected from"))

Or do it the more "common" way:

(filter
 (lambda (line)
   (and (ppcre:scan "sshd" line)
        (ppcre:scan "Disconnected from" line)))
 journal)

Now, POSIX shell people write their intermediate result into a
temporary file:

$ ssh myserver 'journalctl | grep sshd | grep "Disconnected from"' > ssh.log
$ less ssh.log

We have our intermediate result ready for use in our REPL via
*,**,*** variables. We can also store them into variables via def and
alike. If we really want to store it persistently, just

(defile ssh.log ...)

POSIX shell people now use sed magic to pick out user names.

ssh myserver journalctl
 | grep sshd
 | grep "Disconnected from"
 | sed -E 's/.*Disconnected from (invalid |authenticating )?user (.*) [^ ]+ port [0-9]+( \[preauth\])?$/\2/'

We still use regex magic here (using cl-ppcre), but no need for sed
line editing witchery. We collect the matches for the user field into
a list:

(def users
  (collecting
   (do-each (line (ssh myserver journalctl))
     (ppcre:register-groups-bind (_ user)
         (".*Disconnected from (invalid |authenticating )?user (.*) [^ ]+ port [0-9]+( \\[preauth\\])?$"
          line)
       (when user (collect user))))))

Now, POSIX shell people use sort and uniq to count time of
occurrences:

ssh myserver journalctl
 | grep sshd
 | grep "Disconnected from"
 | sed -E 's/.*Disconnected from (invalid |authenticating )?user (.*) [^ ]+ port [0-9]+( \[preauth\])?$/\2/'
 | sort | uniq -c

We may do something just as insane as the above:

(pipe users (sort) (uniq -c))

The output is a list of table rows, each of them is a string
separated by white spaces. Rather than meddling with these
unstructured text, I recommend using structured data consisting of
real lists and numbers.

(def fail-users-alist (hash-table-alist (frequencies users)))

In POSIX shell, they use some more cryptic text-manipulating spells
to sort the table and display the 10 users with most occurrences:

ssh myserver journalctl
 | grep sshd
 | grep "Disconnected from"
 | sed -E 's/.*Disconnected from (invalid |authenticating )?user (.*) [^ ]+ port [0-9]+( \[preauth\])?$/\2/'
 | sort | uniq -c
 | sort -nk1,1 | tail -n10

In Lisp, we do it the structured data way:

(def sus-users (subseq (sort fail-users-alist #'> :key #'cdr) 0 10))

In POSIX shell, awk and paste to construct a comma separated list:

ssh myserver journalctl
 | grep sshd
 | grep "Disconnected from"
 | sed -E 's/.*Disconnected from (invalid |authenticating )?user (.*) [^ ]+ port [0-9]+( \[preauth\])?$/\2/'
 | sort | uniq -c
 | sort -nk1,1 | tail -n10
 | awk '{print $2}' | paste -sd,

In Lisp, we:

(string-join (mapcar #'car sus-users) ",")

Analyzing data

 

Now the POSIX crowd spirals deeper into insanity, constructing math
expressions using string concatenation and passing to bs:

... | paste -sd+ | bc -l

In Lisp, we simply:

(reduce #'+ (mapcar #'cdr fail-users-alist))

Data wrangling to make arguments

 

The POSIX shell way:

rustup toolchain list | grep nightly | grep -vE "nightly-x86" | sed 's/-x86.*//' | xargs rustup toolchain uninstall

The Lisp way:

(def unused-crabs
  (mapcan
   (lambda (l)
     (when (and (ppcre:scan "nightly" l)
                (not (ppcre:scan "nightly-x86" l)))
       (list (ppcre:regex-replace "-x86.*" l ""))))
   (rustup toolchain list)))
(rustup toolchain uninstall ,@unused-crabs)

Footer

 (c) 2024 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact
  * Manage cookies
  * Do not share my personal information

You can't perform that action at this time.