---
layout: ../Site.layout.js
---
# C++ and embeddable common lisp habitat example ~ base64 encoding

[Habitat being jeremy_list's HaikuOS secure scuttlebutt C++ gossip protocol client](https://codeberg.org/jeremylist/habitat). Of course, I use ANSI CL and not C++, and openbsd or maybe debian and not haikuos. So let's see if we can just make the C++ bits into lisp functions in that magical [embeddable common lisp](https://ecl.common-lisp.dev/) way.

Here we continue directly from [making the conventional BSD port of `ecl` embeddable common lisp `--with-cxx`](/fundamental/cxx-ecl-openbsd-port). I'm aware that at least the [gentoo linux distribution has a C++ ecl port](https://packages.gentoo.org/packages/dev-lisp/ecl) (in gentoo, called an `ebuild`).

## Set up common lisp / eev / emacs

`#P"~/ecl-habitat-test/eat-base64-habitat.ecl.lisp"` was my working file. This is the adaptation of eev's `M-x eeit` generated infrastructure I have found myself using.

```
#|
• (setq inferior-lisp-program "ecl")
• (slime)
• (setq eepitch-buffer-name "*slime-repl ECL*")

(compile-file "~/ecl-habitat-test/eat-base64-habitat.ecl.lisp" :load t)
|#
```

`compile-file` in embeddable common lisp provides a very nice low-level-ffi, sffi into either C or C++ (whatever `ecl` had been built with (`--with-cxx` or not)).

NOTE all of ecl's `sffi` stuff is `compile-file`-only. So we have to extract the `ffi` of this article into a lisp file and call `compile-file` .. `:load t` as indicated above. The interpreter doesn't know how to compile C (it only interprets lisp).

## Taking C++ `base64::encode1` from habitat's source

I eventually came to terms with the fact that habitat used a nonportable HaikuOS C++ string library. Instead, I pulled out the base64 encoding source C++ namespace fragment. I promoted the enums and integer-like types to ints, though ecl's sffi directly supports all these common C/C++ types, but they are a bit extraneous to this minimal ecl C++ sffi example.

(Edit: Simple fix mentioned everywhere else right after publishing) Actually, from what I can tell, at least my extracted version of b64 encoding here doesn't work, but the C++ embeddable common lisp works which is our focus. [jeremy_list](https://thunix.net/~jeremylist/) will take a look once this article is up and point out where I went wrong here.

```
(ffi:clines "
namespace base64 {

enum Variant {
  STANDARD,
  URL,
};

int
encode1(int byte, int variant)
{
  if (byte <= 25) {
    return byte + 'A';
  } else if (byte <= 51) {
    return byte - 26 + 'a';
  } else if (byte <= 61) {
    return byte - 52 + '0';
  } else if (byte == 62) {
    if (variant == STANDARD)
      return '+';
    else
      return '-';
  } else {
    if (variant == STANDARD)
      return '/';
    else
      return '_';
  }
}

}
")
```

## A lisp defun whose body is that C++ function

`ffi:defentry` ([sffi docs](https://ecl.common-lisp.dev/static/manual/Foreign-Function-Interface.html#SFFI-Reference)) attaches a C/C++ function as body to a common lisp function. This is just what the form is like.

```
(ffi:defentry c-encode1
    (:int :int)
  (:int |base64::encode1|))
```

`ffi:defentry`'s arguements are

1. name (so the lisp function will be named `c-encode1`)
1. C/C++ argument types (I just promoted them to ints - if you want to [you can get the underlying types and enum right](https://ecl.common-lisp.dev/static/manual/Foreign-Function-Interface.html#Primitive-Types))
1. `(return-type c-or-c++-function-name)`

## A common lisp stab at b64 encoding using that

In one big common lisp loop sorry. I just tried to literally match jeremy_list's code rather than refer to openbsd's b64encode myself. So I am not sure what translation mistake has crept in: Anyway, the function 'works' for what it is it happens to do.

```
(defun b64encode (bytes &optional (variant 0))
  (loop
    :with mask6 := #b111111

    :for n :from 0 :by 4
    :for byte-1 :in bytes :by #'cdddr
    :for byte-2 :in (cdr bytes) :by #'cdddr ; I forgot this cdr
    :for byte-3 :in (cddr bytes) :by #'cdddr ; and cddr originally here.

    :for result-1
      := (c-encode1 (ash byte-1 -2) variant)
    :for partial-1
      := (logand (ash byte-1 4)
		 mask6)

    :for result-2
      := (c-encode1 (logior partial-1
			    (ash byte-2 -4))
		    variant)
    :for partial-2
      := (logand (ash byte-2 2)
		 mask6)

    :for result-3
      := (c-encode1 (logior (ash byte-3 -6)
			    partial-2)
		    variant)
    :for result-4
      := (c-encode1 (logand byte-3 mask6)
		    variant)

    :nconcing
    (list
     result-1 result-2 result-3 result-4)
      :into b64s
    :finally
       (return
	 (coerce
	  (mapcar 'code-char b64s)
	  'string))))
```

## Seeing that working ("working")

```
#|
• (setq inferior-lisp-program "ecl")
• (slime)
• (setq eepitch-buffer-name "*slime-repl ECL*")
(load "eat-base64-habitat.ecl.lisp")

"abcdefghijkl"
;; "abcdefghijkl"
(coerce * 'list)
;; (#\a #\b #\c #\d #\e #\f #\g #\h #\i #\j #\k #\l)
(mapcar 'char-code *)
;; (97 98 99 100 101 102 103 104 105 106 107 108)
(b64encode *)
;; YWJjZGVmZ2hpamts ; now works.
;; "YWFhZGRkZ2dnampq" ; When I forgot those cdrs.
```

Well, this (Edit: Now works, simple typo above) seem[ed] wrong viz openbsd [b64encode(1)](https://man.openbsd.org/b64encode):

```
• (eepitch-shell)
printf "abcdefghijkl" | b64encode lowercase
;; " | b64encode lowercase
;; begin-base64 644 lowercase
;; YWJjZGVmZ2hpamts
;; ====
|#
```
Edit: Now it's working. Forgot three `cdr`s.

Formerly: Though it seems oddly close doesn't it. But it's difficult for me to guess what has gone wrong.

Still, the C++ program code was made available in common lisp. So that has gone right.

# Conclusions

We saw C++ embeddable common lisp compile and connect common lisp to jeremy_list's original C++ namespace using its `sffi` (which is `ecl`; not the external `libffi` - I have met people who just guess only libffi exists), and used it to write a portable lisp version of the base64 encoding rather than a nonportable haikuos C++ one. So we can straightforwardly salvage other projects as starting points; especially C++ programs and libs.

A simple sanity check with openbsd shows something like 2/4 bytes seem to agree with openbsd's b64encode implementation. Edit: This turned out to be a simple fix.

Threeish future implications being ecl inhabiting a C++ program to serve [McCLIM application-frames](https://mcclim.common-lisp.dev/main.html) into the running C++ program ; relatedly, starting a swank server inside a pre-existing C++ program and [slime-connecting](https://slime.common-lisp.dev/doc/html/) into the running program from emacs ; lastly, and it's been on my todo-list a while - using the [enzyme library](https://enzyme.mit.edu/) for automatic differentiation and friends with the previous, especially together with [my Leonardo system](/lispgames/LCKR-object-oriented-simulation-simulation/).

# Fin.

[Talk on the Mastodon as always please](https://gamerplus.org/@screwlisp/114963441276577775).

Oh! One hour from right now 8am Zulu Sunday I'm going to have a peertube live with [Kasper Galkowski](https://itch.io/profile/kgal) about common lisp indie gamedev with opengl, which is at least an ffi adjacent topic. Watch the Mastodon, again please.

[Edrx](https://anggtwu.net/) has been working on emacs eev-mode [kitten](https://kitten.small-web.org/) and has one [for me personally](https://anggtwu.net/elisp/screwlispkitten.el.html) I need to work through - this will shape what these articles are like going forward.