# Webassembly architecture

## Execution model

Webassembly follows a Harvard architecture (code lives in wasm modules, data
live in linear memory), and there are only structured jumps (out of blocks,
into branches, or to start of loops) and calls (to functions) allowed, no
arbitrary branching.  Also, there is no access to the native return stack.

Since Forth depends on being able to arbitrarily jump and also manipulate the
return stack before returning, the Webassembly architecture uses a hybrid
execution model consisting of emulation (similar to the POSIX VM), WASM
fragments (written by the kernel) and whole-word JIT optimization (implemented
in the JavaScript driver).

The kernel is compiled (currently from AssemblyScript) into a .wasm module and
loaded by the driver. It (currently) resides outside the disk image. Execution
starts in emulated mode. When the kernel compiles code to JIT fragments, the
driver collects them into a single WASM module (JIT module), which gets
dynamically recompiled in the background. In case a WASM fragment is to be
executed and the compilation is currently still ongoing, execution has to wait
on the current version of the module to be compiled.

During execution, every call to the WASM module from the emulation layer
requires one context switch from the kernel module to the JavaScript driver,
which then switches to the JIT module, and after the fragment finished,
execution gets passed back to the kernel module via the JS driver in a similar
fashion. As many JIT fragments are really small (consisting of a single HAL
operation), this execution mode is rather slow (yet it is the only mode that is
guaranteed to behave exactly like other architectures).

To speed up execution, all words that can be called and/or exist in sysdict
start with a word mark (see doc/hal.txt). When first executed, the system will
analyze the opcodes of this word (on all execution paths, until it reaches the
RET instruction or an unsupported opcode) and try to build a wasm function for
it (JIT compile it), causing a noticable speed improvement.

For example, on my machine

: foo 0 10000000 for i + next . ;

takes about 10 seconds to execute in emulated mode, and 100 milliseconds to
execute in JIT mode.

While compilation is ongoing, the word mark is replaced by a different word
mark and execution continues in interpreted mode.  All words that are called
from the word being JIT compiled will be also JIT compiled if they are not
already. The JIT process will also keep track if the word is known to never
return (like `(cell)`) or otherwise messes with the return stack in a way that
does not allow the JIT compiled word to be called from another JIT compiled
word (e. g.  the RSP stack depth is different at a code point after different
branches have been executed, the code directly accesses the RSP value, or it
accesses the stack at a location that is higher or equal to the RSP location at
function entry) and takes appropriate actions. The compiled words will be added
to the JIT wasm module and reloaded asynchronously, i.e. the interpreted
version may run for a few more times until the module it is reloaded.

When you have a very slow word, and want to ensure it is not interpreted on the
first run, you can run code like this to force its compilation:

: _ 0 if very_slow_word then ; _ _

There are three possible forms a word can be compiled to:

If the word structure is simple enough (Forth words should usually allow this)
and all calls are to words that have been JIT compiled and are safe for being
called, blocks, loops and branches are identified and the word is compiled as a
native word. This achieves the best performance.

If the word structure is more complicated (or just includes cases I did not
consider), but still all calls are to words that have been JIT compiled and are
safe for being called, the word is compiled into a state machine, with each
state representing a "label" in the code that is actually branched to.

In all other cases, the code is also compiled to a state machine, but the state
machine will not only exit if the return statement is reached, but also
whenever a vmop is reached that cannot be compiled. All those vmops are written
into a special region allocated by the kernel (currently 128 KiB just to be
safe) called the JIT vmop area. The compiled JIT word will use the special
"state.T" register to return the pointer to the JIT vmop area to the kernel,
which will switch to emulation mode and continue emulation. At the end of the
vm operations in the JIT vmop area there is a special vmop (vmop_JIT_CONTINUE)
which takes the address of the word and the next state number of the state
machine, which will pass execution back into the JIT compiled word. Unless the
vmop is a vmop_JIT_BRA, which will always has to be emulated but will never
return.


## Event handling

While a JITed WASM word is executing, no events are passed between the executed
code and the browser. This affects both events from kernel to browser (e.g.
screen updates) and events from browser to kernel (e.g. updateing timer ticks).
Usually, this is no big deal, as the code will eventually read from e.g. the
keyboard and switch back to emulation mode. In some microbenchmarks, having no
accurate timer or not updating the screen can be an issue, though.  For this
use case, the native word `wasmDoEvents` (named after its infamous predecessor
from Visual Basic 6) can be called, which forces a switch to emulated mode and
wait until the event queue has been processed before switching back to JIT
mode.
