# Units and dependencies

In Dusk, we call a "unit" a collection of words tied to a certain name (the unit
name). For example, the unit "lib/str" provides the word "s)". Its structure is
described in [doc/data].

In their primary use case units are tied to a source file of the same name.  For
example, the source file associated with unit "lib/str" is lib/str.fs. Its
associated documentation lives at doc/lib/str.txt.

A unit is declared with the word "unit". For example, "unit lib/foo" declares
the unit "lib/foo". All following dictionary entries following this declaration
will be associated with the unit "lib/foo". This association stops when the word
"endunit" is called. Following words will be unassociated.

Units can be nested, but use cases for this feature are rare. See below.

Declaring a unit creates a word of the same name. That word is an immediate
that, when called, "bubbles" its associated words on top of the system
dictionary.

This simple little trick allows complex dependency chains to occur while
avoiding word shadowing problems that inevitably crop up as a Forth codebase
becomes wider.

For example, let's say that you loaded "lib/str" earlier, but then created a new
word called "s)" that has completely different semantics that "lib/str"'s "s)".

So far, so good. You can do that and everything will work fine. However, now you
want to load "app/foobar" which depends on "lib/str" and it just so happens that
it needs to use "s)". Guess what? it's going to pick up the wrong "s)" and
"app/foobar" is going to be broken.

But if "app/foobar" calls "lib/str", then all those words are going to be
bubbled up to the top in the system dictionary. The words don't actually move,
only the dictionary chain links change (so all compiled references to those
words are still valid, of course). But because "app/foobar" declared that need,
the the "s)" it actually needs will re-shadow the "s)" that shadowed it earlier.

Let's see an example where "lib/a" define a single word, "a", "lib/b", "b" and
"lib/c", "c":

  f<< lib/a.fs
  f<< lib/b.fs
  f<< lib/c.fs
  words \ prints "c b a (... rest of system dict)"
  lib/a
  words \ prints "a c b ..."
  lib/c lib/b
  words \ prints "b c a ..."

## The "needs" word

But actually, calling unit words directly is not how most of Dusk's code declare
its dependencies. It rather uses the word "needs". When called, "needs" reads
the rest of the line and then expects a list of unit names separated by
whitespace. The last element of the line must be directly followed by a LF (no
whitespace in between).

As long as [fs/core] is not loaded, "needs ..." is the equivalent to calling
each of the following words directly. Once [fs/core] is initialized, it checks
whether the name exists in the system dict and automatically loads the unit from
file if it doesn't. The associated file is "<unitname>.fs". For example, doing
"needs lib/str" when "lib/str" doesn't exist in the system dictionary would
do "f<< lib/str.fs" first, then call "lib/str".

"needs" cannot be called inside a unit definition.

Sometimes, the "needs" list can be quite lengthy and span over multiple lines.
Rather than calling "needs" more than once, which is incorrect, you can insert
a "\" at the end of a line to indicate that the unit list continues on the
following line.

  needs a \
    b \
    c

is exactly equivalent to:

  needs a b c

## The two stages of needs

Naively, you could think that:

  needs lib/a lib/b lib/c

and:

  needs lib/a
  needs lib/b
  needs lib/c

are equivalent. You'd *almost* be right. If all libraries are already loaded in
memory, they are equivalent. If one of the libraries is not, then the two forms
are not equivalent because the "bubbling up" that is a side effect of the
loading will mess up our own top level "bubbling up".

For example, if "lib/c" wasn't already in memory and that it depended on
"lib/a", then "lib/a" is going to end up "on top" of "lib/b", which is not the
intent.

It sets the stage for some really tricky bugs and we don't want that. We want a
given namespace following a "needs" to be deterministic.

Therefore, needs operates in two stages. First, it goes through the specified
units and ensure that they're loaded in memory. Then, it goes through the list
again, from left to right, then "bubble up" each unit one after the other.

This ensures a deterministic namespace, independently of prior "loaded" state of
the specified dependencies.

## Handling name clashes

In Dusk's code, we avoid both namespaces and word name prefixes. Each unit
chooses its name for "maximum local elegance", that is, the code should be
understandable locally, without adding spurious prefixes to avoid global name
clashes.

In practice, we find that such clashes rarely happen because those clashing
words are rarely used in the same context.

... but it *can* happen. When that happens, you have two choices: calling the
unit word or using unitalias.

For example, let's imagine that both "lib/foo" and "lib/bar" have a "xlerb"
word. As long as they're not used together, no problem. They could even be used
together, but only one of the "xlerb" is used, so it's only a matter of
correctly ordering the "needs".

But if we need to use them both, we could do either:

  needs lib/foo lib/bar
  : callbothxlerbs lib/foo xlerb lib/bar xlerb ;

or you could do:

  needs lib/foo lib/bar
  unitalias lib/foo xlerb fooxlerb
  unitalias lib/bar xlerb barxlerb
  : callbothxlerbs fooxlerb barxlerb ;

The first option is the easiest, but it shouldn't be used inside another unit
because otherwise, you "import" that unit's words into your present unit.

Using "unitalias" is more verbose, but it does the right thing when called
inside a unit. As with regular aliases, unit aliases copy their target immediate
and findselector flags.

## System unit

There is a builtin unit called "system". When called, it bubbles up all kernel
words as well as the words defined in xcomp/boot.fs.

You usually don't need to do this because most code avoid shadowing core words,
but some do. For example, assemblers often have words shadowing the HAL. It's
usually with words that have the same semantics, but if you're loading an
assembler that is for a different CPU than the live one, then you're breaking
your HAL. You can fix it with "system".

## Nested units

Units can be nested:

  unit foo
    : x ;
    unit bar
      : y ;
    endunit
    : z ;
  endunit

In that example above, "x", "y", "z" and "bar" are initially all part of the
"foo" unit and "y" is part of the "bar unit.

As soon as "bar" is called once, however, "y" is bubbled up and leaves the "foo"
unit. "bar" stays in "foo".

Nestability can be used to wrap Big Moustache [lib/bm] fields in named structs.
Most of the time, a given unit handles one, or very few structs, so the
namespace noise of a nested unit makes it not worth it.

But some units, such as drv/usb/usb.fs manage many structs with conflicting
field names. In those rare cases, nested units are useful. For example:

  unit lib/foo
    unit Struct1
      value{} id
      value{++} foo
    endunit
    unit Struct2
      value{} id
      value{++} bar
    endunit
    : showstructs ( s2 s1 -- )
      Struct1 bi foo | id .f"Struct1: id %d foo %d"
      Struct2 bi bar | id .f"Struct2: id %d bar %d" ;
  endunit

Nested units, in these cases, allowed us to keep the "id" fields named "id"
rather than naming them "struct1id" and "struct2id".

An implementation detail brings in an important caveat with nested units: a
nested unit cannot directly "end" its parent unit. There must be a word between
the nested "endunit" and its parent "endunit". If there isn't weird dictionary
bugs will show up.

So, that's the spirit of nested units. But generally, avoid them.

## Units and compile time aborts

When developing a unit, it can happen that you stumble on a compile time abort.
When this happens, the unit will be left "opened" and you'll be stuck with an
error "needs inside unit!" whenever you try to load another unit. Just call
"endunit" to fix this.
