https://9p.io/sys/doc/mk.html
Maintaining Files on Plan 9 with Mk
Andrew G. Hume
andrew@research.att.com
Bob Flandrena
bobf@plan9.bell-labs.com
ABSTRACT
Mk is a tool for describing and maintaining dependencies between
files. It is similar to the UNIX program make, but provides several
extensions. Mk's flexible rule specifications, implied dependency
derivation, and parallel execution of maintenance actions are
well-suited to the Plan 9 environment. Almost all Plan 9 maintenance
procedures are automated using mk.
1. Introduction
This document describes how mk, a program functionally similar to
make [Feld79], is used to maintain dependencies between files in Plan
9. Mk provides several extensions to the capabilities of its
predecessor that work well in Plan 9's distributed,
multi-architecture environment. It exploits the power of
multiprocessors by executing maintenance actions in parallel and
interacts with the Plan 9 command interpreter rc to provide a
powerful set of maintenance tools. It accepts pattern-based
dependency specifications that are not limited to describing rules
for program construction. The result is a tool that is flexible
enough to perform many maintenance tasks including database
maintenance, hardware design, and document production.
This document begins by discussing the syntax of the control file,
the pattern matching capabilities, and the special rules for
maintaining archives. A brief description of mk's algorithm for
deriving dependencies is followed by a discussion of the conventions
used to resolve ambiguous specifications. The final sections describe
parallel execution and special features.
An earlier paper [Hume87] provides a detailed discussion of mk's
design and an appendix summarizes the differences between mk and make
.
2. The Mkfile
Mk reads a file describing relationships among files and executes
commands to bring the files up to date. The specification file,
called a mkfile, contains three types of statements: assignments,
includes, and rules. Assignment and include statements are similar to
those in C. Rules specify dependencies between a target and its
prerequisites. When the target and prerequisites are files, their
modification times determine if they are out of date. Rules often
contain a recipe, an rc(1) script that produces the target from the
prerequisites.
This simple mkfile produces an executable from a C source file:
CC=pcc
f1: f1.c
$CC -o f1 f1.c
The first line assigns the name of the portable ANSI/POSIX compiler
to the mk variable CC; subsequent references of the form $CC select
this compiler. The only rule specifies a dependence between the
target file f1 and the prerequisite file f1.c. If the target does not
exist or if the prerequisite has been modified more recently than the
target, mk passes the recipe to rc for execution. Here, f1.c is
compiled and loaded to produce f1.
The native Plan 9 environment requires executables for all
architectures, not only the current one. The Plan 9 version of the
same mkfile looks like:
$objtype/mkfile
f1: f1.$O
$LD $LDFLAGS -o f1 f1.$O
f1.$O: f1.c
$CC $CFLAGS f1.c
The first line is an include statement that replaces itself with the
contents of the file /$objtype/mkfile. The variable $objtype is
inherited from the environment and contains the name of the target
architecture. The prototype mkfile for that architecture defines
architecture-specific variables: CC and LD are the names of the
compiler and loader, O is the code character of the architecture. The
rules compile the source file into an object file and invoke the
loader to produce f1. Invoking mk from the command line as follows
% objtype=mips mk
vc -w f1.c
vl $LDFLAGS -o f1 f1.k
%
produces the mips executable of program f1 regardless of the current
architecture type.
We can extend the mkfile to build two programs:
$objtype/mkfile
ALL=f1 f2
all:V: $ALL
f1: f1.$O
$LD $LDFLAGS -o f1 f1.$O
f1.$O: f1.c
$CC $CFLAGS f1.c
f2: f2.$O
$LD $LDFLAGS -o f2 f2.$O
f2.$O: f2.c
$CC $CFLAGS f2.c
The target all, modified by the attribute V, builds both programs.
The attribute identifies all as a dummy target that is not related to
a file of the same name; its precise effect is explained later. This
example describes cascading dependencies: the first target depends on
another which depends on a third and so on. Here, individual rules
build each program; later we'll see how to do this with a general
rule.
3. Variables and the environment
Mk does not distinguish between its internal variables and rc
variables in the environment. When mk starts, it imports each
environment variable into a mk variable of the same name. Before
executing a recipe, mk exports all variables, including those
inherited from the environment, to the environment in which rc
executes the recipe.
There are several ways for a variable to take a value. It can be set
with an assignment statement, inherited from the environment, or
specified on the command line. Mk also maintains several special
internal variables that are described in mk(1). Assignments have the
following decreasing order of precedence:
1) Command line assignment
2) Assignment statement
3) Imported from the environment
4) Implicitly set by mk
For example, a command line assignment overrides a value imported
from the environment.
All variable values are strings. They can be used for pattern
matching and comparison but not for arithmetic. A list is a string
containing several values separated by white space. Each member is
handled individually during pattern matching, target selection, and
prerequisite evaluation.
A namelist is a list produced by transforming the members of an
existing list. The transform applies a pattern to each member,
replacing each matched string with a new string, much as in the
substitute command in sam(1) or ed(1). The syntax is
${var:A%B=C%D}
where var is a variable. The pattern A%B matches a member beginning
with the string A and ending with the string B with any string in
between; it behaves like the regular expression A.*B. When a member
of the var list matches this pattern, the string C replaces A, D
replaces B, and the matched string replaces itself. Any of A, B, C,
or D may be the empty string. In effect, a namelist is generated by
applying the ed(1) substitute command
s/A(.*)B/C\1D/
to each member of a variable list.
Namelists are useful for generating a list based on a predictable
transformation. For example,
SRC=a.c b.c c.c
OBJ=${SRC:%.c=%.v}
assigns the list (a.v b.v c.v) to OBJ. A namelist may be used
anywhere a variable is allowed except in a recipe.
Command output is assigned to a variable using the normal rc syntax:
var='{rc command}
The command executes in an environment populated with previously
assigned variables, including those inherited from mk's execution
environment. The command may be arbitrarily complex; for example,
TARG='{ls -d *.[cy] | sed 's/..$//'}
assigns a list of the C and yacc source files in the current
directory, stripped of their suffix, to the variable TARG.
4. The include statement
The include statement replaces itself with the contents of a file. It
is functionally similar to the C #include statement but uses a
different syntax:
$target
produces the message
mk: pic mk.ms | ... : exit status=rc 685: deleting 'pic.out'
if any program in the recipe exits with an error status.
14. Unspecified dependencies
The -w command line flag forces the files following the flag to be
treated as if they were just modified. We can use this flag with a
command that selects files to force a build based on the selection
criterion. For example, if the declaration of a global variable named
var is changed in a header file, all source files that reference it
can be rebuilt with the command
$ mk -w'{grep -l var *.[cyl]}
15. Conclusion
There are many programs related to make, each choosing a different
balance between specialization and generality. Mk emphasizes
generality but allows customization through its pattern
specifications and include facilities.
Plan 9 presents a difficult maintenance environment with its
heterogeneous architectures and languages. Mk's flexible
specification language and simple interaction with rc work well in
this environment. As a result, Plan 9 relies on mk to automate almost
all maintenance. Tasks as diverse as updating the network data base,
producing the manual, or building a release are expressed as mk
procedures.
16. References
[Cmel86] R. F. Cmelik, ''Concurrent Make: A Distributed Program in
Concurrent C'', AT&T Bell Laboratories Technical Report, 1986.
[Feld79] S. I. Feldman, ''Make -- a program for maintaining computer
programs'', Software Practice & Experience , 1979 Vol 9 #4, pp.
255-266.
[Flan95] Bob Flandrena, ''Plan 9 Mkfiles'', this volume.
[Hume87] A. G. Hume, ''Mk: A Successor to Make'', USENIX Summer Conf.
Proc., Phoenix, Az.
17. Appendix: Differences between make and mk
The differences between mk and make are:
[?] Make builds targets when it needs them, allowing systematic use of
side effects. Mk constructs the entire dependency graph before
building any target.
[?] Make supports suffix rules and % metarules. Mk supports % and
regular expression metarules. (Older versions of make support only
suffix rules.)
[?] Mk performs transitive closure on metarules, make does not.
[?] Make supports cyclic dependencies, mk does not.
[?] Make evaluates recipes one line at a time, replacing variables by
their values and executing some commands internally. Mk passes the
entire recipe to the shell without interpretation or internal
execution.
[?] Make supports parallel execution of single-line recipes when
building the prerequisites for specified targets. Mk supports
parallel execution of all recipes. (Older versions of make did not
support parallel execution.)
[?] Make uses special targets (beginning with a period) to indicate
special processing. Mk uses attributes to modify rule evaluation.
[?] Mk supports virtual targets that are independent of the file
system.
[?] Mk allows non-standard out-of-date determination, make does not.
It is usually easy to convert a makefile to or from an equivalent
mkfile.