1. Interscript Version
----------------------

This document describes Interscript version 1.0a8 build
1384 on ruby by root.It was generated by Interscript
version 1.0a8, build 1383 on ruby by root at Mon Nov
30, 1998 at 05:02 AM (UTC).

1.1. Contents
-------------

   Interscript Version .............................. 1
      Contents ...................................... 9
   Introduction .................................... 74
      Interscript ................................. 102
      Features .................................... 134
   Requirements ................................... 350
      Functionality ............................... 385
      Interface ................................... 770
      Management .................................. 864
      Implementation pragmatics .................. 1021
   Design Fundamentals ........................... 1064
      Devices .................................... 1070
      Processors ................................. 1251
      Tanglers ................................... 1282
      Lexical Scoping ............................ 1324
      Weaver Control ............................. 1356
      Source Tracking ............................ 1377
      Parsing .................................... 1434
      Documentation Constructions ................ 1460
      Microformatting shortcuts .................. 1495
      Weaver Architecture ........................ 1586
   Tutorial ...................................... 1718
      Weaving a document ......................... 1726
      Tangling code .............................. 1933
      Scripting .................................. 2014
      Unit tests ................................. 2122
      Fonts ...................................... 2240
      Lists ...................................... 2268
      Tables ..................................... 2338
      Citations .................................. 2358
      Cross References ........................... 2368
      Including Files ............................ 2394
      Translating Html ........................... 2436
      Special constructions ...................... 2448
      File Names ................................. 2547
      Questions and Answers ...................... 2596
   Implementation ................................ 2857
      Interscript Module ......................... 2862
      Core Subpackage ............................ 3534
      Drivers Subpackage ......................... 3679
      Weavers .................................... 4245
      Weaver Filters ............................. 5577
      Tanglers ................................... 5625
      Tokenisers ................................. 5994
      Html parser test ........................... 6182
      LALR(1) Parser table generator ............. 6214
      Introduction to categories ................. 7017
      Felix Object protocol ...................... 7061
      Felix Package .............................. 7138
      Architecture ............................... 7668
      Module getoptions .......................... 8806
      Get Options ................................ 8839
      Utility Modules ............................ 8892
      Application and tool directory ............. 9181
      Test package ............................... 9229
   Appendices .................................... 9240
      File List .................................. 9244
      Source List ................................ 9867
      Include List ............................... 9920
      Bugs (etc) for Version 1a7 ................. 9974
      Installation Guide ........................ 10432

2. Introduction
---------------

Interscript is a component of a new style of
programming environment incorporating the ideas of
literate programming and category theory. Literate
programming was first embodied in significant code by
Donald Knuth in a program called Web, to support
publication of the sources for his TeX typesetting
system.

The fundamental precept of literate programming is that
documentation should not be written after software
development, but be an integral part of it. A literate
programming source is primarily a document in which
source is embedded; the source is extracted by the
literate programming tool by a process called tangling,
while the file is typeset by a process called weaving.

Unlike traditional documentation, whether it be design
specifications, or, code or user documentation,
literate programming offers the opportunity for the
documentation and software to remain synchronised
because the two are lexically close and easy to
maintain together. Furthermore, the published sources
not only match the software exactly, but are sure to be
complete.

2.1. Interscript
----------------

Interscript is a major breakthough in the design of
literate programming tools. Other tools such as Web,
C-Web, and FunnelWeb, have limited functionality, and
are mainly restricted to partitioning code and
documentation.

Interscript is different because it embodies a complete
and fully functional programming language, namely
Python. An Interscript source file consists of three,
not two, kinds of source: the target source code,
documentation, and executable Python script. Because of
this feature, the user source can extend the tool in
arbitrary ways at _run time_ (without modifying the
original tool).

In addition, the basic Interscript tool has an advanced
object based design, including pluggable drivers for
data sources and sinks, including automatic file
downloading, tanglers for various programming languages
including C, C++, Python, generic script , and raw
data, and plugable weavers for various typesetting
systems including HTML, TeX, Postscript, and plain
text.

Interscript includes functionality designed to support
building, testing, installation, and verion control of
software. In particular, the standard system includes
functionality designed to replace _make_.

2.2. Features
-------------

Interscript supports many powerful features.

2.2.1. Python Scripting
-----------------------

Interscript uses Python 1.5.1 both for its
implementation and to supply scripting services to
client sourceware.

2.2.2. Platform independent filenames
-------------------------------------

Are required in interscript sourceware. Filenames must
be unix style relative filenames; interscript maps
these to host operating system filenames automatically.

2.2.3. Documentation Constructions
----------------------------------

Several specialised documentation constructions are
provided apart from paragraphs and headings.

2.2.3.1. Automatic heading numbering
------------------------------------

Headings are automatically numbered, and may be nested
to any depth.

2.2.3.2. Cross references
-------------------------

References to any labelled part of the document can be
cited. Headings may be labelled, and a label may be set
within ordinary text.

2.2.3.3. Tables
---------------

Basic tables with headings are provided.

2.2.3.4. Nestable Lists
-----------------------

Lists are provided in bullet, numbered and keyed
styles. Lists can be nested.

2.2.3.5. URL citation
---------------------

A specialised contruction for citing a URL becomes an
active link in HTML documents.

2.2.3.6. Code and Prose displays
--------------------------------

Display constructions for quoting code and prose are
provided. [There is no support for mathematics yet.]

2.2.3.7. Basic font selection
-----------------------------

For emphasised, strong, italic, bold, or code fonts.

2.2.4. Advanced Web Weaving
---------------------------

The advanced web weaver supports many features.

2.2.4.1. Automatic pagination on headings
-----------------------------------------

The web weaver spawns child pages on each heading.

2.2.4.2. Syntax Highlighting
----------------------------

For tokenising tanglers, syntax highlighting is
supported by use of the SPAN tag with CLASS attribute.

2.2.4.3. CSS1 Cascading Style Sheets
------------------------------------

A standard interscript style sheet, interscript.css is
provided. A dummy style sheet, user.css, is provided to
support client style overrides.

2.2.4.4. Folding Table of Contents
----------------------------------

Using ECMAscript, operates conditionally on presence of
Internet Explorer style document object model (DOM).

2.2.4.5. Frame presentation
---------------------------

A simple three frame presentation with a master index
control panel, an index frame for holding various cross
reference tables, and a document view frame.

2.2.4.6. Flexible navigation
----------------------------

Navigation is via a cross reference index, or via
standard navigation links per page or per code section.

2.2.4.7. Table of classes
-------------------------

Points of definition of classes.

2.2.4.8. Table of functions
---------------------------

Points of definition of functions and methods.

2.2.4.9. Table of identifiers
-----------------------------

Points of definition and use for identifiers.

2.2.4.10. Table of sections
---------------------------

List of sections making up a code file.

2.2.4.11. Table of tests
------------------------

Table of tests with test results where applicable.

2.2.4.12. Convergence status
----------------------------

List of all output files with an indication of whether
the file was changed.

2.2.4.13. Source tree
---------------------

Heirachical list of all input files.

2.2.5. PerlPOD support
----------------------

Perl POD is recognized and woven into documents.

2.2.6. Extensive Python Support
-------------------------------

Specialised Python support includes numerous general
purpose extension modules, a function generator
supporting Eiffel style assertions including protocol
verification, preconditions, postconditions, and per
argument documentation.

2.2.7. Latex, Plain text and flat Html Weavers
----------------------------------------------

Weavers are provided for Latex2e, plain text, and a
single flat html file.

2.2.8. Html input filter
------------------------

Allows simple Html to be processed as input.

2.2.9. XML support planned
--------------------------

XML is not currently supported, but and XML weaver and
input filter is planned.

2.2.10. Unit testing
--------------------

On the fly unit tests can be embedded in code and are
marshalled in a table. Special support for Python is
available to execute the tests.

2.2.10.1. Test output verification
----------------------------------

Comparison of expected and actual output presented in a
difference table if gnu diff is available.

2.2.11. Option help
-------------------

Is provided with the --help command line option.

2.2.12. Tutorial
----------------

A tutorial is provided which covers some basic
concepts.

2.2.13. Statement of Requirements
---------------------------------

Discusses and tabulates some specific requirements.

2.2.14. Design Document
-----------------------

Discusses some design and implementation issues.

2.2.15. Full source listing
---------------------------

Interscript is literate programmed with itself, so that
a full source listing is embedded in the implementation
documentation.

3. Requirements
---------------

Interscript is a software component intended to assist
in the publishing and development of software by
providing integrated source code, documentation,
testing and project management facilities.

The detailed requirements stem from fundamental design
decisions including the choice of python as an
implementation language, the decision to restrict the
user interface to batch processing of text files, and
the embodiment of the principal notions of literate
programming.

The sections below discuss the requirements from the
point of view of desired functionality, interface,
implementation pragmatics, and management.

However, it is not possible to entirely separate
discussion of requirements from design, because
detailed requirements stem from design decisions, and
guide more detailed design. Nor is it possible to
separate implementation from design, since
implementation details have design consequences -- and
also provided unexpected opportunities for
functionality not in the original requirements.

In other words you can expect some discussion of design
and implementation in the section, although the
emphasis is on requirements. See the next major section
for a discussion focusing on design, and the following
section for a detailed description of the
implementation.

3.1. Functionality
------------------

This section discusses the functionality Interscript
requires.

The overall functionality is easy to describe:
interscript must process input source files and from
them extract the target program files, and extract and
format a suitable set of documentation files. In
addition it must be able to compile and build the
target software, execute test code, and present the
results of all these processes in a comprehensible
format.

We will call the goal of program code extraction
_tangling_, that of document construction _weaving_,
and that of compilation and test code execution
_building_.

3.1.1. Tangling
---------------

This section describes the requirements of the tangling
goal, and is by far the simplest to state (and
implement).

The client will largely present files containing one or
more program code files, separated by documentation
sections. The tangler basically extracts this code,
concatenates the sections sequentially, and writes the
output verbatim to code files. Code sections targeting
a single output file must be associated, and the target
file identified and its location determined.

The detailed requirements for tangling, therefore,
largely focus on deviations from normal processing.

3.1.1.1. Supported Programming Languages
----------------------------------------

Special parsers shall be supplied for C, C++, Python,
Java, Eiffel, Pascal, Modula, Perl. Tcl, interscript,
and Unix shell scripts are too quirky. Basic is too
ugly :-) Cobol, Fortran and PL/1 are too archaic.

3.1.1.2. Source Tracking
------------------------

Source tracking is the ability to determine where
generated program and documentation sections came from
in the original source files. It is vital at the
building stage, so that errors can be corrected. Where
target tools identify the source of errors, they should
be guided if possible, to point at the original
sources.

3.1.1.2.1. C and C++
--------------------

For C and C++, interscript must generate #line
preprocessing directives.

3.1.1.3. Chunking
-----------------

The ability to build program files out of order is
sometimes called chunking. It is sometimes useful, for
example, to define functions after they are used even
when the target programming language requires
otherwise. However, chunking is not restricted to
reordering code sections, but may be considered to
include the ability to nest sections hierarchically.
This permits the author to represent program structure
in manner not provided by the target programming
language.

There are some further variations on chunking. The
first consideration is that code often has to be
repeated, and so a section may be used more than once,
possibly in distinct program files, but sometimes even
in the same file. Because hierarchical chunking
requires naming chunks, reuse is available
automatically; the principal issue here is the
converse: to ensure when required that a chunk is used
exactly once.

Stemming from the use of labelled chunks is the need to
be able to locate them: when a chunk is used, one needs
to find where it is defined, and sometimes conversely.

More generally: a simple verbatim code file is a
special case of breaking a code file into sections
separated by documentation but without reordering,
which is a special case of reorderable sections which
is a special case of hierarchical chunking. Again, a
tree is a special case of a directed acyclic graphs,
which embodies the notion that a chunk may be used
twice and also that it may not be used at all.

Some conventional literate programming tools go further
in permitting what might be called macros: chunks
containing unbound variables which can be bound at the
point of use: a simple case of parameterized macro
processing. Macros are particularly useful for
generating repetitive forms such as tables.

In addition, we might consider conditional compilation,
which is useful for controlling software variations
such as platform dependencies and debugging versions.
Conditional compilation is heavily used in C.

There is yet a more general form of chunking ... in
which the notion of the chunk begins to degenerate,
namely generation or arbitrary code by executable
script. Such facility is essential, for example, where
the code to be generated is sensitive to the
environment, for example the inclusion of the current
date. Perhaps more interesting is the ability of
executable script to generate several files
simultaneously, for example both the declaration in a
header file and the definition in the body file, of a
function in the C programming language. Building
specialized scripts tailored to the client's
requirement is an essential facility of literate
programming tools because, although rarely used, it can
save a lot of work and provide considerable coherence,
as well as generating tabulated documentation.

To provide all these facilities, so the most
specialized, and most common case integrates seamlessly
with the most complex and general, interscript
leverages the Python scripting engine. The most complex
code generation is supported almost effortlessly by
simply allowing the client to write arbitrary python
script, while the more common simpler requirements are
simply provided as pre-built routines: all the
facilities are accessed in precisely the same way, by
script execution.

This feature is central to the interscript design and
cannot be isolation from a discussion of requirements,
which can be recast, in some sense, to a discussion of
the architectural framework in which such script
executes, and the set of pre-built functions which
ought be made available.

Macros with parameters, however, are not especially
good at expression skeletons. A skeleton, or
boilerplate, is essentially a macro with large
arguments.

3.1.1.4. Parsing for reference tables
-------------------------------------

Where possible, interscript should tokenise and parse
or partially parse program files to extract summary
data.

3.1.1.4.1. Tokenisable Languages
--------------------------------

The following languages can be tokenised easily: C,
C++, Python, Java, Eiffel, Pascal, Modula. [Only the
Python tokeniser has been written].

3.1.1.4.1.1. Identifier reference
---------------------------------

For each language which can be tokenised, produce an
identifier reference: C, C++, Python, Java, Eiffel.
[Table generator done for web, html, latex]

3.1.1.4.2. Fully Parsable Languages
-----------------------------------

Python and Java. C and C++ cannot be fully parsed
without semantic analysis, and even then, it is tricky:
it is hard to determine if a statement is a function
declaration, variable declaration, or executable. [Only
partial python parsing has been implemented]

3.1.1.4.3. Partially Parsable Languages
---------------------------------------

C and C++ can be partially parsed: information
extracted may not be completely correct.

3.1.1.4.3.1. Class reference
----------------------------

For each language which supports user defined types,
particularly classes, provide a table of classes.
Python, Java, C, C++, Eiffel, Pascal, Modula. [Table
generator done for web, html, latex]

3.1.1.4.4. Function reference
-----------------------------

For languages supporting functions, provide a function
reference. This includes methods/member functions.
Provide for C++, Python, Java, Eiffel, Pascal, Modula.
[Table generator done for web, html]

3.1.1.5. Parsing for embedded documentation
-------------------------------------------


3.1.1.5.1. Perl DOC
-------------------

Perl DOC is arbitrary documentation provided in
comments, to be converted to interscript method calls.
[Done]

3.1.1.5.2. Java DOC
-------------------

Java DOC is a commenting protocol, and depends on
parsing to identify the entity to which the
documentation refers. [Not implemented]

3.1.1.5.3. Eiffel
-----------------

Eiffel provides dedicated documentation constructions.
[Not implemented]

3.1.1.5.4. Python Doc strings
-----------------------------

Python supports Doc strings with a standard protocol
for module and class documentation. Although the Python
runtime supports module, class, and function Doc
strings, there is no standard way to relate Doc strings
to functions. [Not implemented]

3.1.1.6. Internationalisation
-----------------------------

Interscript tanglers should provide support for
multiple human languages. There are two kinds of
support determined by the binding time:

Interscript time binding 
     Interscript time binding allows generating target
    code for a specified language. If multiple
    languages are specified, multiple versions of the
    target code are generated.

Run time binding 
     Run time binding allows generating target code for
    a specified language set. Even if multiple
    languages are specified, only a single version of
    the target code is generated.

There are two levels of support which can be provided,
which are not independent of binding time.

Identifiers 
     Using names in the programmers native language to
    aid maintenance. This is clearly an interscript
    time option.

Strings 
     Using strings in the client native language to aid
    use. This can be done either at interscript time or
    target software run time.

See alsoline 722 for details on the documentation
aspects of internationalisation.

3.1.2. Weaving
--------------

Weaving is the most complex subsystem.

3.1.2.1. Individual Weavers
---------------------------

There shall be weavers producing plain text [done],
simple HTML 4 (html) [done], advanced HTML 4 multi page
(web) [done], XML [not implemented], latex2e [done].

3.1.2.2. Individual weaver capabilities
---------------------------------------

These capabilities must be provided in each separate
weaver.

3.1.2.2.1. Fonts
----------------

Weavers should support italic, bold, emphasized, strong
and code fonts.

3.1.2.2.2. Lists
----------------

Weavers shall support bullet, numbered, and keyed
lists.

3.1.2.2.3. Displays
-------------------

Weavers shall support displays for code and prose, both
inline and from a separate source. Latex, shall support
math display.

3.1.2.2.4. Tables
-----------------

Weavers shall support simple tables with horizontal
spanning and column headings.

3.1.2.2.5. Headings
-------------------

Weavers shall support multilevel headings with
automatic heading number generation, and labels
(anchors).

3.1.2.2.6. Code echo
--------------------

Weavers shall provide a method to displayed line
numbered code lines with labels (anchors).

3.1.2.2.7. Citations
--------------------

Weavers shall support citation of URLs, print media,
code files, and interscript generated documents.

3.1.2.3. Internationalisation
-----------------------------

Internationalisation of documentation consists of two
facets: automatic substitution of fixed literals such
as the titles of tables, and providing for alternate
translations of client documentary text.

The former requirement should be met by run time
tangler binding of interscript itself, seeline 625 for
details.

3.1.3. Building
---------------


3.1.3.1. Python hosting
-----------------------

Interscript shall be able to host python script,
optionally capturing standard output.

3.1.3.2. Compilers
------------------

Interscript shall host system compilers.

3.1.3.2.1. CPython
------------------

For CPython, C and C++ compilers shall be hosted. Both
executable applications and dynamically loadable python
modules shall be supported.

3.1.3.2.2. JPython
------------------

For JPython, the javac compiler shall be hosted.

3.1.3.3. Diff/Patch
-------------------

Interscript shall provide a file comparison/change tool
similar to Unix Diff and patch. The tools must
provide/apply reversible differentials.

3.1.3.4. Executing external tools
---------------------------------

Interscript shall host external applications.

3.2. Interface
--------------

The normal understanding of this topic requires
discussion of how interscript is launched. As a command
line tool it provides a standard interface, an
equivalent GUI hosted tool would be little different.
Interscript also provides a Python callable API, which
is more interesting, but largely unimportant to most
users who will not be embedding it.

As a batch oriented text file processing tool, the
secondary interface requirements which describe the
organization and format of the input source files, are
considerably more important, since it is this interface
that most clients will use most of the time.

However, we cannot relegate the presentation of outputs
-- both documents and program files -- to discussion of
functionality because, as a specialized tool,
interscript must constrain -- or at least guide --
presentation to suit its purpose as a development
environment.

Finally, we cannot omit consideration of how the
implementation interfaces to the underlying operating
system and its tools, because, in the development
process, the client must use interscript to host the
launching and management of these tools.

In summary: interscript interfacing involves everything
which has visual appearance including input source
files, output documents and program files, and
presentation of client tool interfaces, especially
error output.

3.2.1. Unified command interface
--------------------------------

One of the principal goals of interface design is to
provide an API which is simple and comprehensible,
while at the same time providing comprehensive access
to the underlying functionality.

It is a basic lesson of software modelling that
interfaces reflect architectural structure (or
conversely that the API design influences the system
architecture.)

Interscript maintains a current state analogous to a
graphics device context: instead of pens, brushes, and
canvases, the state can be partitioned into objects
like the current weaver and current tangler.

Thus, as for graphics state, interscript provides a
@select() command, to select objects into the current
context.

Interscript requires clients construct individual
tanglers, and provides the @tangler() command to
facilitate this. In addition, some tanglers provide
specialised tanglers for parts of the target
programming language, such as strings and comments, or
for extensions to the programming language.

On the other hand, weavers are generally not
constructed individually by the LP author. Instead, the
command line processor, or other launch script which
invokes interscript, determines which documents formats
the client desires: a default of none makes sense as
this effectively turns the interscript process into
tangle only mode.

The command line then constructs weavers for each
desired format and hooks these weavers onto a
multiplexor device, called a weaver loom, which
delegates method calls to each of these weavers.

By contrast, typesetting different versions of a
document requires that the user construct and manage a
multiple looms. For example, if a document is written
in one language and a translation into another is also
provided, we need two looms, one of which is selected
when a translation is available. When common text, such
as code, is typeset, both weaver looms are selected.

It is necessary that the author construct these looms,
and do so in a way that depends on the formats selected
by the command line options. Furthermore, user command
line options must be available to determine which
whether to typeset the document in English, French,
both, or neither: it is not the job of the author to
make this decision.

3.3. Management
---------------

All software projects, even small ones, can benefit
from good management; and management depends on
availability not only of technical documentation, but
also meta-information.

Project management meta-information includes
statistical measures of volume and complexity
(so-called software metrics),

3.3.1. Software Metrics
-----------------------

Interscript shall provide some built-in reporting of
project meta-information, including conventional
statistics reporting the number of lines of code (LOC)
for inputs, tangled code, and documentation, number of
classes, functions, and other constructions defined.

Perhaps more significant, however, are measures of unit
test results.

3.3.1.1. Change Impact Analysis
-------------------------------

It is my personal opinon, that these primitive metrics
are only of minimal utility: they_re provided because
they_re easy to compute, rather than because they're
particularly useful.

In my opinion, the kind of metric which is actually a
useful measure of progress and software quality is what
I'll call a change impact analysis.

Extensive changes are indicators of poor software
quality. Basically, if a programmer is changing many
files, the system is unstable and poorly structured.

If, on the other hand, changes are intensive, and
isolated, the system is likely to be robust and well
structured. The intense work being done indicates new
functionality, improved performance, or correction of a
bug, rather than tediously re-engineering many
interfaces as a consequence of a minor change to one
module.

The theory behind this kind of metric is based on
notions of coupling and modular dependencies: if,
according to principles of loose coupling, as espoused
by Bertrand Meyer in Object Oriented Software
Construction, a program is well modularised, then
changes to a module should have a limited impact on
other modules. Indeed, a fundamental tenant of the
Open/Closed principal (again from Meyer), and notions
of information hiding, is that variations in the
implementation of an interface should have no impact on
other modules at all.

These indicators include documentation. Therefore,
traditional development technology is certain to get a
lousy rating by an impact change analysis: requirements
are written first, then the system designed, then
implemented, then tested, and finally documented (if
there is any time left).

This is a woeful strategy. I subscribe to the
philosophy most strongly emphasised by Robert Martin:
that all aspects of software development should be done
together, because they feed back into each other.

Interscript is especially designed to support this
superior strategy by providing integrated
documentation, design, programming, and testing
facilities, along with progress indicators: it is
designed to support ongoing simultaneous development of
requirements, analysis, design, implementation,
testing, and management. As such, there is no notion of
_maintenance_: the same strategy is used throught the
lifetime of the software, although the investment of
resources will likely vary over this period.

Change impact analysis is best based on interscript
sources, because target programming languages provide
dubious support for proper localisation. For example, C
and C++ require separate header and body files:
parallel maintenance of function signatures is an
impediement to quality imposed by the language. An
interscript programmer will write generating tools to
unify declaration and definition of a C function, so
the parallel maintenance will be handled by the
computer system.

It would be entirely wrong to interpret extensive
changes as bad practice, however. On the contrary,
historical impact change analysis ought to show
alternating phases of intensive and extensive changes.
The reason is that while extensive changes indicate
poor software quality and lack of robustness, such
indication is a good thing if the changes are followed
by intensive changes, because they may indicate the
programmer has recognized poor structure, devised a
solution, and implemented it.

On multi-programmer projects, it is essential to
coordinate the extensive change phase; whereas during
times of intensive development, programmers can largely
be left to work independently.

Any sizeable project which fails to go through at least
one complete reorganisation is likely to be of very
dubious quality. For example: interscript went through
a complete reoganisation before 1.0a7 was released; the
original single file was restructured into a package of
separate modules, and the single global scope was
partitioned into several distinct frames.

3.3.1.2. Measuring Change Impact
--------------------------------

Interscript shall provide change impact metrics. It
isn't clear to me at this time exactly what to measure
and how. The basic idea is to compare all the source
files before each run, and record which ones have been
changed. If, after some time period, one observes
changes to a small number of files, the development is
intensive, whereas if one observes changes to a large
number of files, the development is extensive. If there
are a large number of changes, the project is in a
development phase, whereas if there are a small number
of changes, debugging or tuning is indicated.

Management should correlate these predictors with oral
communication and written status reports. Furthermore,
examining the particular location of changes may help
pinpoint a problem and suggest reallocation of human
resources.

3.3.2. Programming metrics
--------------------------

Because projects and management styles differ, and any
notion of software metrics is new enough to be classed
a black art rather than a science, interscript shall
support user programmable metrics. Just as programmers
can use the Python scripting engine to generate code or
documentation, in accordance with their needs, so too
can new software metrics be devised and programmed.

For this reason, interscript shall provide a basic
statistical analysis package. In addition, apart from
standard metrics, interscript shall provide hooks for
gleaning other data required for computation of these
new metrics. [It is not clear at this time how to do
this.]

3.4. Implementation pragmatics
------------------------------

This section discusses requirements in terms of the
constraints imposed by an implementation. There are
several requirements all software must meet:

Performance 
     Interscript must be very fast because it must be
    executed after every change to program source code
    or documentation. If this additional processing
    time is significant compared to the time other
    build components such as compilers take, it is
    likely to be an impediment to practical software
    development, especially that for which
    documentation is not as highly rated as
    functionality.

Portability 
     Interscript must be highly portable: it must at
    least be able to execute on Unix, Windows, and Mac
    platforms with minimal installation and maintenance
    hassles. The installation and configuration must be
    manageable by people who are specialists in some
    computing field -- not necessarily the interscript
    implementation language or the host platform.

Accessibility 
     Interscript must be easy to get hold of.

Client side maintenance 
     Because the repertoire of facilities a literate
    programming tool could provide are huge, upgrades
    must be easy to install, must not compromise client
    configuration, and must remain largely compatible
    with sources already developed by interscript.

Third party development 
     Interscript must provide third party development
    opportunities. This is essential when required
    functionality is certain to be highly deviant, and
    also very specific.

4. Design Fundamentals
----------------------

This part looks at various issues in the design of
Interscript.

4.1. Devices
------------

A _device_ is an object representing an actual input,
output, or execution stream such as a disk file.
Devices are simple objects which read, write, or
execute data without parsing or interpretation.
Instead, they interface to actual system device objects
such as disk files, remote files, or text stored in
memory. In Interscript parlance, an input device is
called a _source_, an output device is called a _sink_,
and a device which acts first as sink, and subsequently
sources data that was sunk into it, is called a
_store_.

The input to interscript is represented by a source
driver object. Both the tanglers and weavers write
output via sink driver objects.

Currently implemented sources:

named_file_source 
     The most commonly used source driver reads a disk
    file named by a native filename.

ftp_file_source 
     This device initially reads files from an FTP host
    and creates a local copy. The local copy is read
    until it is deemed too old, at which point it is
    fetched again by FTP.

http_file_source 
     This device initially reads files from an HTTP
    host and creates a local copy. The local copy is
    read until it is deemed too old, at which point it
    is fetched again by HTTP.

url_file_source 
     Reads a file given a URL. Currently, ftp, http,
    gopher, and local files are supported. There is no
    caching.

stdin_source 
     Reads a file from standard input.

null_sink 
     Throws away data.

simple_named_file_sink 
     Writes a disk file given a native filename.

named_file_sink 
     This device writes a temporary disk file, and
    copies it to a nominated file if the two differ,
    both to prevent touching unchanged files, and to
    ensure the previous file is available during
    parsing, allowing generated files to be included
    back into the document.

stdout_sink 
     Writes to standard output.

memory 
     Writes and reads from memory.

disk 
     Writes and reads to disk (with separate read and
    write heads).

Under development are patch readers and writers. A
patch writer compares two files, and, instead of
replacing the old file with the new one if they differ,
writes a patch file instead. The corresponding reader
applies the patches to a copy of the old file and reads
that instead. This mechanism provides rudimentary
version control, allows stable files to be write
protected, and permits posting and using patches by
email or news.

Writers for news and email are also in the works. The
email sink device is particularly useful to allow
automatic regular updates (run as chron jobs) to send
advice to the client.

The URL reader currently doesn't use a cache, because
it uses the standard Python function _urlopen_, which
doesn't use a cache. Hopefully this function will be
upgraded to check expiration headers on http servers,
and cache files locally.

A _tee_ writer --- a device which writes to several
other devices -- is planned.

4.1.1. File names
-----------------

Common interscript commands always refer to files using
Unix relative filename convention, even on non-Unix
platforms; the interscript command line processor also
requires master filenames given on the command line to
follow this convention. These filenames are converted
to the native format internally. The purpose of this
mechanism is to ensure distributed source documents are
platform independent.

Interscript requires the native operating system
support long case sensitive filenames including the
upper and lower case latin-1 (ASCII) letters, digits,
and underscore, and a heirarchical directory system
with some kind of current directory concept: these
features are supported by all modern Unix , Windows,
and Macintosh platforms. Note that interscript cannot
operate on DOS or Win3.x platforms.

4.1.1.1. Named File Source Names
--------------------------------

Interscript _named_file_source_ takes two arguments
identifying a local disk file. The first, mandatory,
component is name in Unix relative filename format, and
the second, optional, component, is a prefix in native
operating system format.

This pair of names is the interscript file name
convention. If the prefix is empty, it is replaced by
the absolute pathname of the current directory in
native operating system format, including a trailing
directory separator. Then the Unix filename is
converted to a native operating system filename, and
appended to the prefix.

Commands such as _include_file_ which refer to sources
supply the directory of the current file as the prefix
so that the argument filename is relative to the
location of the including input file. All such
arguments must be relative filenames in Unix format for
this reason.

Note that the command line provides the
_--source-prefix=_ option for the same reason: a
filename given on the command line as a master source
file must be given as a relative filename in Unix
format, even on non-Unix systems. The source-prefix
option permits the native operating system name of the
directory containing the file to be specified.

4.1.1.2. Named File Sink Names
------------------------------

Interscript _named_file_sink_ takes two arguments
identifying a local disk file. The first, mandatory,
component is name in Unix relative filename format, and
the second, optional, component, is a prefix in native
operating system format.

This pair of names is the interscript file name
convention. If the prefix is empty, it is replaced by
the absolute pathname of the current directory in
native operating system format, including a trailing
directory separator. The the Unix filename is then
converted to a native operating system filename, and
appended to the prefix.

The command line supports four options,
_--tangler-prefix=_, _--tangler-directory=_,
_--weaver-prefix=_, and _--weaver-directory=_, which
facilitate placement of outputs.

The two directory options allow a Unix relative
filename (or other prefixing characters) to be
prepended to all filenames used for tangler or weaver
outputs, respectively.

The two prefix options allow a native operating system
format filename to specify the output directory for
tangler and weaver files. If the prefix is empty, the
absolute pathname of the current directory, including
trailing directory separator, is used. The Unix
filename is converted to native operating system format
and appended to the prefix.

4.2. Processors
---------------

A _processor_ is an object that performs some
processing function. Processors are generally hooked up
to to devices to read or write data. The separation of
processing data streams, and sourcing and sinking those
streams, is the traditional operating system facility
known as device independence.

Whereas a file, denoted by a filename, is used in an
operating system to implement device independence, in
Interscript, a Python object is used instead.

The design of Interscript consists of four processors:
the input, the code tangler, the document weaver, and
the Python engine.

The Interscript parser reads and parses the input and
sends lines to one of the three outputs. If the line
begins with the special warning character @ it is sent
to Python, otherwise if the tangler is not None it is
sent to the tangler, otherwise it is sent to the
weaver. If the tangler receives the line, it is written
to a code file and echoed to the weaver.

The main source of power in this system is the ability
to execute arbitrary Python script. Interscript has
some builtin commands and data structures to facilitate
control.

4.3. Tanglers
-------------

A tangler is an object that is designed to process
source code in some programming language for which the
tangler is specialised. Tanglers are generally selected
by the @select() command in the source. Output to a
tangler is disabled by most documentation commands, so
the system reverts to generating documentation.

Tanglers can be stacked. Typically, test code or header
code will be embedded in files containing definitions.

Interscript comes with specialised tanglers for,
several languages. The list below shows the currently
implemented special features of these tanglers.

data 
     No special features.

C    Tracks source with #linedirectives. Associated
    string and comment tanglers. Parses identifiers
    (badly).

C++  Tracks source with #line directives. Associated
    string and comment tanglers. Parses identifiers
    (badly).

python 
     Tracks source with #line directives. Associated
    comment tangler. Parses identifiers properly.

perl 
     Tracks source with #line directives. Parses and
    processes Plain Old Documentation constructions.

java 
     Associated string and comment tanglers.

interscript 
     No special features.

4.4. Lexical Scoping
--------------------

The system maintains a stack of objects called input
frames to track input sources. Input can be stacked
using the @include_file() command, which is equivalent
to a subroutine call. The stack is popped when the
included file is exhausted.

The commands @begin() and @end() can also be used to
push and pop the input stack, this is equivalent to a
nested block.

User defined symbols are lexically scoped. The system
currently maintains a dictionary of user symbols with
each stack frame: all assignments enter the symbol into
the dictionary of the top of stack frame.

When the stack is pushed, the new top of stack
dictionary is initialised by a copy of the old top of
stack dictionary. User symbols are searched for first
in the top of stack user dictionary, and then in the
global interscript namespace.

As well as supporting scoped symbols, the parser is
scoped. That is, changes to lexicology or processing
mode made by modifying the parser tables are lost when
the frame is dropped. This ensures that, for example, a
change to the warning character in an included file
does not affect the interpretation of the including
file.

4.5. Weaver Control
-------------------

Weaver control is the most complex. Weavers can be
stacked, for example to allow summary files or notes to
be built incrementally. For HTML, detail pages can be
stacked on a master, with hypertext links in the master
to access them.

More than one weaver can be active at once, so that,
for example, a HTML Web site, a LaTeX book text, and a
plain text news article can be generated
simultaneously. Document lines are sent to all the
active weavers for processing.

Some typesetting constructions are too complex or
specialised to be represented in all weavers. In this
case, verbatim text can be sent to a particular weaver,
or sent to any active weavers obeying a specific
protocol.

4.6. Source Tracking
--------------------

In the beginning, every system is a collection of
original source files, one of which is designated as
the initial (master) source. Interscript begins
processing the initial source, and may switch to
another source as a result of executing some command
such as _include_file_. This can be done to break a
long work into sections, or to include common _macros_
(python script).

Each source must have a definite name, and each line
read is counted. This allows references to the original
source to be generated in the code (and documentation)
files, so that errors reported by language processors
can be corrected. The code files cannot be edited,
because they are generated by Interscript and any
changes would be overwritten by the next processing
run.

A related reason for source tracking is to generate
cross reference tables. For example some tanglers
generate an identifier cross reference, which can be
used, for example, to disentangle duplicated names. The
tables can also be used to generate vi editor tags, for
example.

Every line of output must have an associated original
source. That includes code that is saved temporarily
(in memory, or to a disk file), and it includes
generated code.

In order to accurately track original source lines,
each input driver must have a name, usually the input
file name, and must count lines read. When a line is
read by the parser, the filename and line number are
returned as well. The parser passes this information on
to the relevant processor, usually the executor,
tangler, or weaver.

In turn, the tangler, for example, will check the sink
driver to see if the otput is synchronised with the
original source, and then store the source filename and
line number in the sink driver.

If since the last write to the sink, not necessarily by
this tangler, the original source filename has changed,
or the line number is not one more than last time, then
a section header is written to the sink file before the
data. In C and C++ a #line directive is generated, in
languages not supporting such directives (such as HTML
and Python), a comment is generated instead.
(Coincidentally, many scripting languages use a leading
# for a comment and so the C style #line directives is
generated)

4.7. Parsing
------------

Interscript is a language, and needs a parser. Parsing
is a complex task. The main control algorithm, however,
uses a very simple syntax driven parsing engine. The
parser table is a Python list of pairs. Each pair is a
tuple consisting of a compiled regular expression and a
function. The parser gets one line at a time from the
source, and runs through the list, attempting to match
the line against a regular expression. When a match is
found, the corresponding function is called with the
match and source reference data as an argument.

Because this scheme is very simple, it can be extended
or modified easily by the end user.

Because the function invoked can read further lines
from the input, more sophisticated parsing can be
programmed. For example, the Python suite execution
function matches against a line starting with an @ and
ending with an : or other character that indicates an
incomplete suite. The function reads further lines up
to the end of the script before executing the lot as a
single Python suite.

4.8. Documentation Constructions
--------------------------------

Interscript supplies uses with various documentation
constructions. These include standard constructions
such as a title, multilevel document headings, page
headings, table of contents, index, nestable
enumerated, bulletted, and keyed lists, displays (long
quotations), footnotes, and tables. For a full list of
supported constructions see below.

The requirements here are to support a rich enough set
of constructions, with a fine enough level of control
of details, to do handle the bulk of work which would
be required by a serious author, while at the same time
providing the casual programmer simple enough tools to
typeset basic program documentation.

In addition, Interscript supplies constructions
specialised to literate programming. Naturally, there
is a specialised construction for source code display,
and some tables such as a list of files, and identifier
index unique to literate programmed code. In addition,
it is necessary to be able to typeset code fragments
for _examples of use_, even though literate programming
discourages this. (Give real examples!)

These requirements have to balanced against the
efficiency of the translator, the ease of
implementation of the constructions, and the
availability of features in typesetting systems. How do
you typeset diagrams in plain text? (With difficulty)
What about diagrams, colour, pictures, and font
control?

4.9. Microformatting shortcuts
------------------------------

By microformatting, I mean things like emphasising a
single word. Most typsetters can set a plain, bold,
italic, underlined, and mono-spaced font (but not plain
text). Support for mathematics, however, is more
limited.

The general solution to microformatting is to have
specialised parsers. Just as the line by line parser
can be extended by the user, specialised
microformatting can be provided by the end user by
writing a parser to further translate document source.
Naturally, the weavers to be used will have to be
support the constructions.

Weavers already perform some parsing. For example the
Latex weaver has to translate the characters #$%^_ into
the Latex macros that produce them, since they're
reserved characters in standard Latex.

The biggest problem here is to specify a standard
microformatting language. It is not too onerous to
reserve the @ character at the beginning of a line, but
how does one designate three special fonts (bold,
italic and monospaced) and the scope they apply to?
What about font size? For something more difficult,
mathematics?

Any such language must eat up characters which can be
typeset _as is_: the fewer such reserved characters,
the more cluttered the source will become, whereas if
more are reserved, the more likely the user is to
forget to quote them properly when the character itself
is required instead of _magic_.

HTML reserves <> and uses tag pairs to do detailed
markup, and uses & to allow quoting. Latex reserves
#$%^&_\. Interscript reserves @ at the beginning of the
line.

It is possible to do all formatting using lines. But
that leads to a _troff_ like solution, which is
extremely ugly. It should be possible to write normal
text and have it print properly -- and for a programmer
that will include setting special characters.
Typesetting C code documentation in plain Latex is a
pain because underscore means subscript and is an error
outside maths mode: but underscore is more or less the
C version of a hyphen, and more or less an alphabetic
character.

The characters we can afford to reserve are those not
commonly used in program documentation. There aren_t
any. Here_s the proof by analogy: if we reserve @, for
example, then in the very documentation describing the
construction implemented using the @ character, the
most commonly used special character will, of course,
be @.

The solution I have adopted to this intransigent
problem is as follows. First, all the constructions
have to be provided as commands. That means that
irrespective of other details, all the constructions
are available, even if it is a pain to typeset them.

Secondly, we provide regular expression matching
technology to extract microformatting details using
some standard forms, but we will not enable it by
default.

I_ll call these things _shortcuts'. For example, the
first shortcut for code is an @ followed by a C
identifier. An @ in any other context is typeset as an
@.

Shortcuts are implemented by weavers. (The control loop
never sees them). To provide typesetter independent
shortcuts, we need a special kind of weaver: a filter.
A filtering weaver translates shortcuts and then calls
the normal weaver.

Interscript comes with a standard filtering weaver, and
is equipped with a user programmable table of shortcuts
based on regular expression matching. The default
version of this weaver does not do any shortcuts,
however. Shortcuts must be explicitly enabled by the
programmer. However, there is a table of standard
shortcuts prepared, and a command to enable them.

4.10. Weaver Architecture
-------------------------

Interscript operates on the assumption that there is
exactly one weaver that can process all weaver
commands.

4.10.1. Multiple Typesetters
----------------------------

Often, we want to weave the same document in different
formats, for example, using Latex for book output, HTML
for a web, and plain text for email and news.

This facility is provided by a weaver front end called
a multiplexor. The multiplexor keeps a list of active
weavers, and sends every method call to all weavers in
the list that supports that method. It_s not an error
if the weaver doesn_t have the method, but it certainly
is an error if it does, but the call fails.

4.10.1.1. Raw output
--------------------

When Interscript does not support detailed
constructions, it is necssary to hard code them into
the document. For example, if you really want frames in
HTML, or category theory diagrams in LaTex, you have to
code raw HTML or LaTex (probably XYpic) because
Interscript doesn't provide a generic interface to
support.

The principal mechanism for raw output is to put it
between a @rawif(protocol) command, and a @translate()
command. The rawif command disables a target weaver
unless it supports the nominated protocol, in which
case, it is put in raw mode, whereas the enable command
enables the weaver and put it in translating mode.

If the weaver is the multiplexor, it dispatches these
commands to all the weavers attached to it, thereby
allowing raw output to be written to the subset of
weavers supporting it.

Generally, you should provide raw typesetter data for
every possible typesetter so that _something_ is
typeset in every format of the document. (even if it is
just _paste the diagram here_ :-)

Every weaver constructs a protocol list when it is
created, but the method _add_tag_ can be called to add
another protocol to a weaver. The standard protocol
names are html, latex, and text for the html and web
weavers, the latex weaver, and the plain text weaver,
respectively.

This mechanism is design to be used in documents of
parts of documents without requiring knowledge of which
weavers are active. If a particular weaver is active
and accessible, it can be controlled directly instead,
but this is recommended only for specialised documents.

4.10.2. Multiple Human Languages
--------------------------------

Sometimes, we want to prepare the same document in
several human languages. Interscript cannot translate
English to German, for example, so it is necessary to
provide documentation text in both languages. The
program codes, however, are usually in common, except
possibly for comments.

Interscript can generate a multiple versions of a
single document (and each version will be generated in
all the selected formats) using the same tagging
mechanism that is used to control raw output. You can
write sections of English documentation after that
command @enableif(_English_) and sections of German
version after the command @enableif(_German_), and this
will disable all weavers not supporting the nominated
protocol.

This mechanism applies to interscript program code
comment commands. The commands generate ordinary woven
text, but are also inserted into the tangled output
files. In this case, comments will be inserted in the
selected language or languages. Be aware that while
this will not change program semantics it will change
the physical source file.

It is also possible to generate string constants in
different human languages with interscript, but this is
a tangler function, having no special effect on
weaving: because of the complexities of this issue, it
must be effected using python script crafted by the
author for this purpose ... in other words there are no
special commands for it :-)

4.10.3. Multiple Documents
--------------------------

Sometimes, we wish to construct several documents
simultaneously. For example, we may have a short and
long version of a document. We need to select which
weavers to write to. It is easy to do this in simple
cases by just assigning the weaver. For example:

  @both = multiplexor((long, short))
  @weaver = both
  This document describes Interscript.
  @weaver = long
  Here are some gory details.

Because a multiplexor represents a set of documents,
and because one can multiplex multiplexors, it is easy
to create small sets of weavers, and then create
various unions of these sets.

4.10.4. Cumulated Appendices
----------------------------

Sometimes it is useful to accumulate the text for a
document thru the source: the table of contents is an
example of this. Another important example is an issues
list: details of bugs or issues are written near the
relevant source, collected, and printed as an appendix.

Other examples include footnotes, which are usually
printed all togther at the end of articles or chapters
in some styles, summaries of test results, etc. [To be
continued and stuff implemented]

5. Tutorial
-----------

An easy introduction to the Interscript literate
programming tool and environment. Please note
Interscript is still experimental, and the command set
and architecture are not frozen.

5.1. Weaving a document
-----------------------

To create a plain document is easy. First, you should
create a heading like this, then type documentation.
On-the-fly interscript for test 1 follows.    12: @head(1,'My Document')
    13: This is a document describing Interscript, which
    14: is a literate programming tool. You can use
    15: any characters you like in the document,
    16: such as ~!@#$%^&*(), with one exception:
    17: you should not start a documentation line with @ in
    18: column 1. The @ character in column 1 is used to flag a command.
Test output at ../tests/output/test_1.html. Logfile at
../tests/output/test_1.log.

5.1.1. Headings
---------------

You can create sub-headings, and sub-subheadings, just
use a @head(n,'Heading') command with n set to the
heading level you want. Headings are numbered from 1,
level 1 is the biggest heading. (Technically, the
document title is a level 0 heading.)

Heading levels should go up consecutively, because
Interscript numbers all headings automatically. Here's
a document with several headings. On-the-fly
interscript for test 2 follows.    33: @head(1,'Several Headings')
    34: Test with several headings.
    35: 
    36: @head(2,'First Subheading')
    37: Under the first subheading is a subsubheading.
    38: 
    39: @head(3,'First Subsubheading')
    40: Some details here.
    41: 
    42: @head(3,'Second Subsubheading')
    43: Some more details here.
    44: 
    45: @head(2,'Second Subheading')
    46: Here's the second subhead. And the last
    47: text in the document.
Test output at ../tests/output/test_2.html. Logfile at
../tests/output/test_2.log.

5.1.2. Separating paragraphs
----------------------------

You can separate paragraphs with the command

  @p()

Blanks lines do not separate paragraphs. This is
deliberate. Any number of blank lines translates to a
single space. This allows you to separate parts of your
Interscript document with vertical white space. It is
particularly useful to add blank lines before headings.
For example: On-the-fly interscript for test 3 follows.    61: @head(1,'A document')
    62: We are flying to the moon today.
    63: @p()
    64: But not just any moon. The moon of Mars.
    65: 
    66: @head(1,'Phobos')
    67: Actually, there are two Martian moons.
    68: @p()
    69: @p()
    70: @p()
    71: 
    72: @p()
    73: 
    74: @p()
    75: One of them is Phobos.
Test output at ../tests/output/test_3.html. Logfile at
../tests/output/test_3.log.

You should note that @p() is idempotent, which is a
fancy way of saying two or more of them in a row are
the same as one. You can't add extra space between
paragraphs. Not even by putting dummy blank lines in
between.

5.1.3. Line and Page breaks
---------------------------

You can force a line and page break with the commands

  @line_break()
  @page_break()

respectively. On-the-fly interscript for test 4
follows.    90: @head(1,'A break test')
    91: Here is a short line
    92: @line_break()
    93: and another
    94: @line_break()
    95: and another.
    96: End of page.
    97: @page_break()
    98: Now a new page.
Test output at ../tests/output/test_4.html. Logfile at
../tests/output/test_4.log.

5.1.4. Displaying code examples
-------------------------------

In this tutorial, I've been showing you some example
code. That is something most documentation writers want
to do. You can do it too, like this: On-the-fly
interscript for test 5 follows.   107: @head(1,'Code displays')
   108: Here is a code display:
   109: @begin_displayed_code()
   110:   while 1:
   111:     print 'Hello again and again'
   112: @end_displayed_code()
Test output at ../tests/output/test_5.html. Logfile at
../tests/output/test_5.log.

5.1.5. Running Interscript
--------------------------

Well, you should try an example file. To process a file
we'll say:

  python iscr.py --weaver=html example.pak

where 'example.pak' is the name of the Interscript
document. This will create a single file
"example.html". Try it!

If you prefer HTML split into lots of little pages,
try:

  python iscr.py --weaver=web example.pak

This produces a file "example_top.html", and a number
of auxiliary files, and a file for each heading.

You can also generate latex2e and plain text with the
commands:

  python iscr.py --weaver=latex example.pak

which produces a file "example.tex", and,

  python iscr.py --weaver=text example.pak

which produces a file example.txt. you can put more
than one file name at the end too. Each such document
will be processed separately.

5.1.5.1. Option help
--------------------

If you type:

  python iscr.py --help

you will get a complete list of available options.

5.1.5.2. Passes
---------------

The passes options causes Interscript to process files
more than once. This is sometimes necessary to get
cross reference information right. The default is
currently 1.

  python iscr.py --weaver=text --passes=2 example.pak

Interscript may stop before running the specified
number of passes. It will do this if, and only if,
every buffered disk file ("named_file_sink") would
write an output the same as the existing file. In that
case, it assumes further passes wouldn't change
anything, and stops. This is called convergence.

5.1.5.3. Tangling parts
-----------------------

If you construct your Interscript sources as a tree,
using the "@include_file()" command, and you follow
certain rules, you can run Interscript on the included
file to extract the code, for just that file. The
Interscript sources are constructed this way. This
feature is vital for building big systems because it
allows you to extract the code from files you have
changed, without extracting code from those that have
not.

You must ensure that code files are lexically contained
entirely in a single include file. More generally, the
include file does not rely on any context from its
parent (except for that which is determined from the
command line).

If you weave an include file, you will get a separate
document for that include file which will, in general,
not be linked to the master document: it will be in a
separate file, named after the include file, and
headings will be numbered separately.

[There is currently no simple way to require a separate
source be built entirely independently so that the
master document can link to it. This would be
especially useful, because it would also permit time
stamps to be checked and avoid unnecessary processing.]

5.2. Tangling code
------------------

So far, we have just produced a document. What about
programming? Here's a sample document with tangling.
On-the-fly interscript for test 6 follows.   196: @py = python_output('interscript/tests/output/mymodule.py')
   197: @head(1,'My Module')
   198: This is my very own module.
   199: @select(py)
   200: import sys
   201: class myclass:
   202:   def __init__(self, name):
   203:     self.name = name
   204: @head(2,'hello method')
   205: Just says hello.
   206: @select(py)
   207:   def hello(self):
   208:     print 'hello','self.name
   209: @doc()
   210: And now back to doco.
Test output at ../tests/output/test_6.html. Logfile at
../tests/output/test_6.log.

5.2.1. Original Source References
---------------------------------

Here's the code from the previous example again:

Start python section to interscript/tests/output/mymodule2.py[1]

     1: #line 217 "tutorial.pak"


End python section to interscript/tests/output/mymodule2.py[1]

Notice the #line directives. They're called _original
source references _because they refer to the original,
editable, source file containing the code. If you are
creating C programs, the compiler will recognize them
and report compiler errors in the original file.
Integrated development environments will put the cursor
right in the middle of the Interscript source. This is
necessary, you must not edit the code file. Your
changes will get clobbered next time you run
Interscript on the original source.

5.2.2. Code sections
--------------------

As you can see, the code is displayed with line
numbers, and the file is named at the beginning and
ending of each chunk. Interscript calls these chunks
_code sections _and the doco between them _document
sections. _ So basically, Interscript allows you to
_interleave _code and document sections. This is called
_gathering _ in some other literate programming tools.
You can end a code section with an @head() command, or
a @doc() command, which switches back to document mode.

Just so you can see it again, here is some code
interleaved with documentation.

Start python section to interscript/tests/output/mymodule3.py[1]

     1: #line 266 "tutorial.pak"


End python section to interscript/tests/output/mymodule3.py[1]

And now for the hello method:

Start python section to interscript/tests/output/mymodule3.py[2]

     6: #line 273 "tutorial.pak"


End python section to interscript/tests/output/mymodule3.py[2]

You should see two code sections, embedded in
documentation.

5.3. Scripting
--------------

Traditional literate programming tools have two
conceptual processes: _weaving _ (a document) and
_tangling _ (a code file) which separate out
interleaved document and code sections (respectively).

Interscript has a third kind of section, the _script
section. _In case you're wondering what script sections
look like, well, you've already used them. All those
lines starting with @ are just executable python
script. They aren't really special magical commands,
just function calls to predefined python functions.

You can write any python script you like in a script
section. On-the-fly interscript for test 7 follows.   303: @name = 'John Skaller'
   304: @print 'Hello',name
   305: @print 'Running Python',sys.version
   306: @head(1,'Hello World from '+name)
   307: This is a scripting test.
   308: @weave('Written by '+name+'.')
Test output at ../tests/output/test_7.html. Logfile at
../tests/output/test_7.log.

Notice you don't have to import sys: it is already
imported, because it is used in Interscript. [Add list
of imported modules here]

You will also notice the

  @weave(text)

command, which weaves its argument: use this when you
want to calculate text, as in the example. For more
information on the weave command, seeUnknown Label:weave command
.

You should be careful with this feature. It is
immensely powerful! You can use it to test programs,
and to extend Interscript for you needs in a
_particular _document -- without changing the actual
source code for Interscript. See http://www.python.org
to find out more about python.

5.3.1. Long script sections
---------------------------

You can code long script sections such as a class
definition. The rule is: a long script section is
started by a line starting with @ and ending with :,(
or one of the characters that python would recognize
that signifies that there is more to come. You must
then indent the code with exactly one extra space. A
long script section is ended by the first line not
having a space character in column 1 (or the end of
file).

The whole of a long script section is collected and
then executed at once. On-the-fly interscript for test
8 follows.   349: @head(1,'Long script sections')
   350: Here is a long script. We define a class MyClass.
   351: @class MyClass:
   352:    def __init__(self, name):
   353:      self.name = name
   354:    def hello(self):
   355:      print 'Hello',self.name
   356: 
   357:  # test it
   358:  me = MyClass('John')
   359:  me.hello()
   360:  deliberate error
   361: @doc()
   362: After all that, the deliberate error is ignored.
Test output at ../tests/output/test_8.html. Logfile at
../tests/output/test_8.log.

Errors in script sections are reported with a traceback
to the logfile, but do not halt processing. You cannot
terminate an Interscript processing run inside a script
section, not even with sys.exit(). [Interscript can be
terminated with a keyboard interrupt or system abort
signal, however.]

5.3.2. Very Long script sections
--------------------------------

There's a better way to code long script sections,
using the "python()" command. Here's an example:
On-the-fly interscript for test 9 follows.   376: @head(1,'Long Script test')
   377: Some script should generate a string of 3 'My names',
   378: separated by 3 dashes.
   379: @python('//')
   380: x = 2
   381: y = 3
   382: z = 'My name'
   383: weave((z + '-' * x) * y)
   384: //
Test output at ../tests/output/test_9.html. Logfile at
../tests/output/test_9.log.

The 'python' command accepts a string argument which is
a terminator line for the script section. The whole
section is gathered, without any processing, and then
executed. There is a danger to be aware of: if you
don't put the terminator in correctly, the command will
read all the way to the end of the file.

5.4. Unit tests
---------------

It is possible to test python script 'on the fly' as in
the example: On-the-fly interscript for test 10
follows.   396: @head(1,'A python test')
   397: Test the test_python function.
   398: @test_python(hlevel=2,descr='A simple test',source_terminator='//')
   399: print 'A simple test'
   400: //
Test output at ../tests/output/test_10.html. Logfile at
../tests/output/test_10.log.

The test is also registered in a table of tests.

It is also possible to provide expected output:
Interscript will verify your code by comparing the
expected and actual output, and print a difference
table if the test failed. Note that the difference
table is only available if the module
"interscript.utilities.diff" is available and operates
correctly: the current implementation uses GNU diff
invoked using "os.system()". Here's an example that
veifies OK: On-the-fly interscript for test 11 follows.   413: @head(1,'A diff OK test')
   414: Test the test_python function.
   415: @test_python(hlevel=2,\
   416:   descr='A diff OK test',source_terminator='//', expected_terminator='//')
   417: print 'A simple diff test'
   418: print 'A simple diff test line 2'
   419: print 'A simple diff test line 3'
   420: //
   421: A simple diff test
   422: A simple diff test line 2
   423: A simple diff test line 3
   424: //
Test output at ../tests/output/test_11.html. Logfile at
../tests/output/test_11.log.

And here's one that should fail: On-the-fly interscript
for test 12 follows.   428: @head(1,'A diff fail test')
   429: Test the test_python function.
   430: @test_python(hlevel=2,\
   431:   descr='A diff test',source_terminator='//', expected_terminator='//')
   432: print 'A simple diff test'
   433: print 'A simple diff test line 2'
   434: print 'A simple diff test line 3'
   435: //
   436: A simple diff test
   437: A simple diff test line 2 CHANGED
   438: A simple diff test line 3
   439: //
Test output at ../tests/output/test_12.html. Logfile at
../tests/output/test_12.log.

5.4.1. Weaver Control
---------------------

The python function 'get_weaver()' refers to the
current weaver. You can use it to call methods directly
on the weaver. For example:

  @weaver = get_weaver()
  @weaver.write('Antidis')
  @weaver.begin_bold()
  @weaver.write('establishmentarianism')
  @weaver.end_bold()
   is a long word. Note the space on this line!

which comes out as Antidisestablishmentarianism is a
long word. Note the space on this line!

You can also set the weaver. Suppose you have a weaver
mynotes_weaver, then you can write:

  @old_weaver = get_weaver()
  @set_weaver(mynotes_weaver)
  This is woven into notes.
  @set_weaver(old_weaver)

To make this more convenient, the set_weaver function
returns the current weaver so you can write:

  @old_weaver = set_weaver(mynotes_weaver)
  This is woven into notes.
  @set_weaver(old_weaver)

Even more convenient, you can push and pop weavers onto
a stack using

  @push_weaver(mynotes_weaver)
  This is woven into notes.
  @pop_weaver()

The current weaver is lexically scoped.

5.4.2. Perl hates @
-------------------

If you are programming Perl (or Interscript!) you will
hate having @ as the warning character for script
sections. There are two ways around this.

You can use two @ characters at the beginning of a
line.

  @@p = @x

or you can use a command like:

  @set_warning_character(python='!')

which will set the python warning character to !
instead of @. Advanced Note. This change applies only
to the current file, and only to the end of the
containing block, if any. The effect will not be passed
up to an including file, and it won't be inherited by
an included file either.

5.5. Fonts
----------

The following commands change font. Note that begin/end
pairs must be balanced, and nesting may or may not be
supported, depending on the weaver.

  @begin_emphasize()
  @end_emphasize()

  @begin_strong()
  @end_strong()

  @begin_code()
  @end_code()

  @begin_small()
  @end_small()

  @begin_big()
  @end_big()

  @begin_italic()
  @end_italic()

  @begin_bold()
  @end_bold()

5.6. Lists
----------

The following commands build lists. Note that begin/end
pairs must be balanced, and nesting is supported. There
are three kinds of lists: numbered lists, which are
numbered automatically, bullet lists, which display
some special bullet symbol, and keyed lists, which
display a string of text for each item.

  @begin_numbered_list(start=1):
  @end_numbered_list():
  @begin_numbered_list_item():
  @end_numbered_list_item():

  @begin_bullet_list():
  @end_bullet_list():
  @begin_bullet_list_item():
  @end_bullet_list_item():

  @begin_keyed_list():
  @end_keyed_list():
  @begin_keyed_list_item(key):
  @end_keyed_list_item():

Here's an example:

  @begin_keyed_list()
  @begin_keyed_list_item('bullet')
  A bullet or similar character at the start of each item.
  @end_keyed_list_item()
  @begin_keyed_list_item('numbered')
  A number at the start of each item.
  @end_keyed_list_item()
  @begin_keyed_list_item('keyed')
  A key, or definition term, at the start of each item.
  @end_keyed_list_item()
  @end_keyed_list()

which comes out like:

bullet 
     A bullet or similar character at the start of each
    item.

numbered 
     A number at the start of each item.

keyed 
     A key, or definition term, at the start of each
    item.

5.6.1. Easier lists
-------------------

By default, Interscript installs a special weaver
called 'multiplexor' which delegates commands to zero,
one, or more weavers. This weaver also supports
simplified list definitions. Here's the example above,
simplified.

  @begin_list('keyed')
  @item('bullet')
  A bullet or similar character at the start of each item.
  @item('numbered')
  A number at the start of each item.
  @item('keyed')
  A key, or definition term, at the start of each item.
  @end_list()

5.7. Tables
-----------

Interscript supports tables, although currently the
support is fairly primitive. Here's how to create a
table:

  @begin_table('Column 1','Column 2','Column 3')
  @table_row('Data 11', 'Data 12','Data13')
  @table_row('Data 21', 'Data 22','Data23')
  @end_table()

which looks like this:
+----------+----------+----------+
| Column 1 | Column 2 | Column 3 |
+----------+----------+----------+
| Data 11  | Data 12  | Data13   |
| Data 21  | Data 22  | Data23   |
+----------+----------+----------+

5.8. Citations
--------------

You can cite a URL like:

  @cite_url('http://www.triode.net.au/~skaller')

which will appear as a hyper link in HTML files like
http://www.triode.net.au/~skaller.

5.9. Cross References
---------------------

Interscript supports intra-document (internal) cross
referencing using the commands:

  @set_anchor('MyLabel')
  ...
  please see
  @ref_anchor('MyLabel')
   for details.

The label must be a string, and is currently required
to be a valid identifier since it is used literally by
the HTML weaver in an anchor tag. Latex imposes no such
restrictions, nor does the plain text weaver. For HTML,
the label is set as an anchor; for latex, the page
number is given, for plain text the line number.

Note that inter-document (external) cross references
are different to intra-document cross references. For
truly external references to existing published works,
use citations or bibliographic references. References
across volumes of a work or project are not yet
supported.

5.10. Including Files
---------------------

Interscript allows you to include Interscript files
with the command

  @include_file(filename)

When you do this, you should be aware that it is
treated like a subroutine call: a stack frame is
created, and any symbols bound in script sections are
local to the file. In addition, various parameters are
localized. Therefore, you cannot define a new command
or variable in an include file and expect it to persist
past the end of the file.

5.10.1. Including code
----------------------

You can include existing code directly to the current
tangler like:

  @select(py)
  @include_code(filename)

This is not the same as including an Interscript file.
The contents of the code inclusion 'filename' are
copied verbatim to the tangler 'py'. Leading @
characters are not detected. The contents of the file
are still woven into the document.

5.10.2. Displaying Code
-----------------------

You can display a file as code like this:

  @display_code(filename)

This is very useful for printing the results of a
processing run, or weaving example programs into a
book.

5.11. Translating Html
----------------------

Interscript normally reads Interscript. But it can also
read (a small subset of) HTML. The command:

  @include_html(filename)

will read an HTML file and translate the tags to
Interscript. In this manner, you can convert flat HTML
into stacked HTML, or into Latex or plain text.

5.12. Special constructions
---------------------------


5.12.1. Table of Contents
-------------------------

To print the table of contents, you can say:

  @print_table_of_contents()

For Latex, the native table of contents construction is
used. For flat HTML, the weaver generates a hyper
linked table of headings, but two passes are required
to get it right. For plain text, two passes are also
required.

For stacked HTML, a separate contents page is created
automatically. For this reason, the table of contents
command is disabled for that weaver.

5.12.2. Identifier Index
------------------------

This is table of all the identifiers used in your
programs. For HTML, the entries are hyper linked to
each occurrence of the identifier. Finding identifiers
is the task of tanglers. At this time, the Python
tangler can find most of them by tokenising python
script. None of the other tanglers support this feature
properly yet.

  @print_identifier_cross_reference()

5.12.3. File list
-----------------

A list of all the generated files. Use

  @print_file_list()

to generate it. You can use this to assemble the
generated files into a tar ball.

5.12.4. Source list
-------------------

A list of all the input files. Use

  @print_source_file_list()

to generate it. This command is useful so you know what
files are required for a package.

5.12.5. The Web Weaver
----------------------

Here are some special features of the 'web' weaver.

5.12.5.1. Automatic table generation
------------------------------------

As of version 1.0a6, the web weaver automatically
produces a number of tables: the table of contents,
index of classes, index of functions, index of
identifiers, index of unit tests, and file convergence
status report. None of these tables can be disabled at
present.

5.12.5.2. Mandatory Frames
--------------------------

The web weaver also produces several framesets. There
is no 'noframes' option at present.

5.12.5.3. Internet Explorer DHTML support
-----------------------------------------

In addition, the table of contents uses ECMAscript to
determine if it is running on a version Microsoft
Internet Explorer; if so, it enables dynamic HTML
features unique to the Microsoft object model which
permit dynamic expansion or contraction of the table of
contents tree at each branch.

5.12.5.4. Cascading Style Sheets
--------------------------------

Both the web and html weavers uses CLASS attributes in
tags, and the web weaver in particular makes fairly
heavy use of the generic DIV and SPAN tags.

A standard Cascading Style Sheet called Interscript.css
can be found in the directory interscript/doc. It
colours various elements in a suggestive way. Do not
change interscript.css; instead, supply user.css; it
should override interscript.css even if your browser
finds both files.

5.13. File Names
----------------

The names of files used in interscript documents should
be relative pathnames obeying the Unix convention, even
on other platforms such as NT or the Mac: separate
components with a / character. Don't use silly
characters such as : in components names.

I plan to upgrade the file naming convention to use
URLs with 'interscript' addressing scheme, in which the
'network' component is treated as a logical location
identifier; the client will map these locations to
physical ones.

The current version of interscript does not provide
this mechanism yet. Instead, there are four command
line options:

  --weaver-prefix=nativepath
  --tangler-prefix=nativebspath
  --weaver-directory=relpath
  --tangler-directory=relpath

where the nativepath is a prefix in native operating
system format, and the relpath is a prefix in Unix
format. For an interscript file given as 'basename',
the resulting actual filename is:

  abspath+ (string.join(string.split(relpath+basename,'/').os.sep)

Note that if you use 'a' and 'b' as the prefix and
directory a filename base will be called 'abbase': no
separators are put between the prefix, directory and
base. Here's an example for Windows:

  python iscr.py                        \
    --tangler-prefix=c:\mydevelopment\  \
    --tangler-directory=code/           \
    example.pak

Note that interscript creates directories automatically
for the 'Unix' part of the filename, but _not_ the
native prefix. Thus in the example 'c:\mydevelopment'
must exist, whereas 'code'is created within it
automatically. If example.pak tangles a file
'package/module.py', then 'package' is also created
automatically.

5.14. Questions and Answers
---------------------------


5.14.1. Why are HTML tags printed verbatim, even by the html weaver?
--------------------------------------------------------------------

Weavers are _supposed_ to print all printable
characters literally. So tagged text such as
<EM>tagged</EM> is printed exactly as you wrote it.

5.14.2. OK, so how do I put HTML into a document?
-------------------------------------------------

If you want to write specialised HTML and put it into a
document, there are several ways to do it.

5.14.2.1. Tag method
--------------------

This is the prefered method. You say:

  @weaver.rawif('html')
  '<B>raw html</B>'
  @weaver.enable()

What this does is disable the weaver except unless it
is tagged 'html'. Then you raw-write the html, and
finally re-enable the weaver. This works even when you
have multiple weavers configured with filters, because
the multiplexor and markup filters delegate these
commands to their clients.

You add tags to a weaver with anything by saying

  @weaver.add_tag(something)

The built in weavers are tagged 'text', 'html' 'latex'
and 'raw' as appropriate.

I plan to make raw writing a bit more convenient!

5.14.2.2. Cheat method 1
------------------------

If you know you have a particular weaver set, you can
just raw write to it like

  @weaver._write('<B>raw html</B>')

and that will go straight to the output device. It will
look funny if you have a Latex weaver set! You can also
say:

  @weaver.raw()
  '<B>raw html</B>'
  @weaver.translate()

to force the current weaver into raw mode and back to
translation mode.

5.14.2.3. Cheat method 2
------------------------

What you do is construct a raw weaver bound to the same
sink device as your html weaver, and then set it to be
the current weaver. This weaver thinks it is writing a
raw data file, and copies all input straight to the
sink device. So your HTML tags will go straight through
to the output file, uninterpreted.

5.14.3. How can I use existing HTML as a source?
------------------------------------------------

You can construct an _input filter_ that parses HTML,
translating into to Interscript. The Interscript parser
then processes this data, calling the current weaver
with appropriate methods. If the weaver is an html
weaver, it will then translate the calls back to html
tags.

The advantage of this method, however, is that if the
current weaver is, say, a Latex weaver, then the output
will be the Latex equivalent of the original HTML. In
other words, you will have translated the original HTML
into Latex.

5.14.4. Isn't it hard to create an HTML filter?
-----------------------------------------------

No, it is very easy, because a class,
html_input_filter, to do exactly this is standard in
Interscript. This class uses the Python standard
library module htmllib to parse the input.

5.14.5. Why are blank lines ignored?
------------------------------------

They're not, they're converted to spaces, which is the
HTML convention.

5.14.6. Why do I get multiple spaces in my document?
----------------------------------------------------

This is probably a bug in Interscript. In general,
multiple spaces in input files are passed on to the
typesetter 'as is'. HTML and Latex will reduce these to
a single space automatically, but not all typesetters
do this.

Generally, weavers write lines out as they came in, in
other words, a word at the end of one line will be
separated from a word starting the next line because
the weaver writes out two lines, and the typesetter
replaces them with a space.

However, there are some nasty cases. In the html
weaver, the @begin_bold() and @end_bold() commands
write out the <B> and </B> tags without any spaces
before or after. But the text to be boldened is always
a sequence of lines, so there will always be a newline
before the </B> tag. So there's no easy way to bold a
single letter of a word using the bold command in an
Interscript input.

However, this is _not_ the case if you implement a
filter, because the filter can write out the argument
using the weaver method write(), rather than
writeline().

5.14.7. The indentation of my programs is all wrong. Why?
---------------------------------------------------------

Interscript does not allow tabs. You should never ever
use tabs, they're an extremely stupid idea. Interscript
replaces tabs by spaces automatically. This feature
_cannot_ be disabled (without hackery), because it is
implemented in the main control loop.

The variable g.tabwidth can be changed from the default
value of 2 (which is suitable for publication) to some
other value (such as 8) with a command like @g.tabwidth
= 8.

You can, and sometimes must, generate tabs in code
files. For example, tabs are required in 'make' files.
You can do this in many ways, for example by
implementing a front end for a tangler that replaces
leading spaces, or perhaps \t sequence, by tabs.

5.14.8. How do I get existing code into Interscript?
----------------------------------------------------

Generally, commands turn off tangling. However, the
command @include_code(filename) leaves the current
tangler enabled. This causes the contents of the file
to be typeset as code (even if it contains leading @
characters) _and_ included in the output for the
tangler.

5.14.9. What if I just want to print it?
----------------------------------------

If you don't want to write code to any output file,
just use a tangler with a null sink device. That device
just throws away all the data written to it. For
example:

Start C section to null sink[1]

     1: #line 128 "faq.ipk"
     2: @from interscript.tanglers.c import c_tangler
     3: @from interscript.drivers.sinks.null import null_sink
     4: @dummy = c_tangler(null_sink(), weaver)
     5: @select(dummy)
     6: /* These comments do not get written out */

5.14.10. How do I debug Interscript
-----------------------------------

This is a very hard question. With some vexation, I
have decided to make Interscript robust rather than
fragile. (Generally, this is a bad idea).

First, the standard weaver is an object of the class
multiplexor. That object is a proxy which dispatches
method calls to every weaver in a list of delegates,
but only if the delegate supports that method.

This means that the default behaviour of a method is to
do nothing, which seems appropriate for weavers. For
example the plain text weaver cannot do bolding, so
rather than implement a 'pass' method, you just don't
bother to implement the method at all.

When the Python engine detects an unexpected exception,
usually coming from Python script in your document, but
sometimes from I/O failures, and sometimes from bugs,
Interscript attempts to put debugging information into
the current document and continue. So, for example, if
you try to include a file that doesn't exist, you'll
get a message in the current document.

There are two excuses for doing this. The first is that
it allows the rest of the process to continue, in
particular, it prevents errors in documentation
constructions holding up code development.

This feature was crucial in developing Interscript
itself for the following interesting reason: when I
added a new feature to a weaver I would have to use the
existing version of Interscript to tangle the code
containing the implementation of that feature. If I
tried to use that feature in the documentation, it
would always crash, because the feature wasn't
implemented in the old version.

So, I would build the new version, then start using the
new constructions. The problem was that there would be
bugs in the implementation. If I just fixed the bugs,
I'd be trying to tangle the code using the old bugged
version, which would crash. On the other hand, if I
used an older backup version of the program, it would
also crash because there was _no_ implementation of the
construction.

Developing a program generator which generates itself
is an excellent way to test the program generator --
but there it is sometimes difficult to recover from
faults!

So, I decided that documentation errors shouldn't stop
the system generating code, as, in my case, the fixes
would be in the code I was trying to generate!

Instead, error messages are printed in the
documentation if possible. One excuse for this is that
such messages are easy to find, and where they occur is
a good indication of the fault. Separate logs are much
harder to use, and if the process is to continue in the
presence of a documentation fault, there's no harm from
corrupting an already corrupt document.

One downside of this features comes if you don't read
your documents: you may not notice an error. Another
downside is that a corrupt weaver may not be able to
print the error into the document, and it may not be
there to read.

I'd be very interested hearing reports of how this
facility trips people up: I'm sure it will, but not
sure of the details. (Putting errors after a corrupt
'</HTML>' tag, for example, means some browsers will
never show the error!)

Most diagnostics are also written to a log file and
standard output.

NOTE: the details described herein are currently not
properly implemented!

6. Implementation
-----------------

Literate programmed by itself, of course!

6.1. Interscript Module
-----------------------

The top level interscript module.

Start data section to interscript.README[1]

     1: The literate programming tool Interscript
     2:  version '1.0a8'
     3:  has been unpacked here.
     4: It consists of two text files, and a directory:
     5: 
     6:   interscript.README (this file)
     7:   interscript.pth    (the Python package control file)
     8:   interscript/       (the interscript package directory)
     9: 
    10: Find the basic HTML documentation at
    11: 
    12:   interscript/doc/iscrp.html
    13: 
    14: Find the enhanced HTML documentation at
    15: 
    16:   interscript/doc/iscr_top.html
    17: 
    18: Find the text documentation at
    19: 
    20:   interscript/doc/iscr.txt
    21: 
    22: Find the latex2e documentation at
    23: 
    24:   interscript/doc/iscr.tex
    25: 
    26: Find the interscript home page at
    27: 
    28:   http://www.triode.net.au/~skaller/interscript
    29: 


End data section to interscript.README[1]

Start data section to interscript.pth[1]

     1: interscript


End data section to interscript.pth[1]

Start python section to interscript/__init__.py[1]

     1: #line 237 "iscr.pak"


End python section to interscript/__init__.py[1]

6.1.1. Construct Global Frame
-----------------------------

This frame is shared between all processes, and is
initialised at module load time. It hooks crucial
resources and identification information. The
attributes of the global frame class are used as the
globals() dictionary for executing user scripts.

[This is probably a bad idea, because it allows the
user to change the attributes using the global
declaration. On the other hand, it provides a method
for sharing between processes.]

For some weird reason, the global frame is a python
class, not a module, and not a class instance. It's not
a plain module, because python termination is somewhat
indeterminate, and it isn't an instance, because
there's only ever one of them. Clients of the global
frame keep a reference to it explicitly to prevent
premature deletion by the python run time on program
termination.

The global fram in turn keeps references to a set of
important resources, so that they're not deleted
prematurely either. I'm doing this because interscript
__del__ methods are often used to do substantial work,
and it's imperative that system resources are available
until all dynamically created objects are destroyed.

Start python section to interscript/__init__.py[2]

     3: #line 268 "iscr.pak"


End python section to interscript/__init__.py[2]

6.1.2. Set version
------------------

This is an experimental release only, intended as an
act of advocacy and to solicit comments and support.

The code below is tricky to understand. There are three
distinct sets of version information: data describing
the version being generated, data describing the
version doing the generating, and data describing the
version which generated the generator.

Python script at the top of this file computes or fixes
the version information for the version being
generated, that information is bound into the source of
the generated version.

When that version is itself used to generated yet
another version the data bound into it by the previous
generation describes the currently executing version,
which is now the generator.

When the version being generated is yet again used to
generate another version, the original data now
describes the version that generated the version
generating the new version.

In order to make it easy to add or change
identification attributes, they're initialised to dummy
values, in case the generating version doesn't has the
same set of values. This is always the case when a new
attribute is introduced: the attribute won't make it
into the generating version until a second pass is
executed and the version being generated itself becomes
the generator.

Note that knowing the version that generated the
executing version is important for bug tracking, since
a problem may be due to a bug in the source for the
version, or in the generator which processed it: a bug
can persist even when the source is correct for many
generations if it is not processed correctly by the
generator. Bootstrapped code can be a real nightmare to
debug.

Start python section to interscript/__init__.py[3]

    64: #line 366 "iscr.pak"


End python section to interscript/__init__.py[3]

6.1.3. Listing of iscr.pak source
---------------------------------

Here is the fully listings of the top level interscript
source file, iscr.pak.

@# Interscript

@# set to 1/0 to enable/disable debugging
@debug_version = 0

@# --------------------------------------------------------------------
@# Get standard modules.
@import os
@import time
@from interscript.weavers.multiplexor import multiplexor

@# --------------------------------------------------------------------
@# Set the source version.
@gen_version = '1.0a8'

@# --------------------------------------------------------------------
@# Get the build number. Update it, but only in pass 0.
@try:
   gen_buildno = 0
   print 'Trying to open iscrbldn.dat, which contains the build number..'
   temp = open('iscrbldn.dat','r')
   line = temp.readline()
   line = line[:-1]
   gen_buildno = int(line)
   temp.close()
   print 'OK, got last build number',gen_buildno
 except:
   print "Couldn't open build number file. Must be a virgin build."
   pass

@if get_pass_frame().passno == 0:
   gen_buildno = gen_buildno + 1
   temp = open('iscrbldn.dat','w')
   temp.write(str(gen_buildno)+'\n')
   temp.close()
   del temp

@# --------------------------------------------------------------------
@# Try to get the hostname.
@# Should work on Unix, but don't worry if it fails.
@gen_hostname = 'unknown'
@try:
   print 'Tring to get host name with "hostname" command'
   f = os.popen('hostname','r')
   gen_hostname = string.strip(f.readline())
   f.close()
   print 'Got hostname',gen_hostname
 except:
   print "Couldn't get hostname, probably not Unix."
   pass

@# --------------------------------------------------------------------
@# Try to get the username.
@# Should work on Unix, but don't worry if it fails.
@gen_username = 'unknown'
@try:
   print 'Trying to get username from environment variable USERNAME'
   gen_username = os.environ['USERNAME']
   print 'Got username',gen_username
 except:
   print "Couldn't get username, probably not Unix."
   pass

@# --------------------------------------------------------------------
@# Print the current working directory.
@try:
   print 'Current Directory is',os.getcwd()
 except:
   pass

 #print 'Python Path is'
 #for path in sys.path:
 #  if path == '': print '  '+os.curdir+' ('+os.getcwd()+')'
 #  else: print ' ',path

@# --------------------------------------------------------------------
@# Get the build time, but only during pass 0.
@if get_pass_frame().passno == 0:
   get_master_frame().now = time.time()
@now = get_master_frame().now
@localtime = time.localtime(now)
@utctime = time.gmtime(now)
@lfmtime = time.strftime('%a %b %d, %Y at %I:%M %p (%Z)',localtime)
@ufmtime = time.strftime('%a %b %d, %Y at %I:%M %p (UTC)',utctime)

@# --------------------------------------------------------------------
@# Get the identification information from the currently
@# executing version of interscript.
@global_frame = get_process_frame().global_frame
@print 'This is Interscript',global_frame.version,'build',global_frame.buildno
@print 'built on',global_frame.generator_hostname,'by',global_frame.generator_username
@print
@print 'Running on',gen_hostname,'by',gen_username,'at',lfmtime
@print "and building Interscript",gen_version,'build',gen_buildno
@print

@author = 'John Max Skaller'
@title = 'Interscript'
@pfx = get_master_frame().weaver_directory

@# --------------------------------------------------------------------
@# Use the 'weavers' user option to decide whether to generate documentation
@# of not. Issue a warning to the user if no weaver is set.
@# Also, warn if '--weaver=' is used instead of '--weavers'

@weavers = []
@weaver_opt = []
@def print_weaver_help():
   print '  Please use "--weavers=[\'html\',\'latex\']" (etc) instead of "--weaver=".'
   print '  Note: you may need to escape the quotes (or quote the argument).'
   print '  The argument must be an evaluable python list of strings,'
   print '  or the string "all".'

@if get_master_frame().autoweave != []:
   print 'WARNING: Interscript source uses document defined weavers.'
   print_weaver_help()
 else:
   useropt = get_master_frame().useropt
   if not useropt.has_key('weavers'):
     print 'No Weaver in use.'
     print_weaver_help()
   else:
     weaver_opt  = useropt['weavers']
     try:
       weaver_opt = eval(weaver_opt)
     except:
       print ' ERROR: The argument "'+weaver_opt+'" to --weavers= must be a python list of strings.'
       print_weaver_help()

@if weaver_opt == 'all':
   weaver_opt = ['html','text','web','latex']

@if 'text' in weaver_opt:
   plain_text_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscr.txt')
   plain = plain_text_weaver(get_pass_frame(),plain_text_output,title=title,author=author)
   weavers.append(plain)
   print 'Generating text document.'

@if 'html' in weaver_opt:
   simple_html_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscrp.html')
   simple_html = html_weaver(get_pass_frame(),simple_html_output,title=title,author=author)
   weavers.append(simple_html)
   print 'Generating simple HTML document.'

@if 'web' in weaver_opt:
   stacked_html_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscrs.html')
   stacked_html = html_weaver(get_pass_frame(),stacked_html_output,title=title,author=author)
   stacked = stacking_weaver(stacked_html,'iscr_%s.html',(1,2,3,4,5,6,7,99),toc_depth=3)
   weavers.append(stacked)
   print 'Generating complex WEB/HTML document.'

@if 'latex' in weaver_opt:
   latex_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscr.tex')
   latex = latex_weaver(get_pass_frame(),latex_output,title=title,author=author, heading_level_offset=1)
   weavers.append(latex)
   print 'Generating latex document.'

@# terminate the if :-)
@multiplex = multiplexor(get_pass_frame(),weavers)
@emph = markup_filter(get_pass_frame(),"'([^']+)'",'begin_emphasize','end_emphasize',[multiplex])
@code = markup_filter(get_pass_frame(),'"([^"]+)','begin_code','end_code',[emph])
@master_weaver = code
@set_weaver(master_weaver)

@# --------------------------------------------------------------------
@# Begin construction of the new version of interscript.
@head(1,'Interscript Version')
@# --------------------------------------------------------------------
@# Print information identifying the version of intercsript which
@# is being generated, and the version which is generating it.
@try:
   weave('This document describes Interscript version '+gen_version)
   weave(' build '+str(gen_buildno))
   weave(' on '+gen_hostname+' ')
   weave(' by '+gen_username+'.')

   weave('It was generated by Interscript version '+global_frame.version+', build '+str(global_frame.buildno))
   weave(' on '+global_frame.hostname)
   weave(' by '+global_frame.username)
   weave(' at '+global_frame.buildtime+'.')
 except: pass

@# --------------------------------------------------------------------
@# Print various tables.
@p()
@print_contents(hlevel=2,maxlev=2)

@# --------------------------------------------------------------------
@# Print the introduction and design principles chapters.
@include_file('introduction.ipk')
@include_file('requirements.ipk')
@include_file('design.ipk')
@include_file('tutorial.pak')


@# --------------------------------------------------------------------
@# Generate the source code and program documentation.
@head(1,'Implementation')
Literate programmed by itself, of course!

@head(2,'Interscript Module')
The top level interscript module.
@select(data_output('interscript.README'))
The literate programming tool Interscript
@tangle(' version '+repr(gen_version))
 has been unpacked here.
It consists of two text files, and a directory:

  interscript.README (this file)
  interscript.pth    (the Python package control file)
  interscript/       (the interscript package directory)

Find the basic HTML documentation at

  interscript/doc/iscrp.html

Find the enhanced HTML documentation at

  interscript/doc/iscr_top.html

Find the text documentation at

  interscript/doc/iscr.txt

Find the latex2e documentation at

  interscript/doc/iscr.tex

Find the interscript home page at

  http://www.triode.net.au/~skaller/interscript

@select(data_output('interscript.pth'))
interscript
@iscr_pkg = python_output('interscript/__init__.py')
@select(iscr_pkg)
# interscript package
@if debug_version:
  # announce the version when the module is imported.
  tangle('print "Interscript Package: version '+gen_version+'",')
  tangle('print "build '+str(gen_buildno)+'"')

@head(3,'Construct Global Frame')
This frame is shared between all processes, and is initialised
at module load time. It hooks crucial resources and identification
information. The attributes of the global frame class are used
as the globals() dictionary for executing user scripts.
@p()
[This is probably a bad idea, because it allows the user to change
the attributes using the global declaration. On the other hand,
it provides a method for sharing between processes.]
@p()
For some weird reason, the global frame is a python class,
not a module, and not a class instance. It's not a plain module,
because python termination is somewhat indeterminate, and it isn't
an instance, because there's only ever one of them. Clients of the
global frame keep a reference to it explicitly to prevent
premature deletion by the python run time on program termination.
@p()
The global fram in turn keeps references to a set of important resources,
so that they're not deleted prematurely either.
I'm doing this because interscript __del__ methods are often used to
do substantial work, and it's imperative that system resources are
available until all dynamically created objects are destroyed.


@select(iscr_pkg)
class global_frame:

  from interscript.drivers.sinks.bufdisk import named_file_sink
  from interscript.drivers.sinks.disk import simple_named_file_sink
  from interscript.drivers.sinks.null import null_sink
  from interscript.drivers.sinks.cache import cache_sink

  from interscript.drivers.sources.base import eoi, eof
  from interscript.drivers.sources.disk import named_file_source
  from interscript.drivers.sources.url import url_source
  from interscript.drivers.sources.ftp import ftp_file_source
  from interscript.drivers.sources.http import http_file_source
  from interscript.drivers.sources.cache import cache_source

  from interscript.drivers.storage.memory import memory

  from interscript.weavers.text import plain_text_weaver
  from interscript.weavers.latex import latex_weaver
  from interscript.weavers.html import html_weaver
  from interscript.weavers.raw import raw_weaver
  from interscript.weavers.web import stacking_weaver
  from interscript.weavers.auto import auto_weaver
  from interscript.weavers.filter import markup_filter
  from interscript.weavers.multiplexor import multiplexor

  from interscript.parsers.html import sgml_wrapper, html_filter

  from interscript.tanglers.c import c_tangler
  from interscript.tanglers.cpp import cpp_tangler
  from interscript.tanglers.java import java_tangler
  from interscript.tanglers.perl import perl_tangler
  from interscript.tanglers.data import data_tangler
  from interscript.tanglers.python import python_tangler
  from interscript.tanglers.null import null_tangler
  from interscript.tanglers.doc import doc_tangler
  from interscript.tanglers.tcl import tcl_tangler

  import sys
  import os
  import string
  import re
  import time
  from interscript.utilities import commands
  from interscript.core.sets import set
  from interscript.core.stacks import stack
  import interscript.core.protocols
  protocol = interscript.core.protocols
  import getoptions

  import __builtin__
  __builtins__ = __builtin__
  del __builtin__

  try:
    import thread
    #print 'thread available'
  except:
    #print 'thread NOT available'
    pass

@head(3,'Set version')
This is an experimental release only,
intended as an act of advocacy and to solicit comments and support.
@p()
The code below is tricky to understand. There are three distinct
sets of version information: data describing the version
being generated, data describing the version doing the generating,
and data describing the version which generated the generator.
@p()
Python script at the top of this file computes or fixes
the version information for the version being generated,
that information is bound into the source of the generated
version.
@p()
When that version is itself used to generated yet another
version the data bound into it by the previous generation
describes the currently executing version, which is now
the generator.
@p()
When the version being generated is yet again used to generate
another version, the original data now describes the version
that generated the version generating the new version.
@p()
In order to make it easy to add or change identification attributes,
they're initialised to dummy values, in case the generating version
doesn't has the same set of values. This is always the case
when a new attribute is introduced: the attribute won't make it into
the generating version until a second pass is executed and the version
being generated itself becomes the generator.
@p()
Note that knowing the version that generated the executing version
is important for bug tracking, since a problem may be due to a bug
in the source for the version, or in the generator which processed
it: a bug can persist even when the source is correct for many generations
if it is not processed correctly by the generator.
Bootstrapped code can be a real nightmare to debug.

@select(iscr_pkg)
# first a hack to help bootstrapping work
# if any of the variable in the second section don't exist.
# then the at least some value is set in the generated code.
# Iterated bootstrapping should eventually fix the problem.

@tangle('  buildno=0')
@tangle('  version=0')
@tangle('  hostname="unknown"')
@tangle('  username="unknown"')
@tangle('  buildtime="unknown"')
@tangle('  generator_buildno=0')
@tangle('  generator_hostname="unknown"')
@tangle('  generator_username="unknown"')
@tangle('  generator_version="unknown"')
@tangle('  generator_buildtime="unknown"')

# now the real data
@tangle('  buildno='+str(gen_buildno))
@tangle('  version='+repr(gen_version))
@tangle('  hostname='+repr(gen_hostname))
@tangle('  username='+repr(gen_username))
@tangle('  buildtime='+repr(ufmtime))
@tangle('  generator_buildno='+str(global_frame.buildno))
@tangle('  generator_hostname='+repr(global_frame.hostname))
@tangle('  generator_username='+repr(global_frame.username))
@tangle('  generator_version='+repr(global_frame.version))
@tangle('  generator_buildtime='+repr(global_frame.buildtime))

# now print the current version information
# wrapped in try/except clause in case any of the variables didn't get set
try:
  print 'Interscript version',global_frame.version,
  print 'build',global_frame.buildno
  print 'Built by',global_frame.username,
  print 'on',global_frame.hostname,
  print 'at',global_frame.buildtime
  print 'Generated by',global_frame.generator_version,
  print 'buildno',global_frame.generator_buildno,
  print 'host',global_frame.generator_hostname
  print 'at',global_frame.buildtime
except: pass

# This is a utility function that makes it easy to use interscript
# givem options in a standard form. The arguments are a list as
# would be entered on a unix or nt command line.
# Mac (or Tkinter) users can create a GUI interface to set the options
# and then call this function to run interscript.

def run_from_options(arguments):
  from interscript.getframes import getoption_frames
  from interscript.frames.processf import process_frame
  process_options, master_options = getoption_frames(arguments)
  process = process_frame(global_frame, process_options, master_options)
  process.run()
  del process

@head(3,'Listing of iscr.pak source')
Here is the fully listings of the top level interscript source file, iscr.pak.
@display_code('iscr.pak')

@head(2,'Core Subpackage')
@select(python_output('interscript/core/__init__.py'))
# interscript core modules

@push_head(2)
@include_file('sets.ipk')     # sets
@include_file('stacks.ipk')   # stacks
@include_file('protocols.ipk')# protocols
@pop_head(2)

@push_head()
@include_file('drivers.ipk')  # drivers
@include_file('weavers.ipk') # weavers
@include_file('weaver_filters.ipk') # weaver filters
@include_file('tanglers.ipk')  # tanglers
@include_file('tokenizers.ipk')  # tokenisers
@include_file('parsers.ipk') # parsers
@include_file('felix.pak') # felix category stuff
@include_file('frames.ipk') # architecture
@include_file('options.ipk') # generic option processing
@include_file('interscript_options.ipk') # interscript option processing
@pop_head()


@head(2,'Utility Modules')
@select(python_output('interscript/utilities/__init__.py'))
# interscript utilities

@push_head(2)
@include_file('compilers.ipk')
@include_file('diff.ipk')
@include_file('commands.ipk')
@pop_head(2)


@head(2,'Application and tool directory')
This is where the interscript mainline and any other shell commands go.
@select(python_output('interscript/bin/__init__.py'))
# dummy interscript.bin

@head(3,'Stand alone unix/nt mainline')
Command line stub.

@select(python_output('interscript/bin/iscr.py'))
#!/usr/bin/env python
import sys
args = sys.argv[1:]
if len(args)>0 and args[0]=='--test':
  print 'Interscript test mode, loading interscript from current directory'
  if sys.path[0]!='':
    sys.path = ['']+ sys.path
  args = args[1:]
import interscript
interscript.run_from_options(args)

@head(3,'Windows launcher')
A batch file to launch interscript.
You will need to edit the file!
@select(python_output('interscript/bin/iscr.bat'))
python iscr.py %1 %2 %3 %4 %5 %6 %7 %8 %9

@p()
There's a better way in Windows NT!
(SOMEONE CHECK THIS!)
Using the 'ASSOC' and 'FTYPE' commands,
you can specify the type of .pak files, and what to do with them:
@begin_displayed_code()
  ASSOC .pak=Interscript.Master.Document
  FTYPE Interscript.Master.Document=D:\interscript\interscript\bin\iscr.py %*
@end_displayed_code()
and then you can just type the name of the interscript
file with options to process it. This should already
have been done for python .py files if you ran a recent
version of the Windows installer. If you also do:
@begin_displayed_code()
  set PATHEXT=.pak;%PATHEXT%
@end_displayed_code()
you won't need to type the extension. To make this persistent,
you can use the Control Panel System/Environment editor.

@head(2,'Test package')
@select(python_output('interscript/tests/__init__.py'))
# dummy interscript/test

@head(1,'Appendices')
@# --------------------------------------------------------------------
@# Print the table of identifiers and class reference
@print_identifier_cross_reference(hlevel=2)
@print_class_reference(hlevel=2)
@print_file_list(hlevel=2)
@print_source_list(hlevel=2)
@print_include_list(hlevel=2)
@push_head()
@include_file('bugs.ipk')
@include_file('installation.ipk')
@pop_head()


6.2. Core Subpackage
--------------------


Start python section to interscript/core/__init__.py[1]

     1: #line 428 "iscr.pak"

6.2.1. Sets
-----------

This class implements mathematical sets using a
dictionary.

Start python section to interscript/core/sets.py[1]

     1: #line 6 "sets.ipk"

6.2.1.1. Test source
--------------------

A simple test of sets.

Start python section to interscript/tests/test_sets.py[1]

     1: #line 163 "sets.ipk"

6.2.1.2. Test output
--------------------

The result of running the test.

Start output section of "/usr/local/bin/python" interscript/tests/test_sets.py

     1: Interscript version 1.0a8 build 1378
     2: Built by root on ruby at Sun Nov 29, 1998 at 08:48 PM (UTC)
     3: Generated by 1.0a8 buildno 1376 host ruby
     4: at Sun Nov 29, 1998 at 08:48 PM (UTC)
     5: s1= set(3, 2, 1)
     6: s2= set(30, 20, 3, 10)
     7: s1|s2= set(10, 30, 20, 3, 2, 1)
     8: s1&s2= set(3)
     9: s1^s2= set(30, 20, 10, 1, 2)
    10: s1+s2= set(10, 30, 20, 3, 2, 1)
    11: s1-s2= set(2, 1)
    12: s1==s1= 1
    13: s1[:]= set(3, 2, 1)
    14: s1[1:-1]= set(2)
    15: s1==s2= 0
    16: s1<s2= 1
    17: s1>s2= 0
    18: s1.le(s2)= 0
    19: s1.lt(s2)= 0
    20: s1.ge(s2)= 0
    21: s1.gt(s2)= 0
    22: s1= set(9, 8, 7, 6, 5, 4, 3, 2, 1)
    23: s2= set(7, 5, 3, 9, 1)
    24: s1.le(s2)= 0
    25: s1.lt(s2)= 0
    26: s1.ge(s2)= 1
    27: s1.gt(s2)= 1

End output section of "/usr/local/bin/python" interscript/tests/test_sets.py

6.2.2. Stack
------------

Simple stack class.

Start python section to interscript/core/stacks.py[1]

     1: #line 5 "stacks.ipk"

6.2.3. protocol module
----------------------

This modules implements protocols. Install in
site-packages.

6.2.3.1. Standard Protocol Rules
--------------------------------

Every object o has the protocol type(o). Every instance
of class X, has protocol X. An object has protocol p,
if p is a class protocol of its class, or any direct or
indirect base of that class.

An object o of type t has protcol p if
add_type_proto(t,p) has_been called. An object o which
is an instance of class c, has protocol p if
add_class_protol(b,p) has been called, where b is c or
one of its direct or indirect bases.

An object o has protocol p if add_obj_proto(o,p) has
been called and returned normally.

It is not possible to attach protocols to an object
using add_obj_proto unless the object either has a
readable attribute '__protocols__' which is a mutable
sequence, or will set an attribute '__protocols__'
(which can subsequently be read).

Please note the crucial distinction between the
protocols of a class object c, which include
c.__protocols__, and the protocols of instances of c,
which include c.__class_protocols__. A class object is
an object in its own right and obeys protocols distinct
from its instances.

Similarly, note that a type object has different
protocols than the instances of that type.

6.2.3.2. Source
---------------


Start python section to interscript/core/protocols.py[1]

     1: #line 38 "protocols.ipk"


End python section to interscript/core/protocols.py[1]

6.2.3.3. Test
-------------


Start python section to interscript/tests/test_protocol.py[1]

     1: #line 151 "protocols.ipk"


End python section to interscript/tests/test_protocol.py[1]

Start output section of "/usr/local/bin/python" interscript/tests/test_protocol.py

     1: Interscript version 1.0a8 build 1378
     2: Built by root on ruby at Sun Nov 29, 1998 at 08:48 PM (UTC)
     3: Generated by 1.0a8 buildno 1376 host ruby
     4: at Sun Nov 29, 1998 at 08:48 PM (UTC)

End output section of "/usr/local/bin/python" interscript/tests/test_protocol.py

     4: #line 435 "iscr.pak"

6.3. Drivers Subpackage
-----------------------

There are three classes of drivers: source drivers,
sink drivers, and storage drivers.

Start python section to interscript/drivers/__init__.py[1]

     1: #line 6 "drivers.ipk"

6.3.1. Source Drivers Subpackage
--------------------------------

This subpackage contains components to read lines from
various sources. Note that an interscript line is not
terminated by a newline, and it never ends in spaces.
Control characters are not permitted in lines.

It should be noted that it is the responsibility of the
driver to translate the underlying file into a source
of whole lines according to these requirements.

Source drivers supply the following standard python
methods:

read(size=None) 
     Return the whole file as a single string with
    lines terminated by newline characters. Whitespace
    is stripped from the end of each line before the
    newline is appended. At least one line will be
    returned, and there will always be a newline at the
    end of the string. The optional size argument is
    ignored. This method throws an eof exception if
    there are no lines in the file. An exception is
    thrown if the file is has been closed.

readline(sizehint=None) 
     Read a up to the end of a line, return the read
    data with a newline appended. Whitespace is
    stripped from the end of the line before the
    newline is appended. This method throws an eof
    exception if there are no lines in the file. An
    exception is thrown if the file is has been closed.

readlines(sizehint=None) 
     Read to the end of the file, returning a list of
    lines, each of which has had trailing whitespace
    stripped and a newline character appended. An empty
    list is returned if there are no lines in the file.
    An exception is thrown if the file is has been
    closed.

flush() 
     May flush input device.

In addition, the following methods are provided:

raw_readline() 
     Return next line. The line has trailing whitespace
    stripped, no newline character is appended. Throws
    an exception if there are no lines left in the
    file.

raw_readlines() 
     Return a list of all lines to the end of the file,
    with trailing whitespace stripped and no newlines
    added. Returns an empty list if there are no lines
    in the file. This routine is intended to be
    supplied by derivers of subclasses of the standard
    base.

6.3.1.1. Source Drivers Module
------------------------------

The source drivers module contains the class for the
exception to be thrown when a source resource cannot be
acquired, usually a file which cannot be opened because
it doesn't exist.

Start python section to interscript/drivers/sources/__init__.py[1]

     1: #line 64 "source_drivers.ipk"


End python section to interscript/drivers/sources/__init__.py[1]

6.3.1.2. Source Base
--------------------


6.3.1.2.1. End Exceptions
-------------------------

A class to be used as an end of file exception, and a
class to be used as an end of information exception.
The distinction is: any input driver (source) file
could throw an eof. An eoi is thrown by a higher level
routine to indicate end of logical input frame.

These exceptions, when thrown, do not represent an
error, they're just used to facilitate alternate block
exits.

Start python section to interscript/drivers/sources/base.py[1]

     1: #line 80 "source_drivers.ipk"


End python section to interscript/drivers/sources/base.py[1]

6.3.1.2.2. Source Base Class
----------------------------

Interscript source drivers provide all the facilities
required of standard python file objects opened for
input.

Start python section to interscript/drivers/sources/base.py[2]

     5: #line 88 "source_drivers.ipk"


End python section to interscript/drivers/sources/base.py[2]

6.3.1.3. Disk File Input
------------------------

This driver uses a filename in your local operating
system's convention.

Start python section to interscript/drivers/sources/disk.py[1]

     1: #line 162 "source_drivers.ipk"


End python section to interscript/drivers/sources/disk.py[1]

6.3.1.4. URL input
------------------

This driver accepts a URL. Since it uses the Python
urllib library, it has the same limitations. From the
manual:

*    Currently, only the following protocols are
    supported: HTTP, (versions 0.9 and 1.0), Gopher
    (but not Gopher-+), FTP, and local files.

*    The caching feature of urlretrieve() has been
    disabled until I find the time to hack proper
    processing of Expiration time headers.

*    There should be a function to query whether a
    particular URL is in the cache.

*    For backward compatibility, if a URL appears to
    point to a local file but the file can't be opened,
    the URL is re-interpreted using the FTP protocol.
    This can sometimes cause confusing error messages.

*    The urlopen() and urlretrieve() functions can
    cause arbitrarily long delays while waiting for a
    network connection to be set up. This means that it
    is difficult to build an interactive web client
    using these functions without using threads.

*    The data returned by urlopen() or urlretrieve() is
    the raw data returned by the server. This may be
    binary data (e.g. an image), plain text or (for
    example) HTML. The HTTP protocol provides type
    information in the reply header, which can be
    inspected by looking at the Content-type header.
    For the Gopher protocol, type information is
    encoded in the URL; there is currently no easy way
    to extract it. If the returned data is HTML, you
    can use the module htmllib to parse it.

*    Although the urllib module contains (undocumented)
    routines to parse and unparse URL strings, the
    recommended interface for URL manipulation is in
    module urlparse.

Start python section to interscript/drivers/sources/url.py[1]

     1: #line 260 "source_drivers.ipk"


End python section to interscript/drivers/sources/url.py[1]

6.3.1.5. FTP input
------------------

This object fetches a file by ftp on construction. The
file is then read from the local copy. The host name
and remote filename arguments are mandatory. Optional
arguments given by keywords are:

port 
     The port number of the host to use for FTP.
    Defaults to 21.

user 
     User name. Defaults to 'anonymous'.

account 
     User's account name.

password 
     User's password. If user is anonymous, the default
    password is `realuser@host' where realuser is the
    real user name (glanced from the `LOGNAME' or
    `USER' environment variable) and host is the
    hostname as returned by socket.gethostname().

remote_directory 
     The directory to 'cd' to to get the nominated
    file. If not specified, no 'cd' is done and the
    file is expected in the FTP root.

local_filename 
     The name of the file to which the download should
    be written. Defaults to 'remote_filename'.

refresh_interval 
     The is the number of days old the local file can
    be before ftp is done. The default is 28. If this
    value is set to 0, a download is always done. If
    the value is set to -1, a download is done only if
    the local file does not exist; delete the file to
    force another download.

An exception is thrown if there is no local file, and
the attempt to ftp the remote file fails. No exception
is thrown if the local file exists, even if it is out
of date.

Start python section to interscript/drivers/sources/ftp.py[1]

     1: #line 317 "source_drivers.ipk"


End python section to interscript/drivers/sources/ftp.py[1]

6.3.1.6. HTTP input
-------------------

This object fetches a file by http on construction. The
file is then read from the local copy. The host name
and remote filename arguments are mandatory. Optional
arguments given by keywords are:

port 
     The port number of the host to use for FTP.
    Defaults to 21.

local_filename 
     The name of the file to which the download should
    be written. Defaults to 'remote_filename'.

refresh_interval 
     This is the number of days old the local file can
    be before ftp is done. The default is 28. If this
    value is set to 0, a download is always done. If
    the value is set to -1, a download is done only if
    the local file does not exist; delete the file to
    force another download.

An exception is thrown if there is no local file, and
the attempt to ftp the remote file files. No exception
is thrown if the local file exists, even if it is out
of date.

Start python section to interscript/drivers/sources/http.py[1]

     1: #line 472 "source_drivers.ipk"


End python section to interscript/drivers/sources/http.py[1]

6.3.1.7. Standard Input
-----------------------


Start python section to interscript/drivers/sources/stdin.py[1]

     1: #line 590 "source_drivers.ipk"


End python section to interscript/drivers/sources/stdin.py[1]

6.3.1.8. Cache
--------------


Start python section to interscript/drivers/sources/cache.py[1]

     1: #line 616 "source_drivers.ipk"

6.3.2. Sink Drivers
-------------------

Sink drivers support the standard Python file protocol
for output of text. They supply the standard python
file methods:

write 
     Write text to output device. Repeatedly raw_write
    characters upto either the end of the string, or a
    newline character. If a newline character is found,
    increment the line count amd call raw_eol.
    Precondition: no control characters other than
    newline in the input.

writelines 
     Accepts a list of strings which are concatenated
    and passed to the write method. No newline
    characters are added. Precondition: no control
    characters other than newline in the input.

close 
     If the sink is not flagged as closed, flag the
    sink as closed for access to the device and call
    raw_close. Note this need not close the underlying
    device.

flush 
     May flush data to the device.

isatty 
     May return 1 if the underlying device is a tty
    type device.

The methods read, readline and readlines are note
provided because a sink need not support read
operations. The methods seek and tell are not provided
because sinks need not support positioning. The method
truncate is not provided because a sink need not be
storage device. There is no mode attribute, sinks
support writing, but need not be files.

The following methods augment the standard python
methods.

writeline 
     Append a newline character to the argument and
    pass to write method.

raw_writelines 
     Accepts a list of strings which are written in
    turn using writeline. Use with a list of printable
    strings; that is, without trailing newlines.

The following methods should be provided by subclass
derivers, if the default implementation (if it exists)
is inappropriate. They are intended as private
implementation details, not to be called by the user.

raw_write 
     Write raw text to the device. Precondition: only
    printable characters in the input (space is
    allowed). [All characters are written verbatim to
    the underlying device without incrementing the line
    count.]

raw_close 
     May close the underlying device. Called by the
    close method if the closed flag is false.

raw_flush 
     May flush the underlying device device.

raw_eol 
     Ends a line on the underlying device.

In addition, the following attributes are available:

lines_written 
     The total number of calls to writeline.

last_source_file 
     Provided for the client to store the name of the
    original source of the last line written.

last_source_line 
     Provided for the client to store the line number
    of the last line written in the original source
    file.

last_inhibit_sref 
     Provided for the client to store whether source
    referencing was disabled on the last write.

closed 
     Indicates if close was called.

Start python section to interscript/drivers/sinks/__init__.py[1]

     1: #line 96 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/__init__.py[1]

Start python section to interscript/drivers/sinks/util.py[1]

     1: #line 102 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/util.py[1]

6.3.2.1. Sink Base Class
------------------------


Start python section to interscript/drivers/sinks/base.py[1]

     1: #line 181 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/base.py[1]

6.3.2.2. Null Sink
------------------


Start python section to interscript/drivers/sinks/null.py[1]

     1: #line 231 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/null.py[1]

6.3.2.3. Tee Sink
-----------------


Start python section to interscript/drivers/sinks/tee.py[1]

     1: #line 242 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/tee.py[1]

6.3.2.4. Simple Disk File Sink
------------------------------


Start python section to interscript/drivers/sinks/disk.py[1]

     1: #line 253 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/disk.py[1]

6.3.2.5. Complex Disk File Sink
-------------------------------

This class provides more advanced facilities than the
simple disk file. Output is written to a temporary file
until the driver is closed. At that point, the
temporary file is compared line by line with the target
file, and if the files differ, the temporary file is
copied onto the target, and then, in either case the
temporary deleted.

This mechanism has several purposes. First, it is used
to prevent 'make' systems recompiling unchanged code
files. Second, it is used so that the output of a
previous run can be read back in while the current run
is generating a new copy. Finally, the comparisons on
all these files can be used to determine if the
literate programming process has converged to a stable
point or may require another pass.

Please note that some care is required to ensure
convergence is possible. Note also that when line
numbers are generated, a single change to the source
file will cause comparisons to fail for all files with
sections defined after that point, often causing an
apparently redundant rebuild. This is not alsways the
case, for example, in C, the assert macro reports line
numbers at run time, and the rebuild is necessary. It
may be useful to turn off line number generation for
files once they have compiled successfully (until the
production build).

Finally, note that if you put the time into an output
file, it is likely to be different each run!

Start python section to interscript/drivers/sinks/bufdisk.py[1]

     1: #line 311 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/bufdisk.py[1]

6.3.2.6. Standard Output Sink
-----------------------------


Start python section to interscript/drivers/sinks/stdout.py[1]

     1: #line 388 "sink_drivers.ipk"


End python section to interscript/drivers/sinks/stdout.py[1]

6.3.2.7. Persistent Storage Sink
--------------------------------

This is really the same as a disk file, except the data
is stored in an array, and stashed into the cache.

Start python section to interscript/drivers/sinks/cache.py[1]

     1: #line 402 "sink_drivers.ipk"

6.3.3. Storage Drivers
----------------------

These drivers support both reading and writing
operations.

Start python section to interscript/drivers/storage/__init__.py[1]

     1: #line 4 "storage_drivers.ipk"


End python section to interscript/drivers/storage/__init__.py[1]

6.3.3.1. Memory Driver
----------------------

The memory driver is both a source and sink and can is
used as a simple form of macro. By tangling sections of
code into memory, a sction can be built which is later
emitted in a single block. In this sense the tangler
driving the memory device is defining a macro, and its
later inclusion in te source stream can be considered
expanding the macro.

Start python section to interscript/drivers/storage/memory.py[1]

     1: #line 16 "storage_drivers.ipk"


End python section to interscript/drivers/storage/memory.py[1]

6.3.3.2. Disk Driver
--------------------

The disk driver is also both a source and sink and can
is used as a simple form of macro. By tangling sections
of code onto disk, a section can be built which is
emitted in a single block.

By using differential files drivers, and performing
multiple passes on the source, disk based macros can be
emitted before they are defined, or even in the middle
of their definition: the result will be the filw
written from the last run.

Start python section to interscript/drivers/storage/disk.py[1]

     1: #line 61 "storage_drivers.ipk"
     3: #line 11 "drivers.ipk"

6.4. Weavers
------------


Start python section to interscript/weavers/__init__.py[1]

     1: #line 3 "weavers.ipk"


End python section to interscript/weavers/__init__.py[1]

6.4.1. Weaver Base
------------------


Start python section to interscript/weavers/base.py[1]

     1: #line 8 "weavers.ipk"


End python section to interscript/weavers/base.py[1]

6.4.2. Raw weaver
-----------------

The raw weaver outputs text exactly as it is input,
special characters included. It cannot do any
formatting.

Use of raw weavers is vital for output of typesetter
specialised constructions. For example, If you are
generating output for both HTML and Latex, and you have
a complex table to print, you can set the weaver to a
raw weaver attached to the same sink file as the HTML
weaver and write HTML directly, then you can do the
same for Latex. In that way, you get tables optimised
for the various typesetting systems.

Start python section to interscript/weavers/raw.py[1]

     1: #line 50 "weavers.ipk"


End python section to interscript/weavers/raw.py[1]

6.4.3. Multiplex Weaver
-----------------------

This class is used as a base for classes that represent
multiple weavers. It supports only one operation:
method call with no return value, because only this
functionality is transparent; that is, operates the
same for a single object, or a set for which the
multiplexor is a proxy.

The multiplex weaver also supports list shortcuts.

The technology is unfortunately not re-entrant, the
multiplex call method return must be invoked
immediately. This makes multi-threaded calls to
multiplexed objects unsafe. (Python lacks appropriate
categorical constructions.) We could use a thread lock
to fix this, but it doesn't seem worth the effort.

Start python section to interscript/weavers/multiplexor.py[1]

     1: #line 78 "weavers.ipk"

6.4.4. Plain text weaver
------------------------

This nasty weaver tries to output plain text that can
be published on the Internet. It can underline
headings, and flow text with left justification (and
should be able to format tables), but the results are
rather crude. Right justification is possible, but not
implemented because, in my experience, the results are
woeful.

Start python section to interscript/weavers/text.py[1]

     1: #line 11 "text_weaver.ipk"


End python section to interscript/weavers/text.py[1]

6.4.4.1. reference processor
----------------------------


Start python section to interscript/weavers/text.py[2]

    47: #line 58 "text_weaver.ipk"


End python section to interscript/weavers/text.py[2]

6.4.4.2. Table of Contents
--------------------------


Start python section to interscript/weavers/text.py[3]

    70: #line 82 "text_weaver.ipk"


End python section to interscript/weavers/text.py[3]

6.4.4.3. Code File List
-----------------------


Start python section to interscript/weavers/text.py[4]

    83: #line 96 "text_weaver.ipk"


End python section to interscript/weavers/text.py[4]

6.4.4.4. Code File Status
-------------------------


Start python section to interscript/weavers/text.py[5]

    94: #line 108 "text_weaver.ipk"


End python section to interscript/weavers/text.py[5]

6.4.4.5. Input File List
------------------------


Start python section to interscript/weavers/text.py[6]

   148: #line 163 "text_weaver.ipk"


End python section to interscript/weavers/text.py[6]

6.4.4.6. Include List
---------------------


Start python section to interscript/weavers/text.py[7]

   159: #line 175 "text_weaver.ipk"


End python section to interscript/weavers/text.py[7]

6.4.4.7. Tables
---------------


Start python section to interscript/weavers/text.py[8]

   170: #line 187 "text_weaver.ipk"


End python section to interscript/weavers/text.py[8]

6.4.4.8. Code Output
--------------------


Start python section to interscript/weavers/text.py[9]

   226: #line 244 "text_weaver.ipk"


End python section to interscript/weavers/text.py[9]

6.4.4.9. Citations
------------------


Start python section to interscript/weavers/text.py[10]

   393: #line 412 "text_weaver.ipk"


End python section to interscript/weavers/text.py[10]

6.4.4.10. Citations
-------------------


Start python section to interscript/weavers/text.py[11]

   437: #line 457 "text_weaver.ipk"
    72: #line 151 "weavers.ipk"

6.4.5. HTML Weaver
------------------

The most commonly used weaver is sure to be the HTML
weaver. This weaver attempts to generate HTML compliant
with the proposed strict HTML 4.0 DTD. See
http://www.w3.org/TR/REC/-html40

Interscript uses a different markup model to HTML.
While interscript uses begin and end methods like many
HTML tags, HTML paragraph elements are not indicated by
such methods. Instead, text written in many places is
implicitly a paragraph element, and interscript must
detect these places and insert an appropriate start
tag. Similarly, interscript paragraphs are separated by
@p() commands, and terminated by some other block level
construction like a heading or table.

On the other hand, not all text lines are paragraph
content, for example list body content.

Start python section to interscript/weavers/html.py[1]

     1: #line 21 "html_weaver.ipk"


End python section to interscript/weavers/html.py[1]

6.4.5.1. Initialisation
-----------------------


Start python section to interscript/weavers/html.py[2]

    31: #line 52 "html_weaver.ipk"


End python section to interscript/weavers/html.py[2]

6.4.5.2. Termination
--------------------


Start python section to interscript/weavers/html.py[3]

    55: #line 77 "html_weaver.ipk"


End python section to interscript/weavers/html.py[3]

6.4.5.3. Body Output and Mode Control
-------------------------------------


Start python section to interscript/weavers/html.py[4]

    60: #line 83 "html_weaver.ipk"


End python section to interscript/weavers/html.py[4]

6.4.5.4. Reference Processor
----------------------------


Start python section to interscript/weavers/html.py[5]

   212: #line 236 "html_weaver.ipk"


End python section to interscript/weavers/html.py[5]

6.4.5.5. Headings
-----------------


Start python section to interscript/weavers/html.py[6]

   226: #line 251 "html_weaver.ipk"


End python section to interscript/weavers/html.py[6]

6.4.5.6. Lists
--------------

Interscript has a problem here. HTML 4.0 allows the
content of a list item to be either inline text or
block elements.

Interscript has no way to distinguish between inline
text and a paragraph block. Even if we look ahead, a
plain text stream could be intended as either a
paragraph or inline text.

Unfortunately, NS3 and IE4 put blank lines in on </P>
tags, which is wrong: the presence or absence of an
optional tag ought not affect rendering. For this
reason, I've been forced to remove the wrapping of text
in <P> and </P> tags; the @p() command inserts a plain
<P> after the _first_ paragraph. Technically, this is
invalid HTML, the first pargraph is not wrapped in
start and end P tags.

6.4.5.6.1. Numbered Lists
-------------------------


Start python section to interscript/weavers/html.py[7]

   268: #line 311 "html_weaver.ipk"


End python section to interscript/weavers/html.py[7]

6.4.5.6.2. Bullet Lists
-----------------------


Start python section to interscript/weavers/html.py[8]

   282: #line 326 "html_weaver.ipk"


End python section to interscript/weavers/html.py[8]

6.4.5.6.3. Keyed Lists
----------------------


Start python section to interscript/weavers/html.py[9]

   297: #line 342 "html_weaver.ipk"


End python section to interscript/weavers/html.py[9]

6.4.5.7. Code Output
--------------------


Start python section to interscript/weavers/html.py[10]

   311: #line 357 "html_weaver.ipk"


End python section to interscript/weavers/html.py[10]

6.4.5.8. Citations
------------------


Start python section to interscript/weavers/html.py[11]

   336: #line 383 "html_weaver.ipk"


End python section to interscript/weavers/html.py[11]

6.4.5.9. Paragraphs
-------------------


Start python section to interscript/weavers/html.py[12]

   340: #line 388 "html_weaver.ipk"


End python section to interscript/weavers/html.py[12]

6.4.5.10. Fonts
---------------


Start python section to interscript/weavers/html.py[13]

   356: #line 405 "html_weaver.ipk"


End python section to interscript/weavers/html.py[13]

6.4.5.11. Identifier Cross Reference Table
------------------------------------------


Start python section to interscript/weavers/html.py[14]

   399: #line 449 "html_weaver.ipk"


End python section to interscript/weavers/html.py[14]

6.4.5.12. Class Reference Table
-------------------------------


Start python section to interscript/weavers/html.py[15]

   423: #line 474 "html_weaver.ipk"


End python section to interscript/weavers/html.py[15]

6.4.5.13. Table of Contents
---------------------------


Start python section to interscript/weavers/html.py[16]

   451: #line 503 "html_weaver.ipk"


End python section to interscript/weavers/html.py[16]

6.4.5.14. Code File List
------------------------


Start python section to interscript/weavers/html.py[17]

   462: #line 515 "html_weaver.ipk"


End python section to interscript/weavers/html.py[17]

6.4.5.15. Code File Status
--------------------------


Start python section to interscript/weavers/html.py[18]

   472: #line 526 "html_weaver.ipk"


End python section to interscript/weavers/html.py[18]

6.4.5.16. Input File List
-------------------------


Start python section to interscript/weavers/html.py[19]

   525: #line 580 "html_weaver.ipk"


End python section to interscript/weavers/html.py[19]

6.4.5.17. Include List
----------------------


Start python section to interscript/weavers/html.py[20]

   535: #line 591 "html_weaver.ipk"


End python section to interscript/weavers/html.py[20]

6.4.5.18. Tables
----------------


Start python section to interscript/weavers/html.py[21]

   545: #line 602 "html_weaver.ipk"


End python section to interscript/weavers/html.py[21]

6.4.5.19. HTML Header
---------------------


Start python section to interscript/weavers/html.py[22]

   570: #line 628 "html_weaver.ipk"


End python section to interscript/weavers/html.py[22]

6.4.5.20. Interscript Style Sheet
---------------------------------

The file 'interscript/doc/interscript.css' is a CSS1
style sheet for interscript documents. It is <LINK>ed
by interscript documentation, but is not currently
copied to the weaver output directory.

HTML documents also <LINK> a stub file, 'user.css'
which allows the user to override the interscript
standard one.

Start data section to interscript/doc/interscript.css[1]

     1: BODY {
     2:   background-color : #FFFFF0;
     3: }
     4: BODY EM {
     5:   color: #A01010;
     6: }
     7: 
     8: BODY CODE {
     9:   color: #101080;
    10: }
    11: DIV.CODE {
    12:   color : #101080;
    13:   background-color: #E0FFFF;
    14:   margin-top: 0.0 ex;
    15:   padding-top: 0.4 ex;
    16:   padding-bottom: 0.2 ex;
    17:   margin-bottom: 0.0 ex;
    18:   margin-right: 2 ex;
    19:   border: thin solid gray;
    20:   display: block;
    21: }
    22: SPAN.LINENO {
    23:   color : #101010;
    24:   font-size: 80%;
    25: }
    26: 
    27: DIV.CODE SPAN.KEYWORD {
    28:   color : #000000;
    29:   font-weight: bold;
    30: }
    31: DIV.CODE SPAN.NAME {
    32:   color : #000000;
    33: }
    34: DIV.CODE SPAN.STRING{
    35:   color : #004000;
    36: }
    37: DIV.CODE SPAN.NUMBER {
    38:   color : #002020;
    39: }
    40: DIV.CODE SPAN.BRACKET {
    41:   color : #800000;
    42: }
    43: DIV.CODE SPAN.PUNCT {
    44:   color : #802000;
    45: }
    46: DIV.CODE SPAN.OP {
    47:   color : #400000;
    48: }
    49: DIV.CODE SPAN.COMMENT {
    50:   color : #000000;
    51:   background-color : #FFF0FF;
    52:   font-size: 80%;
    53:   font-family: sans-serif;
    54: }
    55: 
    56: TABLE.DEFAULT_TABLE_CLASS {
    57:   color: #101010;
    58:   background-color: #F0F0E0;
    59: }
    60: 
    61: TABLE.DIFF {
    62:   color: #204060;
    63:   background-color: #FFE0E0;
    64: }
    65: 
    66: DIV.CODE_SECTION_HEAD {
    67:   margin-top: 1 ex;
    68:   padding-top: 0.2 ex;
    69:   border-top: 0 ex;
    70:   padding-bottom: 0.0 ex;
    71:   border-bottom: 0 ex;
    72:   margin-bottom: -1 ex;
    73:   padding-left: 0.1 em;
    74:   display: block;
    75: }
    76: DIV.CODE_SECTION_FOOT {
    77:   margin-top: -1.0 ex;
    78:   padding-top: 0.0 em;
    79:   border-top-width: 0.0 em;
    80:   border-bottom-width: 0.0 em;
    81:   padding-bottom: 0.0 em;
    82:   margin-bottom: 0.0 em;
    83:   display: block;
    84:   display:none;
    85: }
    86: 
    87: DIV.CODE_SECTION_HEAD SMALL, DIV.CODE_SECTION_FOOT SMALL{
    88:   color : #503020;
    89:   font-size: 80%;
    90:   display: inline;
    91:   color: #101080;
    92: }
    93: DIV.CODE_SECTION_HEAD STRONG, DIV.CODE_SECTION_FOOT STRONG {
    94:   font-family : monospace, courier;
    95:   font-weight: normal;
    96:   font-size: 100%;
    97:   display: inline;
    98:   color : black;
    99: }
   100: DIV.CODE_SECTION_HEAD EM, DIV.CODE_SECTION_FOOT EM {
   101:   display: inline;
   102: }
   103: 
   104: DIV.TEST_OUTPUT {
   105:   color : #101080;
   106:   background-color: #E0FFE0;
   107:   margin-top: 0.0 ex;
   108:   padding-top: 0.4 ex;
   109:   padding-bottom: 0.2 ex;
   110:   margin-bottom: 0.0 ex;
   111:   margin-right: 2 ex;
   112:   border: thin solid gray;
   113:   display: block;
   114: }
   115: DIV.BAD_TEST_OUTPUT {
   116:   color : #101080;
   117:   background-color: #FFE0E0;
   118:   margin-top: 0.0 ex;
   119:   padding-top: 0.4 ex;
   120:   padding-bottom: 0.2 ex;
   121:   margin-bottom: 0.0 ex;
   122:   margin-right: 2 ex;
   123:   border: thin solid gray;
   124:   display: block;
   125: }
   126: DIV.TEST_OUTPUT_RESULT {
   127:   margin-top: 1.5 ex;
   128:   padding-top: 0.2 ex;
   129:   border-top: 0 ex;
   130:   padding-bottom: 0.0 ex;
   131:   border-bottom: 0 ex;
   132:   margin-bottom: -1 ex;
   133:   padding-left: 0.1 em;
   134:   display: block;
   135: }
   136: DIV.TEST_OUTPUT_SECTION_HEAD {
   137:   margin-top: 1 ex;
   138:   padding-top: 0.2 ex;
   139:   border-top: 0 ex;
   140:   padding-bottom: 0.0 ex;
   141:   border-bottom: 0 ex;
   142:   margin-bottom: -1 ex;
   143:   padding-left: 0.1 em;
   144:   display: block;
   145: }
   146: 
   147: DIV.TEST_OUTPUT_SECTION_FOOT {
   148:   margin-top: -1.0 ex;
   149:   padding-top: 0.0 em;
   150:   border-top-width: 0.0 em;
   151:   border-bottom-width: 0.0 em;
   152:   padding-bottom: 0.0 em;
   153:   margin-bottom: 0.0 em;
   154:   display: block;
   155:   display:none;
   156: }
   157: 
   158: DIV.TEST_OUTPUT_SECTION_HEAD SMALL, DIV.TEST_OUTPUT_SECTION_FOOT SMALL{
   159:   color : #503020;
   160:   font-size: 80%;
   161:   display: inline;
   162:   color: #101080;
   163: }
   164: DIV.TEST_OUTPUT_SECTION_HEAD STRONG, DIV.TEST_OUTPUT_SECTION_FOOT STRONG {
   165:   font-family : monospace, courier;
   166:   font-weight: normal;
   167:   font-size: 100%;
   168:   display: inline;
   169:   color : black;
   170: }
   171: DIV.TEST_OUTPUT_SECTION_HEAD EM, DIV.TEST_OUTPUT_SECTION_FOOT EM {
   172:   display: inline;
   173: }
   174: 
   175: DIV.EXPECTED_OUTPUT {
   176:   color : #101080;
   177:   background-color: #E0FFFF;
   178:   margin-top: 0.0 ex;
   179:   padding-top: 0.4 ex;
   180:   padding-bottom: 0.2 ex;
   181:   margin-bottom: 0.0 ex;
   182:   margin-right: 2 ex;
   183:   border: thin solid gray;
   184:   display: block;
   185: }
   186: 
   187: DIV.EXPECTED_OUTPUT_SECTION_HEAD {
   188:   margin-top: 1 ex;
   189:   padding-top: 0.2 ex;
   190:   border-top: 0 ex;
   191:   padding-bottom: 0.0 ex;
   192:   border-bottom: 0 ex;
   193:   margin-bottom: -1 ex;
   194:   padding-left: 0.1 em;
   195:   display: block;
   196: }
   197: 
   198: DIV.EXPECTED_OUTPUT_SECTION_FOOT {
   199:   margin-top: -1.0 ex;
   200:   padding-top: 0.0 em;
   201:   border-top-width: 0.0 em;
   202:   border-bottom-width: 0.0 em;
   203:   padding-bottom: 0.0 em;
   204:   margin-bottom: 0.0 em;
   205:   display: block;
   206:   display:none;
   207: }
   208: 
   209: DIV.EXPECTED_OUTPUT_SECTION_HEAD SMALL, DIV.EXPECTED_OUTPUT_SECTION_FOOT SMALL{
   210:   color : #503020;
   211:   font-size: 80%;
   212:   display: inline;
   213:   color: #101080;
   214: }
   215: DIV.EXPECTED_OUTPUT_SECTION_HEAD STRONG, DIV.EXPECTED_OUTPUT_SECTION_FOOT STRONG {
   216:   font-family : monospace, courier;
   217:   font-weight: normal;
   218:   font-size: 100%;
   219:   display: inline;
   220:   color : black;
   221: }
   222: DIV.EXPECTED_OUTPUT_SECTION_HEAD EM, DIV.EXPECTED_OUTPUT_SECTION_FOOT EM {
   223:   display: inline;
   224: }
   225: 
   226: DIV.DIFF {
   227:   color : #101080;
   228:   background-color: #E0FFE0;
   229:   margin-top: 0.0 ex;
   230:   padding-top: 0.4 ex;
   231:   padding-bottom: 0.2 ex;
   232:   margin-bottom: 0.0 ex;
   233:   margin-right: 2 ex;
   234:   border: thin solid gray;
   235:   display: block;
   236: }
   237: 
   238: DIV.DIFF_SECTION_HEAD {
   239:   margin-top: 1 ex;
   240:   padding-top: 0.2 ex;
   241:   border-top: 0 ex;
   242:   padding-bottom: 0.0 ex;
   243:   border-bottom: 0 ex;
   244:   margin-bottom: -1 ex;
   245:   padding-left: 0.1 em;
   246:   display: block;
   247: }
   248: 
   249: DIV.DIFF_SECTION_FOOT {
   250:   margin-top: -1.0 ex;
   251:   padding-top: 0.0 em;
   252:   border-top-width: 0.0 em;
   253:   border-bottom-width: 0.0 em;
   254:   padding-bottom: 0.0 em;
   255:   margin-bottom: 0.0 em;
   256:   display: block;
   257:   display:none;
   258: }
   259: 
   260: DIV.DIFF_SECTION_HEAD SMALL, DIV.DIFF_SECTION_FOOT SMALL{
   261:   color : #503020;
   262:   font-size: 80%;
   263:   display: inline;
   264:   color: #101080;
   265: }
   266: DIV.DIFF_SECTION_HEAD STRONG, DIV.DIFF_SECTION_FOOT STRONG {
   267:   font-family : monospace, courier;
   268:   font-weight: normal;
   269:   font-size: 100%;
   270:   display: inline;
   271:   color : black;
   272: }
   273: DIV.DIFF_SECTION_HEAD EM, DIV.DIFF_SECTION_FOOT EM {
   274:   display: inline;
   275: }
   276: 


End data section to interscript/doc/interscript.css[1]

Start data section to interscript/doc/user.css[1]

     1: // dummy: to be replaced by the user

6.4.6. Web Weaver
-----------------

This weaver creates a _single_ document consisting of a
heirarchy of HTML pages , using hyperlinks from parent
pages to child pages. The control weaver maintains a
stack of weavers, delegating most commands to the
weaver on the top of the stack.

However, when a heading command at a specified level is
trapped, a new weaver is created and pushed onto the
stack. When a heading at that same level is trapped
again, the weaver on the top of the stack is popped,
and a new weaver pushed in its place.

In this way, all headings at a certain level in a
document will create a new html document.

A stacking weaver requires a list of montonic
increasing integers to determine when to spawn a child
page. When a child is spawned by a level n heading,
that heading becomes the title of the spawned page, and
a heading of level n+m is sent to the child as a level
m heading.

On the other hand, it has to create separate sinks for
each new weaver, except the top level (main page). This
is currently done by supplying a prefix for the child
page filenames, the stacking weaver appends a number
representing the child number, in order the children
were spawned.

Implementation notes. We maintain a document stack.
Each document is triggered by the corresponding heading
level from the supplied trigger list (which had better
be strictly montonic increasing).

When we get a heading level n, we pop off weavers until
n is greater than the trigger level of that weaver.
(The test is performed first).

Then, we push documents onto the stack, until n is
greater than or equal to the document trigger level.
(The test is performed first).

Then, if n is the trigger level of the document, we use
it as a title and hyperlink the document into it's
parent, otherwise we just typeset the heading.

Note that the first loop will destroy a document whose
trigger level is equal to the heading level, and the
second loop will create a new document at this level.

6.4.6.1. Bugs
-------------

Summary: This weaver is not as flexible as it should
be.

Child document filenames are simply the given prefix
plus a sequentially generated count. The child weaver
is always the system html weaver.

Headings in child pages are adjusted to start at level
1. The 'print level' offset is not inherited from the
parent or controllable in any way.

Heading numbers in child documents never reflect the
position in the whole document, only the child page;
that is, the first heading in a child spawned by 2.3.4
will be 1, not 2.3.4.1.

Each page will have it's own table of contents. There's
no overall table for the document. (This will require
links to pages other than 'self').

There are no 'next, previous, home' type links
anywhere, these are necessary.

The spawning weaver assumes HTML. There's no 'in
principle' reason it shouldn't be possible to generate
other documents this way. Although, for example,
'hyperlinks' in printed documents as would be produced
by LaTeX will be of the form, 'see page 99 of such and
such', they're hyperlinks never the less, even if they
have to be followed by the 'manual' browser. :-)

In light of these comments, one can say the current
system confuses production of multi-page web documents,
and production of distinct separate documents (which
happen to be hyperlinked).

6.4.6.2. Code
-------------


Start python section to interscript/weavers/web.py[1]

     1: #line 87 "web_weaver.ipk"


End python section to interscript/weavers/web.py[1]

6.4.6.2.1. reference processor
------------------------------


Start python section to interscript/weavers/web.py[2]

   189: #line 276 "web_weaver.ipk"


End python section to interscript/weavers/web.py[2]

6.4.6.2.2. section processor
----------------------------

Process code sections.

Start python section to interscript/weavers/web.py[3]

   217: #line 306 "web_weaver.ipk"


End python section to interscript/weavers/web.py[3]

6.4.6.2.3. heading processor
----------------------------

The title of the main document is level 0. The top
level content of that document is level 1. A heading
inserted into the document is level 1. After a level 1
heading is inserted, the immediately following content
is level 2.

The hcount list is a list of current heading numbers at
each level, including 0. The current document level is
the length of this list, minus 1. The last entry in the
list is always 0, except briefly when a heading is
being generated at that level.

Start python section to interscript/weavers/web.py[4]

   256: #line 355 "web_weaver.ipk"


End python section to interscript/weavers/web.py[4]

6.4.6.2.4. document control
---------------------------

The break level of a page is the level of it title, so
the top level content of that page is 1 more than its
break level.

Start python section to interscript/weavers/web.py[5]

   275: #line 377 "web_weaver.ipk"


End python section to interscript/weavers/web.py[5]

6.4.6.2.5. Table of contents
----------------------------


6.4.6.2.5.1. MSIE DOM ECMAScript
--------------------------------

ECMAscript for Microsoft Internet Explorer Docuement
Object Model.

Start python section to interscript/weavers/web.py[6]

   408: #line 513 "web_weaver.ipk"


End python section to interscript/weavers/web.py[6]

6.4.6.2.5.2. Table of contents generator
----------------------------------------

Make the table of contents.

Start python section to interscript/weavers/web.py[7]

   450: #line 557 "web_weaver.ipk"


End python section to interscript/weavers/web.py[7]

6.4.6.2.6. Identifier Cross references
--------------------------------------

This is a bit hacky. When a tangler calls
'set_fc_anchor' with a file name and line number, an
anchor with the name

  original_file_name + ':' + str(original_lineno)

is generated. Tanglers which call this function supply
the original source file name and line number (not the
file and line being written to).

Tanglers generate an entry for the identifier occurence
which contains both the original source file and line
number, and the output code file and line number.

The cross reference table the html weaver generates
displays the output code file and line number (not the
orginal file and line number), but it generates a
reference of the form

  '#' + original_file_name + ':' + str(original_lineno)

using the original file name and line number.

The stacking weaver must trap calls to
'identifier_reference' because the hyperlinks the child
weaver charged with generating this table would fail to
refer to the specific html file containing the anchor.
Instead, the stacking weaver must build the table.

To do that, the stacking weaver must know in which html
child document the reference was generated so it can
generate a hyperlink of the form:

  childname + '#' + original_file_name + ':' + str(original_lineno)

Unfortunately, the ids dictionary in the global frame
currently being used to hold identifier cross
references on a document wide basis, does not have this
information available. It wouldn't make sense either,
the information in that table is generated by tanglers,
and has nothing to do with the weaver.

Therefore, the stacking weaver must _also_ intercept
calls to 'set_fc_anchor', and create an entry in a
dictionary keyed by the original filename and original
line number, which tells in which html file the anchor
was set: the stacking weaver knows that!

Start python section to interscript/weavers/web.py[8]

   504: #line 658 "web_weaver.ipk"


End python section to interscript/weavers/web.py[8]

6.4.6.3. Test Code
------------------


Start python section to interscript/tests/test_stacking_weaver.pak[1]

     1: #line 815 "web_weaver.ipk"


End python section to interscript/tests/test_stacking_weaver.pak[1]

Click
interscript/tests/output/test_stacking_weaver.html for
result. See diagnostics from the run below.

Start output section of "/usr/local/bin/python" interscript/bin/iscr.py --test interscript/tests/test_stacking_weaver.pak

     1: Interscript test mode, loading interscript from current directory
     2: Interscript version 1.0a8 build 1384
     3: Built by root on ruby at Mon Nov 30, 1998 at 08:56 AM (UTC)
     4: Generated by 1.0a8 buildno 1383 host ruby
     5: at Mon Nov 30, 1998 at 08:56 AM (UTC)
     6: 
     7: ---------------------------------
     8: Interscript 1.0a8[1384] Process Mon 30 Nov, 1998 20:44:56 (EST)
     9: Processing test_stacking_weaver.pak Pass 0
    10: Creating file status file: swt_filestatus.html
    11: Of 11 files, only 0 were stable on pass 0
    12: There were 11 new files,
    13: and 0 unstable files.
    14: Elapsed Process Time 0 seconds
    15: ================================
    16: 

End output section of "/usr/local/bin/python" interscript/bin/iscr.py --test interscript/tests/test_stacking_weaver.pak

6.4.7. LaTeX Weaver
-------------------

This weaver generates LaTeX2e codes. You'll require a
full LaTeX2e installation to typeset it, along with
common standard packages such as makeidx.

Start python section to interscript/weavers/latex.py[1]

     1: #line 8 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[1]

6.4.7.1. Initialisation
-----------------------


Start python section to interscript/weavers/latex.py[2]

    21: #line 29 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[2]

6.4.7.2. Termination
--------------------


Start python section to interscript/weavers/latex.py[3]

    39: #line 48 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[3]

6.4.7.3. Identifier Cross Reference Table
-----------------------------------------


Start python section to interscript/weavers/latex.py[4]

    43: #line 53 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[4]

6.4.7.4. Class Reference Table
------------------------------


Start python section to interscript/weavers/latex.py[5]

    65: #line 76 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[5]

6.4.7.5. Table of Contents
--------------------------


Start python section to interscript/weavers/latex.py[6]

    91: #line 103 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[6]

6.4.7.6. Code File List
-----------------------


Start python section to interscript/weavers/latex.py[7]

    95: #line 108 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[7]

6.4.7.7. Source List
--------------------


Start python section to interscript/weavers/latex.py[8]

   101: #line 115 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[8]

6.4.7.8. Latex Preamble
-----------------------

This is a very complex section that provides the latex
preamble and document front matter. The basic command
is like: @w = latex_weaver(sink,documentclass='book',
documentclass_options=['a4paper','11pt'],topmargin='4pt').
[Typeset this as a display .. when the code is done for
that feature]

Start python section to interscript/weavers/latex.py[9]

   107: #line 127 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[9]

6.4.7.9. Body Output and Mode Control
-------------------------------------


Start python section to interscript/weavers/latex.py[10]

   197: #line 218 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[10]

6.4.7.10. Headings
------------------


Start python section to interscript/weavers/latex.py[11]

   258: #line 280 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[11]

6.4.7.11. Tables
----------------


Start python section to interscript/weavers/latex.py[12]

   281: #line 304 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[12]

6.4.7.12. Lists
---------------


6.4.7.12.1. Numbered Lists
--------------------------


Start python section to interscript/weavers/latex.py[13]

   303: #line 328 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[13]

6.4.7.12.2. Bullet Lists
------------------------


Start python section to interscript/weavers/latex.py[14]

   316: #line 342 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[14]

6.4.7.12.3. Keyed Lists
-----------------------


Start python section to interscript/weavers/latex.py[15]

   329: #line 356 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[15]

6.4.7.13. Code Output
---------------------


Start python section to interscript/weavers/latex.py[16]

   344: #line 372 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[16]

6.4.7.14. Paragraphs
--------------------


Start python section to interscript/weavers/latex.py[17]

   352: #line 381 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[17]

6.4.7.15. Fonts
---------------


Start python section to interscript/weavers/latex.py[18]

   365: #line 395 "latex_weaver.ipk"


End python section to interscript/weavers/latex.py[18]

6.4.7.16. Citations
-------------------


Start python section to interscript/weavers/latex.py[19]

   408: #line 439 "latex_weaver.ipk"
    75: #line 158 "weavers.ipk"


End python section to interscript/weavers/multiplexor.py[1]

6.4.8. Unimplemented Weavers
----------------------------

In the following sections, I describe some weavers that
are to be considered standard. Some ought to be
implemented but haven't been yet, including Postscript,
Nroff for man pages, and some Microsoft formats. There
are some typesetting systems I don't think need
support, however. I think texinfo is archaic, as is
troff.

6.4.8.1. microsoft word weaver
------------------------------

isn't implemented yet. produces word files for some
popular version of word. help please!

6.4.8.2. microsoft help weaver
------------------------------

isn't implemented yet. produces micorsoft help files.

6.4.8.3. nroff weaver
---------------------

this unimplemented weaver is useful for producing man
pages on unix systems. man pages are considered
essential base documentation by some people.

6.4.8.4. lout weaver
--------------------

this weaver isn't implemented yet.

6.4.8.5. postscript weaver
--------------------------

this weaver isn't implemented yet. it should output
encapsulated postscript in some sensible format. it
shouldn't be too hard to write, but the results will
probably not be as pleasing as postscript produced from
a proper typesetting system.

6.4.9. auto weaver
------------------

make a weaver loom from the list of descriptors.

Start python section to interscript/weavers/auto.py[1]

     1: #line 193 "weavers.ipk"

6.5. Weaver Filters
-------------------


6.5.1. Markup filtering weaver
------------------------------

This weaver is a multiplexor with the additional
facility to perform search for markup codes in input
lines, and execute appropriate weaver constructions.

Note that only one regexp can be matched: chain filters
together to parse multiple markups shortcuts. Note that
the regexp must not be nullable, that is, it must never
match an empty string.

Note that the start and end method must be weaver
method names, not actual methods.

Start python section to interscript/weavers/filter.py[1]

     1: #line 17 "weaver_filters.ipk"


End python section to interscript/weavers/filter.py[1]

6.5.2. Convenience commands
---------------------------

These functions are conveniences for creating filter
chains. The function chain_filters chains a sequence of
filters together, the first filter is called first, and
calls the second filter, etc.

the function create filters takes a sequence of triples
as an argument, and returns a sequence of filters: each
triple should consist of a regexp, start method name,
and end method name. The filters are not connected.

The function create_filter_chain combines these two
functions, it takes a sequence of triples and
constructs a filter chain. Such a connected chain is
also known as a pipe.

Start python section to interscript/weavers/filter.py[2]

    69: #line 99 "weaver_filters.ipk"

6.6. Tanglers
-------------

A tangler is a driver that accepts lines of code in a
particular language or class of languages, writes it to
a code file, possibly pretty printing it, and displays
it in a weaver fixed at construction time.

Tanglers may also build tables of information about the
code.

The main method of a tangler is writeline, which
requires three arguments: the line of code to be
tangled, and the name of the original file from which
the line came, and the line number within the original
file from which the line came.

If the target language supports original source
references (like c's #line directive), the tangler
should also insert a directive into the code file using
the supplied arguments, unless the file name is the
same as the previous line's original source file, or
the line number is not one more than the previous
line's original line number.

The tangler should attach the original filename and
line number to it's sink device, since more than one
tangler may write to the same sink.

A tangler may supply a write_comment method accepting a
single argument, some text to be comments. The tangler
may assume the text does not contain any end of line
characters. It may write the text into the code file as
comments, not necessarily on one line; however, it may
not write anything which affects the semantics of the
program being generated. The comment should be woven as
ordinary program text.

A tangler may supply a method get_comment_tangler. If
it does, then, when called with no arguments, it shall
return a tangler which can be used to write a single
block of comments to the same sink and weaver as
itself.

A tangler may supply a method get_string_tangler. If it
does, it shall accept two or more arguments, the first
being an end of line string, and the second a width. A
string tangler so returned shall produce a string in
the target language, whose text in the target language
shall have the same natural language interpretation as
the text fed to the string tangler via calls to its
writeline method.

The string tangler may strip trailing, leading, or
interior whitespace from the data supplied via the
writeline method, and it may insert the supplied end of
line string in appropriate places. It may also
translate the encoding of the input to another
encoding, or translate the natural language input to
another natural language.

The string tangler may not insert end of line markers
other than the supplied marker. (The target operating
system for execution of the code is not known to the
string tangler. Also, the caller may supply a blank, to
inhibit multi-line output, or may supply a null string,
to force concatenation of non-blank text.

The width parameter is a hint to the string tangler
indicating approximately how long lines should be if
the string tangler is flowing, justifying, or otherwise
formatting the text as lines of a paragraph.

Note: the purpose of string tanglers is simply to allow
the programmer to write natural language text in
Interscript source files without having to worry about
escape characters, quoting, encodings, or line endings.
Text so provided may be intended to be output for user
help, as a message, or, as part of a run-time
documentation system.

Although Python currently supports only strings of 8
bit characters, a Japanese programmer might construct a
c string tangler that translated some multi-byte
encoding of Japanese characters into a c 'wide' string
literal with a 16 bit encoding.

Start python section to interscript/tanglers/__init__.py[1]

     1: #line 84 "tanglers.ipk"


End python section to interscript/tanglers/__init__.py[1]

6.6.1. Tangler Base class
-------------------------


Start python section to interscript/tanglers/base.py[1]

     1: #line 89 "tanglers.ipk"


End python section to interscript/tanglers/base.py[1]

6.6.2. Null Tangler
-------------------

A null tangler eats up input without writing it to
either a sink file or the documentation file. Note that
the start and end section markers are still written
(showing something was skipped over).

Start python section to interscript/tanglers/null.py[1]

     1: #line 144 "tanglers.ipk"


End python section to interscript/tanglers/null.py[1]

6.6.3. Document Tangler
-----------------------

A document tangler writes to the documentation file
without actually writing any source file: it can be
used to present examples or code snippets, for example.

Start python section to interscript/tanglers/doc.py[1]

     1: #line 170 "tanglers.ipk"


End python section to interscript/tanglers/doc.py[1]

6.6.4. Data Tangler
-------------------

The data tangler simply write output to the attached
sink file and weaver, it does not parse the input in
any way or emit any line numbers. There is no
associated comment tangler either.

Start python section to interscript/tanglers/data.py[1]

     1: #line 192 "tanglers.ipk"


End python section to interscript/tanglers/data.py[1]

6.6.5. c Tangler
----------------

The c tangler is used to write c code. It supports
#line directives refering to the original input.

The write_comment method can be used to insert comments
into a code file without them appearing in the
documentation file.

There is an associated comment tangler which writes
text as block of comments to the same sink: the whole
block is enclosed in a single /* */ pair and nicely
formatted. (The c comment tangler cannot interleave
comments).

Design note: The idea of comment tanglers was to get
rid of a 'mode' for tanglers. But the idea seems to
have problems.

There is also an associated string tangler which
generates native strings.

Start python section to interscript/tanglers/c.py[1]

     1: #line 220 "tanglers.ipk"


End python section to interscript/tanglers/c.py[1]

6.6.5.1. c comment tangler
--------------------------


Start python section to interscript/tanglers/c.py[2]

    29: #line 249 "tanglers.ipk"


End python section to interscript/tanglers/c.py[2]

6.6.5.2. c string tangler
-------------------------


Start python section to interscript/tanglers/c.py[3]

    45: #line 266 "tanglers.ipk"


End python section to interscript/tanglers/c.py[3]

6.6.6. c++ Tangler
------------------

The c++ tangler is the same as the c tangler except it
writes c++ style comments and is associated with a
corresponding c++ comment tangler.

Start python section to interscript/tanglers/cpp.py[1]

     1: #line 293 "tanglers.ipk"


End python section to interscript/tanglers/cpp.py[1]

6.6.6.1. Hash Tangler
---------------------


Start python section to interscript/tanglers/cpp.py[2]

    30: #line 323 "tanglers.ipk"


End python section to interscript/tanglers/cpp.py[2]

6.6.6.2. c++ comment tangler
----------------------------


Start python section to interscript/tanglers/cpp.py[3]

    41: #line 335 "tanglers.ipk"


End python section to interscript/tanglers/cpp.py[3]

6.6.7. Java Tangler
-------------------

The Java tangler is the same as the c++ tangler, except
there are no #line directives. This is because I don't
know enough about Java to write a Java tangler. Help!

Start python section to interscript/tanglers/java.py[1]

     1: #line 345 "tanglers.ipk"


End python section to interscript/tanglers/java.py[1]

6.6.8. Tcl Tangler
------------------


Start python section to interscript/tanglers/tcl.py[1]

     1: #line 373 "tanglers.ipk"


End python section to interscript/tanglers/tcl.py[1]

Start python section to interscript/tanglers/tcl.py[2]

     4: #line 376 "tanglers.ipk"

6.6.9. Python Tangler
---------------------

The Python tangler is particularly important for
Interscript, because it is the native script language
in which Interscript is written, in which commands are
issued in source files, and with which it can be
extended by the user.

This very document is itself the Python source for
Interscript, and that leads to interesting
bootstrappping problems in development.

6.6.9.1. Tokeniser interface
----------------------------


Start python section to interscript/tanglers/python.py[1]

     1: #line 14 "python_tangler.ipk"


End python section to interscript/tanglers/python.py[1]

6.6.9.2. Function generator
---------------------------


Start python section to interscript/tanglers/python.py[2]

    36: #line 50 "python_tangler.ipk"


End python section to interscript/tanglers/python.py[2]

6.6.9.3. Code generator
-----------------------


Start python section to interscript/tanglers/python.py[3]

    48: #line 63 "python_tangler.ipk"


End python section to interscript/tanglers/python.py[3]

6.6.9.4. Document generator
---------------------------


Start python section to interscript/tanglers/python.py[4]

   176: #line 192 "python_tangler.ipk"


End python section to interscript/tanglers/python.py[4]

6.6.10. Python Tangler
----------------------


Start python section to interscript/tanglers/python.py[5]

   266: #line 283 "python_tangler.ipk"


End python section to interscript/tanglers/python.py[5]

6.6.11. Python comment tangler
------------------------------


Start python section to interscript/tanglers/python.py[6]

   416: #line 434 "python_tangler.ipk"

6.6.12. Perl Tangler
--------------------

This tangler attempts to support Perl's POD
constructions. It implements the commands pod, cut,
head1, head2, over, back, item, for, begin, and end.

Contrary to the perlpod manpage, Interscript pod
commands are terminated at the end of a line, not the
end of a 'paragraph'. It shouldn't make the slightest
bit of difference, however, since weavers ignore blank
lines anyhow.

When the tangler is in paragraph mode, blank lines are
translated to end of paragraph commands. Paragraph mode
is triggered by any non-command non-blank data lines,
so you won't get an end of paragraph after a command
before the first text.

Currently, support for the for/begin/end pod
constructions is a hack. Interscript uses a different
(better!) mechanism.

Start python section to interscript/tanglers/perl.py[1]

     1: #line 22 "perl_tangler.ipk"

6.7. Tokenisers
---------------

Tokenise various languages.

Start python section to interscript/tokenisers/__init__.py[1]

     1: #line 4 "tokenizers.ipk"

6.7.1. Python Tokeniser
-----------------------

This module was modified from tokenize.py of the
standard library marked

  __version__ = "Ka-Ping Yee, 26 October 1997; patched, GvR 3/30/98'

The module provides tokenisation of python source code.

The module provides a class 'python_tokenize' and a
function 'tokenize'.

The function tokenize is provided for compatibility
with the original tokenize.py. It accepts up to four
arguments. The first argument, readline, is required
and is a callback function which fetches a line for
tokenisation. It should return a line with a trailing
newline character, or an empty string to indicate end
of input. The second argument, tokeneater, is a
callback which is called with each token as an
argument. If omitted, it defaults to a pretty-printing
routine which writes a formatted display of the token
to sys.stdout.

The class constructor and function accept three
optional arguments.

squashop 
     The argument squashop defaults to 0 for the class
    and 1 for the function. If set, all special tokens
    are reported as token OP.

report_comments 
     The argument report_comments defaults to 0 for the
    class and 1 for the function. If set, comments are
    reported as COMMENTS, and blanks lines and
    mid-statement end of lines are reported as NL.

split_multiline_strings 
     If set, multiline strings are reported using one
    token per line. The default is 0 because partial
    strings aren't really tokens.

If the optional arguments are zero, the result is a
'pure' token stream suitable for parsing, if they're
all set the result is more suitable for pretty
printing.

The class provides the following methods. The method
reset() resets the tokenizer state. The method write
accepts arbitrary text data. The method writeline shall
be called with a single line including a trailing
newline character, or with an empty string, indicating
end of input. The method get_tokens is called to fetch
tokens which have been produced and clears the token
queue. The method close signals end of input and
returns any trailing tokens. The method tokenize
accepts any text data and returns the tokens from the
queue. Tokens which span lines are report after the
line in which they are terminated is processed.

The format of a token consists of an integer token
index corresponding to python tokens as listed in the
file token.py, the lexeme which the token represents,
the starting and ending positions of the lexeme as
(line, column) pairs, and the source containing the
lexeme. Lines are numbered from 1.

Start python section to interscript/tokenisers/python.py[1]

     1: #line 57 "python_tokeniser.ipk"


End python section to interscript/tokenisers/python.py[1]

6.7.1.1. Callback Interface
---------------------------

This function does callbacks from a supplier to get
data, and sends tokens to a consumer callback. State is
preserved on the system stack.

Start python section to interscript/tokenisers/python.py[2]

   120: #line 180 "python_tokeniser.ipk"


End python section to interscript/tokenisers/python.py[2]

6.7.1.2. Server Interface
-------------------------

This interface is driven by the client. State is
preserved in a class instance. Tokens are buffered
until the client fetches them.

Start python section to interscript/tokenisers/python.py[3]

   137: #line 200 "python_tokeniser.ipk"


End python section to interscript/tokenisers/python.py[3]

6.7.1.2.1. Main Routine
-----------------------

This is the main function to supply a line of data to
the tokeniser. It was adapted from the original
function by control inversion: when it runs out of
data, instead of fetching more, it just returns.

Start python section to interscript/tokenisers/python.py[4]

   183: #line 251 "python_tokeniser.ipk"
     4: #line 8 "tokenizers.ipk"

6.7.2. Parsers
--------------

While a tangler is used to 'pretty print' code in some
language, a parser, or 'meta-tangler' is used to
actually execute it. A meta-tangler has a writeline
method just like any other tangler object, however
instead of writing output to a code file and weaver, it
translates the input and executes it.

Start python section to interscript/parsers/__init__.py[1]

     1: #line 12 "parsers.ipk"

6.7.2.1. HTML Parser
--------------------

We provide an HTML meta tangler. The sgml_wrapper class
maps writeline calls from the control algorithm to feed
calls of on the parser.

Construction of an html_filter object feeds the parser
with an initial <HTML> tag. Termination semantics are
as follows: if the external data source becomes
exhausted, processing of buffered data should be forced
by calling the close method of the sgml_wrapper or
html_filter object.

If a </HTML> ending tag is detected, an eoi exception
is thrown.

In either case, reset() me be called to reinitialise
the object state, or the object can be destroyed. Note
that the current implementation dispatches tags to
global methods, rather than to a weaver bound to the
object. This is to permit bindings to Interscript
operations other than weaving.

Embedded Python and Tcl (if supported) can be executed
in the Interscript environment using the <SCRIPT> tag
as follows:

  <SCRIPT LANGUAGE="Python"><!--
    print "Hello World"
  #-->
  </SCRIPT>
  <SCRIPT LANGUAGE="Tcl"><!--
    puts "Hello World"
  #-->

Note that the used of comments is _not_ optional. If an
error is detected during execution, a diagnostic will
be printed but will not terminate continued processing
of the document beyond the ending SCRIPT tag.

Start python section to interscript/parsers/html.py[1]

     1: #line 39 "html_parser.ipk"


End python section to interscript/parsers/html.py[1]

6.8. Html parser test
---------------------

    This is an Html test for bold and italicsand
    STRONG, _emphasised_ and code.Sizes too: big and
    small.We can also do lists: an ordered list

 1.  an ordered list item

 2.  another ordered list item

    and an unordered list

*    an unordered list item

*    another unordered list item

    and a description list:

dl key 1 
     description 1

dl key 2 
     description 2

Here's a code display, using PRE tag:

A code example.With PRE tags.

Now for some script.print "Python script"weave("Some
Python script made this.")

6.9. LALR(1) Parser table generator
-----------------------------------


6.9.1. Credits
--------------

This code is heavily adapted from the original by

  mailto:scott@chronis.icgroup.com
__version__ = "$Id: Grammar.py,v 0.2 1997/12/13 03:02:13 scott Exp scott $"

Scott has all the credit for implementing the very
complex LALR parser table generator. The algorithms are
from the Dragon Book, 'Compilers: Principles,
Techniques and Tools' by Aho Sethi and Ullman, Addison
Wesley. Mine has ISBN 0-201-10194-7, but is an old
(1987) edition. Page numbers herein refer to that.

6.9.1.1. Dependencies
---------------------

This module requires the Maxtal 'sets' module which
contains the class 'set', used to represent a set. See
'sets.pak'.

Start python section to interscript/parsers/lalr1.py[1]

     1: #line 21 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[1]

6.9.1.2. Reserved Symbols
-------------------------

The following symbols are reserved by the system. EPS
is a nonterminal which derives nothing, commonly called
epsilon. It can be used in the client grammar, but must
not be the LHS of a production.

EOF is a terminal, marking the end of the input stream.
It must not be used in the client grammar. Some parsers
may prohibit the inclusion of this symbol in the input
stream, others may require it, and others may require
an infinite stream of them at the end of the input.

In addition, the empty string and -1 are reserved as
symbols by the system.

Note: this is ugly. We should use a class, but the
repr() of a class is ugly.

Start python section to interscript/parsers/lalr1.py[2]

     7: #line 42 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[2]

6.9.1.3. Grammar Productions
----------------------------

A production of a grammar is any object with two
attributes, 'LHS' and 'RHS' where LHS is a nonterminal
of the grammar, and the RHS is a sequence of grammar
symbols, possibly empty.

There is no restriction on what kind of sequence is
used, nor on what kind of objects the grammar symbols
are, except that the string "<EPS>" may not be the LHS
symbol of a production, and "<EOF>" and None may not be
used at all.

It is recommended that interned strings or integers be
used as grammar symbols. Strings make debugging easy
because they can be read. Integers are commonly
produced by other generating software.

Note that a production may have other attributes. In
particular, the attribute 'func' may be used by parsers
as a function to be invoked when deriving a
nonterminal, thus implementing syntax directed parsing
for S-attributed grammars.

Note: There may be a prohibition against using -1 as a
grammar symbol. This should be fixed, it should be
permitted.

Start python section to interscript/parsers/lalr1.py[3]

    11: #line 67 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[3]

6.9.1.4. Grammar
----------------

A grammar object is initialised by a nonempty sequence
of productions, and a start symbol. The start symbol is
required and must equal the LHS of at least one
production.

An optional argument with keyword 'verbosity' defaults
to 0, and may be used to control the amount of
debugging information output during the initialisation
process.

The sequence of productions is copied, but the
productions objects are not.

The constructor builds some tables of auxilliary
information about the grammar. After construction the
following attributes are defined:

symbols 
     The set of symbols used in the client grammar,
    excluding "<EPS>". These are called the client
    symbols.

nonterms 
     The set of symbols found on the LHS of some
    production.

terms 
     The set of client symbols of the grammar excluding
    those found on the LHS of a production.

epslhs 
     The subset of client nonterminals which directly
    derive either nothing or "<EPS>".

lhsdereps 
     The subset of client nonterminals which derive
    nothing. This is a (possibly improper) superset of
    'epslhs'.

lhsprods 
     A dictionary of keyed by client nonterminals,
    returning the set of productions of the
    nonterminal.

productions 
     A list of productions in the same order as the
    sequence passed to the constructor.

start 
     The nonterminal which the client denoted as the
    start symbol of the grammar.

firstmap 
     A dictionary keyed by grammar symbols, defining
    the FIRST set of that symbol. The dictionary
    includes entried for "<EOF>", "<EPS>", and a
    special entry for -1 (which is used internally as
    an error state).

    The elements of the FIRST sets will be either
    client terminals or "<EOF>".

    The FIRST set of a symbol is defined as follows:
    for a terminal, the singleton set containing just
    that terminal, for a nonterminal, the set of
    terminals which could be the first symbol of a
    derivation of the nonterminal, and including
    "<EPS>" if the nonterminal can derive nothing. See
    the Dragon book [4.4 p188].

followmap 
     A dictionary keyed by client nonterminals defining
    the FOLLOW set of that nonterminal.

    The FOLLOW set is defined as the set of terminals
    which can follow the nonterminal in some sentential
    form of the grammar, and including "<EOF>" if the
    nonterminal can appear at the right of a sentential
    form. See the Dragon book [4.4 p189].

Start python section to interscript/parsers/lalr1.py[4]

    28: #line 154 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[4]

6.9.1.4.1. Constructor
----------------------


Start python section to interscript/parsers/lalr1.py[5]

    32: #line 159 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[5]

6.9.1.4.2. Find Nullable NonTerminals
-------------------------------------

Non-optimal!

Start python section to interscript/parsers/lalr1.py[6]

    84: #line 213 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[6]

6.9.1.4.3. Find FIRST sets
--------------------------

Calculcate FIRST(sym) for each symbol in the grammar,
including epsilon, EOF, and DummyLA (whatever that is).

Start python section to interscript/parsers/lalr1.py[7]

   108: #line 240 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[7]

6.9.1.4.4. Augment grammar
--------------------------

This function adds a production S' -> S to the grammar
where S was the start symbol.

Start python section to interscript/parsers/lalr1.py[8]

   157: #line 292 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[8]

6.9.1.4.5. Find FOLLOW sets
---------------------------

Calculcate FOLLOW(sym) for each symbol in the grammar,
including epsilon,

Start python section to interscript/parsers/lalr1.py[9]

   167: #line 304 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[9]

6.9.1.4.6. LALR Closure
-----------------------

Compute the LALR Closure. An 'item' is logically a pair
consisting of a production and an indictor of a
position in the RHS. It is represented here as a pair
of integers: an index into the sequence of productions,
and an integer between 0 and the length of the RHS of
the production, inclusive.

Start python section to interscript/parsers/lalr1.py[10]

   217: #line 360 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[10]

6.9.2. LALR Parser Table generator
----------------------------------

This is the efficient LALR parser generator described
in the Dragon book.

Start python section to interscript/parsers/lalr1.py[11]

   256: #line 401 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[11]

6.9.3. mkntfirstmap
-------------------

Computes all nonterms A, first of (strings n) such that
some nonterminal B derives [A, n] in zero or more steps
of (rightmost) derivation. used to help make epsilon
productions quickly calculable. (B may == A)

This is to help mak epsilon productions work with
kernel items and to compute goto transitions from
kernel.

Start python section to interscript/parsers/lalr1.py[12]

   269: #line 422 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[12]

6.9.3.1. newmkntfirstmap
------------------------

computes all nonterms A, first of (strings n) such that
some nonterminal B derives [A, n] in zero or more steps
of (rightmost) derivation. used to help make epsilon
productions quickly calculable. (B may == A)

Start python section to interscript/parsers/lalr1.py[13]

   316: #line 473 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[13]

6.9.3.2. mktfirstmap
--------------------

This is to help make shifts work with only kernel
items.

For each nonterminal C, compute the set of all
terminals a, such that C derives ax in zero or more
steps of (rightmost) derivation where the last
derivation is not an epsilon (empty) production.

assumes .mkfirstntmap() has been run and has already
produced self.ntfirstmap

Start python section to interscript/parsers/lalr1.py[14]

   352: #line 517 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[14]

6.9.3.3. goto
-------------


Start python section to interscript/parsers/lalr1.py[15]

   374: #line 540 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[15]

6.9.3.4. lookaheads
-------------------


Start python section to interscript/parsers/lalr1.py[16]

   392: #line 559 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[16]

6.9.3.5. kernelsoflalr1items
----------------------------


Start python section to interscript/parsers/lalr1.py[17]

   419: #line 587 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[17]

6.9.3.6. Calculate LALR1items
-----------------------------


Start python section to interscript/parsers/lalr1.py[18]

   441: #line 610 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[18]

6.9.3.7. Calculate Action Table
-------------------------------


Start python section to interscript/parsers/lalr1.py[19]

   516: #line 686 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[19]

6.9.3.8. Calculate goto table
-----------------------------


Start python section to interscript/parsers/lalr1.py[20]

   623: #line 794 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[20]

6.9.3.9. Parser
---------------

As described in the Dragon book, 4.7 Fig 4.30. p219,
but with the addition of action functions on reduce
corresponding to evaluation of S-attributes.

We hope to upgrade this to L-attributes later.

Start python section to interscript/parsers/lalr1.py[21]

   637: #line 814 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[21]

6.9.4. LR parser engine
-----------------------

The 'parse' function is a temporary test parser to
checkout the tables produced by LR parser generators.

data 
     A sequence of pairs. The first item in each pair
    should usually be a terminal symbol of the grammar,
    and the second item some value or other data
    associated with it.

gram 
     A sequence of productions.

tab  A pair (action, goto) containing LR parsing tables
    as follows: each table consists of a sequence of
    rows, one row for each non-error state of the
    parser. Each row is a single dictionary, keyed by a
    grammar symbol.

    For the action table, the keys are terminals of the
    grammar, and the values are pairs consisting of an
    action indicator and an integer. The action
    indicator must be an empty string for an error, the
    letter 'r' for a reduce operation, the letter 's'
    for a shift operation, or the letter 'a' for an
    accept operation.

    For a reduce, the integer is the index of the
    production in the grammar to be reduced, plus one.
    (The plus one is an artefact of the use of an
    augmented grammar in the LALR1 parser generator.
    This needs to be fixed.)

    For a shift, the integer is a state to be shifted.

    For the goto table, the keys are nonterminals of
    the grammar, the value is simply an integer
    denoting the state to be shifted.

start 
     The initial state of the parser. Defaults to 0.

reduce 
     The reduction functor to use.

    This is the name of the attribute of a production
    which will be used to locate a function to be
    executed when the production is reduced, and
    defaults to 'func'. This facility is useful to
    allow a single grammar to have multiple action
    categories. For example, a grammar for expressions
    could build a parse tree, emit the input in reverse
    polish order, evaluate the expression, or generate
    code to evaluate the expression.

6.9.5. Test function
--------------------

This is a very simple test. Much more is needed.

Start python section to interscript/parsers/lalr1.py[22]

   692: #line 921 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[22]

6.9.6. Bootstrapping
--------------------

The function below is used to bootstrap the parser. We
use the standard Python tokeniser, and contruct a
grammar by hand to recognize grammars. The reduction
functor is used to build a list of productions, the
functions named in the grammar are attached to these
productions. The grammar we will use is:

  G -> Plist
  Plist -> Plist newline P
  Plist -> P
  P -> LHS = RHS { func }
  RHS -> RHS sym
  RHS -> sym

We will immediately test the generated parser to parse
a string representing a grammar, namely, the same
grammar, which should generate an equivalent parser.

  G = Plist { build_grammar }
  Plist = Plist newline P { add_production_to_list }
  Plist = P { new_production_list }
  P = LHS = RHS "{" func "}" { build_production }
  RHS = RHS sym { add_sym_to_list }
  RHS = sym { new_RHS_list}

6.9.6.1. Test Tokeniser
-----------------------


Start python section to interscript/parsers/lalr1.py[23]

   776: #line 1032 "lalr1_parser.ipk"


End python section to interscript/parsers/lalr1.py[23]

6.9.6.2. Bootstrapping test
---------------------------


Start python section to interscript/parsers/lalr1.py[24]

   829: #line 1086 "lalr1_parser.ipk"

Start python section to interscript/tests/tgram.py[1]

     1: #line 1205 "lalr1_parser.ipk"

Start output section of "/usr/local/bin/python" interscript/tests/tgram.py

     1: Interscript version 1.0a8 build 1378
     2: Built by root on ruby at Sun Nov 29, 1998 at 08:48 PM (UTC)
     3: Generated by 1.0a8 buildno 1376 host ruby
     4: at Sun Nov 29, 1998 at 08:48 PM (UTC)
     5: actions
     6: 0 : {'id': ('s', 11), '(': ('s', 9)}
     7: 1 : {'+': ('s', 3), '<EOF>': ('a', None)}
     8: 2 : {'+': ('s', 3), ')': ('s', 10)}
     9: 3 : {'id': ('s', 11), '(': ('s', 9)}
    10: 4 : {'<EOF>': ('r', 1), '*': ('s', 6), '+': ('r', 1), ')': ('r', 1)}
    11: 5 : {'<EOF>': ('r', 2), '*': ('s', 6), '+': ('r', 2), ')': ('r', 2)}
    12: 6 : {'id': ('s', 11), '(': ('s', 9)}
    13: 7 : {'<EOF>': ('r', 3), '*': ('r', 3), '+': ('r', 3), ')': ('r', 3)}
    14: 8 : {'<EOF>': ('r', 4), '*': ('r', 4), '+': ('r', 4), ')': ('r', 4)}
    15: 9 : {'id': ('s', 11), '(': ('s', 9)}
    16: 10 : {'<EOF>': ('r', 5), '*': ('r', 5), '+': ('r', 5), ')': ('r', 5)}
    17: 11 : {'<EOF>': ('r', 6), '*': ('r', 6), '+': ('r', 6), ')': ('r', 6)}
    18: gotos
    19: 0 : {'F': 8, 'T': 5, 'E': 1}
    20: 1 : {}
    21: 2 : {}
    22: 3 : {'F': 8, 'T': 4}
    23: 4 : {}
    24: 5 : {}
    25: 6 : {'F': 7}
    26: 7 : {}
    27: 8 : {}
    28: 9 : {'F': 8, 'T': 5, 'E': 2}
    29: 10 : {}
    30: 11 : {}
    31: calling fdummy with args (('id', 3),)
    32: calling fdummy with args (('F', 3),)
    33: calling fdummy with args (('T', 3),)
    34: calling fdummy with args (('id', 4),)
    35: calling fdummy with args (('F', 4),)
    36: calling fdummy with args (('id', 2),)
    37: multiplying 4 with 2
    38: calling fdummy with args (('T', 8),)
    39: handling parens, returning whats in between
    40: calling fdummy with args (('F', 8),)
    41: adding 3 with 8
    42: handling parens, returning whats in between
    43: calling fdummy with args (('F', 11),)
    44: calling fdummy with args (('id', 2),)
    45: multiplying 11 with 2
    46: calling fdummy with args (('id', 5),)
    47: multiplying 22 with 5
    48: calling fdummy with args (('T', 110),)
    49: Accept
    50: RESULT= ('E', 110)
    51: bootgram= [('G', ['Plist'], <function just_copy at 8176c58>), ('Plist', ['Plist', 'P'], <function add_production_to_list at 8172860>), ('Plist', ['P'], <function new_production_list at 817ad10>), ('P', ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'], <function build_production at 8186350>), ('RHS', ['RHS', 'SYM'], <function add_symbol_to_list at 8172500>), ('RHS', ['SYM'], <function new_symbol_list at 8172820>), ('SYM', ['NAME'], <function name_as_symbol at 80ba370>), ('SYM', ['STRING'], <function string_as_symbol at 80ba848>)]
    52: actions
    53: 0 : {'NAME': ('s', 5)}
    54: 1 : {'<EOF>': ('a', None)}
    55: 2 : {'<EOF>': ('r', 1), 'NAME': ('s', 5)}
    56: 3 : {'<EOF>': ('r', 2), 'NAME': ('r', 2)}
    57: 4 : {'<EOF>': ('r', 3), 'NAME': ('r', 3)}
    58: 5 : {'=': ('s', 6)}
    59: 6 : {'NAME': ('s', 14), 'STRING': ('s', 15)}
    60: 7 : {'{': ('s', 8), 'NAME': ('s', 14), 'STRING': ('s', 15)}
    61: 8 : {'NAME': ('s', 9)}
    62: 9 : {'}': ('s', 10)}
    63: 10 : {'NEWLINE': ('s', 11)}
    64: 11 : {'<EOF>': ('r', 4), 'NAME': ('r', 4)}
    65: 12 : {'{': ('r', 5), 'NAME': ('r', 5), 'STRING': ('r', 5)}
    66: 13 : {'{': ('r', 6), 'NAME': ('r', 6), 'STRING': ('r', 6)}
    67: 14 : {'{': ('r', 7), 'NAME': ('r', 7), 'STRING': ('r', 7)}
    68: 15 : {'{': ('r', 8), 'NAME': ('r', 8), 'STRING': ('r', 8)}
    69: gotos
    70: 0 : {'G': 1, 'Plist': 2, 'P': 4}
    71: 1 : {}
    72: 2 : {'P': 3}
    73: 3 : {}
    74: 4 : {}
    75: 5 : {}
    76: 6 : {'RHS': 7, 'SYM': 13}
    77: 7 : {'SYM': 12}
    78: 8 : {}
    79: 9 : {}
    80: 10 : {}
    81: 11 : {}
    82: 12 : {}
    83: 13 : {}
    84: 14 : {}
    85: 15 : {}
    86: 
    87: ---------------------------------------------------
    88:   TABLES FOR MANUALLY CONSTRUCTED GRAMMAR GENERATED
    89: ---------------------------------------------------
    90: 
    91: tokens= [('NAME', 'G'), ('=', '='), ('NAME', 'Plist'), ('{', '{'), ('NAME', 'just_copy'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'Plist'), ('=', '='), ('NAME', 'Plist'), ('NAME', 'P'), ('{', '{'), ('NAME', 'add_production_to_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'Plist'), ('=', '='), ('NAME', 'P'), ('{', '{'), ('NAME', 'new_production_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'P'), ('=', '='), ('NAME', 'NAME'), ('STRING', '='), ('NAME', 'RHS'), ('STRING', '{'), ('NAME', 'NAME'), ('STRING', '}'), ('NAME', 'NEWLINE'), ('{', '{'), ('NAME', 'build_production'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'RHS'), ('=', '='), ('NAME', 'RHS'), ('NAME', 'SYM'), ('{', '{'), ('NAME', 'add_symbol_to_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'RHS'), ('=', '='), ('NAME', 'SYM'), ('{', '{'), ('NAME', 'new_symbol_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'SYM'), ('=', '='), ('NAME', 'NAME'), ('{', '{'), ('NAME', 'name_as_symbol'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'SYM'), ('=', '='), ('NAME', 'STRING'), ('{', '{'), ('NAME', 'string_as_symbol'), ('}', '}'), ('NEWLINE', '\012')]
    92: G = Plist { just_copy }
    93: Plist = Plist P { add_production_to_list }
    94: Plist = P { new_production_list }
    95: P = NAME = RHS { NAME } NEWLINE { build_production }
    96: RHS = RHS SYM { add_symbol_to_list }
    97: RHS = SYM { new_symbol_list }
    98: SYM = NAME { name_as_symbol }
    99: SYM = STRING { string_as_symbol }
   100: 
   101: Accept
   102: RESULT= ['G' -> ['Plist'] {'func': 'just_copy'}, 'Plist' -> ['Plist', 'P'] {'func': 'add_production_to_list'}, 'Plist' -> ['P'] {'func': 'new_production_list'}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': 'build_production'}, 'RHS' -> ['RHS', 'SYM'] {'func': 'add_symbol_to_list'}, 'RHS' -> ['SYM'] {'func': 'new_symbol_list'}, 'SYM' -> ['NAME'] {'func': 'name_as_symbol'}, 'SYM' -> ['STRING'] {'func': 'string_as_symbol'}]
   103: BOUND= ['G' -> ['Plist'] {'func': <function just_copy at 8176c58>}, 'Plist' -> ['Plist', 'P'] {'func': <function add_production_to_list at 8172860>}, 'Plist' -> ['P'] {'func': <function new_production_list at 817ad10>}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': <function build_production at 8186350>}, 'RHS' -> ['RHS', 'SYM'] {'func': <function add_symbol_to_list at 8172500>}, 'RHS' -> ['SYM'] {'func': <function new_symbol_list at 8172820>}, 'SYM' -> ['NAME'] {'func': <function name_as_symbol at 80ba370>}, 'SYM' -> ['STRING'] {'func': <function string_as_symbol at 80ba848>}]
   104: 0 : 'G' -> ['Plist'] {'func': <function just_copy at 8176c58>}
   105: 1 : 'Plist' -> ['Plist', 'P'] {'func': <function add_production_to_list at 8172860>}
   106: 2 : 'Plist' -> ['P'] {'func': <function new_production_list at 817ad10>}
   107: 3 : 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': <function build_production at 8186350>}
   108: 4 : 'RHS' -> ['RHS', 'SYM'] {'func': <function add_symbol_to_list at 8172500>}
   109: 5 : 'RHS' -> ['SYM'] {'func': <function new_symbol_list at 8172820>}
   110: 6 : 'SYM' -> ['NAME'] {'func': <function name_as_symbol at 80ba370>}
   111: 7 : 'SYM' -> ['STRING'] {'func': <function string_as_symbol at 80ba848>}
   112: Symbols set('SYM', 'Plist', '=', 'NEWLINE', 'RHS', '}', 'STRING', 'NAME', 'P', '{')
   113: Terminals set('STRING', '}', 'NEWLINE', 'NAME', '=', '{')
   114: NonTerminals set('G', 'Plist', 'SYM', 'RHS', 'P')
   115: Directly Derive epsilon set()
   116: Derive epsilon set()
   117: First Sets:
   118: -1 -> set(-1)
   119: Plist -> set('NAME')
   120: } -> set('}')
   121: { -> set('{')
   122: P -> set('NAME')
   123: <EPS> -> set('<EPS>')
   124: NAME -> set('NAME')
   125: = -> set('=')
   126: SYM -> set('STRING', 'NAME')
   127: <EOF> -> set('<EOF>')
   128: NEWLINE -> set('NEWLINE')
   129: RHS -> set('STRING', 'NAME')
   130: G -> set('NAME')
   131: STRING -> set('STRING')
   132: Follow Sets:
   133: G -> set('<EOF>')
   134: Plist -> set('<EOF>', 'NAME')
   135: SYM -> set('{', 'STRING', 'NAME')
   136: RHS -> set('{', 'STRING', 'NAME')
   137: P -> set('<EOF>', 'NAME')
   138: found 4 more kernels
   139: found 2 more kernels
   140: found 4 more kernels
   141: found 2 more kernels
   142: found 1 more kernels
   143: found 1 more kernels
   144: found 1 more kernels
   145: initLALR1items, kernels done, calculating propagations and spontaneous lookaheads
   146: . . . . . . . . . . . . . . . . done init LALR1items
   147: done with lalr1items, reorganizing the data
   148: actions
   149: 0 : {'NAME': ('s', 5)}
   150: 1 : {'<EOF>': ('a', None)}
   151: 2 : {'<EOF>': ('r', 1), 'NAME': ('s', 5)}
   152: 3 : {'<EOF>': ('r', 2), 'NAME': ('r', 2)}
   153: 4 : {'<EOF>': ('r', 3), 'NAME': ('r', 3)}
   154: 5 : {'=': ('s', 6)}
   155: 6 : {'NAME': ('s', 14), 'STRING': ('s', 15)}
   156: 7 : {'{': ('s', 8), 'NAME': ('s', 14), 'STRING': ('s', 15)}
   157: 8 : {'NAME': ('s', 9)}
   158: 9 : {'}': ('s', 10)}
   159: 10 : {'NEWLINE': ('s', 11)}
   160: 11 : {'<EOF>': ('r', 4), 'NAME': ('r', 4)}
   161: 12 : {'{': ('r', 5), 'NAME': ('r', 5), 'STRING': ('r', 5)}
   162: 13 : {'{': ('r', 6), 'NAME': ('r', 6), 'STRING': ('r', 6)}
   163: 14 : {'{': ('r', 7), 'NAME': ('r', 7), 'STRING': ('r', 7)}
   164: 15 : {'{': ('r', 8), 'NAME': ('r', 8), 'STRING': ('r', 8)}
   165: gotos
   166: 0 : {'G': 1, 'Plist': 2, 'P': 4}
   167: 1 : {}
   168: 2 : {'P': 3}
   169: 3 : {}
   170: 4 : {}
   171: 5 : {}
   172: 6 : {'RHS': 7, 'SYM': 13}
   173: 7 : {'SYM': 12}
   174: 8 : {}
   175: 9 : {}
   176: 10 : {}
   177: 11 : {}
   178: 12 : {}
   179: 13 : {}
   180: 14 : {}
   181: 15 : {}
   182: 
   183: --------------------------------------------------------
   184:   TABLES FOR AUTOMATICALLY CONSTRUCTED GRAMMAR GENERATED
   185: --------------------------------------------------------
   186: 
   187: Accept
   188: RESULT= ['G' -> ['Plist'] {'func': 'just_copy'}, 'Plist' -> ['Plist', 'P'] {'func': 'add_production_to_list'}, 'Plist' -> ['P'] {'func': 'new_production_list'}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': 'build_production'}, 'RHS' -> ['RHS', 'SYM'] {'func': 'add_symbol_to_list'}, 'RHS' -> ['SYM'] {'func': 'new_symbol_list'}, 'SYM' -> ['NAME'] {'func': 'name_as_symbol'}, 'SYM' -> ['STRING'] {'func': 'string_as_symbol'}]
   189: --------------------------------------------------------
   190:  IF THAT WORKED, WE HAVE A BOOTSTRAP
   191: --------------------------------------------------------
   192: Accept
   193: RESULT= ['G' -> ['S'] {'func': 'dummy'}, 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': 'dummy'}, 'S' -> ['if', 'E', 'then', 'S'] {'func': 'dummy'}, 'S' -> ['X'] {'func': 'dummy'}, 'E' -> ['X'] {'func': 'dummy'}]
   194: BOUND= ['G' -> ['S'] {'func': <function dummy at 80f3d60>}, 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 80f3d60>}, 'S' -> ['if', 'E', 'then', 'S'] {'func': <function dummy at 80f3d60>}, 'S' -> ['X'] {'func': <function dummy at 80f3d60>}, 'E' -> ['X'] {'func': <function dummy at 80f3d60>}]
   195: 0 : 'G' -> ['S'] {'func': <function dummy at 80f3d60>}
   196: 1 : 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 80f3d60>}
   197: 2 : 'S' -> ['if', 'E', 'then', 'S'] {'func': <function dummy at 80f3d60>}
   198: 3 : 'S' -> ['X'] {'func': <function dummy at 80f3d60>}
   199: 4 : 'E' -> ['X'] {'func': <function dummy at 80f3d60>}
   200: Symbols set('if', 'X', 'E', 'else', 'S', 'then')
   201: Terminals set('if', 'then', 'X', 'else')
   202: NonTerminals set('S', 'G', 'E')
   203: Directly Derive epsilon set()
   204: Derive epsilon set()
   205: First Sets:
   206: if -> set('if')
   207: -1 -> set(-1)
   208: <EOF> -> set('<EOF>')
   209: X -> set('X')
   210: G -> set('if', 'X')
   211: <EPS> -> set('<EPS>')
   212: E -> set('X')
   213: else -> set('else')
   214: S -> set('if', 'X')
   215: then -> set('then')
   216: Follow Sets:
   217: G -> set('<EOF>')
   218: E -> set('then')
   219: S -> set('else', '<EOF>')
   220: found 4 more kernels
   221: found 2 more kernels
   222: found 1 more kernels
   223: found 1 more kernels
   224: found 1 more kernels
   225: found 1 more kernels
   226: initLALR1items, kernels done, calculating propagations and spontaneous lookaheads
   227: . . . . . . . . . . . done init LALR1items
   228: done with lalr1items, reorganizing the data
   229: Shift/Reduce Conflict, Use Shift [6,'else']: ('r', 3) -> ('s', 7)
   230: actions
   231: 0 : {'if': ('s', 3), 'X': ('s', 9)}
   232: 1 : {'<EOF>': ('a', None)}
   233: 2 : {'<EOF>': ('r', 1)}
   234: 3 : {'X': ('s', 10)}
   235: 4 : {'then': ('s', 5)}
   236: 5 : {'if': ('s', 3), 'X': ('s', 9)}
   237: 6 : {'else': ('s', 7), '<EOF>': ('r', 3)}
   238: 7 : {'if': ('s', 3), 'X': ('s', 9)}
   239: 8 : {'else': ('r', 2), '<EOF>': ('r', 2)}
   240: 9 : {'else': ('r', 4), '<EOF>': ('r', 4)}
   241: 10 : {'then': ('r', 5)}
   242: gotos
   243: 0 : {'S': 2, 'G': 1}
   244: 1 : {}
   245: 2 : {}
   246: 3 : {'E': 4}
   247: 4 : {}
   248: 5 : {'S': 6}
   249: 6 : {}
   250: 7 : {'S': 8}
   251: 8 : {}
   252: 9 : {}
   253: 10 : {}
   254: [('if', 'if'), ('X', 'X'), ('then', 'then'), ('X', 'X'), ('else', 'else'), ('X', 'X')]
   255: dummy:reduce 'E' -> ['X'] {'func': <function dummy at 80f3d60>} args= (('X', 'X'),)
   256: dummy:reduce 'S' -> ['X'] {'func': <function dummy at 80f3d60>} args= (('X', 'X'),)
   257: dummy:reduce 'S' -> ['X'] {'func': <function dummy at 80f3d60>} args= (('X', 'X'),)
   258: dummy:reduce 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 80f3d60>} args= (('if', 'if'), ('E', None), ('then', 'then'), ('S', None), ('else', 'else'), ('S', None))
   259: dummy:reduce 'G' -> ['S'] {'func': <function dummy at 80f3d60>} args= (('S', None),)
   260: Accept
   261: RESULT= None

End output section of "/usr/local/bin/python" interscript/tests/tgram.py

     3: #line 17 "parsers.ipk"

6.10. Introduction to categories
--------------------------------

A category is a collection of arrows together with a
associative partial binary operator called composition,
and which has unit (identity) arrows called objects.

6.10.1. Category of functions
-----------------------------

An important example is a collection of functions which
(1) is closed under function composition, and (2)
includes the identity mapping for each set which is a
domain or codomain of some function.

6.10.2. Category of strings
---------------------------

The set of all strings over some character set is a
category, with the single object the empty string (and
the composition operation being string concatenation).

More generally, given the empty string and any finite
set of strings, the set of all combinations of these
strings is said to be the category generated by that
set, which is called the generator set.

6.10.3. Paths on graphs
-----------------------

The set of all paths on a graph is a category, with the
vericies being considered as identity paths, and
composition being path composition. The category is
said to be generated by the graph.

6.10.4. Groups
--------------

All groups are categories, for example integers,
rationals, and reals under addition (or multiplication
if zero is removed), invertible matrices and linear
operators including stretching and rotation in
n-dimensional space.

6.11. Felix Object protocol
---------------------------

Felix is a language for programming with categories. It
is characterized by the exclusive use of function
composition, having no concept of function application:
functions do not have arguments, they're arrows of
categories which are composed with other functions.

6.11.1. Category Protocol
-------------------------

A category "c" shall support the following methods:

c.is_arrow(x) 
     Shall return 1 if the argument is an arrow of the
    category, and 0 otherwise.

c.is_object(x) 
     Shall return 1 if the argument is an object of the
    category, and 0 otherwise.

c.domain(x) 
     Shall return the domain object of arrow x.

c.codomain(x) 
     Shall return the codomain object of arrow x.

c.can_compose(x,y) 
     If x and y are arrows, shall return 1 if x can be
    composed with y, or 0 if they cannot be composed,
    else undefined.

c.compose(x,y) 
     Shall return the composition of x and y if x and y
    are composable.

c(x1, x2, ...) 
     Shall return the composition of the arguments if
    well defined.

The following methods are optional:

c.inverse(x) 
     Shall return the inverse of an arrow x if it has
    one.

c.get_arrows() 
     Shall return a sequence of all the arrows of a
    category. If defined, the representation of the
    category is said to be enumerable.

c.get_objects() 
     Shall return a sequence of all the objects of a
    category. Shall be defined if "get_arrows()" is
    defined.

c.get_nonobjects() 
     Shall return a sequence of all the non-object
    arrows of a category. Shall be defined if
    "get_arrows()" is defined.

Composition is a partial binary operator, and shall be
associative. The objects shall be precisely theidentity
arrows. In particular:

  if c.is_object(p):
    assert c.is_arrow(p)
  if c.is_arrow(x) and c.is_arrow(y) and c.is_arrow(z):
    if c.can_compose(x,y) and c.can_compose(y,z):
      assert c(c(x,y),z) == c(x,c(y,z) # associative
  if c.is_arrow(x):
    assert c.is_object(c.domain(x))    # domain and codomain are objects
    assert c.is_object(c.codomain(x))
    assert c.compose(c.domain(x), x) == x
    assert c.compose(x, c.codomain(x)) == x

6.12. Felix Package
-------------------

Herein, the python prototype.

Start python section to interscript/felix/__init__.py[1]

     1: #line 5 "felix_package.ipk"


End python section to interscript/felix/__init__.py[1]

Start python section to interscript/felix/model/__init__.py[1]

     1: #line 7 "felix_package.ipk"


End python section to interscript/felix/model/__init__.py[1]

6.12.1. Category Base
---------------------


Start python section to interscript/felix/model/basecat.py[1]

     1: #line 12 "felix_package.ipk"

6.12.2. Finite Category model
-----------------------------

We build a finite category in two stages. First we
construct an empty pre-category, and add arrows and
composition rules to it. Then, we check it is indeed a
category. Unlike most of the classes in core felix,
"fincat" does some argument checking.

6.12.2.1. Implementation
------------------------


Start python section to interscript/felix/model/fincat.py[1]

     1: #line 11 "felix_fincat.ipk"


End python section to interscript/felix/model/fincat.py[1]

6.12.2.1.1. Data structure
--------------------------


Start python section to interscript/felix/model/fincat.py[2]

     5: #line 16 "felix_fincat.ipk"


End python section to interscript/felix/model/fincat.py[2]

6.12.2.1.2. Accessors
---------------------


Start python section to interscript/felix/model/fincat.py[3]

    18: #line 30 "felix_fincat.ipk"


End python section to interscript/felix/model/fincat.py[3]

6.12.2.1.3. Predicates
----------------------


Start python section to interscript/felix/model/fincat.py[4]

    71: #line 84 "felix_fincat.ipk"


End python section to interscript/felix/model/fincat.py[4]

6.12.2.1.4. Mutators
--------------------


Start python section to interscript/felix/model/fincat.py[5]

    78: #line 92 "felix_fincat.ipk"

6.12.3. Standard Categories
---------------------------


6.12.3.1. Implementation
------------------------


Start python section to interscript/felix/model/stdcat.py[1]

     1: #line 6 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[1]

6.12.3.1.1. Empty Category
--------------------------


Start python section to interscript/felix/model/stdcat.py[2]

     4: #line 10 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[2]

6.12.3.1.2. Universal Set
-------------------------


Start python section to interscript/felix/model/stdcat.py[3]

     9: #line 16 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[3]

6.12.3.1.3. Monoids
-------------------

A monoid is a category with one object.

Start python section to interscript/felix/model/stdcat.py[4]

    15: #line 24 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[4]

6.12.3.1.4. Category with one arrow
-----------------------------------

class cat_one(cat_monoid): def __init__(self, unit):
cat_monoid.__init__(self, unit) def is_arrow(self,
arrow): return arrow == self.unit

6.12.3.1.5. Integer Addition
----------------------------

The integers under addition are a group; that is, every
arrow is invertible. The additive inverse of an integer
is its negation.

Start python section to interscript/felix/model/stdcat.py[5]

    31: #line 49 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[5]

6.12.3.1.6. Integer Multiplication
----------------------------------

Note that this monoid is a group if and only if the
modulus is a prime.

Start python section to interscript/felix/model/stdcat.py[6]

    52: #line 74 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[6]

6.12.3.1.7. Sequence concatenation
----------------------------------


Start python section to interscript/felix/model/stdcat.py[7]

    69: #line 92 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[7]

6.12.3.1.7.1. Tuples
--------------------


Start python section to interscript/felix/model/stdcat.py[8]

    79: #line 103 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[8]

6.12.3.1.7.2. Lists
-------------------


Start python section to interscript/felix/model/stdcat.py[9]

    86: #line 111 "felix_stdcat.ipk"


End python section to interscript/felix/model/stdcat.py[9]

6.12.3.1.7.3. Strings
---------------------


Start python section to interscript/felix/model/stdcat.py[10]

    93: #line 119 "felix_stdcat.ipk"

6.12.3.2. Test standard categories
----------------------------------


Start python section to interscript/felix/tests/stdcat_test.py[1]

     1: #line 130 "felix_stdcat.ipk"


End python section to interscript/felix/tests/stdcat_test.py[1]

6.12.3.2.1. Test Output
-----------------------


Start output section of "/usr/local/bin/python" interscript/felix/tests/stdcat_test.py

     1: Interscript version 1.0a8 build 1384
     2: Built by root on ruby at Mon Nov 30, 1998 at 08:56 AM (UTC)
     3: Generated by 1.0a8 buildno 1383 host ruby
     4: at Mon Nov 30, 1998 at 08:56 AM (UTC)
     5: Hello world
     6: 28L
     7: 8L
     8: 5040L
     9: 0L
    10: (1, 2, 3, 4, 5, 6, 7)
    11: [1, 2, 3, 4, 5, 6, 7]

End output section of "/usr/local/bin/python" interscript/felix/tests/stdcat_test.py

6.12.4. Categories with simple representation models
----------------------------------------------------


Start python section to interscript/felix/model/stdcat2.py[1]

     1: #line 5 "felix_stdcat2.ipk"


End python section to interscript/felix/model/stdcat2.py[1]

6.12.4.1. Sets
--------------

All the arrows are objects.

Start python section to interscript/felix/model/stdcat2.py[2]

     5: #line 11 "felix_stdcat2.ipk"


End python section to interscript/felix/model/stdcat2.py[2]

6.12.4.2. Category generated by a directed graph
------------------------------------------------

The category generated by a graph is the set of all
paths of the graph plus the set of vertices. Note that
a path can't contain an edge more than once.

A graph is represented by a dictionary: the key is a
vertex, the value is a list of vertices which there is
an edge to. [This model is attributed to Guido van
Rossum]

An arrow is a non-empty sequence of vertices such that
for each adjacent pair a, b, there is an edge from a to
b. (Note that a sequence of one vertex meets this
condition).

Start python section to interscript/felix/model/stdcat2.py[3]

    16: #line 36 "felix_stdcat2.ipk"

6.12.5. Function category
-------------------------

There are many possibly representations. We will use a
monoidal category whose arrows are class instances of
class 'function'. These instances can be called with a
single argument, and composed with the infix addition
operator.

6.12.5.1. Implementation
------------------------


6.12.5.1.1. Composable Function
-------------------------------

First, we need to be able to compose functions. The
function class accepts a list of functions as
arguments, saves the list, and calls the functions from
left to right when required, passing the single allowed
argument to the first function, and the result of that
along the chain. The result returned is a traditional
value, as is the argument.

Start python section to interscript/felix/model/funcat.py[1]

     1: #line 21 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[1]

6.12.5.1.2. Identity
--------------------


Start python section to interscript/felix/model/funcat.py[2]

    31: #line 52 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[2]

6.12.5.1.3. Input Adapter
-------------------------

Wrap any function into a function that accepts a single
object which is a pair. The first element of the pair
is the tuple of positional arguments and the second the
dictionary of keyword arguments. The pair is called the
normalised form of a value.

Start python section to interscript/felix/model/funcat.py[3]

    38: #line 66 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[3]

6.12.5.1.4. Output Adapter
--------------------------

Wrap any function into one that returns a normalised
value.

Start python section to interscript/felix/model/funcat.py[4]

    46: #line 76 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[4]

6.12.5.1.5. Constant Function
-----------------------------

A constant is a function which is a wrapper for a
value. It ignores its argument when called, and returns
the fixed value it was initialised with.

Start python section to interscript/felix/model/funcat.py[5]

    54: #line 89 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[5]

6.12.5.1.6. Function category
-----------------------------

The category of all functions.

Start python section to interscript/felix/model/funcat.py[6]

    60: #line 97 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[6]

6.12.5.1.7. Function Product
----------------------------

A callable object that applies each function in a list
to each argument in a list, returning a tuple of
results.

Start python section to interscript/felix/model/funcat.py[7]

    69: #line 109 "felix_func.ipk"


End python section to interscript/felix/model/funcat.py[7]

6.12.5.1.8. Function Sum
------------------------

A callable object that calls one function in a list to
the argument, depending on the discriminant integer.

Start python section to interscript/felix/model/funcat.py[8]

    80: #line 123 "felix_func.ipk"

6.12.5.2. Test
--------------


Start python section to interscript/felix/tests/func_test.py[1]

     1: #line 135 "felix_func.ipk"

6.12.5.2.1. Test output
-----------------------


Start output section of "/usr/local/bin/python" interscript/felix/tests/func_test.py

     1: Interscript version 1.0a8 build 1384
     2: Built by root on ruby at Mon Nov 30, 1998 at 08:56 AM (UTC)
     3: Generated by 1.0a8 buildno 1383 host ruby
     4: at Mon Nov 30, 1998 at 08:56 AM (UTC)
     5: 0.479425538604
     6: 0.87758256189
     7: 0.25
     8: 0.229848847066
     9: 0.770151152934
    10: 3
    11: (7, 7)
    12: 14
    13: (4, 9)
    14: 1.0
    15: (0, 1) (1, -1)
    16: -1 1
    17: 4 4
    18: 1

End output section of "/usr/local/bin/python" interscript/felix/tests/func_test.py

6.12.6. Constructions
---------------------

Herein some ways to make new categories from old ones.

Start python section to interscript/felix/model/cons.py[1]

     1: #line 6 "felix_cons.ipk"


End python section to interscript/felix/model/cons.py[1]

6.12.6.1. Product
-----------------

The term product is the categorical word for tuple, for
sets it is the cartesian product. We form the product
by delegation. The element arrows are tuples of arrows.
Objects are tuples of objects. Composition is
componentwise.

Start python section to interscript/felix/model/cons.py[2]

     4: #line 16 "felix_cons.ipk"


End python section to interscript/felix/model/cons.py[2]

6.12.6.2. Dual
--------------

The dual of a category is just the same category with
all the arrows reversed.

Start python section to interscript/felix/model/cons.py[3]

    43: #line 58 "felix_cons.ipk"


End python section to interscript/felix/model/cons.py[3]

6.12.6.3. Undiscriminated Union
-------------------------------

The argument categories must have disjoint arrows sets
for this construction to work. Use the discriminated
union if there is an arrow in common.

The categorical sum, or union, of two categories is
just the complete set of all the arrows.

Start python section to interscript/felix/model/cons.py[4]

    66: #line 88 "felix_cons.ipk"


End python section to interscript/felix/model/cons.py[4]

6.12.6.4. Tagged category
-------------------------

The same category, with an arrow a converted to a pair
(tag, a).

Start python section to interscript/felix/model/cons.py[5]

   107: #line 131 "felix_cons.ipk"


End python section to interscript/felix/model/cons.py[5]

6.12.6.5. Discriminated union
-----------------------------

We implement a discriminated union by tagging the
argument categories with unique tags, and forming the
undiscriminated union. It would be better to use the
actual categories as their own tags but that would not
work if the same category was presented twice in the
argument list, which is likely to be the most common
case.

Start python section to interscript/felix/model/cons.py[6]

   130: #line 160 "felix_cons.ipk"
    10: #line 26 "felix_package.ipk"

6.13. Architecture
------------------

This part discusses the architectural design of
Interscript. Here is a simplified picture of which
objects refer to which others. Interscript maintains
several objects that act as _frames_ (or namespaces)
for symbol lookup.

  FRAMES
    process_frame  --> global_frame
    master_frame   --> process_frame
    pass_frame     --> master_frame
    input_frame    --> pass_frame
                   --> weaver_filter
                   --> stack of tangler

    tangler        --> multiplexor
                   --> sink

    weaver_filter  --> multiplexor
    multiplexor    --> weaver
    weaver         --> sink

6.13.1. The universal frame
---------------------------

The most global frame is the _universal frame_, and
contains fixed information such as interscript code,
constant limits of the implementation, and options
which are hardwired into the software. There's no
actual universal frame object, but the Python and
Interscript 'standards' may be considered universals.

6.13.2. Install Frame
---------------------

The _install frame_ is used to control how generated
files are installed.

An _installation point_ is a directory which will
contain a certain class of files. An _installation
mode_ is an indication of the permissions an installed
file should have.

The install frame contains the default places where
development occurs, and where code and documentation
should be installed depending on the designation
ascribed to the file in interscript sources.

Installation is controlled in several ways. The _kind_
of file is identified in the interscript source, this
tells the language used for script, the kind of script,
or whether the final target is an executable of
dynamically loadable binary. (This information is
generally gleaned from the tangler used, plus options
if required).

The _stability_ of the version is also identifed as
experimental, test, alpha, beta, or production in the
interscript source. This information must be supplied
by the vendor.

The _domain of applicability_ is identified in the
interscript source, this tells if the code is
univerally useful, platform specific, should be
tailored per site, or tailored per user-group or user.
This information must be supplied by the vendor.

Start python section to interscript/frames/install.py[1]

     1: #line 62 "frames.ipk"


End python section to interscript/frames/install.py[1]

6.13.3. The user frame
----------------------

The _user frame_ contains data and functions related to
a particular user, such as the users email address. In
addition, Interscript imports the module 'user', which
executes the file 'pythonrc.py' in the user's home
directory under Unix.

Start python section to interscript/frames/user.py[1]

     1: #line 169 "frames.ipk"


End python section to interscript/frames/user.py[1]

6.13.4. The project frame
-------------------------

The _project frame_ contains data and functions related
to a specific project. A project may consist of a
number of users, and it may contain configuration
information such as the style of page headings used,

that relate to all the software of a given project.
There is currently no project frame, because the
architecture of projects has not yet been designed,
however comments from users are solicited.

Start python section to interscript/frames/project.py[1]

     1: #line 182 "frames.ipk"


End python section to interscript/frames/project.py[1]

Start python section to interscript/frames/__init__.py[1]

     1: #line 188 "frames.ipk"

6.13.5. The platform frame
--------------------------

The _platform frame_ contains platform specific data
and functions, such as the name of the operating
system. These include build options of Python itself,
and the platform dependent modules such as 'posix'
representing platform specific operating system
services and access paths.

Note the tmp.tmp file which is used as a bit bucket for
stderr. This works on NT and Unix. It's ugly.

Start python section to interscript/frames/platform.py[1]

     1: #line 13 "platform_frame.ipk"

6.13.6. The site frame
----------------------

The _site frame_ contains data and functions which
configure Interscript to a specific installation. For
example, the availability of various tanglers and
weavers is likely to be site specific. The specific
location of the Python and Interscript systems is part
of the conceptual site frame.

Start python section to interscript/frames/site.py[1]

     1: #line 10 "site_frame.ipk"

6.13.7. The process frame
-------------------------

The _process frame_ contains data configuring a
particular processing run. That data is gleaned from
command line switches, the environment, defaults from
other frames, and statistics gathered during
processing, and also some data configured by user
commands in input files.

User commands may read symbols from this dictionary,
and may modify the objects to which the symbol names
refer, but are protected from rebinding the names to
other objects or values.

The organisation of the process frame is currently
haphazard and may change from version to version.

Make an object to hold global variables that user
objects can hang onto, to ensure these variables live
during the execution of the object destructors.

Start python section to interscript/frames/processf.py[1]

     1: #line 22 "process_frame.ipk"


End python section to interscript/frames/processf.py[1]

6.13.7.1. Execute Python Script
-------------------------------


Start python section to interscript/frames/processf.py[2]

    64: #line 86 "process_frame.ipk"

6.13.8. Master Frame
--------------------

There is a _master frame_ for each master document file
read by Interscript. It contains data for the document
which persists between passes on the source. For
example the master table of contents is part of the
master frame. Information in the master frame is often
collected during pass and set into the frame at the end
of the pass so it can be used by the next pass: the
table of contents is such an example.

Conceptually, if multiple master documents are
processed by interscript, elaboration of each document
is performed in a separate thread. The master frame
contains thread local data, whereas the process frame
is shared by all threads.

At the 'document' level, the failure to generate a
document correctly is at most fatal to that thread of
control because documents are in some senses
independent. However, when multiple documents form a
project, inter-document dependencies may dictate
abortion of the whole processes if one of the master
threads fails.

Start python section to interscript/frames/masterf.py[1]

     1: #line 23 "master_frame.ipk"


End python section to interscript/frames/masterf.py[1]

6.13.8.1. Run the passes
------------------------

At present, we just run the number of passes specified
as the limit. This will change when convergence testing
is implemented.

The pass loop moves the identifier list, input and
output file lists, and table of contents from the pass
frame to the master frame after each pass. This permits
this data to be read by the next pass (while it is also
regenerating the data for the pass after that).

Conceptually, the loop is said to converge when this
data is the same twice in a row. However we do not test
this yet because the state information is not complete.

To be complete, it must determine whether the input and
output files have changed between passes. This will be
done by comparing the file lists, and comparing the
last modification dates on the files between passes.

Finally, output to simple files will be ignored. It is
necessary to also add user controlled hooks into the
convergence tests, because user script can have
arbitrary side effects.

Start python section to interscript/frames/masterf.py[2]

    36: #line 81 "master_frame.ipk"

6.13.9. The pass frame
----------------------

The _pass frame_ contains data gleaned during a single
pass over a master document. The pass frame is
reinitialised for each pass, so that defaults are
re-established.

For each master document thread, multiple passes on a
document are performed sequentially: a pass may depend
on previous passes.

The thread generating a document is said to converge if
an successive passes reach a fixed point; that is, if
all the outputs from one pass are the same as from the
previous pass.

Conceptually, interscript repeats passes indefinitely
until the passes converge; in practice, a limit is
placed on the number of passes for two reasons: the
first is to prevent unstable definitions locking up the
computer, and the second is to allow for the fact that
detection of convergence is, in general, difficult and
not fully implemented.

For example, although comparison of output files is a
useful tool, minor variations between passes, such as
time stamps, may lead to failed comparisons based on
representation rather than semantic content.

Start python section to interscript/frames/passf.py[1]

     1: #line 26 "pass_frame.ipk"


End python section to interscript/frames/passf.py[1]

6.13.9.1. Construct input object
--------------------------------

We construct an disk source driver by taking the first
command line argument as a disk file name. We construct
an input frame, and attach the source driver to it.

Start python section to interscript/frames/passf.py[2]

    46: #line 76 "pass_frame.ipk"

6.13.10. Input Frame
--------------------

An _input frame_ is created for every document and
subdocument. Input frames are stacked, typically by
@include_file() commands. Input frames implement
lexical scoping and help to isolate lexical information
pertaining to a single file from containing files. For
example, the main parser table, weaver, and tangler
stack are part of an input frame.

When the input frame is stacked, its symbol dictionary
is copied to the new input frame so that included files
inherit the names from the including file. Any
rebindings of those name in the new input file will be
lost when the file ends, however.

For example, if the weaver is set to an HTML weaver,
and an included file sets the weaver to a plain text
weaver, when the included files ends, documentation
will be written to the HTML weaver. The plain text
weaver will have been closed when the last reference to
it disappeared.

You should note, however, that while you cannot change
the bindings of name in parent documents, you certainly
can change the attributes of the objects named by them,
and these changes are persistent. For example, if a
child document changes an attribute of weaver inherited
from the parent, the attribute will remain changed even
after the child is exhausted. It's not that this
behaviour is 'deliberate', so much as it's the way
Python works :-)

Start python section to interscript/frames/inputf.py[1]

     1: #line 34 "input_frame.ipk"


End python section to interscript/frames/inputf.py[1]

6.13.10.1. The input frame
--------------------------

Note the horrid hack: the reg_list is a list of pairs,
the second entry is a method which should be bound to
this frame, but the frame isn't built yet. So the
caller must pass an empty list, and populate it after
the frame is constructed.

This has to be fixed.

Start python section to interscript/frames/inputf.py[2]

    57: #line 97 "input_frame.ipk"


End python section to interscript/frames/inputf.py[2]

6.13.10.1.1. Post user methods
------------------------------

This bit is very hacky! We grab bound methods from
input frame and put them in the user dictionary.

Start python section to interscript/frames/inputf.py[3]

    97: #line 141 "input_frame.ipk"


End python section to interscript/frames/inputf.py[3]

6.13.10.1.2. close
------------------

Remove circular references.

Start python section to interscript/frames/inputf.py[4]

   168: #line 214 "input_frame.ipk"


End python section to interscript/frames/inputf.py[4]

6.13.10.1.3. Process file data
------------------------------


Start python section to interscript/frames/inputf.py[5]

   181: #line 228 "input_frame.ipk"


End python section to interscript/frames/inputf.py[5]

6.13.10.2. Commands
-------------------


6.13.10.2.1. Invoke interscript
-------------------------------

This is a blocking call. Beware of side-effects. [Bug
in current implementation].

6.13.10.2.1.1. Implementation
-----------------------------


Start python section to interscript/frames/inputf.py[6]

   222: #line 275 "input_frame.ipk"


End python section to interscript/frames/inputf.py[6]

6.13.10.2.1.2. Test
-------------------

On-the-fly interscript for test 13 follows.   342: @print 'running call test'
   343: @head(1,'Call test')
   344: This is a call test.
Test output at ../tests/output/test_13.html. Logfile at
../tests/output/test_13.log.

6.13.10.2.2. Misc: move these
-----------------------------


Start python section to interscript/frames/inputf.py[7]

   288: #line 349 "input_frame.ipk"


End python section to interscript/frames/inputf.py[7]

6.13.10.2.3. begin/end blocks
-----------------------------

The @begin() command creates a new scope, by pushing a
new input_frame onto the input stack. The @end()
command is used to end a @begin() block. An unbalanced
@end() command can be used to terminate input from a
file before the physical end of file.

Use of begin/end blocks is important to limit the
lifetime of various objects. In particular, if a
tangler is writing code to a named_file_sink, then the
file will not be closed properly until it is destroyed.

Start python section to interscript/frames/inputf.py[8]

   295: #line 367 "input_frame.ipk"


End python section to interscript/frames/inputf.py[8]

6.13.10.2.3.1. Begin/end test
-----------------------------


Start data section to interscript/tests/test_beginend.pak[1]

     1: @head(1,'Begin/End test')
     2: We test the @begin and @end commands with a simple scoping test.
     3: First, a level is set to 1, and we create a class with
     4: a destructor and an object thereof.
     5: @class X:
     6:    def __init__(self, level, weaver):
     7:      self.level = level
     8:      self.current_weaver = weaver
     9:    def __del__(self):
    10:      self.current_weaver.writeline('X object of level '+str(self.level)+' destroyed.')
    11:  level=1
    12:  x = X(level, get_weaver())
    13: @weave('Level is '+str(level)+'. ')
    14: @p()
    15: Now, a new block.
    16: @begin()
    17: Check old value of level.
    18: @weave('Level is '+str(level)+'. ')
    19: Now set level to 2, and create another class object.
    20: @level = 2
    21: @x = X(level,get_weaver())
    22: @weave('Level is '+str(level)+'. ')
    23: @p()
    24: Finally we end the block here. This should kill the inner class object.
    25: @end()
    26: @p()
    27: Back in the old block we test the level.
    28: @weave('Level is '+str(level)+'. ')
    29: That's the simple test. Now, the outer class object should die.


End data section to interscript/tests/test_beginend.pak[1]

We can look at the html results at
../tests/test_beginend.html

6.13.10.2.4. Include file/source
--------------------------------

The include file command is a subroutine call. It
pushes a new input frame associated with the file,
causing input to be read from the file. The new input
frame has a copy of the user dictionary from the
calling frame. When the file ends, the frame is popped
which destroys the dictionary; all newly created
symbols are lost.

The new input frame does not inherit the parent frames
lexicology. In particular, the warning character is
restablished as @. This command reasserts document
mode, both in the included file and the current one.

The command include_source is a generalisation of
include_file which takes any source driver as the
input.

Start python section to interscript/frames/inputf.py[9]

   314: #line 438 "input_frame.ipk"


End python section to interscript/frames/inputf.py[9]

6.13.10.2.5. Include/insert code
--------------------------------

The @insert_code command inserts a code file into the
input stream of the current tangler. It is used to yank
code fragments from external files.

The external file can contain any data, all of which is
interpreted as literal program code: lines beginning
with @, for example, will not be processed as commands.

Note that the file is not written directly to the
tangler's sink driver, it is written to the tangler.
Therefore, if the code contains embedded documentation
constructions in a form the tangler understands,
documentation can still be woven. For example if a Perl
module containing POD constructions is imported, a
perl_tangler will still recognize these constructions
and send appropriate documentation commens to its
associated weaver.

The @include_code command is the same as the
@insert_code command except that the tangler is passed
as an argument, rather than the current tangler being
used.

Start python section to interscript/frames/inputf.py[10]

   337: #line 482 "input_frame.ipk"
On-the-fly interscript for test 14 follows.   536: @head(1,'The main program')
   537: This code includes and external file, chunk.tmp, defining
   538: the body of function f.
   539: You should see the #line directive for it on line 3.
   540: @select(tangler('chunked.tmp','python'))
   541: def f():
   542: @insert_code('chunk.tmp')
   543: def g(): pass
   544: def h():
   545: @sink_verbatim('chunk.tmp')
   546: def k(): pass
   547: @head(1,'The included chunk')
   548: Here's the included chunk, defined after use.
   549: @select(tangler('chunk.tmp','python'))
   550:   pass
   551: @doc()
   552: End example.
   553: @print_source_list(1)
   554: @print_include_list(1)
Test output at ../tests/output/test_14.html. Logfile at
../tests/output/test_14.log.

   391: #line 556 "input_frame.ipk"


End python section to interscript/frames/inputf.py[10]

6.13.10.2.6. Include html
-------------------------

The @html and @include_html commands switch the input
parser from raw interscript to a small subset of HTML.
The parser maps tags to calls to interscript commands.
For example, <B> is translated to call the bold()
command.

Python script can be included in the html using the
tags <SCRIPT LANGUAGE="python"> .. </SCRIPT> At
present, the translator has even less features than
interscript: it's only a stub for a more full scale
translator.

The @html() command is similar, except it takes html
data from the current input source.

In both cases, </HTML> terminates HTML parsing.

Start python section to interscript/frames/inputf.py[11]

   393: #line 573 "input_frame.ipk"


End python section to interscript/frames/inputf.py[11]

6.13.10.2.7. Verbosity
----------------------


Start python section to interscript/frames/inputf.py[12]

   425: #line 606 "input_frame.ipk"


End python section to interscript/frames/inputf.py[12]

6.13.10.2.8. Weaver Control
---------------------------


Start python section to interscript/frames/inputf.py[13]

   431: #line 613 "input_frame.ipk"


End python section to interscript/frames/inputf.py[13]

6.13.10.2.9. Anchors
--------------------


Start python section to interscript/frames/inputf.py[14]

   448: #line 631 "input_frame.ipk"


End python section to interscript/frames/inputf.py[14]

6.13.10.2.10. Parsing
---------------------


Start python section to interscript/frames/inputf.py[15]

   458: #line 642 "input_frame.ipk"


End python section to interscript/frames/inputf.py[15]

6.13.10.2.11. Input functions
-----------------------------

These are private functions used to actually perform
input from the input_frame source driver.

Start python section to interscript/frames/inputf.py[16]

   472: #line 659 "input_frame.ipk"


End python section to interscript/frames/inputf.py[16]

6.13.10.2.12. Untangle
----------------------

This command is used inside an interscript tangler to
put an external file into a .pak archive. It writes out
a 'select' (seeline 8692 ) command, and then the file,
with leading @ characters converted to two @
characters. Use it like this:

  @select(archive)
  @untangle('fred.pak')
  @untangle('joe.pak')

This mechanism is required to support include files,
since a top level document is incomplete otherwise.

Note that the inserted code is not woven! In fact, the
tangler never sees it, it simply supplies the sink
object.

Start python section to interscript/frames/inputf.py[17]

   512: #line 718 "input_frame.ipk"


End python section to interscript/frames/inputf.py[17]

6.13.10.2.13. Parser functions
------------------------------

These function actually execute script.

For example, these methods control the disposition of
the log. This requires switching the sys.stdout and/or
sys.stdlog object.

Similarly, these routines determine the disposition of
keyboard interrupts, which normally terminate the whole
process. It may be necessary to terminate master
documents threads.

Finally, these routines will permits synchronisation of
delayed file updates: in a 'commit/rollback' scenario,
only the process frame has enough information to permit
or deny updates which may depend on all threads
elaborating their client documents correctly.

Note that the verbosity level used here is not the same
as the master or pass frame verbosity: even if these
values are changed by the user document, debugging
script execution is determined by the process level
verbosity setting. This is necessary so user level
document debugging does not interfere with interscript
developer debugging.

Other features controlled at the process level include
the use of threads, the availability of a GUI, etc.

6.13.10.2.13.1. Utilities
-------------------------

Python is announced by a single @. If the line ends in
a : (or some others) the script continues to the first
line with a non-space in column 1. The first space is
remove from each continuation line.

Tcl is announced by a single !. If the line ends in {,
or with a \, then the script is continued as for
Python.

In both cases, the head line can have spaces or a #
initiated comment afterwards and still be detected: for
Tcl, neither is permitted after a slosh and will
generate a Tcl error.

Start python section to interscript/frames/inputf.py[18]

   531: #line 778 "input_frame.ipk"


End python section to interscript/frames/inputf.py[18]

6.13.10.2.14. Collect stuff
---------------------------

This command reads ahead in the input until a line is
found that does not match the supplied continuation
pattern. It then returns all the input lines as a
single string. It can be used to drive user defined
parser objects, which process the returned string.

For example, the built-in python suite processes uses
this function to collect a multiline python suite
before executing it.

Start python section to interscript/frames/inputf.py[19]

   549: #line 805 "input_frame.ipk"


End python section to interscript/frames/inputf.py[19]

6.13.10.2.15. Execute collected python
--------------------------------------


Start python section to interscript/frames/inputf.py[20]

   583: #line 840 "input_frame.ipk"


End python section to interscript/frames/inputf.py[20]

6.13.10.2.15.1. Test python function
------------------------------------

Woven from embedded python script.

6.13.10.2.15.2. Print diff table
--------------------------------

Print out a diff table with side by side comparison of
files. Requires the diff module. No table is generated
if the files compare equal.

Start python section to interscript/frames/inputf.py[21]

   592: #line 860 "input_frame.ipk"


End python section to interscript/frames/inputf.py[21]

6.13.10.2.16. Register Test
---------------------------


Start python section to interscript/frames/inputf.py[22]

   615: #line 884 "input_frame.ipk"


End python section to interscript/frames/inputf.py[22]

6.13.10.2.17. Python Test
-------------------------


Start python section to interscript/frames/inputf.py[23]

   626: #line 896 "input_frame.ipk"


End python section to interscript/frames/inputf.py[23]

6.13.10.2.17.1. Test 15: Python test test
-----------------------------------------

On-the-fly python test script follows.   988: print 'Hello from python test function test.'
   989: print 'Hello again from python test function test.'
Actual output follows.

Start output section of python <<temporary>>

     1: Hello from python test function test.
     2: Hello again from python test function test.

End output section of python <<temporary>>

6.13.10.2.17.2. Test 16: Python diff test
-----------------------------------------

On-the-fly python test script follows.   993: print 'Hello from python diff function test.'
   994: print 'Hello again from python diff function test.'
Actual output follows.

Start output section of python <<temporary>>

     1: Hello from python diff function test.
     2: Hello again from python diff function test.

End output section of python <<temporary>>

On-the-fly expected output follows.

Start expected section of input_frame.ipk

   996: Hello from python diff function test.
   997: Hello error from python diff function test.

End expected section of input_frame.ipk

Differential follows.
+---------------------------------------------------+---------------------------------------------------+
| Actual                                            | Expected                                          |
+---------------------------------------------------+---------------------------------------------------+
|   2:! Hello again from python diff function test. |   2:! Hello error from python diff function test. |
+---------------------------------------------------+---------------------------------------------------+

6.13.10.2.18. Match Processing functions
----------------------------------------

These commands process match objects which the parser
has construted while scanning the parser table trying
to match the input string.

6.13.10.2.18.1. Single Line Python Script
-----------------------------------------


Start python section to interscript/frames/inputf.py[24]

   718: #line 1006 "input_frame.ipk"


End python section to interscript/frames/inputf.py[24]

6.13.10.2.18.2. Multi-Line Python Script
----------------------------------------


Start python section to interscript/frames/inputf.py[25]

   722: #line 1011 "input_frame.ipk"


End python section to interscript/frames/inputf.py[25]

6.13.10.2.18.3. Ordinary Text
-----------------------------


Start python section to interscript/frames/inputf.py[26]

   727: #line 1017 "input_frame.ipk"


End python section to interscript/frames/inputf.py[26]

6.13.10.2.18.4. Doubled Warning character
-----------------------------------------


Start python section to interscript/frames/inputf.py[27]

   731: #line 1022 "input_frame.ipk"


End python section to interscript/frames/inputf.py[27]

6.13.10.2.18.5. Html input
--------------------------


Start python section to interscript/frames/inputf.py[28]

   735: #line 1027 "input_frame.ipk"


End python section to interscript/frames/inputf.py[28]

6.13.10.2.19. The Tangler Stack
-------------------------------

Interscript supports stacking of tanglers. The internal
commands tangler_push, and tangler_pop push and pop the
stack, while tangler_set changes the top of the stack.

These operations call the start and end section methods
of the tangler objects as they become, or cease to
become, the current tangler, this ensures correct
source file references are inserted into the code
files.

Start python section to interscript/frames/inputf.py[29]

   739: #line 1040 "input_frame.ipk"


End python section to interscript/frames/inputf.py[29]

6.13.10.2.20. Tangler Constructors
----------------------------------

These are convenience functions that construct tanglers
and associated sinks. The argument is a filename which
is used to construct a sink to which the tangler is
linked. The tanglers are linked to the current weaver.

Start python section to interscript/frames/inputf.py[30]

   754: #line 1060 "input_frame.ipk"


End python section to interscript/frames/inputf.py[30]

6.13.10.2.21. Managing the tangler stack
----------------------------------------

These commands also push, pop, and set the tangler
stack, but they also generate start and end section
messages in the documentation file. They are intended
for use by the end user.

Pushing and popping the stack is useful, for example,
to temporarily write a C function declaration to the
header file just before it is defined. (In fact,
Interscript has some special case functionality to
support this while avoiding writing the function
signature twice.)

The begin and end comments functions push and pop the
tangler stack, the push comments function pushes the
comment tangler associated withe the current tangler
(if there is a current tangler, otherwise it pushes
None. The comment tangler associated with the current
tangler could also be, and is by default, None.)

Note that the begin and end comments functions invoke
the internal push and pop functions; that is, they do
not write begin and end section messages to the
documentation file. That's because the comments are
actually written to the code file, they're just
formatted by the weaver like documentation rather than
code.

Design note: There is always a current tangler object,
possibly None. Interscript knows it is writing code,
except when the current tangler is None, in which case
it is writing documentation. I'm not so sure this is a
wonderful scheme :-(

The begin and end string functions write the enclosed
lines to the code file formatted as a (single) string.
The resultant output is echoed to the documentation
file.

The begin and end string functions reformat the
enclosed lines as a single string in the target
programming language. In C, for example, special
character such as slosh are replaced by two sloshes.
(Note that a leading @ can be included only by writing
@@).

The begin string function supports two optional
arguments. The eol argument determines what to insert
at the end of a line, it will usually be either an
empty string, a newline character sequence, or a single
space.

Trailing whitespace is removed from each line. It is
not possible to disable this feature, and that is
deliberate. However, a second argument, width, if
positive, will then pad the line to the specified width
with spaces. This feature is designed to support two
dimensional character arrays.

Design note: there is as yet not support for
international character sets.

Start python section to interscript/frames/inputf.py[31]

   795: #line 1151 "input_frame.ipk"
   806: #line 1162 "input_frame.ipk"


End python section to interscript/frames/inputf.py[31]

6.13.10.2.22. Miscellaneous
---------------------------

The usual stuff that is hard to classify.

Start python section to interscript/frames/inputf.py[32]

   888: #line 1246 "input_frame.ipk"


End python section to interscript/frames/inputf.py[32]

6.13.10.2.23. Capture command output
------------------------------------

This is unix dependent at present.

Start python section to interscript/frames/inputf.py[33]

   929: #line 1289 "input_frame.ipk"


End python section to interscript/frames/inputf.py[33]

6.13.10.2.24. Documentation Constructions
-----------------------------------------

The following constructions affect the documentation
file only. They are used to designate headings,
separate paragraphs, print tables, etc.

These functions are members of class input_frame.

6.13.10.2.24.1. Headings
------------------------


Start python section to interscript/frames/inputf.py[34]

   964: #line 1331 "input_frame.ipk"


End python section to interscript/frames/inputf.py[34]

6.13.10.2.24.2. Paragraphs
--------------------------


Start python section to interscript/frames/inputf.py[35]

  1001: #line 1369 "input_frame.ipk"


End python section to interscript/frames/inputf.py[35]

6.13.10.2.24.3. Citations
-------------------------


Start python section to interscript/frames/inputf.py[36]

  1015: #line 1384 "input_frame.ipk"


End python section to interscript/frames/inputf.py[36]

6.13.10.2.24.4. Tables
----------------------


Start python section to interscript/frames/inputf.py[37]

  1019: #line 1389 "input_frame.ipk"


End python section to interscript/frames/inputf.py[37]

6.13.10.2.24.5. Lists
---------------------


Start python section to interscript/frames/inputf.py[38]

  1032: #line 1403 "input_frame.ipk"


End python section to interscript/frames/inputf.py[38]

6.13.10.2.24.6. Fonts
---------------------


Start python section to interscript/frames/inputf.py[39]

  1078: #line 1450 "input_frame.ipk"


End python section to interscript/frames/inputf.py[39]

6.13.10.2.24.7. Displays
------------------------


Start python section to interscript/frames/inputf.py[40]

  1121: #line 1494 "input_frame.ipk"
  1143: #line 1516 "input_frame.ipk"
     3: #line 196 "frames.ipk"

6.14. Module getoptions
-----------------------

Provides generic command line processing. Input: a list
of words, each of which is an option or filename.
Output: a list of pairs, each of which is a list of
option value pairs, and a filename. The format for
options is --longoption, --longoption=value, -abcd,
-abcdef=value.

Start python section to interscript/getoptions.py[1]

     1: #line 10 "options.ipk"

6.14.1. Test 17: generic getoptions module
------------------------------------------

On-the-fly python test script follows.    47: from interscript import getoptions
    48: args = ['--longopt', '--longopt-value=value', '-short', '-shortval=val2','filename']
    49: result = getoptions.getopt(args)
    50: print result
Actual output follows.

Start output section of python <<temporary>>

     1: Interscript version 1.0a8 build 1378
     2: Built by root on ruby at Sun Nov 29, 1998 at 08:48 PM (UTC)
     3: Generated by 1.0a8 buildno 1376 host ruby
     4: at Sun Nov 29, 1998 at 08:48 PM (UTC)
     5: [([('longopt', None), ('longopt-value', 'value'), ('s', None), ('h', None), ('o', None), ('r', None), ('t', None), ('s', None), ('h', None), ('o', None), ('r', None), ('t', None), ('v', None), ('a', None), ('l', 'val2')], 'filename')]

End output section of python <<temporary>>

6.15. Get Options
-----------------

This module gets interscript options from the command
line.

The verbosity option provides coarse generic control
over the amount of information Interscript writes. The
default is 2, which only prints 'error' messages. Level
1 is useful when you're fairly sure there are no
errors, for example, you have a trustworthy readonly
document, but have deleted the output files during a
cleanup. Level 0 is very dangerous, since even
trustworthy documents can unexpectedly cause errors:
perhaps you installed a new version of Python, or
perhaps you forgot that by default downloaded files are
reloaded from the network automatically once a month?

This module is getting too fat. It also has side
effects, making it impossible to use safely within an
interscript source.

Start python section to interscript/getframes.py[1]

     1: #line 20 "interscript_options.ipk"


End python section to interscript/getframes.py[1]

6.15.1. Help Dictionary
-----------------------


Start python section to interscript/getframes.py[2]

    10: #line 30 "interscript_options.ipk"


End python section to interscript/getframes.py[2]

6.15.2. Argument Frame
----------------------

Just an empty class to hook attributes to.

Start python section to interscript/getframes.py[3]

   105: #line 127 "interscript_options.ipk"
     6: #line 448 "iscr.pak"


End python section to interscript/core/__init__.py[1]

6.16. Utility Modules
---------------------


Start python section to interscript/utilities/__init__.py[1]

     1: #line 452 "iscr.pak"

6.16.1. compiler package
------------------------

This package

Start python section to interscript/compilers/c.py[1]

     1: #line 13 "compilers.ipk"


End python section to interscript/compilers/c.py[1]

Start python section to interscript/compilers/cpp.py[1]

     1: #line 97 "compilers.ipk"


End python section to interscript/compilers/cpp.py[1]

Start python section to interscript/compilers/cconfig.py[1]

     1: #line 188 "compilers.ipk"


End python section to interscript/compilers/cconfig.py[1]

Start python section to interscript/compilers/__init__.py[1]

     1: #line 246 "compilers.ipk"


End python section to interscript/compilers/__init__.py[1]

Start python section to interscript/tests/test_compilers.py[1]

     1: #line 248 "compilers.ipk"


End python section to interscript/tests/test_compilers.py[1]

Start C section to interscript/tests/example.c[1]

     1: #line 262 "compilers.ipk"
     2: #include <stdio.h>
     3: void main() {
     4:   printf("Hello World from compiler test\n");
     5: }
     6: 


End C section to interscript/tests/example.c[1]

6.16.1.1. Test
--------------

Test it.

Start output section of C compiler

     1: Hello World from compiler test
     2: Interscript version 1.0a8 build 1384
     3: Built by root on ruby at Mon Nov 30, 1998 at 08:56 AM (UTC)
     4: Generated by 1.0a8 buildno 1383 host ruby
     5: at Mon Nov 30, 1998 at 08:56 AM (UTC)
     6: system 'gcc -g -O2 -fpic -fPIC -pedantic  -I/usr/local/include/python1.5 -c interscript/tests/example.c -o interscript/tests/example.o'
     7: Object file interscript/tests/example.o generated
     8: system 'gcc  -L/usr/local/lib/python1.5 interscript/tests/example.o -o interscript/tests/example.exe'
     9: Executable interscript/tests/example.exe generated
    10: Executable executed

End output section of C compiler

6.16.2. Interscript Standard module diff for Python
---------------------------------------------------

This site depedent module implements diff and patch for
Python.

6.16.2.1. Description
---------------------

The diff module is an optional Interscript component.
however it is required by the version control
subsystem, and may be used by other functions if
present. The module provides several functions.

6.16.2.2. Requirements
----------------------

The function 'compare_files' shall return 1 if two text
files have the same contents, and 0 otherwise. The
comparison may ignore whitespace at the end of lines
but is not required to. The compare files function
shall return 1 if the files are exactly equal, in
particular if the same file is given as both arguments.

The function 'compare_code_files' is the same as
'compare_files' except it may accept keyword arguments
specifying the language of the source code. If such
arguments are supplied the function may perform a
semantic comparison of the code, and return 1 if the
codes are equivalent. The purpose of the function is
primarily to avoid rebuilding files differing only in
comments, blank lines, etc.

The function 'diff_files' compares two text files, the
original file and an upgraded file, and returns a
string describing the differences. The string may be in
any format, but it must be possible for the patch_file
function to modify the original file using this string,
and produce an upgraded file which compares equal using
the compare files function.

The function 'diff_strings' compares two strings,
returning a context diff as a string.

The function 'diff_lines' compares two sequences of
strings, returning a diff descriptor as follows (code,
left, right). Code may be 'c' for change or 's' for
common, left and right are line counts.

The function 'patch_file' takes a string and an
original file, and the name of an upgraded file, and
produces an upgraded file which has been modified by
the patch string in such a way that the compare_files
function will report the files are equal. In addition,
the function shall accept an empty patch string
signifying no differences between the original and
upgraded files.

This function is not required to operate correctly if
the orginal file has been modified since the patch
string was produced by diff_files.

The names of the argument files shall not denote the
same file.

The function shall return a result

The function 'posix_patch' is optional. If this
function is present, it will take a patch string
produced by the standard posix context diff and patch
the file. The purpose of this function is to facilitate
use of posix context diff as an Internet standard
method of transporting source code patches.

The specification of this function is defined as the
result of calling diff with the -C option on a system
compliant with the ISO Posix standard.

6.16.2.3. Posix Implementation
------------------------------

The following implementation utilises the os.popen and
os.system function to dispatch requests to the
operating system diff and patch commands, and is
intended to work on posix compliant systems. Well, it
works on my Linux box :-)

Start python section to interscript/utilities/diff.py[1]

     1: #line 75 "diff.ipk"


End python section to interscript/utilities/diff.py[1]

6.16.2.4. Test routine 1
------------------------

First, create some files to compare!

Start data section to interscript/tests/diff1.dat[1]

     1: A file with several lines of text.
     2: A line to be deleted.
     3: To be used by the diff module test.
     4: This line will be changed.
     5: The diff module is used to compare and patch test files.
     6: 


End data section to interscript/tests/diff1.dat[1]

Start data section to interscript/tests/diff2.dat[1]

     1: A file with several lines of text.
     2: To be used by the diff module test.
     3: This line is changed.
     4: A NEW LINE INSERTED.
     5: The diff module is used to compare and patch test files.
     6: 

Start python section to interscript/tests/test_diff.py[1]

     1: #line 191 "diff.ipk"


End python section to interscript/tests/test_diff.py[1]

     7: Now run it.

Start output section of "/usr/local/bin/python" interscript/tests/test_diff.py
Command returned 256

     1: diff: diff1.dat: No such file or directory
     2: diff: diff1.dat: No such file or directory
     3: diff: diff2.dat: No such file or directory
     4: Interscript version 1.0a8 build 1384
     5: Built by root on ruby at Mon Nov 30, 1998 at 08:56 AM (UTC)
     6: Generated by 1.0a8 buildno 1383 host ruby
     7: at Mon Nov 30, 1998 at 08:56 AM (UTC)
     8: Traceback (innermost last):
     9:   File "interscript/tests/test_diff.py", line 10, in ?
    10:     assert not cmp
    11: AssertionError

End output section of "/usr/local/bin/python" interscript/tests/test_diff.py

     8: 


End data section to interscript/tests/diff2.dat[1]

6.16.2.5. Test Routine 2
------------------------

Use "test_python" function.

6.16.2.5.1. Test 19: Python diff test
-------------------------------------

On-the-fly python test script follows.   230: print 'Hello from python diff function test.'
   231: print 'Hello again from python diff function test.'
   232: print 'Hello third from python diff function test.'
   233: print 'Hello fourth from python diff function test.'
Actual output follows.

Start output section of python <<temporary>>

     1: Hello from python diff function test.
     2: Hello again from python diff function test.
     3: Hello third from python diff function test.
     4: Hello fourth from python diff function test.

End output section of python <<temporary>>

On-the-fly expected output follows.

Start expected section of diff.ipk

   235: Hello from python diff function test.
   236: Hello error from python diff function test.
   237: Hello fourth from python diff function test.
   238: Hello fifth from python diff function test.

End expected section of diff.ipk

Differential follows.
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
| Actual                                                                                              | Expected                                          |
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
|   2:! Hello again from python diff function test.
  3:! Hello third from python diff function test. |   2:! Hello error from python diff function test. |
|                                                                                                     |   4:+ Hello fifth from python diff function test. |
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
Hmmm. There should be a difference table here.

6.16.3. Commands
----------------

This module is a replacement for the python commands
module, which doesn't work properly on NT.

Start python section to interscript/utilities/commands.py[1]

     1: #line 6 "commands.ipk"
     4: #line 459 "iscr.pak"


End python section to interscript/utilities/__init__.py[1]

6.17. Application and tool directory
------------------------------------

This is where the interscript mainline and any other
shell commands go.

Start python section to interscript/bin/__init__.py[1]

     1: #line 464 "iscr.pak"


End python section to interscript/bin/__init__.py[1]

6.17.1. Stand alone unix/nt mainline
------------------------------------

Command line stub.

Start python section to interscript/bin/iscr.py[1]

     2: #line 471 "iscr.pak"


End python section to interscript/bin/iscr.py[1]

6.17.2. Windows launcher
------------------------

A batch file to launch interscript. You will need to
edit the file!

Start python section to interscript/bin/iscr.bat[1]

     1: #line 485 "iscr.pak"

     4: #line 488 "iscr.pak"

     9: #line 493 "iscr.pak"

    12: #line 496 "iscr.pak"

    17: #line 501 "iscr.pak"

    19: #line 503 "iscr.pak"


End python section to interscript/bin/iscr.bat[1]

6.18. Test package
------------------


Start python section to interscript/tests/__init__.py[1]

     1: #line 508 "iscr.pak"


End python section to interscript/tests/__init__.py[1]

7. Appendices
-------------


7.1. File List
--------------

interscript/doc/iscr.txt
interscript/doc/iscrp.html
interscript/doc/iscrs.html
interscript/doc/iscr_indexview.html
interscript/doc/iscr_top.html
interscript/doc/iscr_indexnav.html
interscript/doc/iscr.tex
interscript/doc/iscr_0001.html
interscript/doc/iscr_0002.html
interscript/doc/iscr_0003.html
interscript/doc/iscr_0004.html
interscript/doc/iscr_0005.html
interscript/doc/iscr_0006.html
interscript/doc/iscr_0007.html
interscript/doc/iscr_0008.html
interscript/doc/iscr_0009.html
interscript/doc/iscr_0010.html
interscript/doc/iscr_0011.html
interscript/doc/iscr_0012.html
interscript/doc/iscr_0013.html
interscript/doc/iscr_0014.html
interscript/doc/iscr_0015.html
interscript/doc/iscr_0016.html
interscript/doc/iscr_0017.html
interscript/doc/iscr_0018.html
interscript/doc/iscr_0019.html
interscript/doc/iscr_0020.html
interscript/doc/iscr_0021.html
interscript/doc/iscr_0022.html
interscript/doc/iscr_0023.html
interscript/doc/iscr_0024.html
interscript/doc/iscr_0025.html
interscript/doc/iscr_0026.html
interscript/doc/iscr_0027.html
interscript/doc/iscr_0028.html
interscript/doc/iscr_0029.html
interscript/doc/iscr_0030.html
interscript/doc/iscr_0031.html
interscript/doc/iscr_0032.html
interscript/doc/iscr_0033.html
interscript/doc/iscr_0034.html
interscript/doc/iscr_0035.html
interscript/doc/iscr_0036.html
interscript/doc/iscr_0037.html
interscript/doc/iscr_0038.html
interscript/doc/iscr_0039.html
interscript/doc/iscr_0040.html
interscript/doc/iscr_0041.html
interscript/doc/iscr_0042.html
interscript/doc/iscr_0043.html
interscript/doc/iscr_0044.html
interscript/doc/iscr_0045.html
interscript/doc/iscr_0046.html
interscript/doc/iscr_0047.html
interscript/doc/iscr_0048.html
interscript/doc/iscr_0049.html
interscript/doc/iscr_0050.html
interscript/doc/iscr_0051.html
interscript/doc/iscr_0052.html
interscript/doc/iscr_0053.html
interscript/doc/iscr_0054.html
interscript/doc/iscr_0055.html
interscript/doc/iscr_0056.html
interscript/doc/iscr_0057.html
interscript/doc/iscr_0058.html
interscript/doc/iscr_0059.html
interscript/doc/iscr_0060.html
interscript/doc/iscr_0061.html
interscript/doc/iscr_0062.html
interscript/doc/iscr_0063.html
interscript/doc/iscr_0064.html
interscript/doc/iscr_0065.html
interscript/doc/iscr_0066.html
interscript/doc/iscr_0067.html
interscript/doc/iscr_0068.html
interscript/doc/iscr_0069.html
interscript/doc/iscr_0070.html
interscript/doc/iscr_0071.html
interscript/doc/iscr_0072.html
interscript/doc/iscr_0073.html
interscript/doc/iscr_0074.html
interscript/doc/iscr_0075.html
interscript/doc/iscr_0076.html
interscript/doc/iscr_0077.html
interscript/doc/iscr_0078.html
interscript/doc/iscr_0079.html
interscript/doc/iscr_0080.html
interscript/doc/iscr_0081.html
interscript/doc/iscr_0082.html
interscript/doc/iscr_0083.html
interscript/doc/iscr_0084.html
interscript/doc/iscr_0085.html
interscript/doc/iscr_0086.html
interscript/doc/iscr_0087.html
interscript/doc/iscr_0088.html
interscript/doc/iscr_0089.html
interscript/doc/iscr_0090.html
interscript/doc/iscr_0091.html
interscript/doc/iscr_0092.html
interscript/doc/iscr_0093.html
interscript/doc/iscr_0094.html
interscript/doc/iscr_0095.html
interscript/doc/iscr_0096.html
interscript/doc/iscr_0097.html
interscript/doc/iscr_0098.html
interscript/doc/iscr_0099.html
interscript/doc/iscr_0100.html
interscript/doc/iscr_0101.html
interscript/doc/iscr_0102.html
interscript/doc/iscr_0103.html
interscript/doc/iscr_0104.html
interscript/doc/iscr_0105.html
interscript/doc/iscr_0106.html
interscript/doc/iscr_0107.html
interscript/doc/iscr_0108.html
interscript/doc/iscr_0109.html
interscript/doc/iscr_0110.html
interscript/doc/iscr_0111.html
interscript/doc/iscr_0112.html
interscript/doc/iscr_0113.html
interscript/doc/iscr_0114.html
interscript/doc/iscr_0115.html
interscript/doc/iscr_0116.html
interscript/doc/iscr_0117.html
interscript/tests/output/mymodule2.py
interscript/doc/iscr_0118.html
interscript/tests/output/mymodule3.py
interscript/doc/iscr_0119.html
interscript/doc/iscr_0120.html
interscript/doc/iscr_0121.html
interscript/doc/iscr_0122.html
interscript/doc/iscr_0123.html
interscript/doc/iscr_0124.html
interscript/doc/iscr_0125.html
interscript/doc/iscr_0126.html
interscript/doc/iscr_0127.html
interscript/doc/iscr_0128.html
interscript/doc/iscr_0129.html
interscript/doc/iscr_0130.html
interscript/doc/iscr_0131.html
interscript/doc/iscr_0132.html
interscript/doc/iscr_0133.html
interscript/doc/iscr_0134.html
interscript/doc/iscr_0135.html
interscript/doc/iscr_0136.html
interscript/doc/iscr_0137.html
interscript/doc/iscr_0138.html
interscript/doc/iscr_0139.html
interscript/doc/iscr_0140.html
interscript/doc/iscr_0141.html
interscript/doc/iscr_0142.html
interscript/doc/iscr_0143.html
interscript/doc/iscr_0144.html
interscript/doc/iscr_0145.html
interscript/doc/iscr_0146.html
interscript/doc/iscr_0147.html
interscript/doc/iscr_0148.html
interscript/doc/iscr_0149.html
interscript/doc/iscr_0150.html
interscript/doc/iscr_0151.html
interscript/doc/iscr_0152.html
interscript/doc/iscr_0153.html
interscript/doc/iscr_0154.html
interscript/doc/iscr_0155.html
interscript/doc/iscr_0156.html
interscript/doc/iscr_0157.html
interscript/doc/iscr_0158.html
interscript/doc/iscr_0159.html
interscript/doc/iscr_0160.html
interscript/doc/iscr_0161.html
interscript.README
interscript.pth
interscript/__init__.py
interscript/doc/iscr_0162.html
interscript/doc/iscr_0163.html
interscript/doc/iscr_0164.html
interscript/doc/iscr_0165.html
interscript/core/__init__.py
interscript/doc/iscr_0166.html
interscript/core/sets.py
interscript/doc/iscr_0167.html
interscript/tests/test_sets.py
interscript/doc/iscr_0168.html
interscript/doc/iscr_0169.html
interscript/core/stacks.py
interscript/doc/iscr_0170.html
interscript/doc/iscr_0171.html
interscript/doc/iscr_0172.html
interscript/core/protocols.py
interscript/doc/iscr_0173.html
interscript/tests/test_protocol.py
interscript/doc/iscr_0174.html
interscript/drivers/__init__.py
interscript/doc/iscr_0175.html
interscript/doc/iscr_0176.html
interscript/drivers/sources/__init__.py
interscript/doc/iscr_0177.html
interscript/drivers/sources/base.py
interscript/doc/iscr_0178.html
interscript/doc/iscr_0179.html
interscript/doc/iscr_0180.html
interscript/drivers/sources/disk.py
interscript/doc/iscr_0181.html
interscript/drivers/sources/url.py
interscript/doc/iscr_0182.html
interscript/drivers/sources/ftp.py
interscript/doc/iscr_0183.html
interscript/drivers/sources/http.py
interscript/doc/iscr_0184.html
interscript/drivers/sources/stdin.py
interscript/doc/iscr_0185.html
interscript/drivers/sources/cache.py
interscript/doc/iscr_0186.html
interscript/drivers/sinks/__init__.py
interscript/drivers/sinks/util.py
interscript/doc/iscr_0187.html
interscript/drivers/sinks/base.py
interscript/doc/iscr_0188.html
interscript/drivers/sinks/null.py
interscript/doc/iscr_0189.html
interscript/drivers/sinks/tee.py
interscript/doc/iscr_0190.html
interscript/drivers/sinks/disk.py
interscript/doc/iscr_0191.html
interscript/drivers/sinks/bufdisk.py
interscript/doc/iscr_0192.html
interscript/drivers/sinks/stdout.py
interscript/doc/iscr_0193.html
interscript/drivers/sinks/cache.py
interscript/doc/iscr_0194.html
interscript/drivers/storage/__init__.py
interscript/doc/iscr_0195.html
interscript/drivers/storage/memory.py
interscript/doc/iscr_0196.html
interscript/drivers/storage/disk.py
interscript/doc/iscr_0197.html
interscript/weavers/__init__.py
interscript/doc/iscr_0198.html
interscript/weavers/base.py
interscript/doc/iscr_0199.html
interscript/weavers/raw.py
interscript/doc/iscr_0200.html
interscript/weavers/multiplexor.py
interscript/doc/iscr_0201.html
interscript/weavers/text.py
interscript/doc/iscr_0202.html
interscript/doc/iscr_0203.html
interscript/doc/iscr_0204.html
interscript/doc/iscr_0205.html
interscript/doc/iscr_0206.html
interscript/doc/iscr_0207.html
interscript/doc/iscr_0208.html
interscript/doc/iscr_0209.html
interscript/doc/iscr_0210.html
interscript/doc/iscr_0211.html
interscript/doc/iscr_0212.html
interscript/weavers/html.py
interscript/doc/iscr_0213.html
interscript/doc/iscr_0214.html
interscript/doc/iscr_0215.html
interscript/doc/iscr_0216.html
interscript/doc/iscr_0217.html
interscript/doc/iscr_0218.html
interscript/doc/iscr_0219.html
interscript/doc/iscr_0220.html
interscript/doc/iscr_0221.html
interscript/doc/iscr_0222.html
interscript/doc/iscr_0223.html
interscript/doc/iscr_0224.html
interscript/doc/iscr_0225.html
interscript/doc/iscr_0226.html
interscript/doc/iscr_0227.html
interscript/doc/iscr_0228.html
interscript/doc/iscr_0229.html
interscript/doc/iscr_0230.html
interscript/doc/iscr_0231.html
interscript/doc/iscr_0232.html
interscript/doc/iscr_0233.html
interscript/doc/iscr_0234.html
interscript/doc/iscr_0235.html
interscript/doc/interscript.css
interscript/doc/user.css
interscript/doc/iscr_0236.html
interscript/doc/iscr_0237.html
interscript/doc/iscr_0238.html
interscript/weavers/web.py
interscript/doc/iscr_0239.html
interscript/doc/iscr_0240.html
interscript/doc/iscr_0241.html
interscript/doc/iscr_0242.html
interscript/doc/iscr_0243.html
interscript/doc/iscr_0244.html
interscript/doc/iscr_0245.html
interscript/doc/iscr_0246.html
interscript/doc/iscr_0247.html
interscript/tests/test_stacking_weaver.pak
interscript/doc/iscr_0248.html
interscript/weavers/latex.py
interscript/doc/iscr_0249.html
interscript/doc/iscr_0250.html
interscript/doc/iscr_0251.html
interscript/doc/iscr_0252.html
interscript/doc/iscr_0253.html
interscript/doc/iscr_0254.html
interscript/doc/iscr_0255.html
interscript/doc/iscr_0256.html
interscript/doc/iscr_0257.html
interscript/doc/iscr_0258.html
interscript/doc/iscr_0259.html
interscript/doc/iscr_0260.html
interscript/doc/iscr_0261.html
interscript/doc/iscr_0262.html
interscript/doc/iscr_0263.html
interscript/doc/iscr_0264.html
interscript/doc/iscr_0265.html
interscript/doc/iscr_0266.html
interscript/doc/iscr_0267.html
interscript/doc/iscr_0268.html
interscript/doc/iscr_0269.html
interscript/doc/iscr_0270.html
interscript/doc/iscr_0271.html
interscript/doc/iscr_0272.html
interscript/doc/iscr_0273.html
interscript/doc/iscr_0274.html
interscript/weavers/auto.py
interscript/doc/iscr_0275.html
interscript/doc/iscr_0276.html
interscript/weavers/filter.py
interscript/doc/iscr_0277.html
interscript/doc/iscr_0278.html
interscript/tanglers/__init__.py
interscript/tanglers/base.py
interscript/doc/iscr_0279.html
interscript/doc/iscr_0280.html
interscript/tanglers/null.py
interscript/doc/iscr_0281.html
interscript/tanglers/doc.py
interscript/doc/iscr_0282.html
interscript/tanglers/data.py
interscript/doc/iscr_0283.html
interscript/tanglers/c.py
interscript/doc/iscr_0284.html
interscript/doc/iscr_0285.html
interscript/doc/iscr_0286.html
interscript/tanglers/cpp.py
interscript/doc/iscr_0287.html
interscript/doc/iscr_0288.html
interscript/doc/iscr_0289.html
interscript/tanglers/java.py
interscript/doc/iscr_0290.html
interscript/tanglers/tcl.py
interscript/doc/iscr_0291.html
interscript/tanglers/python.py
interscript/doc/iscr_0292.html
interscript/doc/iscr_0293.html
interscript/doc/iscr_0294.html
interscript/doc/iscr_0295.html
interscript/doc/iscr_0296.html
interscript/doc/iscr_0297.html
interscript/doc/iscr_0298.html
interscript/tanglers/perl.py
interscript/doc/iscr_0299.html
interscript/tokenisers/__init__.py
interscript/doc/iscr_0300.html
interscript/tokenisers/python.py
interscript/doc/iscr_0301.html
interscript/doc/iscr_0302.html
interscript/doc/iscr_0303.html
interscript/doc/iscr_0304.html
interscript/parsers/__init__.py
interscript/doc/iscr_0305.html
interscript/parsers/html.py
interscript/doc/iscr_0306.html
interscript/doc/iscr_0307.html
interscript/doc/iscr_0308.html
interscript/parsers/lalr1.py
interscript/doc/iscr_0309.html
interscript/doc/iscr_0310.html
interscript/doc/iscr_0311.html
interscript/doc/iscr_0312.html
interscript/doc/iscr_0313.html
interscript/doc/iscr_0314.html
interscript/doc/iscr_0315.html
interscript/doc/iscr_0316.html
interscript/doc/iscr_0317.html
interscript/doc/iscr_0318.html
interscript/doc/iscr_0319.html
interscript/doc/iscr_0320.html
interscript/doc/iscr_0321.html
interscript/doc/iscr_0322.html
interscript/doc/iscr_0323.html
interscript/doc/iscr_0324.html
interscript/doc/iscr_0325.html
interscript/doc/iscr_0326.html
interscript/doc/iscr_0327.html
interscript/doc/iscr_0328.html
interscript/doc/iscr_0329.html
interscript/doc/iscr_0330.html
interscript/doc/iscr_0331.html
interscript/doc/iscr_0332.html
interscript/doc/iscr_0333.html
interscript/doc/iscr_0334.html
interscript/tests/tgram.py
interscript/doc/iscr_0335.html
interscript/doc/iscr_0336.html
interscript/doc/iscr_0337.html
interscript/doc/iscr_0338.html
interscript/doc/iscr_0339.html
interscript/doc/iscr_0340.html
interscript/doc/iscr_0341.html
interscript/doc/iscr_0342.html
interscript/felix/__init__.py
interscript/felix/model/__init__.py
interscript/doc/iscr_0343.html
interscript/felix/model/basecat.py
interscript/doc/iscr_0344.html
interscript/felix/model/fincat.py
interscript/doc/iscr_0345.html
interscript/doc/iscr_0346.html
interscript/doc/iscr_0347.html
interscript/doc/iscr_0348.html
interscript/doc/iscr_0349.html
interscript/doc/iscr_0350.html
interscript/doc/iscr_0351.html
interscript/felix/model/stdcat.py
interscript/doc/iscr_0352.html
interscript/doc/iscr_0353.html
interscript/doc/iscr_0354.html
interscript/doc/iscr_0355.html
interscript/doc/iscr_0356.html
interscript/doc/iscr_0357.html
interscript/doc/iscr_0358.html
interscript/doc/iscr_0359.html
interscript/doc/iscr_0360.html
interscript/doc/iscr_0361.html
interscript/doc/iscr_0362.html
interscript/felix/tests/stdcat_test.py
interscript/doc/iscr_0363.html
interscript/doc/iscr_0364.html
interscript/felix/model/stdcat2.py
interscript/doc/iscr_0365.html
interscript/doc/iscr_0366.html
interscript/doc/iscr_0367.html
interscript/doc/iscr_0368.html
interscript/felix/model/funcat.py
interscript/doc/iscr_0369.html
interscript/doc/iscr_0370.html
interscript/doc/iscr_0371.html
interscript/doc/iscr_0372.html
interscript/doc/iscr_0373.html
interscript/doc/iscr_0374.html
interscript/doc/iscr_0375.html
interscript/doc/iscr_0376.html
interscript/doc/iscr_0377.html
interscript/felix/tests/func_test.py
interscript/doc/iscr_0378.html
interscript/doc/iscr_0379.html
interscript/felix/model/cons.py
interscript/doc/iscr_0380.html
interscript/doc/iscr_0381.html
interscript/doc/iscr_0382.html
interscript/doc/iscr_0383.html
interscript/doc/iscr_0384.html
interscript/doc/iscr_0385.html
interscript/doc/iscr_0386.html
interscript/doc/iscr_0387.html
interscript/frames/install.py
interscript/doc/iscr_0388.html
interscript/frames/user.py
interscript/doc/iscr_0389.html
interscript/frames/project.py
interscript/frames/__init__.py
interscript/doc/iscr_0390.html
interscript/frames/platform.py
interscript/doc/iscr_0391.html
interscript/frames/site.py
interscript/doc/iscr_0392.html
interscript/frames/processf.py
interscript/doc/iscr_0393.html
interscript/doc/iscr_0394.html
interscript/frames/masterf.py
interscript/doc/iscr_0395.html
interscript/doc/iscr_0396.html
interscript/frames/passf.py
interscript/doc/iscr_0397.html
interscript/doc/iscr_0398.html
interscript/frames/inputf.py
interscript/doc/iscr_0399.html
interscript/doc/iscr_0400.html
interscript/doc/iscr_0401.html
interscript/doc/iscr_0402.html
interscript/doc/iscr_0403.html
interscript/doc/iscr_0404.html
interscript/doc/iscr_0405.html
interscript/doc/iscr_0406.html
interscript/doc/iscr_0407.html
interscript/doc/iscr_0408.html
interscript/doc/iscr_0409.html
interscript/tests/test_beginend.pak
interscript/doc/iscr_0410.html
interscript/doc/iscr_0411.html
interscript/doc/iscr_0412.html
interscript/doc/iscr_0413.html
interscript/doc/iscr_0414.html
interscript/doc/iscr_0415.html
interscript/doc/iscr_0416.html
interscript/doc/iscr_0417.html
interscript/doc/iscr_0418.html
interscript/doc/iscr_0419.html
interscript/doc/iscr_0420.html
interscript/doc/iscr_0421.html
interscript/doc/iscr_0422.html
interscript/doc/iscr_0423.html
interscript/doc/iscr_0424.html
interscript/doc/iscr_0425.html
interscript/doc/iscr_0426.html
interscript/doc/iscr_0427.html
interscript/doc/iscr_0428.html
interscript/doc/iscr_0429.html
interscript/doc/iscr_0430.html
interscript/doc/iscr_0431.html
interscript/doc/iscr_0432.html
interscript/doc/iscr_0433.html
interscript/doc/iscr_0434.html
interscript/doc/iscr_0435.html
interscript/doc/iscr_0436.html
interscript/doc/iscr_0437.html
interscript/doc/iscr_0438.html
interscript/doc/iscr_0439.html
interscript/doc/iscr_0440.html
interscript/doc/iscr_0441.html
interscript/doc/iscr_0442.html
interscript/doc/iscr_0443.html
interscript/doc/iscr_0444.html
interscript/doc/iscr_0445.html
interscript/doc/iscr_0446.html
interscript/doc/iscr_0447.html
interscript/doc/iscr_0448.html
interscript/getoptions.py
interscript/doc/iscr_0449.html
interscript/doc/iscr_0450.html
interscript/getframes.py
interscript/doc/iscr_0451.html
interscript/doc/iscr_0452.html
interscript/doc/iscr_0453.html
interscript/utilities/__init__.py
interscript/tests/test_compilers.py
interscript/tests/example.c
interscript/compilers/c.py
interscript/compilers/cpp.py
interscript/compilers/cconfig.py
interscript/compilers/__init__.py
interscript/doc/iscr_0454.html
interscript/doc/iscr_0455.html
interscript/doc/iscr_0456.html
interscript/doc/iscr_0457.html
interscript/doc/iscr_0458.html
interscript/doc/iscr_0459.html
interscript/utilities/diff.py
interscript/doc/iscr_0460.html
interscript/tests/diff1.dat
interscript/tests/diff2.dat
interscript/tests/test_diff.py
interscript/doc/iscr_0461.html
interscript/doc/iscr_0462.html
interscript/doc/iscr_0463.html
interscript/utilities/commands.py
interscript/doc/iscr_0464.html
interscript/bin/__init__.py
interscript/doc/iscr_0465.html
interscript/bin/iscr.py
interscript/doc/iscr_0466.html
interscript/bin/iscr.bat
interscript/doc/iscr_0467.html
interscript/tests/__init__.py
interscript/doc/iscr_0468.html
interscript/doc/iscr_0469.html
interscript/doc/iscr_0470.html
interscript/doc/iscr_0471.html
interscript/doc/iscr_0472.html
interscript/doc/iscr_0473.html
interscript/doc/iscr_0474.html
interscript/doc/iscr_0475.html
interscript/doc/iscr_0476.html
interscript/doc/iscr_0477.html
interscript/doc/iscr_0478.html
interscript/doc/iscr_0479.html
interscript/doc/iscr_0480.html
interscript/doc/iscr_0481.html
interscript/doc/iscr_0482.html
interscript/doc/iscr_0483.html
interscript/doc/iscr_0484.html
interscript/doc/iscr_0485.html
interscript/doc/iscr_0486.html
interscript/doc/iscr_0487.html
interscript/doc/iscr_0488.html
interscript/doc/iscr_0489.html
interscript/doc/iscr_0490.html
interscript/doc/iscr_0491.html
interscript/doc/iscr_0492.html
interscript/doc/iscr_0493.html
interscript/doc/iscr_0494.html
interscript/tests/example.pak
interscript/doc/iscr_0495.html
interscript/doc/iscr_0496.html
interscript/doc/iscr_0497.html
interscript/doc/iscr_0498.html
interscript/doc/iscr_0499.html
interscript/doc/iscr_0500.html
interscript/doc/iscr_0501.html
interscript/doc/iscr_0502.html
interscript/doc/iscr_funcref.html
interscript/doc/iscr_classref.html
interscript/doc/iscr_identref.html
interscript/doc/iscr_toc.html
interscript/doc/iscr_testref.html
interscript/doc/iscr_sectionref.html
interscript/doc/iscr_sourceref.html
/mnt/user1/uestl/py/iscr/interscript/doc/iscr_filestatus.html

7.2. Source List
----------------

iscr.pak
introduction.ipk
requirements.ipk
design.ipk
tutorial.pak
faq.ipk
sets.ipk
stacks.ipk
protocols.ipk
drivers.ipk
source_drivers.ipk
sink_drivers.ipk
storage_drivers.ipk
weavers.ipk
text_weaver.ipk
html_weaver.ipk
web_weaver.ipk
latex_weaver.ipk
weaver_filters.ipk
tanglers.ipk
python_tangler.ipk
perl_tangler.ipk
tokenizers.ipk
python_tokeniser.ipk
parsers.ipk
html_parser.ipk
lalr1_parser.ipk
felix.pak
felix_protocol.ipk
felix_package.ipk
felix_fincat.ipk
felix_stdcat.ipk
felix_stdcat2.ipk
felix_func.ipk
felix_cons.ipk
frames.ipk
platform_frame.ipk
site_frame.ipk
process_frame.ipk
master_frame.ipk
pass_frame.ipk
input_frame.ipk
options.ipk
interscript_options.ipk
compilers.ipk
diff.ipk
commands.ipk
bugs.ipk
installation.ipk

7.3. Include List
-----------------

    interscript: iscr.pak
       interscript: introduction.ipk
       interscript: requirements.ipk
       interscript: design.ipk
       interscript: tutorial.pak
          interscript: faq.ipk
       code: iscr.pak
       interscript: sets.ipk
       interscript: stacks.ipk
       interscript: protocols.ipk
       interscript: drivers.ipk
          interscript: source_drivers.ipk
          interscript: sink_drivers.ipk
          interscript: storage_drivers.ipk
       interscript: weavers.ipk
          interscript: text_weaver.ipk
          interscript: html_weaver.ipk
          interscript: web_weaver.ipk
          interscript: latex_weaver.ipk
       interscript: weaver_filters.ipk
       interscript: tanglers.ipk
          interscript: python_tangler.ipk
          interscript: perl_tangler.ipk
       interscript: tokenizers.ipk
          interscript: python_tokeniser.ipk
       interscript: parsers.ipk
          interscript: html_parser.ipk
          interscript: lalr1_parser.ipk
       interscript: felix.pak
          interscript: felix_protocol.ipk
          interscript: felix_package.ipk
             interscript: felix_fincat.ipk
             interscript: felix_stdcat.ipk
             interscript: felix_stdcat2.ipk
             interscript: felix_func.ipk
             interscript: felix_cons.ipk
       interscript: frames.ipk
          interscript: platform_frame.ipk
          interscript: site_frame.ipk
          interscript: process_frame.ipk
          interscript: master_frame.ipk
          interscript: pass_frame.ipk
          interscript: input_frame.ipk
       interscript: options.ipk
       interscript: interscript_options.ipk
       interscript: compilers.ipk
       interscript: diff.ipk
       interscript: commands.ipk
       interscript: bugs.ipk
       interscript: installation.ipk

7.4. Bugs (etc) for Version 1a7
-------------------------------

As a new development, Interscript has a lot of bugs,
design flaws, and omissions. Please send comments to

  skaller@maxtal.com.au

7.4.1. These debugging notes
----------------------------

These notes reflect current status. They're not date
stamped, and there's no indication if a bug has been
fixed (the comment, hopefully, just gets removed). This
is hopeless for bug tracking.

What we need is more like a database with time and
versions stamped comments, organised by topic so fixes
can be noted.

These notes are written separately from the program and
documentation. For a list generated by cummulating
comments made by me throughout the source, see
iscrbuglist.html for this information.

7.4.2. Control System
---------------------

Keyboard interrupts are still dubious. This has been
improved over the 1a1 release.

The frames described in the doco are now implemented,
more or less. Still needs some work.

No threads yet, but the frame implementation is good
preparation for supporting them.

The SGML input filter is too primitive.

There's no error handling system. There's error
handling, but i wouldn't call it a 'system' :-)

The markup filters are in the wrong place.

7.4.3. Tangler Bugs
-------------------

String tanglers add an extra space on the last line.
It's not easy to stop this without buffering. Also, the
Python string tangler is all wrong. Python doesn't
concatenate a sequence of string literals like C does.

The python tangler currently tokenises the generated
code file (as it is being generated). The tokeniser
used is the standard one, tokeniser.py. Unfortunately,
it has the wrong control interface, and we have to map
it to one the tangler can drive. This requires
coroutines, but Python doesn't have them, so we are
forced to emulate them using threads.

The python tokeniser is now fully functional.

The control system has been almost completely redone.
It still uses the lists of regular expressions paired
with functions to execute, but the functions are now
methods bound to frames, usually the current input
frame.

The predefined methods are replaced for every input
frame constructed. This means users cannot override
them properly, since they're not inherited. Really, we
should use the Guido hack: store functions in the user
dictionary, and turn them into bound methods at the
point of lookup.

Correct implementation of the scoping mechanism is
severely hampered by Python's lack of suitable scoping.

7.4.4. Weaver Bugs
------------------

The table of identifiers gets wide under Latex, the
items don't wrap.

7.4.5. Misc Bugs
----------------

Table of contents and file lists are now stored in
memory and preserved between passes, so it is necessary
to run two passes to get them to print out correctly.
Because the storage is weaver independent, being
generated by the multiplexor, there are no references
in the table of contents. This needs to be fixed by
allowing each weaver private storage persistent across
passes. At this time this is not possible because there
is no way to uniquely identifier a weaver instance (a
new object is constructed each pass).

Heading numbers should have user controlled formats.

7.4.6. Design Issues
--------------------

The interactions in this system are VERY complex. For
example, consider an Interscript source which generates
a Interscript source, or a shell archive. In this case
multiple tanglers may be needed to generate a single
file.

It should be possible to perform 'categorical'
constructions on tanglers: chaining tanglers, writing
to multiple outputs, writing to selected outputs, etc.

7.4.7. Omissions
----------------


7.4.7.1. Building
-----------------

There's no build environment.

7.4.7.2. Scripting
------------------

The Tcl interface doesn't map variables.

There's no Perl interface.

7.4.7.3. Parsing
----------------

No support for Internationalisation.

to invoke Interscript this way. More generally, it
would be nice if Interscript could 'automatically
recognize' various input files.

It's tricky to change the warning character (default
@). Perhaps the first character on the first line
should be taken as the warning character (unless the
first line starts with #!, in which case the second
line is used. Curently, Interscript can't work unless
some executable code is at the start of the primary
source file (at least to set up the weaver).

Perl users will hate having @ as the escape character
because it is the first character of all array
identifiers. (Is that right?)

The parser is not modularised properly.

There's no XML filter.

There's no LaTeX filter (that's hard!)

7.4.7.4. Weaving
----------------

There's no weaver for plain TeX, Postscript, or Word,
nroff, Lout.

The support for hypertext is primitive.

The identifier index uses HTML hyperlinks. What should
be displayed as the anchor text: the input source
reference or the output code reference? Or both? The
input source is where editing would be done, but the
output tells where the identifier ended up, which is
probably more useful.

No pictures or drawings.

7.4.7.5. Tangling
-----------------

There's no support for 'chunking' (noweb) or 'macros'
(FunnelWeb): i.e. the ability to define code fragments
to be included inside others, building the code
heirarchical. This is an absolutely fundamental
requirement, and it is not omitted by accident. There
is some primitive macro support: you can tangle a file
and read it in on the next pass, or you can tangle a
memory sink and then use it as a source. It's easy to
write commands that simplify use of these mechanisms,
and I had some, but they've been removed.

The reason there's no chunking is that I'm not happy
with any of the mechanisms yet, nor certain of the
requirements. FunnelWeb, for example, tangles code
after scanning the whole document, and so can include a
macro A in a macro B, even if B is defined _before_ A.
This isn't possible with a simple one pass dynamic
system. On the other hand, it seems obvious that
tanglers that generate chunks should perform language
specific processing, such as formatting comments and
introducing the #line directives, while the generated
output should be included raw (since it has already
been processed).

Unfortunately, this is a gross oversimplification of
the requirements of a meta-programmable system. For
example, a lot of programming would benefit from chunks
with arguments, which FunnelWeb supports (and which is
why they're called macros). As another example,
FunnelWeb tracks usage of chunks, and requires every
chunk be used exactly once, unless otherwise declared
in chunk definition. Yet another example: templates, or
skeleton. This is conceptually the same as a macro with
arguments, but that is a lousy way to actually have to
write the call. The correct method is to use
co-routines (interleaving), diversions, continuations
(etc).

So we see that generating code by expanding with
substitutions, even the simplest kind of parameterless
chunk substitutions, seems to be the thin end of the
wedge leading to the development of a complete
programming language --- and Interscript is already
equipped with at least one of those.

Here's a final 'killer' example. One of the languages
we are sure to want to tangle is ... Interscript.

If a tangler supports source cross references such as
the #line directive, then there will always be such a
directive as the first line of a code file.
Unfortunately, some languages and systems have special
requirements on what's in the first line. Cases I know
of include:

 1.  the Unix #! convention, which could be used by
    Python and Tcl programs,

 2.  vi: set tabwidth type lines in text files
    (including programs),

 3.  and quote a lot of typesetting other systems
    (including compilers) which require global flags to
    be set by pragmas and the like, right on the first
    line.

The solution is to have a special command to generate
the first lines of a code file, which doesn't trigger
generating the #line directive.

There's no tangler for many languages: Modula, Pascal,
Cobol, Scheme, etc etc.

There's no support for bolding keywords.

There's no Tcl string tangler. The Perl string tangler
is bogus.

7.4.8. New in 1a5
-----------------

By far the most important improvement is the
implementing of the frame architecture, and the
repackaging of interscript as a python package.

The bulk of interscript is now built as a package in a
tree structured directory. Using the frames
architecture, the command line interface has been
reduced to few lines, which will allow interscript to
be embedded in other front end programs.

The web (stacking html) weaver now generates a table of
contents which uses javascript to produce a dynamic
tree under Internet Explorer.

There are also two home pages for a document, and two
tables of contents. The alternate home page creates two
framnes, and loads an alternate table of contents into
the first frame, which targets the second frame.

The web weaver now generates <LINK> tags similar
exhibiting the relationship between the documents.

Tcl script has been discarded completely.

The command line launcher now accepts an option
'--test' which pushes the current directory onto the
system path to force package loading to look in the
current directory. This is the place interscript and
user packages will often be built from. The purpose of
this hack is to allow documents containing both code
and tests for the code, to allow the test code to
actually find the newly created modules before they're
installed in standard places. By default, sys.path is
not modified, so unix '#!/usr/local/bin/python'
launched scripts will not search the current directory.
This hopefully improves separation between reliable
installed software and that under development.

If the first line of a Python output starts with '#!',
line numbers are not generated until the next line.

The autoweaver for html and web now makes sinks which
use carriage return linefeed pairs as end of line
characters. This is to fix bugs in stupid Microsoft
software such as Notepad and Wordpad, which refuses to
recognize them as end of line. This is important for
viewing HTML sources from Internet Explorer.

Sink drivers have been rearchitected to support
standard python output file protocols so they can be
used, for example, as the sys.stdout device. The Python
line handling protocol sucks, stupidly returning the
end of line characters at the end of lines on input,
and requiring them to be there already on output.
Interscript's own protocol returns lines without line
terminators, adds them on output, and recognizes
linefeed in strings (only) as it's internal end of line
terminator.

Option processing now uses an interscript module
(getopt wasn't up to it).

There are now four options, '--weaver_prefix',
'--weaver_directory', '--tangler_prefix',
'--tangler_directory', which allow output to be
directed somewhere other than the current directory
from the command line. See next item.

Interscript now fixes file names to use a subset of the
Unix filename convention. All interscript filenames
must use a sensible set of characters for component
names, and separate them with '/'. Filename must be
relative.

A filename given for a tangler or weaver constructed by
the builtin tangler functions or autoweave is prefixed
by the directory option before being split into
components. Then the prefix is prepended to the first
component.

Directories are built automatically (and somewhat
prematurely).

The effect is that the prefix_* should usually denote
an existing directory in native operating system
format, with a directory separator at the end, and the
directory_* should be a unix pathname with a '/' at the
end.

One of the benefits of this arrangement is that Windows
clients can specify output onto another drive, for
example using the option:

  --weaver_prefix="d:\project\document files\"

Note the quotes!

7.4.8.1. Separate compilation
-----------------------------

Interscript is now constructed in such a way that it is
possible to tangle subdocuments to extract individual
modules. For example, if a weaver is changed, it is
possible to update that component by processing the ipk
(included package) file without rebuilding all of
interscript. If weaving is enabled, a separate document
for that component will be generated, it will not be
related to the pages for the same component generated
as part of a full build. How would we get the right
heading numbers? Or predict the filenames to be
generated?

This needs to be fixed. The problem is one of design
rather than implementation: the web weaver might be
adapted to use filenames derived from the current input
source, locking the structure of the generated
documents to the input source structure. At present,
these are independent except that breaks occur on
headings.

Probably, 'include_file' is not enough to support what
is required.

7.4.9. Under development and planned
------------------------------------

Under development is a syntax driven LALR translator
which I hope to use to replace the 'micky mouse'
control system.

The patch/diff module is working. I still haven't
figured out how to use it to implement version control.

A simple Tkinter front end is under development for
launching interscript runs, as an alternative to the
command line launcher 'iscr.py'.

The current prefix and directory options for tanglers
and weavers is effective but too primitive. The
interscript source contains a currently unused
installation frame which illustrates the mechanism
intended to replace this. Interscript filenames will be
extended to full URI syntax, with the protocol 'iscr'
being used to designate vectoring of the output to a
directory which is the value of a dictionary keyed by
the 'machine' component of the URI, together with a
trust level assigned by the client.

This mechanism allows the author to identify an
arbitrary set of logical file systems into which output
is put, and the client to map these logical file
systems onto actual output points.

For example, the client will output stuff to a project
development structure during testing. When happy, the
output will be remapped to the installation structure.
For example, html documentation could be automatically
posted to the live Internet web site.

The main impediment to actually implementing this, is
that to use the mechanism, the user will have to set up
the installation frame to use interscript. It is very
difficult to provide sensible defaults (other than
'current directory'). Doing this requires non-trivial
programming work by the client, constructing or
modifying the installation frame. This is not so bad if
it is done from a GUI interface.

Move to Java. Because advanced interscript use requires
a GUI based architecture to manage whole projects (as
opposed to just processing a single document at a
time), interscript is likely to move to JPython. This
is advance warning to users: try to get JPyton running!
It is also going to use a lot more advanced HTML
features like javascript, style sheets, and other stuff
which is currently a nightmare due to the lack of a
standard Document Object Model for browsers.

Interscript current supports Internet Explorer DOM, in
lieu of a standard. Although it is possible to
incorporate some Netscape DOM features, the principal
feature needed -- outlining -- requires dynamically
setting the display style of an element, something
Netscape doesn't support at all (at the time of
writing, and as far as I can tell). Therefore, Netscape
is treated like a dumb browser, and enhanced DHTML
features will only work on Internet Explorer.

The emphasis on GUI interfaces cannot be
overemphasised. The move to dynamic, fully featured,
control over browsing the generated documents, together
with full scale meta data generation to facilitate
clients such as search engines and web crawlers
indexing the documents, is considered crucial to
getting a viable literate programmed system running.
Use of Java AWT for the input side seems inevitable, as
this appears to be the only GUI which is likely to be
widely supported, standardised, and which also allows
interaction between output and input by way of browser
applet plugins.

7.5. Installation Guide
-----------------------

How to install interscript on your computer.

7.5.1. What you need
--------------------

You need a modern computer running a vaguely posix like
operating system, C, and CPython 1.5.1. Interscript
should run out of the box on a Linux system, and with
minimal effort on most Unix boxes, and without too much
effort on 32 bit MS-Windows systems. Some enhancements
may require site dependent configuration. Please see
below for more details.

7.5.2. Where to get it
----------------------

You can ftp the interscript tar ball anonymously from
ftp://ftp.triode.net.au/ftp/skaller/interscript.tgz
which is a tarred, gziped, archive. WinZip should
handle this format.

7.5.3. Instant install
----------------------

Here are the quick installation instructions.

7.5.3.1. Unpack the archive
---------------------------

Unpack the tar ball somewhere Python will find it. The
result will be

*    a directory "interscript"

*    a file "interscript.pth"

On my linux box, I use the directory:

  /usr/local/lib/python1.5/site-packages/

7.5.3.2. Unix: Make the command line launch script executable
-------------------------------------------------------------

Optional but recommended. Copy the file
"interscript/bin/iscr.py" to someplace on you execution
path, renaming it "iscr" and making it executable. For
example, I use:

  cp /usr/local/lib/python1.5/site-packages/interscript/bin/iscr.py ~/bin/iscr
  chmod a+x iscr

You may need to edit the first line of this script,
which is

  #!/usr/bin/env python

7.5.3.3. Windows: Make the command line launch script executable
----------------------------------------------------------------

Copy the file interscript/bin/iscr.bat to someplace on
your execution path. You will (definitely!) need to
edit this file, replacing "iscr.py" with the absolute
pathname of the file "interscript/bin/iscr.py". You may
also need to change the name of the python application
executable. Note that this file only supports 9
arguments.

7.5.3.4. Mac: write a launch script
-----------------------------------

I am told the Mac does not have a shell tool.
Interscript can be launched from any Python script by
simply importing the module "interscript" and calling
the function

  interscript.run_from_arguments(args)

where "args" is a python list of strings, each of which
is a command line argument as would be parsed by a Unix
system. You will need to write a Python scipt that
gleans this information from somewhere such as a a
popup edit window, or by reading a response file. I
would be interested in including such a script with
interscript if someone wanted to send it to me at
mailto:skaller@maxtal.com.au

It is very likely that the current version of the
interscript package will also require some changes to
work on the Mac. Please email any advice to me at
mailto:skaller@maxtal.com.au

I will be working to isolate platform dependencies
better in the next version, and to try to provide
pre-configured multi-platform support.

7.5.3.5. Install doco
---------------------

If necessary copy, or move, the HTML documentation in
interscript/doc to someplace on your local WWW server.
You will need to copy all the ".html" files, and the
file "interscript.css" if you want to use the
interscript CSS1 cascading style sheet.

7.5.3.6. Test It
----------------

You can check interscript is installed from the command
line like:

  iscr --weaver=html --passes=3 example.pak

where "example.pak" is any interscript master source
file; for example, the file
"interscript/tests/example.pak", which is displayed
below.

Start python section to interscript/tests/example.pak[1]

     1: #line 101 "installation.ipk"


End python section to interscript/tests/example.pak[1]

7.5.4. Configuration
--------------------

Interscript supports some options which may need
configuration.

WARNING: There is currently no standard way to maintain
the interscript package. Any changes you make may get
wiped out by the next interscript installation. Please
keep a copy somewhere to reconfigure interscript after
the next installation! [I'm working on this.]

7.5.4.1. Compilers sub-package
------------------------------

This sub-package provides interscript with the ability
to compile dynmically loadable python modules written
in C or C++, and with the ability to build stand alone
command lines tools using a command line C and C++
compiler.

The files "c.py", "cpp.py", and "cconfig.py" in
"interscript/compilers/" are set up for my Linux box,
and invoke "gcc" and "g++" with appropriate options for
building python modules and command line executables.
You may need to modify (or even rewrite) these modules
to make them work at your site.

These modules are not required by interscript, but they
do allow it to build some speedup code automatically,
they're generally nice to have, and they will be used
by me when distributing C or C++ code hosted by
interscript (instead of makefiles).

These files are generated by interscript from the
source "interscript/src/compilers.ipk".

7.5.4.2. Diff sub-package
-------------------------

This package provides the ability to diff and patch
files. It is not required but is very strongly
recommended! The script is
"interscript/utilities/diff.py", and is sourced from
"interscript/src/diff.ipk" and requires GNU diff and
GNU patch.

7.5.4.3. Command line tool execution
------------------------------------

The python module 'commands' is used to execute client
commands, including python script, and capture its
output. This module is theoretically optional ...

Note that when python is invoked, the string
"sys.executable" is used to name it. This attribute is
not documented in the python documentation.

7.5.4.4. Speedups
-----------------

The module "interscript.core.iscrcopt" is an optional
dynamically loadable C module which speeds up some of
the character by character lexical processing that
interscript has to do. If the compilers module works, a
build of interscript, and in particular the source
"interscript/src/iscrcopt.ipk", will generate the
binary module. This module is already built for Redhat
Linux 5. This file is completely platform independent
(but the binary isn't, of course).

7.5.4.5. Other
--------------

There are currently a few other platform dependencies.
I'm working on fixing this. That is, I will at least
try to isolate such code in such a way that
re-installation or upgrading of interscript will not
clobber client modifications. This WILL happen at
present!

7.5.5. Building Interscript
---------------------------

If you are keen, you can try to rebuild interscript.

YOU HAVE TO BE REALLY KEEN!

Whew. Now I've done my duty and tried to disuade you ..

The whole of the source is in "interscript/src". I
recommend you create a development directory separate
from the initial (pre-built) installation. Copy the
sources into this directory and invoke the pre-built
interscript command line tool 'iscr' on the file
'iscr.pak' like this:

  iscr iscr.pak

This should build a directory 'interscript' in the
current directory. You may notice some 'errors'. Don't
worry! Most of these are normal. Errors occur
'normally' when interscript tries to read files that
don't exist yet, but get created later in the run.

Now issue the command

  iscr --test "--weavers='all'" --passes=4 iscr.pak

This should rebuild interscript, this time using the
package built by the previous run, and also generating
the documentation. If you have problems, try using the
"-v" switch:

  iscr --test -v "--weavers='all'" --passes=4 iscr.pak

To capture the output, use the logfile option:

  iscr --test --logfile=iscr.log -v "--weavers='all'" --passes=4 iscr.pak

Note that the "--test" switch must come first, it is a
command line tool option, not an interscript option.
This option adds the current directory to the python
path before trying to import the interscript module. If
your build of interscript fails, you can edit the
sources and rebuild without the "--test" option. You
can't build with it (unless the build failed utterly--
or you delete the interscript directory or
interscript.pth file first).

To install your new version of interscript, copy the
interscript directory to your standard installation
point. Don't do this unless you're sure it works! (If
it doesn't, you may not be able to rebuild interscript
without re-installing from the tar ball .. because you
need a working copy of interscript to build
interscript).

You can rebuild any code file by just running
interscript on the ".ipk" file defining it. This won't
rebuild the documentation correctly! But it is very
much faster than rebuilding the whole package.

7.5.6. NT Issues
----------------

Interscript builds sucessfully under NT. You can expect
to see a few of these:

  The name specified is not recognized as an
  internal or external command, operable program or batch file.

On the first pass building interscript, I try to get
hold of the user and hostname using external command
line tools. I use 'uname', for example, which isn't
supported under NT. These tools are called using
os.popen. NT writes an error message to the console if
it cannot find the tool.
