1. Interscript Version
----------------------

This document describes Interscript version 1.0a7 build
1302 on ruby by root.It was generated by Interscript
version 1.0a7, build 1300 on ruby by root at Thu Nov
12, 1998 at 12:49 PM (UTC).

1.1. Contents
-------------

   Interscript Version .............................. 1
      Contents ...................................... 9
   Introduction .................................... 70
      Interscript .................................. 98
   Requirements ................................... 130
      Functionality ............................... 165
      Interface ................................... 550
      Implementation pragmatics ................... 586
   Design Fundamentals ............................ 629
      Devices ..................................... 635
      Processors .................................. 816
      Tanglers .................................... 847
      Lexical Scoping ............................. 889
      Weaver Control .............................. 921
      Source Tracking ............................. 942
      Parsing ..................................... 999
      Documentation Constructions ................ 1025
      Microformatting shortcuts .................. 1060
      Weaver Architecture ........................ 1151
   Interscript Tutorial .......................... 1283
      Weaving a document ......................... 1291
      Tangling code .............................. 1483
      hello method ............................... 1518
      Scripting .................................. 1563
      Test 1: A simple test ...................... 1646
      Fonts ...................................... 1733
      Lists ...................................... 1761
      Tables ..................................... 1831
      Citations .................................. 1851
      Cross References ........................... 1861
      Including Files ............................ 1887
      Translating Html ........................... 1929
      Special constructions ...................... 1941
      File Names ................................. 2040
      Questions and Answers ...................... 2089
   Interscript Package Source .................... 2353
      Interscript Module ......................... 2358
      Core Subpackage ............................ 3094
      Drivers Subpackage ......................... 3637
      Weavers .................................... 4791
      Weaver Filters ............................. 8203
      Tanglers ................................... 8334
      Tokenisers ................................. 9307
      Html parser test ........................... 9932
      LALR(1) Parser table generator ............. 9964
      Architecture .............................. 11698
      Module getoptions ......................... 14175
      Get Options ............................... 14249
      Utility Modules ........................... 14453
      Application and tool directory ............ 15124
      Test package .............................. 15176
   Appendicies .................................. 15189
      File List ................................. 15193
      Source List ............................... 15694
      Include List .............................. 15738
      Bugs (etc) for Version 1a7 ................ 15783
      Installation Guide ........................ 16241

2. Introduction
---------------

Interscript is a component of a new style of
programming environment incorporating the ideas of
literate programming and category theory. Literate
programming was first embodied in significant code by
Donald Knuth in a program called Web, to support
publication of the sources for his TeX typesetting
system.

The fundamental precept of literate programming is that
documentation should not be written after software
development, but be an integral part of it. A literate
programming source is primarily a document in which
source is embedded; the source is extracted by the
literate programming tool by a process called tangling,
while the file is typeset by a process called weaving.

Unlike traditional documentation, whether it be design
specifications, or, code or user documentation,
literate programming offers the opportunity for the
documentation and software to remain synchronised
because the two are lexically close and easy to
maintain together. Furthermore, the published sources
not only match the software exactly, but are sure to be
complete.

2.1. Interscript
----------------

Interscript is a major breakthough in the design of
literate programming tools. Other tools such as Web,
C-Web, and FunnelWeb, have limited functionality, and
are mainly restricted to partitioning code and
documentation.

Interscript is different because it embodies a complete
and fully functional programming language, namely
Python. An Interscript source file consists of three,
not two, kinds of source: the target source code,
documentation, and executable Python script. Because of
this feature, the user source can extend the tool in
arbitrary ways at _run time_ (without modifying the
original tool).

In addition, the basic Interscript tool has an advanced
object based design, including pluggable drivers for
data sources and sinks, including automatic file
downloading, tanglers for various programming languages
including C, C++, Python, generic script , and raw
data, and plugable weavers for various typesetting
systems including HTML, TeX, Postscript, and plain
text.

Interscript includes functionality designed to support
building, testing, installation, and verion control of
software. In particular, the standard system includes
functionality designed to replace _make_.

3. Requirements
---------------

Interscript is a software component intended to assist
in the publishing and development of software by
providing integrated source code, documentation and
testing facilities.

The detailed requirements stem from fundamental design
decisions including the choice of python as an
implementation language, the decision to restrict the
user interface to batch processing of text files, and
the embodiment of the principal notions of literate
programming.

The sections below discuss the requirements from the
point of view of desired functionality, interface, and
implementation pragmatics.

However, it is not possible to entirely separate
discussion of requirements from design, because
detailed requirements stem from design decisions, and
guide more detailed design. Nor is it possible to
separate implementation from design, since
implementation details have design consequences -- and
also provided unexpected opportunities for
functionality not in the original requirements.

In other words you can expect some discussion of design
and implementation in the section, although the
emphasis is on requirements. See the next major section
for a discussion focusing on design, and the following
section for a detailed description of the
implementation.

3.1. Functionality
------------------

This section discusses the functionality Interscript
requires.

The overall functionality is easy to describe:
interscript must process input source files and from
them extract the target program files, and extract and
format a suitable set of documentation files. In
addition it must be able to compile and build the
target software, execute test code, and present the
results of all these processes in a comprehensible
format.

We will call the goal of program code extraction
_tangling_, that of document construction _weaving_,
and that of compilation and test code execution
_building_.

3.1.1. Tangling
---------------

This section describes the requirements of the tangling
goal, and is by far the simplest to state (and
implement).

The client will largely present files containing one or
more program code files, separated by documentation
sections. The tangler basically extracts this code,
concatenates the sections sequentially, and writes the
output verbatim to code files. Code sections targeting
a single output file must be associated, and the target
file identified and its location determined.

The detailed requirements for tangling, therefore,
largely focus on deviations from normal processing.

3.1.1.1. Supported Programming Languages
----------------------------------------

Special parsers shall be supplied for C, C++, Python,
Java, Eiffel, Pascal, Modula, Perl. Tcl, interscript,
and Unix shell scripts are too quirky. Basic is too
ugly :-) Cobol, Fortran and PL/1 are too archaic.

3.1.1.2. Source Tracking
------------------------

Source tracking is the ability to determine where
generated program and documentation sections came from
in the original source files. It is vital at the
building stage, so that errors can be corrected. Where
target tools identify the source of errors, they should
be guided if possible, to point at the original
sources.

3.1.1.2.1. C and C++
--------------------

For C and C++, interscript must generate #line
preprocessing directives.

3.1.1.3. Chunking
-----------------

The ability to build program files out of order is
sometimes called chunking. It is sometimes useful, for
example, to define functions after they are used even
when the target programming language requires
otherwise. However, chunking is not restricted to
reordering code sections, but may be considered to
include the ability to nest sections hierarchically.
This permits the author to represent program structure
in manner not provided by the target programming
language.

There are some further variations on chunking. The
first consideration is that code often has to be
repeated, and so a section may be used more than once,
possibly in distinct program files, but sometimes even
in the same file. Because hierarchical chunking
requires naming chunks, reuse is available
automatically; the principal issue here is the
converse: to ensure when required that a chunk is used
exactly once.

Stemming from the use of labelled chunks is the need to
be able to locate them: when a chunk is used, one needs
to find where it is defined, and sometimes conversely.

More generally: a simple verbatim code file is a
special case of breaking a code file into sections
separated by documentation but without reordering,
which is a special case of reorderable sections which
is a special case of hierarchical chunking. Again, a
tree is a special case of a directed acyclic graphs,
which embodies the notion that a chunk may be used
twice and also that it may not be used at all.

Some conventional literate programming tools go further
in permitting what might be called macros: chunks
containing unbound variables which can be bound at the
point of use: a simple case of parameterized macro
processing. Macros are particularly useful for
generating repetitive forms such as tables.

In addition, we might consider conditional compilation,
which is useful for controlling software variations
such as platform dependencies and debugging versions.
Conditional compilation is heavily used in C.

There is yet a more general form of chunking ... in
which the notion of the chunk begins to degenerate,
namely generation or arbitrary code by executable
script. Such facility is essential, for example, where
the code to be generated is sensitive to the
environment, for example the inclusion of the current
date. Perhaps more interesting is the ability of
executable script to generate several files
simultaneously, for example both the declaration in a
header file and the definition in the body file, of a
function in the C programming language. Building
specialized scripts tailored to the client's
requirement is an essential facility of literate
programming tools because, although rarely used, it can
save a lot of work and provide considerable coherence,
as well as generating tabulated documentation.

To provide all these facilities, so the most
specialized, and most common case integrates seamlessly
with the most complex and general, interscript
leverages the Python scripting engine. The most complex
code generation is supported almost effortlessly by
simply allowing the client to write arbitrary python
script, while the more common simpler requirements are
simply provided as pre-built routines: all the
facilities are accessed in precisely the same way, by
script execution.

This feature is central to the interscript design and
cannot be isolation from a discussion of requirements,
which can be recast, in some sense, to a discussion of
the architectural framework in which such script
executes, and the set of pre-built functions which
ought be made available.

Macros with parameters, however, are not especially
good at expression skeletons. A skeleton, or
boilerplate, is essentially a macro with large
arguments.

3.1.1.4. Parsing for reference tables
-------------------------------------

Where possible, interscript should tokenise and parse
or partially parse program files to extract summary
data.

3.1.1.4.1. Tokenisable Languages
--------------------------------

The following languages can be tokenised easily: C,
C++, Python, Java, Eiffel, Pascal, Modula. [Only the
Python tokeniser has been written].

3.1.1.4.1.1. Identifier reference
---------------------------------

For each language which can be tokenised, produce an
identifier reference: C, C++, Python, Java, Eiffel.
[Table generator done for web, html, latex]

3.1.1.4.2. Fully Parsable Languages
-----------------------------------

Python and Java. C and C++ cannot be fully parsed
without semantic analysis, and even then, it is tricky:
it is hard to determine if a statement is a function
declaration, variable declaration, or executable. [Only
partial python parsing has been implemented]

3.1.1.4.3. Partially Parsable Languages
---------------------------------------

C and C++ can be partially parsed: information
extracted may not be completely correct.

3.1.1.4.3.1. Class reference
----------------------------

For each language which supports user defined types,
particularly classes, provide a table of classes.
Python, Java, C, C++, Eiffel, Pascal, Modula. [Table
generator done for web, html, latex]

3.1.1.4.4. Function reference
-----------------------------

For languages supporting functions, provide a function
reference. This includes methods/member functions.
Provide for C++, Python, Java, Eiffel, Pascal, Modula.
[Table generator done for web, html]

3.1.1.5. Parsing for embedded documentation
-------------------------------------------


3.1.1.5.1. Perl DOC
-------------------

Perl DOC is arbitrary documentation provided in
comments, to be converted to interscript method calls.
[Done]

3.1.1.5.2. Java DOC
-------------------

Java DOC is a commenting protocol, and depends on
parsing to identify the entity to which the
documentation refers. [Not implemented]

3.1.1.5.3. Eiffel
-----------------

Eiffel provides dedicated documentation constructions.
[Not implemented]

3.1.1.5.4. Python Doc strings
-----------------------------

Python supports Doc strings with a standard protocol
for module and class documentation. Although the Python
runtime supports module, class, and function Doc
strings, there is no standard way to relate Doc strings
to functions. [Not implemented]

3.1.1.6. Internationalisation
-----------------------------

Interscript tanglers should provide support for
multiple human languages. There are two kinds of
support determined by the binding time:

Interscript time binding 
     Interscript time binding allows generating target
    code for a specified language. If multiple
    languages are specified, multiple versions of the
    target code are generated.

Run time binding 
     Run time binding allows generating target code for
    a specified language set. Even if multiple
    languages are specified, only a single version of
    the target code is generated.

There are two levels of support which can be provided,
which are not independent of binding time.

Identifiers 
     Using names in the programmers native language to
    aid maintenance. This is clearly an interscript
    time option.

Strings 
     Using strings in the client native language to aid
    use. This can be done either at interscript time or
    target software run time.

See alsoline 502 for details on the documentation
aspects of internationalisation.

3.1.2. Weaving
--------------

Weaving is the most complex subsystem.

3.1.2.1. Individual Weavers
---------------------------

There shall be weavers producing plain text [done],
simple HTML 4 (html) [done], advanced HTML 4 multi page
(web) [done], XML [not implemented], latex2e [done].

3.1.2.2. Individual weaver capabilities
---------------------------------------

These capabilities must be provided in each separate
weaver.

3.1.2.2.1. Fonts
----------------

Weavers should support italic, bold, emphasized, strong
and code fonts.

3.1.2.2.2. Lists
----------------

Weavers shall support bullet, numbered, and keyed
lists.

3.1.2.2.3. Displays
-------------------

Weavers shall support displays for code and prose, both
inline and from a separate source. Latex, shall support
math display.

3.1.2.2.4. Tables
-----------------

Weavers shall support simple tables with horizontal
spanning and column headings.

3.1.2.2.5. Headings
-------------------

Weavers shall support multilevel headings with
automatic heading number generation, and labels
(anchors).

3.1.2.2.6. Code echo
--------------------

Weavers shall provide a method to displayed line
numbered code lines with labels (anchors).

3.1.2.2.7. Citations
--------------------

Weavers shall support citation of URLs, print media,
code files, and interscript generated documents.

3.1.2.3. Internationalisation
-----------------------------

Internationalisation of documentation consists of two
facets: automatic substitution of fixed literals such
as the titles of tables, and providing for alternate
translations of client documentary text.

The former requirement should be met by run time
tangler binding of interscript itself, seeline 405 for
details.

3.1.3. Building
---------------


3.1.3.1. Python hosting
-----------------------

Interscript shall be able to host python script,
optionally capturing standard output.

3.1.3.2. Compilers
------------------

Interscript shall host system compilers.

3.1.3.2.1. CPython
------------------

For CPython, C and C++ compilers shall be hosted. Both
executable applications and dynamically loadable python
modules shall be supported.

3.1.3.2.2. JPython
------------------

For JPython, the javac compiler shall be hosted.

3.1.3.3. Diff/Patch
-------------------

Interscript shall provide a file comparison/change tool
similar to Unix Diff and patch. The tools must
provide/apply reversible differentials.

3.1.3.4. Executing external tools
---------------------------------

Interscript shall host external applications.

3.2. Interface
--------------

The normal understanding of this topic requires
discussion of how interscript is launched. As a command
line tool it provides a standard interface, an
equivalent GUI hosted tool would be little different.
Interscript also provides a Python callable API, which
is more interesting, but largely unimportant to most
users who will not be embedding it.

As a batch oriented text file processing tool, the
secondary interface requirements which describe the
organization and format of the input source files, are
considerably more important, since it is this interface
that most clients will use most of the time.

However, we cannot relegate the presentation of outputs
-- both documents and program files -- to discussion of
functionality because, as a specialized tool,
interscript must constrain -- or at least guide --
presentation to suit its purpose as a development
environment.

Finally, we cannot omit consideration of how the
implementation interfaces to the underlying operating
system and its tools, because, in the development
process, the client must use interscript to host the
launching and management of these tools.

In summary: interscript interfacing involves everything
which has visual appearance including input source
files, output documents and program files, and
presentation of client tool interfaces, especially
error output.

3.3. Implementation pragmatics
------------------------------

This section discusses requirements in terms of the
constraints imposed by an implementation. There are
several requirements all software must meet:

Performance 
     Interscript must be very fast because it must be
    executed after every change to program source code
    or documentation. If this additional processing
    time is significant compared to the time other
    build components such as compilers take, it is
    likely to be an impediment to practical software
    development, especially that for which
    documentation is not as highly rated as
    functionality.

Portability 
     Interscript must be highly portable: it must at
    least be able to execute on Unix, Windows, and Mac
    platforms with minimal installation and maintenance
    hassles. The installation and configuration must be
    manageable by people who are specialists in some
    computing field -- not necessarily the interscript
    implementation language or the host platform.

Accessibility 
     Interscript must be easy to get hold of.

Client side maintenance 
     Because the repertoire of facilities a literate
    programming tool could provide are huge, upgrades
    must be easy to install, must not compromise client
    configuration, and must remain largely compatible
    with sources already developed by interscript.

Third party development 
     Interscript must provide third party development
    opportunities. This is essential when required
    functionality is certain to be highly deviant, and
    also very specific.

4. Design Fundamentals
----------------------

This part looks at various issues in the design of
Interscript.

4.1. Devices
------------

A _device_ is an object representing an actual input,
output, or execution stream such as a disk file.
Devices are simple objects which read, write, or
execute data without parsing or interpretation.
Instead, they interface to actual system device objects
such as disk files, remote files, or text stored in
memory. In Interscript parlance, an input device is
called a _source_, an output device is called a _sink_,
and a device which acts first as sink, and subsequently
sources data that was sunk into it, is called a
_store_.

The input to interscript is represented by a source
driver object. Both the tanglers and weavers write
output via sink driver objects.

Currently implemented sources:

named_file_source 
     The most commonly used source driver reads a disk
    file named by a native filename.

ftp_file_source 
     This device initially reads files from an FTP host
    and creates a local copy. The local copy is read
    until it is deemed too old, at which point it is
    fetched again by FTP.

http_file_source 
     This device initially reads files from an HTTP
    host and creates a local copy. The local copy is
    read until it is deemed too old, at which point it
    is fetched again by HTTP.

url_file_source 
     Reads a file given a URL. Currently, ftp, http,
    gopher, and local files are supported. There is no
    caching.

stdin_source 
     Reads a file from standard input.

null_sink 
     Throws away data.

simple_named_file_sink 
     Writes a disk file given a native filename.

named_file_sink 
     This device writes a temporary disk file, and
    copies it to a nominated file if the two differ,
    both to prevent touching unchanged files, and to
    ensure the previous file is available during
    parsing, allowing generated files to be included
    back into the document.

stdout_sink 
     Writes to standard output.

memory 
     Writes and reads from memory.

disk 
     Writes and reads to disk (with separate read and
    write heads).

Under development are patch readers and writers. A
patch writer compares two files, and, instead of
replacing the old file with the new one if they differ,
writes a patch file instead. The corresponding reader
applies the patches to a copy of the old file and reads
that instead. This mechanism provides rudimentary
version control, allows stable files to be write
protected, and permits posting and using patches by
email or news.

Writers for news and email are also in the works. The
email sink device is particularly useful to allow
automatic regular updates (run as chron jobs) to send
advice to the client.

The URL reader currently doesn't use a cache, because
it uses the standard Python function _urlopen_, which
doesn't use a cache. Hopefully this function will be
upgraded to check expiration headers on http servers,
and cache files locally.

A _tee_ writer --- a device which writes to several
other devices -- is planned.

4.1.1. File names
-----------------

Common interscript commands always refer to files using
Unix relative filename convention, even on non-Unix
platforms; the interscript command line processor also
requires master filenames given on the command line to
follow this convention. These filenames are converted
to the native format internally. The purpose of this
mechanism is to ensure distributed source documents are
platform independent.

Interscript requires the native operating system
support long case sensitive filenames including the
upper and lower case latin-1 (ASCII) letters, digits,
and underscore, and a heirarchical directory system
with some kind of current directory concept: these
features are supported by all modern Unix , Windows,
and Macintosh platforms. Note that interscript cannot
operate on DOS or Win3.x platforms.

4.1.1.1. Named File Source Names
--------------------------------

Interscript _named_file_source_ takes two arguments
identifying a local disk file. The first, mandatory,
component is name in Unix relative filename format, and
the second, optional, component, is a prefix in native
operating system format.

This pair of names is the interscript file name
convention. If the prefix is empty, it is replaced by
the absolute pathname of the current directory in
native operating system format, including a trailing
directory separator. Then the Unix filename is
converted to a native operating system filename, and
appended to the prefix.

Commands such as _include_file_ which refer to sources
supply the directory of the current file as the prefix
so that the argument filename is relative to the
location of the including input file. All such
arguments must be relative filenames in Unix format for
this reason.

Note that the command line provides the
_--source-prefix=_ option for the same reason: a
filename given on the command line as a master source
file must be given as a relative filename in Unix
format, even on non-Unix systems. The source-prefix
option permits the native operating system name of the
directory containing the file to be specified.

4.1.1.2. Named File Sink Names
------------------------------

Interscript _named_file_sink_ takes two arguments
identifying a local disk file. The first, mandatory,
component is name in Unix relative filename format, and
the second, optional, component, is a prefix in native
operating system format.

This pair of names is the interscript file name
convention. If the prefix is empty, it is replaced by
the absolute pathname of the current directory in
native operating system format, including a trailing
directory separator. The the Unix filename is then
converted to a native operating system filename, and
appended to the prefix.

The command line supports four options,
_--tangler-prefix=_, _--tangler-directory=_,
_--weaver-prefix=_, and _--weaver-directory=_, which
facilitate placement of outputs.

The two directory options allow a Unix relative
filename (or other prefixing characters) to be
prepended to all filenames used for tangler or weaver
outputs, respectively.

The two prefix options allow a native operating system
format filename to specify the output directory for
tangler and weaver files. If the prefix is empty, the
absolute pathname of the current directory, including
trailing directory separator, is used. The Unix
filename is converted to native operating system format
and appended to the prefix.

4.2. Processors
---------------

A _processor_ is an object that performs some
processing function. Processors are generally hooked up
to to devices to read or write data. The separation of
processing data streams, and sourcing and sinking those
streams, is the traditional operating system facility
known as device independence.

Whereas a file, denoted by a filename, is used in an
operating system to implement device independence, in
Interscript, a Python object is used instead.

The design of Interscript consists of four processors:
the input, the code tangler, the document weaver, and
the Python engine.

The Interscript parser reads and parses the input and
sends lines to one of the three outputs. If the line
begins with the special warning character @ it is sent
to Python, otherwise if the tangler is not None it is
sent to the tangler, otherwise it is sent to the
weaver. If the tangler receives the line, it is written
to a code file and echoed to the weaver.

The main source of power in this system is the ability
to execute arbitrary Python script. Interscript has
some builtin commands and data structures to facilitate
control.

4.3. Tanglers
-------------

A tangler is an object that is designed to process
source code in some programming language for which the
tangler is specialised. Tanglers are generally selected
by the @select() command in the source. Output to a
tangler is disabled by most documentation commands, so
the system reverts to generating documentation.

Tanglers can be stacked. Typically, test code or header
code will be embedded in files containing definitions.

Interscript comes with specialised tanglers for,
several languages. The list below shows the currently
implemented special features of these tanglers.

data 
     No special features.

C    Tracks source with #linedirectives. Associated
    string and comment tanglers. Parses identifiers
    (badly).

C++  Tracks source with #line directives. Associated
    string and comment tanglers. Parses identifiers
    (badly).

python 
     Tracks source with #line directives. Associated
    comment tangler. Parses identifiers properly.

perl 
     Tracks source with #line directives. Parses and
    processes Plain Old Documentation constructions.

java 
     Associated string and comment tanglers.

interscript 
     No special features.

4.4. Lexical Scoping
--------------------

The system maintains a stack of objects called input
frames to track input sources. Input can be stacked
using the @include_file() command, which is equivalent
to a subroutine call. The stack is popped when the
included file is exhausted.

The commands @begin() and @end() can also be used to
push and pop the input stack, this is equivalent to a
nested block.

User defined symbols are lexically scoped. The system
currently maintains a dictionary of user symbols with
each stack frame: all assignments enter the symbol into
the dictionary of the top of stack frame.

When the stack is pushed, the new top of stack
dictionary is initialised by a copy of the old top of
stack dictionary. User symbols are searched for first
in the top of stack user dictionary, and then in the
global interscript namespace.

As well as supporting scoped symbols, the parser is
scoped. That is, changes to lexicology or processing
mode made by modifying the parser tables are lost when
the frame is dropped. This ensures that, for example, a
change to the warning character in an included file
does not affect the interpretation of the including
file.

4.5. Weaver Control
-------------------

Weaver control is the most complex. Weavers can be
stacked, for example to allow summary files or notes to
be built incrementally. For HTML, detail pages can be
stacked on a master, with hypertext links in the master
to access them.

More than one weaver can be active at once, so that,
for example, a HTML Web site, a LaTeX book text, and a
plain text news article can be generated
simultaneously. Document lines are sent to all the
active weavers for processing.

Some typesetting constructions are too complex or
specialised to be represented in all weavers. In this
case, verbatim text can be sent to a particular weaver,
or sent to any active weavers obeying a specific
protocol.

4.6. Source Tracking
--------------------

In the beginning, every system is a collection of
original source files, one of which is designated as
the initial (master) source. Interscript begins
processing the initial source, and may switch to
another source as a result of executing some command
such as _include_file_. This can be done to break a
long work into sections, or to include common _macros_
(python script).

Each source must have a definite name, and each line
read is counted. This allows references to the original
source to be generated in the code (and documentation)
files, so that errors reported by language processors
can be corrected. The code files cannot be edited,
because they are generated by Interscript and any
changes would be overwritten by the next processing
run.

A related reason for source tracking is to generate
cross reference tables. For example some tanglers
generate an identifier cross reference, which can be
used, for example, to disentangle duplicated names. The
tables can also be used to generate vi editor tags, for
example.

Every line of output must have an associated original
source. That includes code that is saved temporarily
(in memory, or to a disk file), and it includes
generated code.

In order to accurately track original source lines,
each input driver must have a name, usually the input
file name, and must count lines read. When a line is
read by the parser, the filename and line number are
returned as well. The parser passes this information on
to the relevant processor, usually the executor,
tangler, or weaver.

In turn, the tangler, for example, will check the sink
driver to see if the otput is synchronised with the
original source, and then store the source filename and
line number in the sink driver.

If since the last write to the sink, not necessarily by
this tangler, the original source filename has changed,
or the line number is not one more than last time, then
a section header is written to the sink file before the
data. In C and C++ a #line directive is generated, in
languages not supporting such directives (such as HTML
and Python), a comment is generated instead.
(Coincidentally, many scripting languages use a leading
# for a comment and so the C style #line directives is
generated)

4.7. Parsing
------------

Interscript is a language, and needs a parser. Parsing
is a complex task. The main control algorithm, however,
uses a very simple syntax driven parsing engine. The
parser table is a Python list of pairs. Each pair is a
tuple consisting of a compiled regular expression and a
function. The parser gets one line at a time from the
source, and runs through the list, attempting to match
the line against a regular expression. When a match is
found, the corresponding function is called with the
match and source reference data as an argument.

Because this scheme is very simple, it can be extended
or modified easily by the end user.

Because the function invoked can read further lines
from the input, more sophisticated parsing can be
programmed. For example, the Python suite execution
function matches against a line starting with an @ and
ending with an : or other character that indicates an
incomplete suite. The function reads further lines up
to the end of the script before executing the lot as a
single Python suite.

4.8. Documentation Constructions
--------------------------------

Interscript supplies uses with various documentation
constructions. These include standard constructions
such as a title, multilevel document headings, page
headings, table of contents, index, nestable
enumerated, bulletted, and keyed lists, displays (long
quotations), footnotes, and tables. For a full list of
supported constructions see below.

The requirements here are to support a rich enough set
of constructions, with a fine enough level of control
of details, to do handle the bulk of work which would
be required by a serious author, while at the same time
providing the casual programmer simple enough tools to
typeset basic program documentation.

In addition, Interscript supplies constructions
specialised to literate programming. Naturally, there
is a specialised construction for source code display,
and some tables such as a list of files, and identifier
index unique to literate programmed code. In addition,
it is necessary to be able to typeset code fragments
for _examples of use_, even though literate programming
discourages this. (Give real examples!)

These requirements have to balanced against the
efficiency of the translator, the ease of
implementation of the constructions, and the
availability of features in typesetting systems. How do
you typeset diagrams in plain text? (With difficulty)
What about diagrams, colour, pictures, and font
control?

4.9. Microformatting shortcuts
------------------------------

By microformatting, I mean things like emphasising a
single word. Most typsetters can set a plain, bold,
italic, underlined, and mono-spaced font (but not plain
text). Support for mathematics, however, is more
limited.

The general solution to microformatting is to have
specialised parsers. Just as the line by line parser
can be extended by the user, specialised
microformatting can be provided by the end user by
writing a parser to further translate document source.
Naturally, the weavers to be used will have to be
support the constructions.

Weavers already perform some parsing. For example the
Latex weaver has to translate the characters #$%^_ into
the Latex macros that produce them, since they're
reserved characters in standard Latex.

The biggest problem here is to specify a standard
microformatting language. It is not too onerous to
reserve the @ character at the beginning of a line, but
how does one designate three special fonts (bold,
italic and monospaced) and the scope they apply to?
What about font size? For something more difficult,
mathematics?

Any such language must eat up characters which can be
typeset _as is_: the fewer such reserved characters,
the more cluttered the source will become, whereas if
more are reserved, the more likely the user is to
forget to quote them properly when the character itself
is required instead of _magic_.

HTML reserves <> and uses tag pairs to do detailed
markup, and uses & to allow quoting. Latex reserves
#$%^&_\. Interscript reserves @ at the beginning of the
line.

It is possible to do all formatting using lines. But
that leads to a _troff_ like solution, which is
extremely ugly. It should be possible to write normal
text and have it print properly -- and for a programmer
that will include setting special characters.
Typesetting C code documentation in plain Latex is a
pain because underscore means subscript and is an error
outside maths mode: but underscore is more or less the
C version of a hyphen, and more or less an alphabetic
character.

The characters we can afford to reserve are those not
commonly used in program documentation. There aren_t
any. Here_s the proof by analogy: if we reserve @, for
example, then in the very documentation describing the
construction implemented using the @ character, the
most commonly used special character will, of course,
be @.

The solution I have adopted to this intransigent
problem is as follows. First, all the constructions
have to be provided as commands. That means that
irrespective of other details, all the constructions
are available, even if it is a pain to typeset them.

Secondly, we provide regular expression matching
technology to extract microformatting details using
some standard forms, but we will not enable it by
default.

I_ll call these things _shortcuts'. For example, the
first shortcut for code is an @ followed by a C
identifier. An @ in any other context is typeset as an
@.

Shortcuts are implemented by weavers. (The control loop
never sees them). To provide typesetter independent
shortcuts, we need a special kind of weaver: a filter.
A filtering weaver translates shortcuts and then calls
the normal weaver.

Interscript comes with a standard filtering weaver, and
is equipped with a user programmable table of shortcuts
based on regular expression matching. The default
version of this weaver does not do any shortcuts,
however. Shortcuts must be explicitly enabled by the
programmer. However, there is a table of standard
shortcuts prepared, and a command to enable them.

4.10. Weaver Architecture
-------------------------

Interscript operates on the assumption that there is
exactly one weaver that can process all weaver
commands.

4.10.1. Multiple Typesetters
----------------------------

Often, we want to weave the same document in different
formats, for example, using Latex for book output, HTML
for a web, and plain text for email and news.

This facility is provided by a weaver front end called
a multiplexor. The multiplexor keeps a list of active
weavers, and sends every method call to all weavers in
the list that supports that method. It_s not an error
if the weaver doesn_t have the method, but it certainly
is an error if it does, but the call fails.

4.10.1.1. Raw output
--------------------

When Interscript does not support detailed
constructions, it is necssary to hard code them into
the document. For example, if you really want frames in
HTML, or category theory diagrams in LaTex, you have to
code raw HTML or LaTex (probably XYpic) because
Interscript doesn't provide a generic interface to
support.

The principal mechanism for raw output is to put it
between a @rawif(protocol) command, and a @translate()
command. The rawif command disables a target weaver
unless it supports the nominated protocol, in which
case, it is put in raw mode, whereas the enable command
enables the weaver and put it in translating mode.

If the weaver is the multiplexor, it dispatches these
commands to all the weavers attached to it, thereby
allowing raw output to be written to the subset of
weavers supporting it.

Generally, you should provide raw typesetter data for
every possible typesetter so that _something_ is
typeset in every format of the document. (even if it is
just _paste the diagram here_ :-)

Every weaver constructs a protocol list when it is
created, but the method _add_tag_ can be called to add
another protocol to a weaver. The standard protocol
names are html, latex, and text for the html and web
weavers, the latex weaver, and the plain text weaver,
respectively.

This mechanism is design to be used in documents of
parts of documents without requiring knowledge of which
weavers are active. If a particular weaver is active
and accessible, it can be controlled directly instead,
but this is recommended only for specialised documents.

4.10.2. Multiple Human Languages
--------------------------------

Sometimes, we want to prepare the same document in
several human languages. Interscript cannot translate
English to German, for example, so it is necessary to
provide documentation text in both languages. The
program codes, however, are usually in common, except
possibly for comments.

Interscript can generate a multiple versions of a
single document (and each version will be generated in
all the selected formats) using the same tagging
mechanism that is used to control raw output. You can
write sections of English documentation after that
command @enableif(_English_) and sections of German
version after the command @enableif(_German_), and this
will disable all weavers not supporting the nominated
protocol.

This mechanism applies to interscript program code
comment commands. The commands generate ordinary woven
text, but are also inserted into the tangled output
files. In this case, comments will be inserted in the
selected language or languages. Be aware that while
this will not change program semantics it will change
the physical source file.

It is also possible to generate string constants in
different human languages with interscript, but this is
a tangler function, having no special effect on
weaving: because of the complexities of this issue, it
must be effected using python script crafted by the
author for this purpose ... in other words there are no
special commands for it :-)

4.10.3. Multiple Documents
--------------------------

Sometimes, we wish to construct several documents
simultaneously. For example, we may have a short and
long version of a document. We need to select which
weavers to write to. It is easy to do this in simple
cases by just assigning the weaver. For example:

  @both = multiplexor((long, short))
  @weaver = both
  This document describes Interscript.
  @weaver = long
  Here are some gory details.

Because a multiplexor represents a set of documents,
and because one can multiplex multiplexors, it is easy
to create small sets of weavers, and then create
various unions of these sets.

4.10.4. Cumulated Appendices
----------------------------

Sometimes it is useful to accumulate the text for a
document thru the source: the table of contents is an
example of this. Another important example is an issues
list: details of bugs or issues are written near the
relevant source, collected, and printed as an appendix.

Other examples include footnotes, which are usually
printed all togther at the end of articles or chapters
in some styles, summaries of test results, etc. [To be
continued and stuff implemented]

5. Interscript Tutorial
-----------------------

An easy introduction to the Interscript literate
programming tool and environment. Please note
Interscript is still experimental, and the command set
and architecture are not frozen.

5.1. Weaving a document
-----------------------

To create a plain document is easy. First, you should
create a heading like this:

  @head(1,'My Document')

Now, you type some documentation. Like this:

  This is a document describing Interscript, which
  is a literate programming tool. You can use
  any characters you like in the document,
  such as ~!@#$%^&*(), with one exception:
  you should not start a documentation line with @ in
  column 1. The @ character in column 1 is used to flag a command.

5.1.1. Headings
---------------

You can create sub-headings, and sub-subheadings, just
use a @head(n,'Heading') command with n set to the
heading level you want. Headings are number from 1,
level 1 is the biggest heading.

Heading levels should go up consecutively, because
Interscript numbers all headings automatically.

5.1.1.0.1. A skipped Level
--------------------------

If you skip a level, you might get a heading like this
one, with a zero in the number. But worse could happen:
some typesetters will not allow skipped levels. The
HTML and plain text weavers provided by Interscript do,
however.

5.1.2. Separating paragraphs
----------------------------

You can separate paragraphs with the command

  @p()

Blanks lines do not separate paragraphs. This is
deliberate. Any number of blank lines translates to a
single space. This allows you to separate parts of your
Interscript document with vertical white space. It is
particularly useful to add blank lines before headings.
For example:

  @head(1,'A document')
  We are flying to the moon today.
  @p()
  But not just any moon. The moon of Mars.

  @head(1,'Phobos')
  Actually, there are two Martian moons.

  One of them is Phobos.

You should note that @p() is idempotent, which is a
fancy way of saying two or more of them in a row are
the same as one.

You can't add extra space between paragraphs. Not even
by putting dummy blank lines in between like this:

  @p()
  @p()
  @p()

  @p()

  @p()

which is exactly what I did before the paragraph
starting 'you can't add extra space between
paragraphs'!

5.1.3. Line and Page breaks
---------------------------

You can force a line and page break with the commands

  @line_break()
  @page_break()

respectively.

5.1.4. Displaying code examples
-------------------------------

In this tutorial, I've been showing you some example
code. That is something most documentation writers want
to do. You can do it too, like this:

  @begin_displayed_code()
    while 1:
       print 'Hello again and again'
  @end_displayed_code()

5.1.5. Running Interscript
--------------------------

Well, you should try an example file. To process a file
we'll say:

  python iscr.py --weaver=html example.pak

where 'example.pak' is the name of the Interscript
document. This will create a single file
"example.html". Try it!

If you prefer HTML split into lots of little pages,
try:

  python iscr.py --weaver=web example.pak

This produces a file "example_top.html", and a number
of auxiliary files, and a file for each heading.

You can also generate latex2e and plain text with the
commands:

  python iscr.py --weaver=latex example.pak

which produces a file "example.tex", and,

  python iscr.py --weaver=text example.pak

which produces a file example.txt. you can put more
than one file name at the end too. Each such document
will be processed separately.

5.1.5.1. Option help
--------------------

If you type:

  python iscr.py --help

you will get a complete list of available options.

5.1.5.2. Passes
---------------

The passes options causes Interscript to process files
more than once. This is sometimes necessary to get
cross reference information right. The default is
currently 1.

  python iscr.py --weaver=text --passes=2 example.pak

Interscript may stop before running the specified
number of passes. It will do this if, and only if,
every buffered disk file ("named_file_sink") would
write an output the same as the existing file. In that
case, it assumes further passes wouldn't change
anything, and stops. This is called convergence.

5.1.5.3. Tangling parts
-----------------------

If you construct your Interscript sources as a tree,
using the "@include_file()" command, and you follow
certain rules, you can run Interscript on the included
file to extract the code, for just that file. The
Interscript sources are constructed this way. This
feature is vital for building big systems because it
allows you to extract the code from files you have
changed, without extracting code from those that have
not.

You must ensure that code files are lexically contained
entirely in a single include file. More generally, the
include file does not rely on any context from its
parent (except for that which is determined from the
command line).

If you weave an include file, you will get a separate
document for that include file which will, in general,
not be linked to the master document: it will be in a
separate file, named after the include file, and
headings will be numbered separately.

[There is currently no simple way to require a separate
source be built entirely independently so that the
master document can link to it. This would be
especially useful, because it would also permit time
stamps to be checked and avoid unnecessary processing.]

5.2. Tangling code
------------------

So far, we have just produced a document. What about
programming? Here's a sample document with tangling.

  @py = python_output('mymodule.py')
  @head(1,'My Module')
  This is my very own module.
  @select(py)
  import sys
  class myclass:
    def __init__(self, name):
      self.name = name
  @head(2,'hello method')
  Just says hello.
  @select(py)
    def hello(self):
      print 'hello','self.name
  @doc()
  And now back to doco.

Here's what the code will look like:

Start python section to mymodule.py[1]

     1: #line 201 "tutorial.pak"
     2: import sys
     3: class my class:
     4:   def __init__(self, name):
     5:     self.name = name


End python section to mymodule.py[1]

5.3. hello method
-----------------

Just says hello.

Start python section to mymodule.py[2]

     6: #line 208 "tutorial.pak"
     7:   def hello(self):
     8:     print 'hello','self.name


End python section to mymodule.py[2]

And now back to doco.

I've not typeset the top level heading, because it
would come out in this tutorial!

5.3.1. Original Source References
---------------------------------

Notice the #line directives. They're called _original
source references _because they refer to the original,
editable, source file containing the code. If you are
creating C programs, the compiler will recognize them
and report compiler errors in the original file.
Integrated development environments will put the cursor
right in the middle of the Interscript source. This is
necessary, you must not edit the code file. Your
changes will get clobbered next time you run
Interscript on the original source.

5.3.2. Code sections
--------------------

As you can see, the code is displayed with line
numbers, and the file is named at the beginning and
ending of each chunk. Interscript calls these chunks
_code sections _and the doco between them _document
sections. _ So basically, Interscript allows you to
_interleave _code and document sections. You can end a
code section with an @head() command, or a @doc()
command, which switch back to document mode.

5.4. Scripting
--------------

Traditional literate programming tools have two
conceptual processes: weaving (a document) and tangling
(a code file) which separate out interleaved document
and code sections.

Interscript has a third kind of section, the _script
section. _In case you're wondering what script sections
look like, well, you've already used them. All those
lines starting with @ are just executable python
script. They aren't really special magical commands,
just function calls to predefined python functions.

You can write any python script you like in a script
section. Like this:

  @name = 'John Skaller'
  @print 'Hello',name
  @print 'Running Python',sys.version

Notice you don't have to import sys: it is already
imported, because it is used in Interscript.

You should be careful with this feature. It is
immensely powerful! You can use it to test programs,
and to extend Interscript for you needs in a
_particular _document -- without changing the actual
source code for Interscript. See http://www.python.org
to find out more about python.

5.4.1. Long script sections
---------------------------

You can code long script sections such as a class
definition. The rule is: a long script section is
started by a line starting with @ and ending with :,(
or one of the characters that python would recognize
that signifies that there is more to come. You must
then indent the code with exactly one extra space. A
long script section is ended by the first line not
having a space character in column 1 (or the end of
file).

The whole of a long script section is collected and
then executed at once.

Errors in script sections are reported with a traceback
but do not halt processing. You cannot terminate an
Interscript processing run inside a script section, not
even with sys.exit().

5.4.2. Very Long script sections
--------------------------------

There's a better way to code long script sections,
using the "python()" command. Here's an example:

  @python('//')
  x = 1
  y = 2
  //

The 'python' command accepts a string argument which is
a terminator line for the script section. The whole
section is gathered, without any processing, and then
executed. There is a danger to be aware of: if you
don't put the terminator in correctly, the command will
read all the way to the end of the file.

5.4.3. Unit tests
-----------------

It is possible to test python script 'on the fly' as in
the example:

  @test_python(hlevel=2,descr='A simple test',source_terminator='//')
  print 'A simple test'
  //

which results in:

5.5. Test 1: A simple test
--------------------------

On-the-fly python test script follows.   328: print 'A simple test'
Actual output follows.

Start output section of /usr/local/bin/python /usr/tmp/@13003.1200_test.py

     1: A simple test

End output section of /usr/local/bin/python /usr/tmp/@13003.1200_test.py

The test is also registered in a table of tests.

It is also possible to provide expected output:
Interscript will verify your code by comparing the
expected and actual output, and print a difference
table if the test failed. Note that the difference
table is only available if the module
"interscript.utilities.diff" is available and operates
correctly: the current implementation uses GNU diff
invoked using "os.system()".

5.5.1. Weaver Control
---------------------

The python function 'get_weaver()' refers to the
current weaver. You can use it to call methods directly
on the weaver. For example:

  @weaver = get_weaver()
  @weaver.write('Antidis')
  @weaver.begin_bold()
  @weaver.write('establishmentarianism')
  @weaver.end_bold()
   is a long word. Note the space on this line!

which comes out as Antidisestablishmentarianism is a
long word. Note the space on this line!

You can also set the weaver. Suppose you have a weaver
mynotes_weaver, then you can write:

  @old_weaver = get_weaver()
  @set_weaver(mynotes_weaver)
  This is woven into notes.
  @set_weaver(old_weaver)

To make this more convenient, the set_weaver function
returns the current weaver so you can write:

  @old_weaver = set_weaver(mynotes_weaver)
  This is woven into notes.
  @set_weaver(old_weaver)

Even more convenient, you can push and pop weavers onto
a stack using

  @push_weaver(mynotes_weaver)
  This is woven into notes.
  @pop_weaver()

The current weaver is lexically scoped.

5.5.2. Perl hates @
-------------------

If you are programming Perl (or Interscript!) you will
hate having @ as the warning character for script
sections. There are two ways around this.

You can use two @ characters at the beginning of a
line.

  @@p = @x

or you can use a command like:

  @set_warning_character(python='!')

which will set the python warning character to !
instead of @. Advanced Note. This change applies only
to the current file, and only to the end of the
containing block, if any. The effect will not be passed
up to an including file, and it won't be inherited by
an included file either.

5.6. Fonts
----------

The following commands change font. Note that begin/end
pairs must be balanced, and nesting may or may not be
supported, depending on the weaver.

  @begin_emphasize()
  @end_emphasize()

  @begin_strong()
  @end_strong()

  @begin_code()
  @end_code()

  @begin_small()
  @end_small()

  @begin_big()
  @end_big()

  @begin_italic()
  @end_italic()

  @begin_bold()
  @end_bold()

5.7. Lists
----------

The following commands build lists. Note that begin/end
pairs must be balanced, and nesting is supported. There
are three kinds of lists: numbered lists, which are
numbered automatically, bullet lists, which display
some special bullet symbol, and keyed lists, which
display a string of text for each item.

  @begin_numbered_list(start=1):
  @end_numbered_list():
  @begin_numbered_list_item():
  @end_numbered_list_item():

  @begin_bullet_list():
  @end_bullet_list():
  @begin_bullet_list_item():
  @end_bullet_list_item():

  @begin_keyed_list():
  @end_keyed_list():
  @begin_keyed_list_item(key):
  @end_keyed_list_item():

Here's an example:

  @begin_keyed_list()
  @begin_keyed_list_item('bullet')
  A bullet or similar character at the start of each item.
  @end_keyed_list_item()
  @begin_keyed_list_item('numbered')
  A number at the start of each item.
  @end_keyed_list_item()
  @begin_keyed_list_item('keyed')
  A key, or definition term, at the start of each item.
  @end_keyed_list_item()
  @end_keyed_list()

which comes out like:

bullet 
     A bullet or similar character at the start of each
    item.

numbered 
     A number at the start of each item.

keyed 
     A key, or definition term, at the start of each
    item.

5.7.1. Easier lists
-------------------

By default, Interscript installs a special weaver
called 'multiplexor' which delegates commands to zero,
one, or more weavers. This weaver also supports
simplified list definitions. Here's the example above,
simplified.

  @begin_list('keyed')
  @item('bullet')
  A bullet or similar character at the start of each item.
  @item('numbered')
  A number at the start of each item.
  @item('keyed')
  A key, or definition term, at the start of each item.
  @end_list()

5.8. Tables
-----------

Interscript supports tables, although currently the
support is fairly primitive. Here's how to create a
table:

  @begin_table('Column 1','Column 2','Column 3')
  @table_row('Data 11', 'Data 12','Data13')
  @table_row('Data 21', 'Data 22','Data23')
  @end_table()

which looks like this:
+----------+----------+----------+
| Column 1 | Column 2 | Column 3 |
+----------+----------+----------+
| Data 11  | Data 12  | Data13   |
| Data 21  | Data 22  | Data23   |
+----------+----------+----------+

5.9. Citations
--------------

You can cite a URL like:

  @cite_url('http://www.triode.net.au/~skaller')

which will appear as a hyper link in HTML files like
http://www.triode.net.au/~skaller.

5.10. Cross References
----------------------

Interscript supports intra-document (internal) cross
referencing using the commands:

  @set_anchor('MyLabel')
  ...
  please see
  @ref_anchor('MyLabel')
   for details.

The label must be a string, and is currently required
to be a valid identifier since it is used literally by
the HTML weaver in an anchor tag. Latex imposes no such
restrictions, nor does the plain text weaver. For HTML,
the label is set as an anchor; for latex, the page
number is given, for plain text the line number.

Note that inter-document (external) cross references
are different to intra-document cross references. For
truly external references to existing published works,
use citations or bibliographic references. References
across volumes of a work or project are not yet
supported.

5.11. Including Files
---------------------

Interscript allows you to include Interscript files
with the command

  @include_file(filename)

When you do this, you should be aware that it is
treated like a subroutine call: a stack frame is
created, and any symbols bound in script sections are
local to the file. In addition, various parameters are
localized. Therefore, you cannot define a new command
or variable in an include file and expect it to persist
past the end of the file.

5.11.1. Including code
----------------------

You can include existing code directly to the current
tangler like:

  @select(py)
  @include_code(filename)

This is not the same as including an Interscript file.
The contents of the code inclusion 'filename' are
copied verbatim to the tangler 'py'. Leading @
characters are not detected. The contents of the file
are still woven into the document.

5.11.2. Displaying Code
-----------------------

You can display a file as code like this:

  @display_code(filename)

This is very useful for printing the results of a
processing run, or weaving example programs into a
book.

5.12. Translating Html
----------------------

Interscript normally reads Interscript. But it can also
read (a small subset of) HTML. The command:

  @include_html(filename)

will read an HTML file and translate the tags to
Interscript. In this manner, you can convert flat HTML
into stacked HTML, or into Latex or plain text.

5.13. Special constructions
---------------------------


5.13.1. Table of Contents
-------------------------

To print the table of contents, you can say:

  @print_table_of_contents()

For Latex, the native table of contents construction is
used. For flat HTML, the weaver generates a hyper
linked table of headings, but two passes are required
to get it right. For plain text, two passes are also
required.

For stacked HTML, a separate contents page is created
automatically. For this reason, the table of contents
command is disabled for that weaver.

5.13.2. Identifier Index
------------------------

This is table of all the identifiers used in your
programs. For HTML, the entries are hyper linked to
each occurrence of the identifier. Finding identifiers
is the task of tanglers. At this time, the Python
tangler can find most of them by tokenising python
script. None of the other tanglers support this feature
properly yet.

  @print_identifier_cross_reference()

5.13.3. File list
-----------------

A list of all the generated files. Use

  @print_file_list()

to generate it. You can use this to assemble the
generated files into a tar ball.

5.13.4. Source list
-------------------

A list of all the input files. Use

  @print_source_file_list()

to generate it. This command is useful so you know what
files are required for a package.

5.13.5. The Web Weaver
----------------------

Here are some special features of the 'web' weaver.

5.13.5.1. Automatic table generation
------------------------------------

As of version 1.0a6, the web weaver automatically
produces a number of tables: the table of contents,
index of classes, index of functions, index of
identifiers, index of unit tests, and file convergence
status report. None of these tables can be disabled at
present.

5.13.5.2. Mandatory Frames
--------------------------

The web weaver also produces several framesets. There
is no 'noframes' option at present.

5.13.5.3. Internet Explorer DHTML support
-----------------------------------------

In addition, the table of contents uses ECMAscript to
determine if it is running on a version Microsoft
Internet Explorer; if so, it enables dynamic HTML
features unique to the Microsoft object model which
permit dynamic expansion or contraction of the table of
contents tree at each branch.

5.13.5.4. Cascading Style Sheets
--------------------------------

Both the web and html weavers uses CLASS attributes in
tags, and the web weaver in particular makes fairly
heavy use of the generic DIV and SPAN tags.

A standard Cascading Style Sheet called Interscript.css
can be found in the directory interscript/doc. It
colours various elements in a suggestive way. Do not
change interscript.css; instead, supply user.css; it
should override interscript.css even if your browser
finds both files.

5.14. File Names
----------------

The names of files used in interscript documents should
be relative pathnames obeying the Unix convention, even
on other platforms such as NT or the Mac: separate
components with a / character. Don't use silly
characters such as : in components names.

I plan to upgrade the file naming convention to use
URLs with 'interscript' addressing scheme, in which the
'network' component is treated as a logical location
identifier; the client will map these locations to
physical ones.

The current version of interscript does not provide
this mechanism yet. Instead, there are four command
line options:

  --weaver-prefix=nativepath
  --tangler-prefix=nativebspath
  --weaver-directory=relpath
  --tangler-directory=relpath

where the nativepath is a prefix in native operating
system format, and the relpath is a prefix in Unix
format. For an interscript file given as 'basename',
the resulting actual filename is:

  abspath+ (string.join(string.split(relpath+basename,'/').os.sep)

Note that if you use 'a' and 'b' as the prefix and
directory a filename base will be called 'abbase': no
separators are put between the prefix, directory and
base. Here's an example for Windows:

  python iscr.py                        \
    --tangler-prefix=c:\mydevelopment\  \
    --tangler-directory=code/           \
    example.pak

Note that interscript creates directories automatically
for the 'Unix' part of the filename, but _not_ the
native prefix. Thus in the example 'c:\mydevelopment'
must exist, whereas 'code'is created within it
automatically. If example.pak tangles a file
'package/module.py', then 'package' is also created
automatically.

5.15. Questions and Answers
---------------------------


5.15.1. Why are HTML tags printed verbatim, even by the html weaver?
--------------------------------------------------------------------

Weavers are _supposed_ to print all printable
characters literally. So tagged text such as
<EM>tagged</EM> is printed exactly as you wrote it.

5.15.2. OK, so how do I put HTML into a document?
-------------------------------------------------

If you want to write specialised HTML and put it into a
document, there are several ways to do it.

5.15.2.1. Tag method
--------------------

This is the prefered method. You say:

  @weaver.rawif('html')
  '<B>raw html</B>'
  @weaver.enable()

What this does is disable the weaver except unless it
is tagged 'html'. Then you raw-write the html, and
finally re-enable the weaver. This works even when you
have multiple weavers configured with filters, because
the multiplexor and markup filters delegate these
commands to their clients.

You add tags to a weaver with anything by saying

  @weaver.add_tag(something)

The built in weavers are tagged 'text', 'html' 'latex'
and 'raw' as appropriate.

I plan to make raw writing a bit more convenient!

5.15.2.2. Cheat method 1
------------------------

If you know you have a particular weaver set, you can
just raw write to it like

  @weaver._write('<B>raw html</B>')

and that will go straight to the output device. It will
look funny if you have a Latex weaver set! You can also
say:

  @weaver.raw()
  '<B>raw html</B>'
  @weaver.translate()

to force the current weaver into raw mode and back to
translation mode.

5.15.2.3. Cheat method 2
------------------------

What you do is construct a raw weaver bound to the same
sink device as your html weaver, and then set it to be
the current weaver. This weaver thinks it is writing a
raw data file, and copies all input straight to the
sink device. So your HTML tags will go straight through
to the output file, uninterpreted.

5.15.3. How can I use existing HTML as a source?
------------------------------------------------

You can construct an _input filter_ that parses HTML,
translating into to Interscript. The Interscript parser
then processes this data, calling the current weaver
with appropriate methods. If the weaver is an html
weaver, it will then translate the calls back to html
tags.

The advantage of this method, however, is that if the
current weaver is, say, a Latex weaver, then the output
will be the Latex equivalent of the original HTML. In
other words, you will have translated the original HTML
into Latex.

5.15.4. Isn't it hard to create an HTML filter?
-----------------------------------------------

No, it is very easy, because a class,
html_input_filter, to do exactly this is standard in
Interscript. This class uses the Python standard
library module htmllib to parse the input.

5.15.5. Why are blank lines ignored?
------------------------------------

They're not, they're converted to spaces, which is the
HTML convention.

5.15.6. Why do I get multiple spaces in my document?
----------------------------------------------------

This is probably a bug in Interscript. In general,
multiple spaces in input files are passed on to the
typesetter 'as is'. HTML and Latex will reduce these to
a single space automatically, but not all typesetters
do this.

Generally, weavers write lines out as they came in, in
other words, a word at the end of one line will be
separated from a word starting the next line because
the weaver writes out two lines, and the typesetter
replaces them with a space.

However, there are some nasty cases. In the html
weaver, the @begin_bold() and @end_bold() commands
write out the <B> and </B> tags without any spaces
before or after. But the text to be boldened is always
a sequence of lines, so there will always be a newline
before the </B> tag. So there's no easy way to bold a
single letter of a word using the bold command in an
Interscript input.

However, this is _not_ the case if you implement a
filter, because the filter can write out the argument
using the weaver method write(), rather than
writeline().

5.15.7. The indentation of my programs is all wrong. Why?
---------------------------------------------------------

Interscript does not allow tabs. You should never ever
use tabs, they're an extremely stupid idea. Interscript
replaces tabs by spaces automatically. This feature
_cannot_ be disabled (without hackery), because it is
implemented in the main control loop.

The variable g.tabwidth can be changed from the default
value of 2 (which is suitable for publication) to some
other value (such as 8) with a command like @g.tabwidth
= 8.

You can, and sometimes must, generate tabs in code
files. For example, tabs are required in 'make' files.
You can do this in many ways, for example by
implementing a front end for a tangler that replaces
leading spaces, or perhaps \t sequence, by tabs.

5.15.8. How do I get existing code into Interscript?
----------------------------------------------------

Generally, commands turn off tangling. However, the
command @include_code(filename) leaves the current
tangler enabled. This causes the contents of the file
to be typeset as code (even if it contains leading @
characters) _and_ included in the output for the
tangler.

5.15.9. What if I just want to print it?
----------------------------------------

If you don't want to write code to any output file,
just use a tangler with a null sink device. That device
just throws away all the data written to it. For
example:

Start C section to null sink[1]

     1: #line 128 "iscrqa.ipk"
     2: @from interscript.tanglers.c import c_tangler
     3: @from interscript.drivers.sinks.null import null_sink
     4: @dummy = c_tangler(null_sink(), weaver)
     5: @select(dummy)
     6: /* These comments do not get written out */


End C section to null sink[1]

5.15.10. How do I debug Interscript
-----------------------------------

This is a very hard question. With some vexation, I
have decided to make Interscript robust rather than
fragile. (Generally, this is a bad idea).

First, the standard weaver is an object of the class
multiplexor. That object is a proxy which dispatches
method calls to every weaver in a list of delegates,
but only if the delegate supports that method.

This means that the default behaviour of a method is to
do nothing, which seems appropriate for weavers. For
example the plain text weaver cannot do bolding, so
rather than implement a 'pass' method, you just don't
bother to implement the method at all.

When the Python engine detects an unexpected exception,
usually coming from Python script in your document, but
sometimes from I/O failures, and sometimes from bugs,
Interscript attempts to put debugging information into
the current document and continue. So, for example, if
you try to include a file that doesn't exist, you'll
get a message in the current document.

There are two excuses for doing this. The first is that
it allows the rest of the process to continue, in
particular, it prevents errors in documentation
constructions holding up code development.

This feature was crucial in developing Interscript
itself for the following interesting reason: when I
added a new feature to a weaver I would have to use the
existing version of Interscript to tangle the code
containing the implementation of that feature. If I
tried to use that feature in the documentation, it
would always crash, because the feature wasn't
implemented in the old version.

So, I would build the new version, then start using the
new constructions. The problem was that there would be
bugs in the implementation. If I just fixed the bugs,
I'd be trying to tangle the code using the old bugged
version, which would crash. On the other hand, if I
used an older backup version of the program, it would
also crash because there was _no_ implementation of the
construction.

Developing a program generator which generates itself
is an excellent way to test the program generator --
but there it is sometimes difficult to recover from
faults!

So, I decided that documentation errors shouldn't stop
the system generating code, as, in my case, the fixes
would be in the code I was trying to generate!

Instead, error messages are printed in the
documentation if possible. One excuse for this is that
such messages are easy to find, and where they occur is
a good indication of the fault. Separate logs are much
harder to use, and if the process is to continue in the
presence of a documentation fault, there's no harm from
corrupting an already corrupt document.

One downside of this features comes if you don't read
your documents: you may not notice an error. Another
downside is that a corrupt weaver may not be able to
print the error into the document, and it may not be
there to read.

I'd be very interested hearing reports of how this
facility trips people up: I'm sure it will, but not
sure of the details. (Putting errors after a corrupt
'</HTML>' tag, for example, means some browsers will
never show the error!)

Most diagnostics are also written to a log file and
standard output.

NOTE: the details described herein are currently not
properly implemented!

6. Interscript Package Source
-----------------------------

Literate programmed by itself, of course!

6.1. Interscript Module
-----------------------

The top level interscript module.

Start data section to interscript.README[1]

     1: The literate programming tool Interscript
     2:  version '1.0a7'
     3:  has been unpacked here.
     4: It consists of two text files, and a directory:
     5: 
     6:   interscript.README (this file)
     7:   interscript.pth    (the Python package control file)
     8:   interscript/       (the interscript package directory)
     9: 
    10: Find the basic HTML documentation at
    11: 
    12:   interscript/doc/iscrp.html
    13: 
    14: Find the enhanced HTML documentation at
    15: 
    16:   interscript/doc/iscr_top.html
    17: 
    18: Find the text documentation at
    19: 
    20:   interscript/doc/iscr.txt
    21: 
    22: Find the latex2e documentation at
    23: 
    24:   interscript/doc/iscr.tex
    25: 
    26: Find the interscript home page at
    27: 
    28:   http://www.triode.net.au/~skaller/interscript
    29: 


End data section to interscript.README[1]

Start data section to interscript.pth[1]

     1: interscript


End data section to interscript.pth[1]

Start python section to interscript/__init__.py[1]

     1: #line 228 "interscript/src/iscr.pak"
     2: # interscript package
     3: #line 234 "interscript/src/iscr.pak"
     4: print "Interscript Package: version 1.0a7",
     5: #line 234 "interscript/src/iscr.pak"
     6: print "build 1302"


End python section to interscript/__init__.py[1]

6.1.1. Construct Global Frame
-----------------------------

This frame is shared between all processes, and is
initialised at module load time. It hooks crucial
resources and identification information. The
attributes of the global frame class are used as the
globals() dictionary for executing user scripts.

[This is probably a bad idea, because it allows the
user to change the attributes using the global
declaration. On the other hand, it provides a method
for sharing between processes.]

For some weird reason, the global frame is a python
class, not a module, and not a class instance. It's not
a plain module, because python termination is somewhat
indeterminate, and it isn't an instance, because
there's only ever one of them. Clients of the global
frame keep a reference to it explicitly to prevent
premature deletion by the python run time on program
termination.

The global fram in turn keeps references to a set of
important resources, so that they're not deleted
prematurely either. I'm doing this because interscript
__del__ methods are often used to do substantial work,
and it's imperative that system resources are available
until all dynamically created objects are destroyed.

Start python section to interscript/__init__.py[2]

     7: #line 259 "interscript/src/iscr.pak"
     8: class global_frame:
     9: 
    10:   from interscript.drivers.sinks.bufdisk import named_file_sink
    11:   from interscript.drivers.sinks.disk import simple_named_file_sink
    12:   from interscript.drivers.sinks.null import null_sink
    13: 
    14:   from interscript.drivers.sources.base import eoi, eof
    15:   from interscript.drivers.sources.disk import named_file_source
    16:   from interscript.drivers.sources.url import url_source
    17:   from interscript.drivers.sources.ftp import ftp_file_source
    18:   from interscript.drivers.sources.http import http_file_source
    19: 
    20:   from interscript.weavers.text import plain_text_weaver
    21:   from interscript.weavers.latex import latex_weaver
    22:   from interscript.weavers.html import html_weaver
    23:   from interscript.weavers.raw import raw_weaver
    24:   from interscript.weavers.web import stacking_weaver
    25:   from interscript.weavers.auto import auto_weaver
    26:   from interscript.weavers.filter import markup_filter
    27:   from interscript.weavers.multiplexor import multiplexor
    28: 
    29:   from interscript.parsers.html import sgml_wrapper, html_filter
    30: 
    31:   import sys
    32:   import os
    33:   import string
    34:   import re
    35:   import time
    36:   import commands
    37:   from interscript.core.sets import set
    38:   from interscript.core.stacks import stack
    39:   import interscript.core.protocols
    40:   protocol = interscript.core.protocols
    41:   import getoptions
    42: 
    43:   import __builtin__
    44:   __builtins__ = __builtin__
    45:   del __builtin__
    46: 
    47:   try:
    48:     import thread
    49:     print 'thread available'
    50:   except:
    51:     print 'thread NOT available'
    52: 
    53: 


End python section to interscript/__init__.py[2]

6.1.2. Set version
------------------

This is an experimental release only, intended as an
act of advocacy and to solicit comments and support.

The code below is tricky to understand. There are three
distinct sets of version information: data describing
the version being generated, data describing the
version doing the generating, and data describing the
version which generated the generator.

Python script at the top of this file computes or fixes
the version information for the version being
generated, that information is bound into the source of
the generated version.

When that version is itself used to generated yet
another version the data bound into it by the previous
generation describes the currently executing version,
which is now the generator.

When the version being generated is yet again used to
generate another version, the original data now
describes the version that generated the version
generating the new version.

In order to make it easy to add or change
identification attributes, they're initialised to dummy
values, in case the generating version doesn't has the
same set of values. This is always the case when a new
attribute is introduced: the attribute won't make it
into the generating version until a second pass is
executed and the version being generated itself becomes
the generator.

Note that knowing the version that generated the
executing version is important for bug tracking, since
a problem may be due to a bug in the source for the
version, or in the generator which processed it: a bug
can persist even when the source is correct for many
generations if it is not processed correctly by the
generator. Bootstrapped code can be a real nightmare to
debug.

Start python section to interscript/__init__.py[3]

    54: #line 343 "interscript/src/iscr.pak"
    55: # first a hack to help bootstrapping work
    56: # if any of the variable in the second section don't exist.
    57: # then the at least some value is set in the generated code.
    58: # Iterated bootstrapping should eventually fix the problem.
    59: 
    60:   buildno=0
    61:   version=0
    62:   hostname="unknown"
    63:   username="unknown"
    64:   buildtime="unknown"
    65:   generator_buildno=0
    66:   generator_hostname="unknown"
    67:   generator_username="unknown"
    68:   generator_version="unknown"
    69:   generator_buildtime="unknown"
    70: 
    71: # now the real data
    72:   buildno=1302
    73:   version='1.0a7'
    74:   hostname='ruby'
    75:   username='root'
    76:   buildtime='Thu Nov 12, 1998 at 12:57 PM (UTC)'
    77:   generator_buildno=1300
    78:   generator_hostname='ruby'
    79:   generator_username='root'
    80:   generator_version='1.0a7'
    81:   generator_buildtime='Thu Nov 12, 1998 at 12:49 PM (UTC)'
    82: 
    83: # now print the current version information
    84: # wrapped in try/except clause in case any of the variables didn't get set
    85: try:
    86:   print 'Interscript version',global_frame.version,
    87:   print 'build',global_frame.buildno
    88:   print 'Built by',global_frame.username,
    89:   print 'on',global_frame.hostname,
    90:   print 'at',global_frame.buildtime
    91:   print 'Generated by',global_frame.generator_version,
    92:   print 'buildno',global_frame.generator_buildno,
    93:   print 'host',global_frame.generator_hostname
    94:   print 'at',global_frame.buildtime
    95: except: pass
    96: 
    97: # This is a utility function that makes it easy to use interscript
    98: # givem options in a standard form. The arguments are a list as
    99: # would be entered on a unix or nt command line.
   100: # Mac (or Tkinter) users can create a GUI interface to set the options
   101: # and then call this function to run interscript.
   102: 
   103: def run_from_options(arguments):
   104:   from interscript.getframes import getoption_frames
   105:   from interscript.frames.processf import process_frame
   106:   process_options, master_options = getoption_frames(arguments)
   107:   process = process_frame(global_frame, process_options, master_options)
   108:   process.run()
   109:   del process
   110: 


End python section to interscript/__init__.py[3]

6.1.3. Listing of iscr.pak source
---------------------------------

Here is the fully listings of the top level interscript
source file, iscr.pak.

@# Interscript

@# set to 1/0 to enable/disable debugging
@debug_version = 1

@# --------------------------------------------------------------------
@# Get standard modules.
@import os
@import time
@from interscript.weavers.multiplexor import multiplexor

@# --------------------------------------------------------------------
@# Set the source version.
@gen_version = '1.0a7'

@# --------------------------------------------------------------------
@# Get the build number. Update it, but only in pass 0.
@try:
   gen_buildno = 0
   temp = open('iscrbldn.dat','r')
   line = temp.readline()
   line = line[:-1]
   gen_buildno = int(line)
   temp.close()
 except:
   pass

@if get_pass_frame().passno == 0:
   gen_buildno = gen_buildno + 1
   temp = open('iscrbldn.dat','w')
   temp.write(str(gen_buildno)+'\n')
   temp.close()
   del temp

@# --------------------------------------------------------------------
@# Try to get the hostname.
@# Should work on Unix, but don't worry if it fails.
@gen_hostname = 'unknown'
@try:
   f = os.popen('hostname','r')
   gen_hostname = string.strip(f.readline())
   f.close()
 except:
   pass

@# --------------------------------------------------------------------
@# Try to get the username.
@# Should work on Unix, but don't worry if it fails.
@gen_username = 'unknown'
@try:
   gen_username = os.environ['USERNAME']
 except:
   pass

@# --------------------------------------------------------------------
@# Print the current working directory.
@try:
   print 'Current Directory is',os.getcwd()
 except:
   pass

 #print 'Python Path is'
 #for path in sys.path:
 #  if path == '': print '  '+os.curdir+' ('+os.getcwd()+')'
 #  else: print ' ',path

@# --------------------------------------------------------------------
@# Get the build time, but only during pass 0.
@if get_pass_frame().passno == 0:
   get_master_frame().now = time.time()
@now = get_master_frame().now
@localtime = time.localtime(now)
@utctime = time.gmtime(now)
@lfmtime = time.strftime('%a %b %d, %Y at %I:%M %p (%Z)',localtime)
@ufmtime = time.strftime('%a %b %d, %Y at %I:%M %p (UTC)',utctime)

@# --------------------------------------------------------------------
@# Get the identification information from the currently
@# executing version of interscript.
@global_frame = get_process_frame().global_frame
@print 'This is Interscript',global_frame.version,'build',global_frame.buildno
@print 'built on',global_frame.generator_hostname,'by',global_frame.generator_username
@print
@print 'Running on',gen_hostname,'by',gen_username,'at',lfmtime
@print "and building Interscript",gen_version,'build',gen_buildno
@print

@author = 'John Max Skaller'
@title = 'Interscript'
@pfx = get_master_frame().weaver_directory

@# --------------------------------------------------------------------
@# Use the 'weavers' user option to decide whether to generate documentation
@# of not. Issue a warning to the user if no weaver is set.
@# Also, warn if '--weaver=' is used instead of '--weavers'

@weavers = []
@weaver_opt = []
@def print_weaver_help():
   print '  Please use "--weavers=[\'html\',\'latex\']" (etc) instead of "--weaver=".'
   print '  Note: you may need to escape the quotes (or quote the argument).'
   print '  The argument must be an evaluable python list of strings,'
   print '  or the string "all".'

@if get_master_frame().autoweave != []:
   print 'WARNING: Interscript source uses document defined weavers.'
   print_weaver_help()
 else:
   useropt = get_master_frame().useropt
   if not useropt.has_key('weavers'):
     print 'No Weaver in use.'
     print_weaver_help()
   else:
     weaver_opt  = useropt['weavers']
     try:
       weaver_opt = eval(weaver_opt)
     except:
       print ' ERROR: The argument "'+weaver_opt+'" to --weavers= must be a python list of strings.'
       print_weaver_help()

@if weaver_opt == 'all':
   weaver_opt = ['html','text','web','latex']

@if 'text' in weaver_opt:
   plain_text_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscr.txt')
   plain = plain_text_weaver(get_pass_frame(),plain_text_output,title=title,author=author)
   weavers.append(plain)
   print 'Generating text document.'

@if 'html' in weaver_opt:
   simple_html_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscrp.html', eol='\r\n')
   simple_html = html_weaver(get_pass_frame(),simple_html_output,title=title,author=author)
   weavers.append(simple_html)
   print 'Generating simple HTML document.'

@if 'web' in weaver_opt:
   stacked_html_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscrs.html', eol='\r\n')
   stacked_html = html_weaver(get_pass_frame(),stacked_html_output,title=title,author=author)
   stacked = stacking_weaver(stacked_html,'iscr_%s.html',(1,2,3,4,5,6,7,99),toc_depth=3)
   weavers.append(stacked)
   print 'Generating complex WEB/HTML document.'

@if 'latex' in weaver_opt:
   latex_output = named_file_sink(get_pass_frame(),pfx+'interscript/doc/iscr.tex')
   latex = latex_weaver(get_pass_frame(),latex_output,title=title,author=author, heading_level_offset=1)
   weavers.append(latex)
   print 'Generating latex document.'

@# terminate the if :-)
@multiplex = multiplexor(get_pass_frame(),weavers)
@emph = markup_filter(get_pass_frame(),"'([^']+)'",'begin_emphasize','end_emphasize',[multiplex])
@code = markup_filter(get_pass_frame(),'"([^"]+)','begin_code','end_code',[emph])
@master_weaver = code
@set_weaver(master_weaver)

@# --------------------------------------------------------------------
@# Begin construction of the new version of interscript.
@head(1,'Interscript Version')
@# --------------------------------------------------------------------
@# Print information identifying the version of intercsript which
@# is being generated, and the version which is generating it.
@try:
   weave('This document describes Interscript version '+gen_version)
   weave(' build '+str(gen_buildno))
   weave(' on '+gen_hostname+' ')
   weave(' by '+gen_username+'.')

   weave('It was generated by Interscript version '+global_frame.version+', build '+str(global_frame.buildno))
   weave(' on '+global_frame.hostname)
   weave(' by '+global_frame.username)
   weave(' at '+global_frame.buildtime+'.')
 except: pass

@# --------------------------------------------------------------------
@# Print various tables.
@p()
@print_contents(hlevel=2,maxlev=2)

@# --------------------------------------------------------------------
@# Print the introduction and design principles chapters.
@include_file('introduction.ipk')
@include_file('requirements.ipk')
@include_file('design.ipk')
@include_file('tutorial.pak')


@# --------------------------------------------------------------------
@# Generate the source code and program documentation.
@head(1,'Interscript Package Source')
Literate programmed by itself, of course!

@head(2,'Interscript Module')
The top level interscript module.
@select(output('interscript.README'))
The literate programming tool Interscript
@tangle(' version '+repr(gen_version))
 has been unpacked here.
It consists of two text files, and a directory:

  interscript.README (this file)
  interscript.pth    (the Python package control file)
  interscript/       (the interscript package directory)

Find the basic HTML documentation at

  interscript/doc/iscrp.html

Find the enhanced HTML documentation at

  interscript/doc/iscr_top.html

Find the text documentation at

  interscript/doc/iscr.txt

Find the latex2e documentation at

  interscript/doc/iscr.tex

Find the interscript home page at

  http://www.triode.net.au/~skaller/interscript

@select(output('interscript.pth'))
interscript
@iscr_pkg = python_output('interscript/__init__.py')
@select(iscr_pkg)
# interscript package
@if debug_version:
  # announce the version when the module is imported.
  tangle('print "Interscript Package: version '+gen_version+'",')
  tangle('print "build '+str(gen_buildno)+'"')

@head(3,'Construct Global Frame')
This frame is shared between all processes, and is initialised
at module load time. It hooks crucial resources and identification
information. The attributes of the global frame class are used
as the globals() dictionary for executing user scripts.
@p()
[This is probably a bad idea, because it allows the user to change
the attributes using the global declaration. On the other hand,
it provides a method for sharing between processes.]
@p()
For some weird reason, the global frame is a python class,
not a module, and not a class instance. It's not a plain module,
because python termination is somewhat indeterminate, and it isn't
an instance, because there's only ever one of them. Clients of the
global frame keep a reference to it explicitly to prevent
premature deletion by the python run time on program termination.
@p()
The global fram in turn keeps references to a set of important resources,
so that they're not deleted prematurely either.
I'm doing this because interscript __del__ methods are often used to
do substantial work, and it's imperative that system resources are
available until all dynamically created objects are destroyed.


@select(iscr_pkg)
class global_frame:

  from interscript.drivers.sinks.bufdisk import named_file_sink
  from interscript.drivers.sinks.disk import simple_named_file_sink
  from interscript.drivers.sinks.null import null_sink

  from interscript.drivers.sources.base import eoi, eof
  from interscript.drivers.sources.disk import named_file_source
  from interscript.drivers.sources.url import url_source
  from interscript.drivers.sources.ftp import ftp_file_source
  from interscript.drivers.sources.http import http_file_source

  from interscript.weavers.text import plain_text_weaver
  from interscript.weavers.latex import latex_weaver
  from interscript.weavers.html import html_weaver
  from interscript.weavers.raw import raw_weaver
  from interscript.weavers.web import stacking_weaver
  from interscript.weavers.auto import auto_weaver
  from interscript.weavers.filter import markup_filter
  from interscript.weavers.multiplexor import multiplexor

  from interscript.parsers.html import sgml_wrapper, html_filter

  import sys
  import os
  import string
  import re
  import time
  import commands
  from interscript.core.sets import set
  from interscript.core.stacks import stack
  import interscript.core.protocols
  protocol = interscript.core.protocols
  import getoptions

  import __builtin__
  __builtins__ = __builtin__
  del __builtin__

  try:
    import thread
    print 'thread available'
  except:
    print 'thread NOT available'


@head(3,'Set version')
This is an experimental release only,
intended as an act of advocacy and to solicit comments and support.
@p()
The code below is tricky to understand. There are three distinct
sets of version information: data describing the version
being generated, data describing the version doing the generating,
and data describing the version which generated the generator.
@p()
Python script at the top of this file computes or fixes
the version information for the version being generated,
that information is bound into the source of the generated
version.
@p()
When that version is itself used to generated yet another
version the data bound into it by the previous generation
describes the currently executing version, which is now
the generator.
@p()
When the version being generated is yet again used to generate
another version, the original data now describes the version
that generated the version generating the new version.
@p()
In order to make it easy to add or change identification attributes,
they're initialised to dummy values, in case the generating version
doesn't has the same set of values. This is always the case
when a new attribute is introduced: the attribute won't make it into
the generating version until a second pass is executed and the version
being generated itself becomes the generator.
@p()
Note that knowing the version that generated the executing version
is important for bug tracking, since a problem may be due to a bug
in the source for the version, or in the generator which processed
it: a bug can persist even when the source is correct for many generations
if it is not processed correctly by the generator.
Bootstrapped code can be a real nightmare to debug.

@select(iscr_pkg)
# first a hack to help bootstrapping work
# if any of the variable in the second section don't exist.
# then the at least some value is set in the generated code.
# Iterated bootstrapping should eventually fix the problem.

@tangle('  buildno=0')
@tangle('  version=0')
@tangle('  hostname="unknown"')
@tangle('  username="unknown"')
@tangle('  buildtime="unknown"')
@tangle('  generator_buildno=0')
@tangle('  generator_hostname="unknown"')
@tangle('  generator_username="unknown"')
@tangle('  generator_version="unknown"')
@tangle('  generator_buildtime="unknown"')

# now the real data
@tangle('  buildno='+str(gen_buildno))
@tangle('  version='+repr(gen_version))
@tangle('  hostname='+repr(gen_hostname))
@tangle('  username='+repr(gen_username))
@tangle('  buildtime='+repr(ufmtime))
@tangle('  generator_buildno='+str(global_frame.buildno))
@tangle('  generator_hostname='+repr(global_frame.hostname))
@tangle('  generator_username='+repr(global_frame.username))
@tangle('  generator_version='+repr(global_frame.version))
@tangle('  generator_buildtime='+repr(global_frame.buildtime))

# now print the current version information
# wrapped in try/except clause in case any of the variables didn't get set
try:
  print 'Interscript version',global_frame.version,
  print 'build',global_frame.buildno
  print 'Built by',global_frame.username,
  print 'on',global_frame.hostname,
  print 'at',global_frame.buildtime
  print 'Generated by',global_frame.generator_version,
  print 'buildno',global_frame.generator_buildno,
  print 'host',global_frame.generator_hostname
  print 'at',global_frame.buildtime
except: pass

# This is a utility function that makes it easy to use interscript
# givem options in a standard form. The arguments are a list as
# would be entered on a unix or nt command line.
# Mac (or Tkinter) users can create a GUI interface to set the options
# and then call this function to run interscript.

def run_from_options(arguments):
  from interscript.getframes import getoption_frames
  from interscript.frames.processf import process_frame
  process_options, master_options = getoption_frames(arguments)
  process = process_frame(global_frame, process_options, master_options)
  process.run()
  del process

@head(3,'Listing of iscr.pak source')
Here is the fully listings of the top level interscript source file, iscr.pak.
@display_code('iscr.pak')

@head(2,'Core Subpackage')
@select(python_output('interscript/core/__init__.py'))
# interscript core modules

@push_head(2)
@include_file('sets.ipk')     # sets
@include_file('stacks.ipk')   # stacks
@include_file('protocols.ipk')# protocols
@pop_head(2)

@push_head()
@include_file('drivers.ipk')  # drivers
@include_file('weavers.ipk') # weavers
@include_file('weaver_filters.ipk') # weaver filters
@include_file('tanglers.ipk')  # tanglers
@include_file('tokenizers.ipk')  # tokenisers
@include_file('parsers.ipk') # parsers
@include_file('frames.ipk') # architecture
@include_file('options.ipk') # generic option processing
@include_file('interscript_options.ipk') # interscript option processing
@pop_head()


@head(2,'Utility Modules')
@select(python_output('interscript/utilities/__init__.py'))
# interscript utilities

@push_head(2)
@include_file('compilers.ipk')
@include_file('diff.ipk')
@pop_head(2)


@head(2,'Application and tool directory')
This is where the interscript mainline and any other shell commands go.
@select(python_output('interscript/bin/__init__.py'))
# dummy interscript.bin

@head(3,'Stand alone unix/nt mainline')
Command line stub.

@select(python_output('interscript/bin/iscr.py'))
#!/usr/bin/env python
import sys
args = sys.argv[1:]
if len(args)>0 and args[0]=='--test':
  print 'Interscript test mode, loading interscript from current directory'
  if sys.path[0]!='':
    sys.path = ['']+ sys.path
  args = args[1:]
import interscript
interscript.run_from_options(args)

@head(3,'Windows launcher')
A batch file to launch interscript.
You will need to edit the file!

@select(python_output('interscript/bin/iscr.bat'))
python iscr.py %1 %2 %3 %4 %5 %6 %7 %8 %9

@head(2,'Test package')
@select(python_output('interscript/tests/__init__.py'))
# dummy interscript/test

@head(1,'Appendicies')
@# --------------------------------------------------------------------
@# Print the table of identifiers and class reference
@print_identifier_cross_reference(hlevel=2)
@print_class_reference(hlevel=2)
@print_file_list(hlevel=2)
@print_source_list(hlevel=2)
@print_include_list(hlevel=2)
@push_head()
@include_file('bugs.ipk')
@include_file('installation.ipk')
@pop_head()


6.2. Core Subpackage
--------------------


Start python section to interscript/core/__init__.py[1]

     1: #line 405 "interscript/src/iscr.pak"
     2: # interscript core modules
     3: 


End python section to interscript/core/__init__.py[1]

6.2.1. Sets
-----------

This class implements mathematical sets using a
dictionary.

Start python section to interscript/core/sets.py[1]

     1: #line 6 "sets.ipk"
     2: class set:
     3:   __class_protocols__= ['sequence','mutable','set']
     4:   def __init__(self,*args):
     5:     self.s = {}
     6:     for e in args: self.s[e]=None
     7: 
     8:   # set contains element
     9:   def contains(self,e):
    10:     return self.s.has_key(e)
    11: 
    12:   # we're an improper subset of the rhs
    13:   def le(self,rhs):
    14:     for e in self.s.keys():
    15:       if e not in rhs: return 0
    16:     return 1
    17: 
    18:   # the rhs is an improper subset of us
    19:   def ge(self,rhs):
    20:     for e in rhs:
    21:       if not self.s.has_key(e): return 0
    22:     return 1
    23: 
    24:   def eq(self,rhs):
    25:     return self.le(rhs) and self.ge(rhs)
    26: 
    27:   def ne(self,rhs):
    28:     for e in rhs:
    29:       if not self.s.has_key(e): return 1
    30:     for e in self.s.keys():
    31:       if not e in rhs: return 1
    32:     return 0
    33: 
    34:   def gt(self,rhs):
    35:     return self.ne(rhs) and self.ge(rhs)
    36: 
    37:   def lt(self,rhs):
    38:     return self.ne(rhs) and self.le(rhs)
    39: 
    40:   def min(self):
    41:     return min(self.s.keys())
    42: 
    43:   def max(self):
    44:     return max(self.s.keys())
    45: 
    46:   def index(self,e):
    47:     return self.s.keys().index(e)
    48: 
    49:   def count(self,e):
    50:     return self.s.has_key(e)
    51: 
    52:   # ensure set contains element; no error if already in set
    53:   def insert(self,e):
    54:     self.s[e]=None
    55: 
    56:   # remove element, must be in set or error
    57:   def remove(self,e):
    58:     del self.s[e]
    59: 
    60:   # remove element if in set
    61:   def excise(self,e):
    62:     if self.s.has_key(e): del self.s[e]
    63: 
    64:   # append all the elements in the sequence
    65:   def append_sequence(self,seq):
    66:     for e in seq: self.s[e]=None
    67: 
    68:   # get list of elements
    69:   def list(self):
    70:     return self.s.keys()
    71: 
    72:   # get tuple of elements
    73:   def tuple(self):
    74:     return tuple(self.s.keys())
    75: 
    76:   # get dictionary of elements
    77:   def dict(self):
    78:     return self.s.copy()
    79: 
    80:   # return a copy of this set
    81:   def copy(self):
    82:     s = set()
    83:     s.s = self.s.copy()
    84:     return s
    85: 
    86:   # repr is set(e1, e2, e3) etc
    87:   def __repr__(self):
    88:     keys = self.s.keys()
    89:     p = 'set('
    90:     if keys: p = p + repr(keys[0])
    91:     for key in keys[1:]: p = p + ', ' + repr(key)
    92:     p = p + ')'
    93:     return p
    94: 
    95:   # 0 if empty, 1 otherwise
    96:   def __nonzero__(self):
    97:     return len(s)!=0
    98: 
    99:   # lexicographical comparison!
   100:   # a < b does NOT mean a is a subset of b!!!
   101: 
   102:   def __cmp__(self,other):
   103:     right = set()
   104:     for e in other: right.insert(e)
   105:     k1 = self.s.keys()
   106:     k1.sort()
   107:     k2 = right.s.keys()
   108:     k2.sort()
   109:     return cmp(k1,k2)
   110: 
   111:   def __len__(self):
   112:     return len(self.s)
   113: 
   114:   def __getitem__(self,index):
   115:     return self.s.keys()[index]
   116: 
   117:   def __delitem__(self,index):
   118:     k = self.s.keys()[index]
   119:     del self.s[k]
   120: 
   121:   def __getslice__(self,i,j):
   122:     return apply(set,tuple(self.s.keys()[i:j]))
   123: 
   124:   def __and__(self,right):
   125:     s = set()
   126:     for e in self.s.keys():
   127:       if e in right: s.insert(e)
   128:     return s
   129: 
   130:   def __or__(self,right):
   131:     s = set()
   132:     s.s = self.s.copy()
   133:     for e in right: s.s[e]=None
   134:     return s
   135: 
   136:   def __xor__(self,right):
   137:     s = set()
   138:     for e in right: s.insert(e)
   139:     for e in self.s.keys():
   140:       if s.s.has_key(e): del s.s[e]
   141:       else: s.s[e]=None
   142:     return s
   143: 
   144:   def __add__(self,right):
   145:     return self.__or__(right)
   146: 
   147:   def __sub__(self,right):
   148:     s = set()
   149:     for e in self.s.keys():
   150:       if e not in right: s.insert(e)
   151:     return s
   152: 


End python section to interscript/core/sets.py[1]

6.2.1.1. Test source
--------------------

A simple test of sets.

Start python section to interscript/tests/test_sets.py[1]

     1: #line 163 "sets.ipk"
     2: import interscript.core.sets
     3: set = interscript.core.sets.set
     4: 
     5: s1 = set(1,2,3,1,2)
     6: s2 = set(10,20,30,3)
     7: print 's1=',s1
     8: print 's2=',s2
     9: print 's1|s2=',s1 | s2
    10: print 's1&s2=',s1 & s2
    11: print 's1^s2=',s1 ^ s2
    12: print 's1+s2=',s1 + s2
    13: print 's1-s2=',s1 - s2
    14: print 's1==s1=',s1 == s1.copy()
    15: assert (s1 ^ s2) == ((s1 | s2) - (s1 & s2))
    16: print 's1[:]=',s1[:]
    17: print 's1[1:-1]=',s1[1:-1]
    18: 
    19: print 's1==s2=',s1 == s2
    20: print 's1<s2=',s1 < s2
    21: print 's1>s2=',s1 > s2
    22: 
    23: print 's1.le(s2)=',s1.le(s2)
    24: print 's1.lt(s2)=',s1.lt(s2)
    25: print 's1.ge(s2)=',s1.ge(s2)
    26: print 's1.gt(s2)=',s1.gt(s2)
    27: 
    28: s1 = set(1,2,3,4,5,6,7,8,9)
    29: s2 = set(1,3,5,7,9)
    30: print 's1=',s1
    31: print 's2=',s2
    32: print 's1.le(s2)=',s1.le(s2)
    33: print 's1.lt(s2)=',s1.lt(s2)
    34: print 's1.ge(s2)=',s1.ge(s2)
    35: print 's1.gt(s2)=',s1.gt(s2)
    36: 


End python section to interscript/tests/test_sets.py[1]

6.2.1.2. Test output
--------------------

The result of running the test.

Start output section of /usr/local/bin/python interscript/tests/test_sets.py

     1: Interscript Package: version 1.0a7 build 1253
     2: Using iscrcopt
     3: thread available
     4: Interscript version 1.0a7 build 1253
     5: Built by root on ruby at Mon Nov 09, 1998 at 07:08 PM (UTC)
     6: Generated by 1.0a7 buildno 1252 host ruby
     7: at Mon Nov 09, 1998 at 07:08 PM (UTC)
     8: s1= set(3, 2, 1)
     9: s2= set(30, 20, 3, 10)
    10: s1|s2= set(10, 30, 20, 3, 2, 1)
    11: s1&s2= set(3)
    12: s1^s2= set(30, 20, 10, 1, 2)
    13: s1+s2= set(10, 30, 20, 3, 2, 1)
    14: s1-s2= set(2, 1)
    15: s1==s1= 1
    16: s1[:]= set(3, 2, 1)
    17: s1[1:-1]= set(2)
    18: s1==s2= 0
    19: s1<s2= 1
    20: s1>s2= 0
    21: s1.le(s2)= 0
    22: s1.lt(s2)= 0
    23: s1.ge(s2)= 0
    24: s1.gt(s2)= 0
    25: s1= set(9, 8, 7, 6, 5, 4, 3, 2, 1)
    26: s2= set(7, 5, 3, 9, 1)
    27: s1.le(s2)= 0
    28: s1.lt(s2)= 0
    29: s1.ge(s2)= 1
    30: s1.gt(s2)= 1

End output section of /usr/local/bin/python interscript/tests/test_sets.py

6.2.2. Stack
------------

Simple stack class.

Start python section to interscript/core/stacks.py[1]

     1: #line 5 "stacks.ipk"
     2: class stack:
     3:   __class_protocols__ = ['sequence','mutable']
     4:   def __init__(self, *args): self.s = list(args)
     5:   def pop(self):
     6:     tmp = self.s[-1]
     7:     del self.s[-1]
     8:     return tmp
     9:   def push(self,x): self.s.append(x)
    10:   def append(self,x): self.s.append(x)
    11:   def __len__(self): return len(self.s)
    12:   def __nonzero__(self): return len(self.s)!=0
    13:   def __getitem__(self,index): return self.s[index]
    14:   def __setitem__(self,index,value): self.s[index]=value
    15:   def __delitem__(self,index): del self.s[index]
    16:   def __getslice__(self,i,j): return self.s[i:j]
    17:   def __setslice__(self,i,j,seq): self.s[i:j]=seq
    18:   def __delslice__(self,i,j): del self.s[i:j]
    19:   def __mul__(self,i): return apply(stack, tuple(self.s * i))
    20:   def __rmul__(self,i): return apply(stack, tuple(self.s * i))
    21:   def __add__(self,s): return apply(stack, tuple(self.s + s.s))
    22:   def __cmp__(self, other): return cmp(self.s,other.s)
    23:   def __repr__(self):
    24:     s = 'stack('
    25:     if self.s: s = s + repr(self.s[0])
    26:     for i in self.s[1:]: s = s + ', '+repr(i)
    27:     return s+')'
    28: 
    29:   def __setattr__(self,attr,value):
    30:     if attr == 'top':
    31:       self.s[-1]=value
    32:     else:
    33:       self.__dict__[attr]=value
    34: 
    35:   def __getattr__(self,attr):
    36:     if attr=='top':
    37:       return self.s[-1]
    38:     else:
    39:       raise AttributeError,attr
    40: 
    41:   def __delattr__(self,attr):
    42:     if attr=='top':
    43:       del self.s[-1]
    44:     else:
    45:       raise AttributeError,attr
    46: 
    47:   def copy(self): return apply(stack,tuple(self.s))
    48:   def count(self, item): return self.s.count(item)
    49:   def index(self, item): return self.s.index(item)
    50:   def sort(self, order=None):
    51:     if order == None:
    52:       self.s.sort()
    53:     else:
    54:       self.s.sort(order)
    55:   def insert(self,index,item): self.s.insert(index,item)
    56:   def remove(self,item): self.s.remove(item)
    57:   def reverse(self): self.s.reverse()
    58: 


End python section to interscript/core/stacks.py[1]

6.2.3. protocol module
----------------------

This modules implements protocols. Install in
site-packages.

6.2.3.1. Standard Protocol Rules
--------------------------------

Every object o has the protocol type(o). Every instance
of class X, has protocol X. An object has protocol p,
if p is a class protocol of its class, or any direct or
indirect base of that class.

An object o of type t has protcol p if
add_type_proto(t,p) has_been called. An object o which
is an instance of class c, has protocol p if
add_class_protol(b,p) has been called, where b is c or
one of its direct or indirect bases.

An object o has protocol p if add_obj_proto(o,p) has
been called and returned normally.

It is not possible to attach protocols to an object
using add_obj_proto unless the object either has a
readable attribute '__protocols__' which is a mutable
sequence, or will set an attribute '__protocols__'
(which can subsequently be read).

Please note the crucial distinction between the
protocols of a class object c, which include
c.__protocols__, and the protocols of instances of c,
which include c.__class_protocols__. A class object is
an object in its own right and obeys protocols distinct
from its instances.

Similarly, note that a type object has different
protocols than the instances of that type.

6.2.3.2. Source
---------------


Start python section to interscript/core/protocols.py[1]

     1: #line 38 "protocols.ipk"
     2: #-------------- protocol.py ---------------------------
     3: import types
     4: 
     5: type_protocols = {
     6:   types.NoneType : [],
     7:   types.TypeType : ['type','immutable'],
     8:   types.IntType : ['integer','number','immutable'],
     9:   types.LongType : ['integer','number','immutable'],
    10:   types.FloatType : ['number','immutable'],
    11:   types.StringType : ['string','immutable','filename','url'],
    12:   types.TupleType : ['sequence','immutable'],
    13:   types.ListType : ['sequence','mutable'],
    14:   types.DictType : ['map','mutable'],
    15:   types.FunctionType : ['function'],
    16:   types.LambdaType : ['function'],
    17:   types.CodeType : ['code'],
    18:   types.ClassType : ['class'],
    19:   types.InstanceType : ['instance'],
    20:   types.MethodType : ['function'],
    21:   types.BuiltinFunctionType: ['function'],
    22:   types.ModuleType: ['module'],
    23:   types.FileType: ['file'],
    24:   types.XRangeType: ['range'],
    25:   types.TracebackType: ['traceback'],
    26:   types.FrameType: ['frame'],
    27:   types.SliceType: ['slice'],
    28:   types.EllipsisType: ['ellipsis']
    29: }
    30: try:
    31:   type_protocols[types.ComplexType]='number'
    32: except NameError:
    33:   pass
    34: 
    35: class provides_attr:
    36:   def __init__(self,name):
    37:     self.name = name
    38: 
    39: def isclass(obj):
    40:   return type(obj) is types.ClassType
    41: 
    42: def isinstance(obj):
    43:   return type(obj) is types.InstanceType
    44: 
    45: def classof(obj):
    46:   if isinstance(obj):
    47:     return obj.__class__
    48:   else:
    49:     return None
    50: 
    51: def add_obj_proto(object,protocol):
    52:   if hasattr(object,'__protocols__'):
    53:     getattr(object,'__protocols__').append(protocol)
    54:   else:
    55:     setattr(object,'__protocols__',[protocol])
    56: 
    57: def add_obj_protos(object,protocols):
    58:   for p in protocols: add_obj_protos(object,p)
    59: 
    60: def add_class_proto(cls,protocol):
    61:   if hasattr(cls,'__class_protocols__'):
    62:     getattr(cls,'__class_protocols__').append(protocol)
    63:   else:
    64:     setattr(cls,'__class_protocols__',[protocol])
    65: 
    66: def add_class_protos(object,protocols):
    67:   for p in protocols: add_class_protos(object,p)
    68: 
    69: def add_type_protos(object,protocols):
    70:   for p in protocols: add_type_protos(object,p)
    71: 
    72: def add_type_proto(typ, protocol):
    73:   if type_protocols.has_key(typ):
    74:     type_protocols[typ].append(protocol)
    75:   else:
    76:     type_protocols[typ] = [protocol]
    77: 
    78: def has_class_proto(cls,protocol):
    79:   if cls is protocol: return 1
    80:   if hasattr(cls,'__class_protocols__'):
    81:     if protocol in getattr(cls,'__class_protocols__'): return 1
    82:   return 0
    83: 
    84: def has_type_proto(object,protocol):
    85:   typ = type(object)
    86:   if typ is protocol: return 1
    87:   if type_protocols.has_key(typ):
    88:     if protocol in type_protocols[typ]: return 1
    89:   return 0
    90: 
    91: def has_protocol(object,protocol):
    92:   if hasattr(object,'__protocols__'):
    93:     if protocol in getattr(object,'__protocols__'): return 1
    94: 
    95:   cls = classof(object)
    96:   if cls:
    97:     v = has_class_proto(cls, protocol)
    98:     if v: return 1
    99:     for base in cls.__bases__:
   100:       if has_class_proto(base,protocol): return 1
   101:   if has_type_proto(object,protocol): return 1
   102:   if type(protocol) is types.InstanceType:
   103:     if protocol.__class__ is provides_attr:
   104:       if hasattr(object,protocol.name): return 1
   105:   return 0
   106: 
   107: def has_protocols(object,protocols):
   108:   for p in protocols:
   109:     if not has_protocol(object,p): return 0
   110:   return 1
   111: 


End python section to interscript/core/protocols.py[1]

6.2.3.3. Test
-------------


Start python section to interscript/tests/test_protocol.py[1]

     1: #line 151 "protocols.ipk"
     2: #-------------- test_protocol.py ---------------------------
     3: import types
     4: import interscript.core.protocols
     5: protocol = interscript.core.protocols
     6: assert protocol.has_protocol(types.IntType,types.TypeType)
     7: assert protocol.has_protocol(1,types.IntType)
     8: assert protocol.has_protocol(1,'integer')
     9: 
    10: class B:
    11:   def __init__(self):
    12:     self.fred = 1
    13: 
    14: class D(B): pass
    15: d = D()
    16: 
    17: protocol.add_class_proto(B,'B')
    18: protocol.add_class_proto(D,'D')
    19: protocol.add_obj_proto(d,'d')
    20: protocol.add_type_proto(types.InstanceType,'my instance')
    21: 
    22: assert protocol.has_protocol(d,'B')
    23: assert protocol.has_protocol(d,'D')
    24: assert protocol.has_protocol(d,'d')
    25: assert protocol.has_protocol(d,'instance')
    26: assert protocol.has_protocol(d,'my instance')
    27: assert protocol.has_protocol(d,protocol.provides_attr('fred'))
    28: 


End python section to interscript/tests/test_protocol.py[1]

Start output section of /usr/local/bin/python interscript/tests/test_protocol.py

     1: Interscript Package: version 1.0a7 build 1253
     2: Using iscrcopt
     3: thread available
     4: Interscript version 1.0a7 build 1253
     5: Built by root on ruby at Mon Nov 09, 1998 at 07:08 PM (UTC)
     6: Generated by 1.0a7 buildno 1252 host ruby
     7: at Mon Nov 09, 1998 at 07:08 PM (UTC)

End output section of /usr/local/bin/python interscript/tests/test_protocol.py

6.3. Drivers Subpackage
-----------------------

There are three classes of drivers: source drivers,
sink drivers, and storage drivers.

Start python section to interscript/drivers/__init__.py[1]

     1: #line 6 "drivers.ipk"
     2: # drivers package


End python section to interscript/drivers/__init__.py[1]

6.3.1. Source Drivers Subpackage
--------------------------------

This subpackage contains components to read lines from
various sources.

6.3.1.1. Source Drivers Module
------------------------------

The source drivers module contains the class for the
exception to be thrown when a source resource cannot be
acquired, usually a file which cannot be opened because
it doesn't exist.

Start python section to interscript/drivers/sources/__init__.py[1]

     1: #line 11 "source_drivers.ipk"
     2: #--- source package ---
     3: class source_open_error(Exception): pass
     4: 


End python section to interscript/drivers/sources/__init__.py[1]

6.3.1.2. Source Base
--------------------


6.3.1.2.1. End Exceptions
-------------------------

A class to be used as an end of file exception, and a
class to be used as an end of information exception.
The distinction is: any input driver (source) file
could throw an eof. An eoi is thrown by a higher level
routine to indicate end of logical input frame.

These exceptions, when thrown, do not represent an
error, they're just used to facilitate alternate block
exits.

Start python section to interscript/drivers/sources/base.py[1]

     1: #line 27 "source_drivers.ipk"
     2: class eof(Exception): pass
     3: class eoi(Exception): pass
     4: 


End python section to interscript/drivers/sources/base.py[1]

6.3.1.2.2. Source Base Class
----------------------------

Interscript source drivers provide all the facilities
required of standard python file objects opened for
input.

Start python section to interscript/drivers/sources/base.py[2]

     5: #line 35 "source_drivers.ipk"
     6: #---------------------------------------------------------
     7: # source base
     8: class source:
     9:   def __init__(self, **kwds):
    10:     self.lines_read = 0
    11:     self.mode = 'r'
    12:     for k in kwds.keys():
    13:       self.__dict__[k]=kwds[k]
    14:     self.closed = 1
    15: 
    16:   def get_source_name(self):
    17:     return self.name
    18: 
    19:   def get_lines_read(self):
    20:     return self.lines_read
    21: 
    22:   def readlines():
    23:     if self.closed:
    24:       raise eof
    25:     lines = []
    26:     try:
    27:       while 1:
    28:         lines.append(self.readline())
    29:     except:
    30:       pass
    31:     return lines
    32: 
    33:   def isatty(self):
    34:     return 0
    35: 
    36:   def close(self):
    37:     self.closed = 1
    38: 
    39:   def flush(self):
    40:     pass
    41: 


End python section to interscript/drivers/sources/base.py[2]

6.3.1.3. Disk File Input
------------------------

This driver uses a filename in your local operating
system's convention.

Start python section to interscript/drivers/sources/disk.py[1]

     1: #line 77 "source_drivers.ipk"
     2: #---------------------------------------------------------
     3: # gets input from a named file
     4: from interscript.drivers.sources import source_open_error
     5: from interscript.drivers.sources.base import source
     6: from interscript.drivers.sources.base import eof
     7: import string
     8: import os
     9: 
    10: def loadfile(filename):
    11:    "return a list of lines, trailing whitespace removed"
    12:    try:
    13:      f = open(filename)
    14:    except:
    15:      raise source_open_error,filename
    16:    data = f.readlines()
    17:    f.close()
    18:    for i in range(len(data)):
    19:      data[i]=string.rstrip(data[i])
    20:    return data
    21: 
    22: def parse_source_filename(filename, prefix):
    23:   pathlist = string.split(filename,'/')
    24:   if prefix == '':
    25:     prefix = os.getcwd()
    26:     if prefix[-1] != os.sep:
    27:       prefix = prefix + os.sep
    28:   directory = prefix + string.join(pathlist[:-1],os.sep)
    29:   if directory[-1] != os.sep:
    30:     directory = directory + os.sep
    31:   basename = pathlist[-1]
    32:   full_filename = directory + basename
    33:   return pathlist, basename, directory, full_filename
    34: 
    35: class named_file_source(source):
    36:   def __init__(self,pass_frame,filename, prefix=''):
    37:     source.__init__(self)
    38:     pass_frame.iflist.append(filename)
    39:     self.name = filename
    40: 
    41:     pathlist, self.basename, self.directory, self.filename =\
    42:       parse_source_filename(filename, prefix)
    43: 
    44:     try:
    45:       self.file = open(self.filename,'r')
    46:       self.closed = 0
    47:     except:
    48:       raise source_open_error,filename
    49: 
    50:   def __del__(self):
    51:     if getattr(self,'file'): self.file.close()
    52: 
    53:   def readline(self):
    54:     if self.closed:
    55:       raise eof
    56:     line = self.file.readline()
    57:     if len(line)==0: raise eof
    58:     self.lines_read = self.lines_read + 1
    59:     return line
    60: 
    61:   def get_filename(self):
    62:     return self.name
    63: 


End python section to interscript/drivers/sources/disk.py[1]

6.3.1.4. URL input
------------------

This driver accepts a URL. Since it uses the Python
urllib library, it has the same limitations. From the
manual:

*    Currently, only the following protocols are
    supported: HTTP, (versions 0.9 and 1.0), Gopher
    (but not Gopher-+), FTP, and local files.

*    The caching feature of urlretrieve() has been
    disabled until I find the time to hack proper
    processing of Expiration time headers.

*    There should be a function to query whether a
    particular URL is in the cache.

*    For backward compatibility, if a URL appears to
    point to a local file but the file can't be opened,
    the URL is re-interpreted using the FTP protocol.
    This can sometimes cause confusing error messages.

*    The urlopen() and urlretrieve() functions can
    cause arbitrarily long delays while waiting for a
    network connection to be set up. This means that it
    is difficult to build an interactive web client
    using these functions without using threads.

*    The data returned by urlopen() or urlretrieve() is
    the raw data returned by the server. This may be
    binary data (e.g. an image), plain text or (for
    example) HTML. The HTTP protocol provides type
    information in the reply header, which can be
    inspected by looking at the Content-type header.
    For the Gopher protocol, type information is
    encoded in the URL; there is currently no easy way
    to extract it. If the returned data is HTML, you
    can use the module htmllib to parse it.

*    Although the urllib module contains (undocumented)
    routines to parse and unparse URL strings, the
    recommended interface for URL manipulation is in
    module urlparse.

Start python section to interscript/drivers/sources/url.py[1]

     1: #line 185 "source_drivers.ipk"
     2: # gets input from a URL
     3: from interscript.drivers.sources.base import source
     4: from interscript.drivers.sources.base import eof
     5: class url_source(source):
     6:   def __init__(self,filename):
     7:     source.__init__(self)
     8:     self.name = filename
     9:     self.file = urllib.urlopen(filename)
    10:     self.closed = 0
    11: 
    12:   def __del__(self):
    13:     self.file.close()
    14: 
    15:   def readline(self):
    16:     line = self.file.readline()
    17:     if len(line)==0: raise eof
    18:     self.lines_read = self.lines_read + 1
    19:     return line
    20: 
    21:   def get_filename(self):
    22:     return self.name
    23: 


End python section to interscript/drivers/sources/url.py[1]

6.3.1.5. FTP input
------------------

This object fetches a file by ftp on construction. The
file is then read from the local copy. The host name
and remote filename arguments are mandatory. Optional
arguments given by keywords are:

port 
     The port number of the host to use for FTP.
    Defaults to 21.

user 
     User name. Defaults to 'anonymous'.

account 
     User's account name.

password 
     User's password. If user is anonymous, the default
    password is `realuser@host' where realuser is the
    real user name (glanced from the `LOGNAME' or
    `USER' environment variable) and host is the
    hostname as returned by socket.gethostname().

remote_directory 
     The directory to 'cd' to to get the nominated
    file. If not specified, no 'cd' is done and the
    file is expected in the FTP root.

local_filename 
     The name of the file to which the download should
    be written. Defaults to 'remote_filename'.

refresh_interval 
     The is the number of days old the local file can
    be before ftp is done. The default is 28. If this
    value is set to 0, a download is always done. If
    the value is set to -1, a download is done only if
    the local file does not exist; delete the file to
    force another download.

An exception is thrown if there is no local file, and
the attempt to ftp the remote file fails. No exception
is thrown if the local file exists, even if it is out
of date.

Start python section to interscript/drivers/sources/ftp.py[1]

     1: #line 252 "source_drivers.ipk"
     2: #---------------------------------------------------------
     3: # gets input by FTP
     4: import ftplib
     5: import time
     6: import os
     7: from interscript.drivers.sources.base import source
     8: from interscript.drivers.sources.base import eof
     9: 
    10: class ftp_file_source(source):
    11:   def __init__(self,host,remote_filename,**kwds):
    12:     source.__init__(self)
    13:     self.name = remote_filename
    14:     self.remote_filename = remote_filename
    15:     self.host = host
    16:     self.g = g
    17:     for k in kwds.keys():
    18:       self.__dict__[k]=kwds[k]
    19:     if not hasattr(self,'local_filename'):
    20:       self.local_filename = self.remote_filename
    21:     self.os = os
    22:     self.fetch()
    23:     self.file = open(self.local_filename,'r')
    24:     self.closed = 0
    25: 
    26:   def transfer(self,data):
    27:     self.file.write(data+'\n')
    28: 
    29:   def fetch(self):
    30:     if not hasattr(self,'refresh_interval'):
    31:       self.refresh_interval = 28
    32:     if self.refresh_interval < 0: self.refresh_interval = 100000
    33:     self.local_file_exists = 1
    34:     try:
    35:       f = open(self.local_filename)
    36:       f.close()
    37:       if verbosity>=4: print 'local file',self.local_filename,'exists'
    38:     except:
    39:       if verbosity>=4: print 'local file',self.local_filename,'does NOT exist'
    40:       self.local_file_exists = 0
    41: 
    42:     if self.local_file_exists:
    43:       self.local_file_modify_time = os.stat(self.local_filename)[stat.ST_MTIME]
    44:       now = time.time()
    45:       age = (now - self.local_file_modify_time)/ (24 * 60 * 60)
    46:       download = age > self.refresh_interval
    47:     else:
    48:       download = 1
    49: 
    50:     if hasattr(self.g,'download'):
    51:       if self.g.download == 'always': download = 1
    52:       if self.g.download == 'never': download = 0
    53: 
    54:     if download:
    55:       try:
    56:         if verbosity>=2: print 'downloading',self.remote_filename
    57:         # create FTP object
    58:         ftp = ftplib.FTP()
    59: 
    60:         # connect to server
    61:         if hasattr(self,'port'):
    62:           ftp.connect(self.host,self.port)
    63:         else:
    64:           ftp.connect(self.host)
    65:         print 'connected to',self.host
    66: 
    67:         # login to server
    68:         if hasattr(self,'user'):
    69:           if hasattr(self,'password'):
    70:             if hasattr(self,'account'):
    71:               ftp.login(self.user,self.password,self.account)
    72:             else: ftp.login(self.user,self.password)
    73:           else: ftp.login(self.user)
    74:         else: ftp.login()
    75:         if verbosity>=4: print 'logged in'
    76: 
    77:         # set remote directory
    78:         if hasattr(self,'remote_directory'):
    79:           ftp.cwd(self.remote_directory)
    80:           print 'changed to remote directory',self.remote_directory
    81: 
    82:         # get file to a temporary
    83:         try:
    84:           tmp_filename = tempfile.mktemp()
    85:           self.file= open(tmp_filename,'w')
    86:           print 'opened',tmp_filename,'for download'
    87:           ftp.retrlines('RETR '+self.remote_filename, self.transfer)
    88:           self.file.close()
    89:           ftp.quit()
    90:           if verbosity>=2: print 'download complete'
    91: 
    92:           file = open(tmp_filename,'r')
    93:           newlines = file.readlines()
    94:           file.close()
    95: 
    96:           if self.local_file_exists:
    97:             file = open(self.local_filename,'r')
    98:             oldlines = file.readlines()
    99:             file.close()
   100: 
   101:             if newlines != oldlines:
   102:               if verbosity>=4: print 'Local file',self.local_filename,'UPDATED from',self.remote_filename
   103:             else:
   104:               if verbosity>=4: print 'Local file',self.local_filename,'unchanged'
   105:           else:
   106:             if verbosity>=4: print 'Writing new local file',self.local_filename
   107: 
   108:           # note that the local file is written even if it isn't changed
   109:           # to update the time stamp
   110:           file = open(self.local_filename,'w')
   111:           file.writelines(newlines)
   112:           file.close()
   113:           self.os.remove(self.tmp_filename)
   114: 
   115:         except:
   116:           print 'Cannot download',self.remote_filename,
   117:           if hasattr(self,'remote_directory'):
   118:             print 'from directory',self.remote_directory
   119:           else: print 'of',self.host
   120:           file.close()
   121:           self.os.remove(tmp_filename)
   122:           ftp.quit()
   123: 
   124:       except:
   125:         pass # ignore errors from ftp attempt
   126:     else:
   127:       print 'Skipping ftp download'
   128: 
   129:   def __del__(self):
   130:     self.file.close()
   131: 
   132:   def readline(self):
   133:     line = self.file.readline()
   134:     if len(line)==0: raise eof
   135:     self.lines_read = self.lines_read + 1
   136:     return line
   137: 
   138:   def get_filename(self):
   139:     return self.name
   140: 


End python section to interscript/drivers/sources/ftp.py[1]

6.3.1.6. HTTP input
-------------------

This object fetches a file by http on construction. The
file is then read from the local copy. The host name
and remote filename arguments are mandatory. Optional
arguments given by keywords are:

port 
     The port number of the host to use for FTP.
    Defaults to 21.

local_filename 
     The name of the file to which the download should
    be written. Defaults to 'remote_filename'.

refresh_interval 
     This is the number of days old the local file can
    be before ftp is done. The default is 28. If this
    value is set to 0, a download is always done. If
    the value is set to -1, a download is done only if
    the local file does not exist; delete the file to
    force another download.

An exception is thrown if there is no local file, and
the attempt to ftp the remote file files. No exception
is thrown if the local file exists, even if it is out
of date.

Start python section to interscript/drivers/sources/http.py[1]

     1: #line 418 "source_drivers.ipk"
     2: #---------------------------------------------------------
     3: # gets input by HTTP
     4: from interscript.drivers.sources.base import source
     5: from interscript.drivers.sources.base import eof
     6: import os
     7: import time
     8: import httplib
     9: 
    10: class http_file_source(source):
    11:   def __init__(self,host,remote_filename,**kwds):
    12:     source.__init__(self)
    13:     self.name = remote_filename
    14:     self.remote_filename = remote_filename
    15:     self.host = host
    16:     self.g = g
    17:     for k in kwds.keys():
    18:       self.__dict__[k]=kwds[k]
    19:     if not hasattr(self,'local_filename'):
    20:       self.local_filename = self.remote_filename
    21:     self.os = os
    22:     self.fetch()
    23:     self.file = open(self.local_filename,'r')
    24:     self.closed = 0
    25: 
    26:   def fetch(self):
    27:     if not hasattr(self,'refresh_interval'):
    28:       self.refresh_interval = 28
    29:     if self.refresh_interval < 0: self.refresh_interval = 100000
    30:     self.local_file_exists = 1
    31:     try:
    32:       f = open(self.local_filename)
    33:       f.close()
    34:       print 'local file',self.local_filename,'exists'
    35:     except:
    36:       print 'local file',self.local_filename,'does NOT exist'
    37:       self.local_file_exists = 0
    38: 
    39:     if self.local_file_exists:
    40:       self.local_file_modify_time = os.stat(self.local_filename)[stat.ST_MTIME]
    41:       now = time.time()
    42:       age = (now - self.local_file_modify_time)/ (24 * 60 * 60)
    43:       download = age > self.refresh_interval
    44:     else:
    45:       download = 1
    46: 
    47:     if hasattr(self.g,'download'):
    48:       if self.g.download == 'always': download = 1
    49:       if self.g.download == 'never': download = 0
    50: 
    51:     if download:
    52:       try:
    53:         print 'downloading',self.remote_filename
    54:         # create HTTP object
    55:         http = httplib.HTTP()
    56: 
    57:         # connect to server
    58:         if hasattr(self,'port'):
    59:           http.connect(self.host+':'+str(self.port))
    60:         else:
    61:           ftp.connect(self.host)
    62:         print 'connected to',self.host
    63: 
    64:         # set remote directory
    65:         to_download = self.remote_filename
    66:         if hasattr(self,'remote_directory'):
    67:           to_download = to_download + '/' + self.remote_directory
    68: 
    69:         # get file to a temporary
    70:         try:
    71:           http.putrequest('GET',to_download)
    72:           http.putheader('Accept','text/html')
    73:           http.putheader('Accept','text/plain')
    74:           http.endheaders()
    75:           errcode, errmsg, headers = http.getreply()
    76:           if errcode != 200: raise 'http error '+str(errcode)+'; '+errmsg
    77:           file = http.getfile()
    78:           newlines = file.readlines()
    79:           file.close()
    80:           print 'download complete'
    81: 
    82:           if self.local_file_exists:
    83:             file = open(self.local_filename,'r')
    84:             oldlines = file.readlines()
    85:             file.close()
    86: 
    87:             if newlines != oldlines:
    88:               print 'Local file',self.local_filename,'UPDATED from',self.remote_filename
    89:             else:
    90:               print 'Local file',self.local_filename,'unchanged'
    91:           else:
    92:             print 'Writing new local file',self.local_filename
    93: 
    94:           # note that the local file is written even if it isn't changed
    95:           # to update the time stamp
    96:           file = open(self.local_filename,'w')
    97:           file.writelines(newlines)
    98:           file.close()
    99: 
   100:         except:
   101:           print 'Cannot download',self.remote_filename,
   102:           if hasattr(self,'remote_directory'):
   103:             print 'from directory',self.remote_directory
   104:           else: print 'of',self.host
   105:           try:
   106:             print 'code',errcode,'msg',errmsg
   107:           except:
   108:             pass
   109:       except:
   110:         pass # ignore errors from ftp attempt
   111:     else:
   112:       print 'Skipping http download'
   113: 
   114:     self.file = open(self.local_filename,'r')
   115: 
   116:   def __del__(self):
   117:     self.file.close()
   118: 
   119:   def readline(self):
   120:     line = self.file.readline()
   121:     if len(line)==0: raise eof
   122:     self.lines_read = self.lines_read + 1
   123:     return line
   124: 
   125:   def get_filename(self):
   126:     return self.name
   127: 


End python section to interscript/drivers/sources/http.py[1]

6.3.1.7. Standard Input
-----------------------


Start python section to interscript/drivers/sources/stdin.py[1]

     1: #line 547 "source_drivers.ipk"
     2: #---------------------------------------------------------
     3: # gets input from _python_ sys.stdin object
     4: # same as named_file_source, except named 'standard input'
     5: # and doesn't close file on destruction
     6: import sys
     7: from interscript.drivers.sources.base import source
     8: from interscript.drivers.sources.base import eof
     9: 
    10: class stdin_source(source):
    11:   def __init__(self):
    12:     source.__init__(self)
    13:     self.name = 'standard input'
    14:     self.closed = 0
    15: 
    16:   def readline(self):
    17:     if self.closed:
    18:       raise eof
    19:     line = sys.stdin.readline()
    20:     if len(line)==0: raise eof
    21:     self.lines_read = self.lines_read + 1
    22:     return line


End python section to interscript/drivers/sources/stdin.py[1]

6.3.2. Sink Drivers
-------------------

Sink drivers support the standard Python file protocol
for output of text. They supply the methods:

write 
     Write text to output device. Newline character is
    parsed as end of line. Precondition: no control
    characters other than newline in the input.

writelines 
     Accepts a list of strings which are concatenated
    and passed to the write method. Precondition: no
    control characters other than newline in the input.

close 
     Flags the sink is closed for access to the device.
    Need not close the device.

flush 
     May flush data to the device.

The methods read, readline and readlines are note
provided because a sink need not support read
operations. The methods seek and tell are not provided
because sinks need not support positioning. The method
truncate is not provided because a sink need not be
storage device. There is no mode attribute, sinks
support writing, but need not be files. The following
methods should be provided by clients if the default
implementation (if it exists) is inappropriate.

raw_write 
     Write raw text to the device. Precondition: only
    printable characters in the input (space is
    allowed).

raw_close 
     May close device.

raw_flush 
     May flush device.

raw_eol 
     Ends a line on a device.

In addition, the following attributes are available:

lines_written 
     The total number of calls to writeline.

last_source_file 
     Provided for the client to store the name of the
    original source of the last line written.

last_source_line 
     Provided for the client to store the line number
    of the last line written in the original source
    file.

closed 
     Indicates if close was called.

Start python section to interscript/drivers/sinks/__init__.py[1]

     1: #line 70 "sink_drivers.ipk"
     2: #--- sink package ---
     3: class sink_open_error(Exception): pass
     4: 
     5: 


End python section to interscript/drivers/sinks/__init__.py[1]

Start python section to interscript/drivers/sinks/util.py[1]

     1: #line 76 "sink_drivers.ipk"
     2: import os
     3: posixpath = os.path
     4: import errno
     5: 
     6: # OS dependent routines.
     7: # Note: we use some posixpath functions, but don't trust them
     8: 
     9: # make the given directory, no error if it exists
    10: # this is posix specific, and should be moved to a platform
    11: # dependent place in the code
    12: 
    13: def create_directory(dir):
    14:   if not posixpath.isdir(dir):
    15:     try: os.mkdir(dir)
    16:     except os.error, data:
    17:       if data[0]!=errno.EEXIST: # File Exists is OK, everything else is fatal
    18:         raise os.error, (data[0],data+': directory "'+dir+'"')
    19:   if not posixpath.isdir(dir):
    20:     raise os.error, (errno.ENOENT, 'Created a directory '+dir+', but it is not there!')
    21: 
    22: # given an os specific prefix and a list of component names,
    23: # make the directory structure in which the last component is contained
    24: # and synthesise and return its full os specific pathname
    25: 
    26: def mk_dir(prefix, pathlist):
    27:   if len(pathlist)>1:
    28:     # there's more than one component in the list
    29:     # so create directories for all but the last component
    30: 
    31:     pathname = prefix+pathlist[0]
    32:     create_directory(pathname)
    33:     for component in pathlist[1:-1]:
    34:       pathname = pathname + os.sep + component
    35:       create_directory(pathname)
    36:     pathname = pathname + os.sep + pathlist[-1]
    37: 
    38:   else:
    39:     # only one component on the list
    40:     pathname = prefix+pathlist[0]
    41: 
    42:   if pathname[0]!=os.sep:
    43:     # the pathname isn't absolute, so make it so
    44:     # get current directory
    45:     curdir = os.getcwd()
    46: 
    47:     # strip trailing separator
    48:     # note this should fix cases like '/' (unix) or 'd:\' (nt)
    49:     # as well as cope with systems that return a trailing separator
    50:     if curdir[-1] == os.sep: curdir = curdir[:-1]
    51: 
    52:     # append a separator and the pathname: there will only be one
    53:     # separator at the join point unless the current directory
    54:     # ends with two separators (like the URL: http://)
    55:     pathname = curdir + os.sep + pathname
    56:   return pathname
    57: 
    58: # this routine checks is a file exists and is readable
    59: # (in the sense that it can be opened for read)
    60: # it returns 1 if the file exist and can be read, 0 if the file
    61: # doesn't exist, and throws an exception if anything else goes wrong
    62: 
    63: def file_exists(pathname):
    64:   try:
    65:     # note this leaks a file if it is opened but not closed :-))
    66:     open(pathname,'r').close()
    67:     return 1
    68:   except IOError, data:
    69:    if data[0] == errno.ENOENT:
    70:      return 0
    71:    raise IOError, data
    72: 
    73: # Note: the intention is to apply mk_dir to ensure the file has a
    74: # parent directory, creating it if necessary, then test if the file
    75: # already exists or has to be created. Even if it exists, it may have
    76: # to be replaced.
    77: 


End python section to interscript/drivers/sinks/util.py[1]

6.3.2.1. Sink Base Class
------------------------


Start python section to interscript/drivers/sinks/base.py[1]

     1: #line 155 "sink_drivers.ipk"
     2: import string
     3: class sink:
     4:   __class_protocols__ = ['sink','file']
     5:   def __init__(self, **kwds):
     6:     self.lines_written = 0
     7:     self.last_source_file = ''
     8:     self.last_source_line = -1
     9:     self.closed = 0
    10:     for k in kwds.keys():
    11:       self.__dict__[k]=kwds[k]
    12: 
    13:   def raw_close(self): pass
    14:   def raw_flush(self): pass
    15:   def raw_eol(self): self.raw_write('\n')
    16:   def isatty(self): return 0
    17: 
    18:   def raw_writeline(self,line):
    19:     self.raw_write(line)
    20:     self.raw_eol()
    21:     self.lines_written = self.lines_written  + 1
    22: 
    23:   def writeline(self,line):
    24:     self.write(line + '\n')
    25: 
    26:   def writelines(self,lines):
    27:     self.write(string.join(lines,''))
    28: 
    29:   def get_sink_name(self):
    30:     return self.name
    31: 
    32:   def write(self,text):
    33:     lines = string.split(text,'\n')
    34:     for line in lines[:-1]:
    35:       self.raw_writeline(line)
    36:     self.raw_write(lines[-1])
    37: 
    38:   def close(self):
    39:     self.closed = 1
    40:     self.raw_close()
    41: 
    42:   def flush(self):
    43:     self.raw_flush
    44: 


End python section to interscript/drivers/sinks/base.py[1]

6.3.2.2. Null Sink
------------------


Start python section to interscript/drivers/sinks/null.py[1]

     1: #line 201 "sink_drivers.ipk"
     2: from interscript.drivers.sinks.base import sink
     3: class null_sink(sink):
     4:   def __init__(self):
     5:     sink.__init__(self,name='null sink')
     6: 
     7:   def raw_write(self,line):
     8:     pass
     9: 


End python section to interscript/drivers/sinks/null.py[1]

6.3.2.3. Simple Disk File Sink
------------------------------


Start python section to interscript/drivers/sinks/disk.py[1]

     1: #line 212 "sink_drivers.ipk"
     2: import string
     3: from interscript.drivers.sinks.base import sink
     4: from interscript.drivers.sinks import sink_open_error
     5: from interscript.drivers.sinks.util import mk_dir
     6: 
     7: class simple_named_file_sink(sink):
     8:   def __init__(self,pass_frame,input_filename, prefix='', eol='\n'):
     9:     self.eol = eol
    10: 
    11:     # compute absolute pathname, and create directories if necessary
    12:     # we don't use posixpath because we're enforcing an _interscript_
    13:     # pathname convention here
    14:     pathlist = string.split(input_filename,'/')
    15:     self.basename = pathlist[-1]
    16:     pathname = mk_dir(prefix, pathlist)
    17:     try:
    18:       file = open(pathname,'w')
    19:     except:
    20:       raise sink_open_error,pathname
    21:     sink.__init__(self, name = input_filename, file = file)
    22:     pass_frame.flist.append(pathname)
    23: 
    24:   def __del__(self):
    25:     self.file.close()
    26: 
    27:   def raw_write(self,line): self.file.write(line)
    28:   def raw_eol(self): self.raw_write(self.eol)
    29: 


End python section to interscript/drivers/sinks/disk.py[1]

6.3.2.4. Complex Disk File Sink
-------------------------------

This class provides more advanced facilities than the
simple disk file. Output is written to a temporary file
until the driver is closed. At that point, the
temporary file is compared line by line with the target
file, and if the files differ, the temporary file is
copied onto the target, and then, in either case the
temporary deleted.

This mechanism has several purposes. First, it is used
to prevent 'make' systems recompiling unchanged code
files. Second, it is used so that the output of a
previous run can be read back in while the current run
is generating a new copy. Finally, the comparisons on
all these files can be used to determine if the
literate programming process has converged to a stable
point or may require another pass.

Please note that some care is required to ensure
convergence is possible. Note also that when line
numbers are generated, a single change to the source
file will cause comparisons to fail for all files with
sections defined after that point, often causing an
apparently redundant rebuild. This is not alsways the
case, for example, in C, the assert macro reports line
numbers at run time, and the rebuild is necessary. It
may be useful to turn off line number generation for
files once they have compiled successfully (until the
production build).

Finally, note that if you put the time into an output
file, it is likely to be different each run!

Start python section to interscript/drivers/sinks/bufdisk.py[1]

     1: #line 270 "sink_drivers.ipk"
     2: import string
     3: from interscript.drivers.sinks.base import sink
     4: from interscript.drivers.sinks import sink_open_error
     5: from interscript.drivers.sinks.util import mk_dir, file_exists
     6: import tempfile
     7: import os
     8: 
     9: class named_file_sink(sink):
    10:   def __init__(self,pass_frame,input_filename, prefix='', eol='\n'):
    11:     self.pass_frame = pass_frame
    12:     self.process = pass_frame.process
    13:     self.verbosity = self.pass_frame.verbosity
    14:     self.eol = eol
    15: 
    16:     # compute absolute pathname, and create directories if necessary
    17:     # we don't use posixpath because we're enforcing an _interscript_
    18:     # pathname convention here
    19:     pathlist = string.split(input_filename,'/')
    20:     self.basename = pathlist[-1]
    21:     pathname = mk_dir(prefix, pathlist)
    22: 
    23:     if file_exists(pathname):
    24:       self.tmp_filename = tempfile.mktemp()
    25:       if self.verbosity>=4:
    26:         print 'Generating temporary',self.tmp_filename,'for',input_filename
    27:       try:
    28:         file =open(self.tmp_filename,'w')
    29:         self.pass_frame.fdict[input_filename]='temporary'
    30:       except:
    31:         raise sink_open_error, self.tmp_filename
    32:       sink.__init__(self, filename = pathname, name = input_filename, file = file )
    33:       self.os = os
    34:       self.pass_frame.flist.append(input_filename)
    35:     else:
    36:       if self.verbosity>=3:
    37:         print 'Generating original',input_filename
    38:       try:
    39:         file = open(pathname,'w')
    40:         self.pass_frame.fdict[input_filename]='original'
    41:       except:
    42:         raise sink_open_error,pathname
    43:       sink.__init__(self, filename = pathname, name = input_filename, file = file)
    44:       self.pass_frame.flist.append(input_filename)
    45: 
    46:   def __del__(self):
    47:     if self.verbosity>=5: print 'closing', self.name
    48:     self.file.close()
    49:     if hasattr(self,'tmp_filename'):
    50:       if self.process.update_files:
    51:         original_file = open(self.filename,'r')
    52:         original_lines = original_file.readlines()
    53:         original_file.close()
    54: 
    55:         new_file = open(self.tmp_filename,'r')
    56:         new_lines = new_file.readlines()
    57:         new_file.close()
    58: 
    59:         if not original_lines == new_lines:
    60:           if self.verbosity>=1: print 'File',self.filename,'is CHANGED'
    61:           self.pass_frame.fdict[self.name]='changed'
    62:           file = open(self.filename,'w')
    63:           file.writelines(new_lines)
    64:           file.close()
    65:         else:
    66:           if self.verbosity>=4: print 'File',self.filename,'is unchanged'
    67:           self.pass_frame.fdict[self.name]='unchanged'
    68:       else:
    69:         if self.verbosity>=1: print '*** System error inhibiting file update for',self.filename,'***'
    70:         self.pass_frame.fdict[self.name]='cancelled'
    71:       self.os.remove(self.tmp_filename)
    72: 
    73:   def raw_write(self,line): self.file.write(line)
    74:   def raw_eol(self): self.raw_write(self.eol)
    75: 


End python section to interscript/drivers/sinks/bufdisk.py[1]

6.3.2.5. Standard Output Sink
-----------------------------


Start python section to interscript/drivers/sinks/stdout.py[1]

     1: #line 347 "sink_drivers.ipk"
     2: import sys
     3: from interscript.drivers.sinks.base import sink
     4: class stdout_sink(sink):
     5:   def __init__(self):
     6:     sink.__init__(self,name='standard output')
     7: 
     8:   def raw_write(self,line):
     9:     sys.stdout.write(line)
    10: 


End python section to interscript/drivers/sinks/stdout.py[1]

6.3.3. Storage Drivers
----------------------

These drivers support both reading and writing
operations.

Start python section to interscript/drivers/storage/__init__.py[1]

     1: #line 4 "storage_drivers.ipk"
     2: # storage drivers


End python section to interscript/drivers/storage/__init__.py[1]

6.3.3.1. Memory Driver
----------------------

The memory driver is both a source and sink and can is
used as a simple form of macro. By tangling sections of
code into memory, a sction can be built which is later
emitted in a single block. In this sense the tangler
driving the memory device is defining a macro, and its
later inclusion in te source stream can be considered
expanding the macro.

Start python section to interscript/drivers/storage/memory.py[1]

     1: #line 14 "storage_drivers.ipk"
     2: from interscript.drivers.sources.base import source
     3: from interscript.drivers.sources.base import eof
     4: from interscript.drivers.sinks.base import sink
     5: 
     6: class memory(source,sink):
     7:   def __init__(self,name,**kwds):
     8:     source.__init__(self)
     9:     self.name = name
    10:     self.saved = ''
    11:     self.list = []
    12:     for k in kwds.keys():
    13:       self.k = kwds[k]
    14: 
    15: 
    16:   def readline(self):
    17:     print 'reading memory object',self.name,'line',self.lines_read
    18:     if len(self.list)>self.lines_read:
    19:       line = self.list[self.lines_read]
    20:     else:
    21:       raise eof
    22:     if len(line) and line[-1]=='\n':
    23:       line = line[:-1]
    24:     self.lines_read = self.lines_read + 1
    25:     return line
    26: 
    27:   def writeline(self,line=''):
    28:     self.list.append(self.saved+line)
    29:     self.saved = ''
    30: 
    31:   def rewind(self):
    32:     self.lines_read = 0
    33: 
    34:   def write(self,data):
    35:     self.saved = self.saved + data
    36: 


End python section to interscript/drivers/storage/memory.py[1]

6.3.3.2. Disk Driver
--------------------

The disk driver is also both a source and sink and can
is used as a simple form of macro. By tangling sections
of code onto disk, a section can be built which is
emitted in a single block.

By using differential files drivers, and performing
multiple passes on the source, disk based macros can be
emitted before they are defined, or even in the middle
of their definition: the result will be the filw
written from the last run.

Start python section to interscript/drivers/storage/disk.py[1]

     1: #line 61 "storage_drivers.ipk"
     2: from interscript.drivers.sources.base import source
     3: from interscript.drivers.sinks.base import sink
     4: #---------------------------------------------------------
     5: # This object is both a reader and a writer!
     6: 
     7: class disk(source,sink):
     8:   pass
     9:   # not implemented yet :-(
    10: 


End python section to interscript/drivers/storage/disk.py[1]

6.4. Weavers
------------


Start python section to interscript/weavers/__init__.py[1]

     1: #line 3 "weavers.ipk"
     2: # weavers package
     3: 


End python section to interscript/weavers/__init__.py[1]

6.4.1. Weaver Base
------------------


Start python section to interscript/weavers/base.py[1]

     1: #line 8 "weavers.ipk"
     2: class weaver_base:
     3:   __class_protocols = ['weaver']
     4:   def __init__(self, pass_frame):
     5:     self.enabled = 1
     6:     self.translating = 1
     7:     self.tags = []
     8:     self.pass_frame = pass_frame
     9:     self.master = pass_frame.master
    10: 
    11:     self.sequence = self.pass_frame.get_new_sequence_number()
    12:     self.persistent_frame = self.master.get_persistent_frame(self.sequence)
    13: 
    14:   def enable(self): self.enabled = 1
    15:   def disable(self): self.enabled = 0
    16:   def translate(self): self.translating = 1
    17:   def raw(self): self.translating = 0
    18:   def add_tag(self,tag): self.tags.append(tag)
    19:   def rawif(self,tag):
    20:     if tag in self.tags: self.raw()
    21:     else: self.disable()
    22: 
    23:   def writeline(self,line=''):
    24:     self.write(line + '\n')
    25: 
    26:   def write(self,line):
    27:     self.sink.write(line)
    28: 
    29: 


End python section to interscript/weavers/base.py[1]

6.4.2. Raw weaver
-----------------

The raw weaver outputs text exactly as it is input,
special characters included. It cannot do any
formatting.

Use of raw weavers is vital for output of typesetter
specialised constructions. For example, If you are
generating output for both HTML and Latex, and you have
a complex table to print, you can set the weaver to a
raw weaver attached to the same sink file as the HTML
weaver and write HTML directly, then you can do the
same for Latex. In that way, you get tables optimised
for the various typesetting systems.

Start python section to interscript/weavers/raw.py[1]

     1: #line 50 "weavers.ipk"
     2: from interscript.weavers.base import weaver_base
     3: class raw_weaver(weaver_base):
     4:   def __init__(self, pass_frame, writer ,**kwds):
     5:     weaver_base.__init__(self, pass_frame)
     6:     if verbosity>=3: print 'initialising raw weaver, writer',writer.get_sink_name()
     7:     self.protocol = 'raw'
     8:     self.tags = ['raw']
     9:     self.sink = writer
    10:     self.name = 'raw weaver v1 for '+self.sink.name
    11: 
    12: 


End python section to interscript/weavers/raw.py[1]

6.4.3. Multiplex Weaver
-----------------------

This class is used as a base for classes that represent
multiple weavers. It supports only one operation:
method call with no return value, because only this
functionality is transparent; that is, operates the
same for a single object, or a set for which the
multiplexor is a proxy.

The multiplex weaver also supports list shortcuts.

The technology is unfortunately not re-entrant, the
multiplex call method return must be invoked
immediately. This makes multi-threaded calls to
multiplexed objects unsafe. (Python lacks appropriate
categorical constructions.) We could use a thread lock
to fix this, but it doesn't seem worth the effort.

Start python section to interscript/weavers/multiplexor.py[1]

     1: #line 78 "weavers.ipk"
     2: import traceback
     3: class multiplexor:
     4:   __class_protocols__ = ['weaver']
     5:   def __init__(self,pass_frame,base):
     6:     self.pass_frame = pass_frame
     7:     self.base=base
     8:     self.name = 'multiplexor v1'
     9:     self.debug_missing_methods = 0
    10:     self.list_stack = []
    11: 
    12:   def __nonzero__(self):
    13:     return 1
    14: 
    15:   def callit(self,*args, **kwds):
    16:     self._callit(self.temp,args,kwds)
    17: 
    18:   def _callit(self,at,args, kwds):
    19:     for b in self.base:
    20:       if hasattr(b,at):
    21:         try:
    22:           apply(getattr(b,at),args,kwds)
    23:         except KeyboardInterrupt:
    24:           raise KeyboardInterrupt
    25:         except:
    26:           protocol = 'No protocol attribute'
    27:           name = 'No name attribute'
    28:           if hasattr(b,'protocol'): protocol = b.protocol
    29:           if hasattr(b,'name'): name = b.name
    30:           print 'Error in call!'
    31:           print '  Method',at
    32:           print '  Args  ',args
    33:           print '  Kwds  ',kwds
    34:           print '  of    ',b.protocol,'weaver',b.name,'ignored'
    35:           traceback.print_exc()
    36:       elif self.debug_missing_methods:
    37:         protocol = 'No protocol attribute'
    38:         sinkname = 'No sink attribute'
    39:         if hasattr(b,'protocol'): protocol = b.protocol
    40:         if hasattr(b,'sink'):
    41:           sinkname = 'no sink name'
    42:           sink = b.sink
    43:           if hasattr(sink,'name'):
    44:             sinkname = sink.name
    45:         print 'Warning: missing method',at,'of weaver type',protocol,'for',sinkname,'ignored'
    46: 
    47:   def __getattr__(self,x):
    48:     self.temp = x
    49:     return self.callit
    50: 
    51:   def begin_list(self,style='bullet'):
    52:      if not style in ('bullet','numbered','keyed'):
    53:        style = 'bullet'
    54:      self.list_stack.append([style,0])
    55:      self._callit('begin_'+style+'_list',(),{})
    56: 
    57:   def end_list(self):
    58:     style, open_item = self.list_stack[-1]
    59:     del self.list_stack[-1]
    60:     if open_item:
    61:       self._callit('end_'+style+'_list_item',(),{})
    62:     self._callit('end_'+style+'_list',(),{})
    63: 
    64:   def item(self, *args, **kwds):
    65:     style, open_item = self.list_stack[-1]
    66:     if open_item:
    67:       self._callit('end_'+style+'_list_item',(),{})
    68:     else:
    69:       self.list_stack[-1][1]=1
    70:     self._callit('begin_'+style+'_list_item',args,kwds)
    71: 


End python section to interscript/weavers/multiplexor.py[1]

6.4.4. Plain text weaver
------------------------

This nasty weaver tries to output plain text that can
be published on the Internet. It can underline
headings, and flow text with left justification (and
should be able to format tables), but the results are
rather crude. Right justification is possible, but not
implemented because, in my experience, the results are
woeful.

Start python section to interscript/weavers/text.py[1]

     1: #line 11 "text_weaver.ipk"
     2: from interscript.weavers.base import weaver_base
     3: import string
     4: 
     5: class table_rule_object: pass
     6: 
     7: class plain_text_weaver(weaver_base):
     8:   def __init__(self, pass_frame,writer ,**kwds):
     9:     weaver_base.__init__(self, pass_frame)
    10:     self.verbosity = pass_frame.verbosity
    11:     if self.verbosity>=3:
    12:       print 'initialising plain text weaver, writer',writer.get_sink_name()
    13:     self.protocol = ('text/plain',1)
    14:     self.width = 55
    15:     self.c = 0
    16:     self.buffer = ''
    17:     self.strong = 0
    18:     self.string = string
    19:     self.code = 0
    20:     self.sink = writer
    21:     self.name = 'plain text weaver v1 for '+self.sink.name
    22:     self.tags = ['text']
    23:     self.margin = 0
    24:     self.numbered_list_stack = []
    25:     self.sop = 1
    26:     self.hcount = []
    27:     self.toc = []
    28: 
    29:   def _write(self,line):
    30:     if self.enabled:
    31:       self.sink.write(line)
    32:       self.c = self.c + len(line)
    33:     if line:
    34:       self.sop = 0
    35: 
    36:   def _writeline(self,line=''):
    37:     if self.enabled:
    38:       self._write(line+'\n')
    39:       self.c = 0
    40: 
    41:   def _goto(self,column):
    42:     if self.enabled:
    43:       if column < 0: column = self.width + column
    44:       if column < self.c: self._writeline()
    45:       if column > self.c: self._write(' '*(column-self.c))
    46: 


End python section to interscript/weavers/text.py[1]

6.4.4.1. reference processor
----------------------------


Start python section to interscript/weavers/text.py[2]

    47: #line 58 "text_weaver.ipk"
    48:   def set_original_filename(self, filename):
    49:     self.original_filename = filename
    50: 
    51:   def set_anchor(self, label):
    52:     if not self.persistent_frame.has_key('anchors'):
    53:       self.persistent_frame['anchors']  = {}
    54:     self.persistent_frame['anchors'][label]=self.sink.lines_written+1
    55: 
    56:   def get_anchor(self, label):
    57:     href = None
    58:     if self.persistent_frame.has_key('anchors'):
    59:       if self.persistent_frame['anchors'].has_key(label):
    60:         href =self.persistent_frame['anchors'][label]
    61:     return href
    62: 
    63:   def ref_anchor(self, label):
    64:     href = self.get_anchor(label)
    65:     if href:
    66:       self._write('line '+str(href))
    67:     else:
    68:       self._write('Unknown Label:'+label)
    69: 


End python section to interscript/weavers/text.py[2]

6.4.4.2. Table of Contents
--------------------------


Start python section to interscript/weavers/text.py[3]

    70: #line 82 "text_weaver.ipk"
    71:   def print_contents(self, hlevel=2, maxlev=3, *args, **kwds):
    72:     if hlevel>0:
    73:       self.head(hlevel,'Contents')
    74:     toc = self.persistent_frame.get('contents',[])
    75:     for level, line, lineno in toc:
    76:       if level <=maxlev:
    77:         prefix = ' '*(level*3)+line+' '
    78:         suffix = ' '+str(lineno+1)
    79:         mid = '.' * (self.width - len(prefix) - len(suffix))
    80:         self._writeline(prefix + mid + suffix)
    81:     self.par()
    82: 


End python section to interscript/weavers/text.py[3]

6.4.4.3. Code File List
-----------------------


Start python section to interscript/weavers/text.py[4]

    83: #line 96 "text_weaver.ipk"
    84:   def print_file_list(self,hlevel=2, *args, **kwds):
    85:     if hlevel>0:
    86:       self.head(hlevel,'File List')
    87:     if self.master.flist:
    88:       for line in self.master.flist:
    89:         self._writeline(line)
    90:     else:
    91:       self._writeline('No data available in pass '+str(self.pass_frame.passno)+'.')
    92:     self.par()
    93: 


End python section to interscript/weavers/text.py[4]

6.4.4.4. Code File Status
-------------------------


Start python section to interscript/weavers/text.py[5]

    94: #line 108 "text_weaver.ipk"
    95:   def print_file_status(self,hlevel=2, *args, **kwds):
    96:     passno = self.pass_frame.passno
    97:     h = 'File Status for pass '+str(passno-1)
    98:     if hlevel>0:
    99:       self.head(hlevel,h)
   100:     if self.master.fdict:
   101:       skeys = self.master.fdict.keys()
   102:       skeys.sort()
   103: 
   104:       h = 'Unchanged Files'
   105:       if hlevel>0:
   106:         self.head(hlevel+1,h)
   107:       else:
   108:         self._writeline(h)
   109:       for key in skeys:
   110:         status,change_passno = self.master.fdict[key]
   111:         if status == 'unchanged' and change_passno==0:
   112:           self._writeline(key)
   113: 
   114:       h = 'Changed Files'
   115:       if hlevel>0:
   116:         self.head(hlevel+1,h)
   117:       else:
   118:         self._writeline(h)
   119:       for key in skeys:
   120:         status,change_passno = self.master.fdict[key]
   121:         if status == 'unchanged' and change_passno == 1:
   122:           self._writeline(key)
   123: 
   124:       h = 'Files which required 2 or more passes to converge'
   125:       if hlevel>0:
   126:         self.head(hlevel+1,h)
   127:       else:
   128:         self._writeline(h)
   129:       for key in skeys:
   130:         status,change_passno = self.master.fdict[key]
   131:         if status == 'unchanged' and change_passno > 1:
   132:           self._writeline(key+' (converged in '+str(change_passno)+' passes)')
   133: 
   134:       h = 'Unstable Files'
   135:       if hlevel>0:
   136:         self.head(hlevel+1,h)
   137:       else:
   138:         self._writeline(h)
   139:       for key in skeys:
   140:         status,change_passno = self.master.fdict[key]
   141:         if status == 'changed':
   142:           self._writeline(key)
   143: 
   144:     else:
   145:       self._writeline('No data available in pass '+str(passno)+'.')
   146:     self.par()
   147: 


End python section to interscript/weavers/text.py[5]

6.4.4.5. Input File List
------------------------


Start python section to interscript/weavers/text.py[6]

   148: #line 163 "text_weaver.ipk"
   149:   def print_source_list(self, hlevel=2, *args, **kwds):
   150:     if hlevel>0:
   151:       self.head(hlevel,'Source List')
   152:     if self.master.iflist:
   153:       for line in self.master.iflist:
   154:         self._writeline(line)
   155:     else:
   156:       self._writeline('No data available in pass '+str(self.pass_frame.passno)+'.')
   157:     self.par()
   158: 


End python section to interscript/weavers/text.py[6]

6.4.4.6. Include List
---------------------


Start python section to interscript/weavers/text.py[7]

   159: #line 175 "text_weaver.ipk"
   160:   def print_include_list(self, hlevel=2, *args, **kwds):
   161:     if hlevel>0:
   162:       self.head(hlevel,'Include List')
   163:     if self.master.include_files:
   164:       for level, type, name in self.master.include_files:
   165:         self._writeline(' '*(level*3)+' '+type+': '+name)
   166:     else:
   167:       self._writeline('No data available in pass '+str(self.pass_frame.passno)+'.')
   168:     self.par()
   169: 


End python section to interscript/weavers/text.py[7]

6.4.4.7. Tables
---------------


Start python section to interscript/weavers/text.py[8]

   170: #line 187 "text_weaver.ipk"
   171:   def __htabrule(self,colw):
   172:     self._write('+')
   173:     for w in colw:
   174:       self._write('-'*(w+2)+'+')
   175:     self._writeline()
   176: 
   177:   def __tabrow(self,colw,data):
   178:     self._write('|')
   179:     for i in range(len(colw)):
   180:       w = colw[i]
   181:       entry = ' '* w
   182:       if i<len(data):
   183:         entry = (data[i]+entry)[:w]
   184:       self._write(' '+entry+' |')
   185:     self._writeline()
   186: 
   187:   def begin_table(self,*headings, **kwds):
   188:     self.table_headings = headings
   189:     self.table_data = []
   190: 
   191:   def table_row(self,data):
   192:     self.table_data.append(data)
   193: 
   194:   def table_rule(self):
   195:     self.table_data.append(table_rule_object)
   196: 
   197:   def end_table(self):
   198:     width = len(self.table_headings)
   199:     for row in self.table_data:
   200:       if row is not table_rule_object:
   201:         w = len(row)
   202:         if w>width : width = w
   203:     colw = [0] * width
   204:     for i in range(len(self.table_headings)):
   205:       w = len(self.table_headings[i])
   206:       if colw[i]<w: colw[i]=w
   207:     for row in self.table_data:
   208:       if row is not table_rule_object:
   209:         for i in range(len(row)):
   210:           w = len(row[i])
   211:           if colw[i]<w: colw[i]=w
   212:     self._flush()
   213:     self._writeline()
   214:     self.__htabrule(colw)
   215:     self.__tabrow(colw,self.table_headings)
   216:     self.__htabrule(colw)
   217:     for data in self.table_data:
   218:       if data is table_rule_object:
   219:         self.__htabrule(colw)
   220:       else:
   221:         self.__tabrow(colw,data)
   222:     self.__htabrule(colw)
   223:     del self.table_headings
   224:     del self.table_data
   225: 


End python section to interscript/weavers/text.py[8]

6.4.4.8. Code Output
--------------------


Start python section to interscript/weavers/text.py[9]

   226: #line 244 "text_weaver.ipk"
   227:   def writecode(self,line):
   228:     self._flush()
   229:     self._writeline(line)
   230: 
   231:   def echotangle(self,count,data):
   232:     self.writecode("%6d: %s" % (count,data))
   233: 
   234:   def _write_word(self,word):
   235:     if self.c == 0:
   236:       self._write((' '*self.margin)+word)
   237:     elif self.c + len(word) < self.width:
   238:       self._write(' '+word)
   239:     else:
   240:       self._writeline()
   241:       self._write((' '*self.margin)+word)
   242: 
   243:   def _flush(self):
   244:     words = self.string.split(self.buffer)
   245:     for w in words:
   246:       self._write_word(w)
   247:     self.buffer = ''
   248: 
   249:   def write(self,line):
   250:     if self.translating:
   251:       if self.strong: line = string.upper(line)
   252:       self.buffer = self.buffer + line
   253:     else:
   254:       self._write(line)
   255: 
   256:   def writeline(self,line = ''):
   257:     if self.translating:
   258:       self.write(line)
   259:       if self.code:
   260:         self._writeline(self.buffer)
   261:         self.buffer = ''
   262:       else:
   263:         self._flush()
   264:     else:
   265:       self._writeline(line)
   266: 
   267:   def par(self):
   268:     self.line_break()
   269:     if not self.sop:
   270:       self._writeline()
   271:       self.sop = 1
   272: 
   273:   def line_break(self):
   274:     self._flush()
   275:     if self.c != 0: self._writeline()
   276: 
   277:   def page_break(self):
   278:     self.par()
   279:     self._writeline()
   280:     self._writeline('-' * self.width)
   281:     self._writeline()
   282:     self.sop = 1
   283: 
   284:   def begin_emphasize(self):
   285:     self.write('_')
   286: 
   287:   def end_emphasize(self):
   288:     self.write('_')
   289: 
   290:   def begin_strong(self):
   291:     self.strong = 1
   292: 
   293:   def end_strong(self):
   294:     self.strong = 0
   295: 
   296:   def begin_displayed_code(self):
   297:     self.par()
   298:     self.code = 1
   299: 
   300:   def end_displayed_code(self):
   301:     self.par()
   302:     self.code = 0
   303: 
   304:   def begin_displayed_text(self):
   305:     self.par()
   306:     self.margin = self.margin + 4
   307: 
   308:   def end_displayed_text(self):
   309:     self.par()
   310:     self.margin = self.margin - 4
   311: 
   312:   def new_heading(self,level):
   313:     while level>len(self.hcount): self.hcount.append(0)
   314:     while level<len(self.hcount): del self.hcount[-1]
   315:     counter = self.hcount[level-1]+1
   316:     self.hcount[level-1] = counter
   317:     return counter
   318: 
   319:   def get_formatted_heading_number(self, sep):
   320:     hnumber = ''
   321:     for i in range(0,len(self.hcount)-1):
   322:       hnumber = hnumber + str(self.hcount[i])+sep
   323:     hnumber = hnumber + str(self.hcount[-1])
   324:     return hnumber
   325: 
   326:   def head(self,level, text, atext='', anchor=''):
   327:     self.par()
   328:     self.strong = 0
   329:     self.new_heading(level)
   330:     h = self.get_formatted_heading_number('.')+'. '+text
   331:     self.toc.append((level,text,self.sink.lines_written))
   332:     self._writeline(h)
   333:     self._writeline('-'*len(h))
   334:     self._writeline()
   335: 
   336:   def code_head(self,tangler, secno):
   337:     if tangler:
   338:       self.par()
   339:       language = tangler.get_language()
   340:       filename  =tangler.sink.get_sink_name()
   341:       self._writeline( 'Start '+language+' section to '+\
   342:         filename+'['+str(secno)+']')
   343:       self._writeline()
   344: 
   345:   def code_foot(self,tangler,secno):
   346:     if tangler:
   347:       self.par()
   348:       language = tangler.get_language()
   349:       filename  =tangler.sink.get_sink_name()
   350:       self._writeline()
   351:       self._writeline( 'End '+language+' section to '+filename+\
   352:         '['+str(secno)+']')
   353:       self.par()
   354: 
   355:   def test_output_head(self,command, status):
   356:     self.par()
   357:     self._writeline( 'Start output section of '+command)
   358:     if status:
   359:       self._writeline( 'Command returned '+str(status))
   360:     self.par()
   361: 
   362:   def test_output_foot(self,command,status):
   363:     self.par()
   364:     self._writeline('End output section of '+command)
   365:     self.par()
   366: 
   367:   def expected_head(self,command):
   368:     self.par()
   369:     self._writeline( 'Start expected section of '+command)
   370:     self.par()
   371: 
   372:   def expected_foot(self,command):
   373:     self.par()
   374:     self._writeline('End expected section of '+command)
   375:     self.par()
   376: 
   377:   def diff_head(self,command):
   378:     self.par()
   379:     self._writeline( 'Start diff section of '+command)
   380:     self.par()
   381: 
   382:   def diff_foot(self,command):
   383:     self.par()
   384:     self._writeline('End diff section of '+command)
   385:     self.par()
   386: 
   387:   def __del__(self):
   388:     self._flush()
   389:     if self.c != 0:
   390:       self._writeline()
   391:     self.persistent_frame['contents']=self.toc
   392: 


End python section to interscript/weavers/text.py[9]

6.4.4.9. Citations
------------------


Start python section to interscript/weavers/text.py[10]

   393: #line 412 "text_weaver.ipk"
   394:   def begin_keyed_list(self):
   395:     self.margin = self.margin + 4
   396: 
   397:   def begin_numbered_list(self,start=1):
   398:     self.margin = self.margin + 4
   399:     self.numbered_list_stack.append(start)
   400: 
   401:   def begin_bullet_list(self):
   402:     self.margin = self.margin + 4
   403: 
   404:   def end_keyed_list(self):
   405:     self.par()
   406:     self.margin = self.margin - 4
   407: 
   408:   def end_numbered_list(self):
   409:     self.par()
   410:     self.margin = self.margin - 4
   411:     del self.numbered_list_stack[-1]
   412: 
   413:   def end_bullet_list(self):
   414:     self.par()
   415:     self.margin = self.margin - 4
   416: 
   417:   def begin_keyed_list_item(self,key):
   418:     self.par()
   419:     self._goto(self.margin-4)
   420:     self._write(key+' ')
   421:     self._goto(self.margin)
   422: 
   423:   def begin_numbered_list_item(self):
   424:     self.par()
   425:     key = "%2d. " % self.numbered_list_stack[-1]
   426:     self.numbered_list_stack[-1] = self.numbered_list_stack[-1] + 1
   427:     self._goto(self.margin-4)
   428:     self._write(key)
   429: 
   430:   def begin_bullet_list_item(self):
   431:     self.par()
   432:     key = '*   '
   433:     self._goto(0)
   434:     self._goto(self.margin-4)
   435:     self._write(key)
   436: 


End python section to interscript/weavers/text.py[10]

6.4.4.10. Citations
-------------------


Start python section to interscript/weavers/text.py[11]

   437: #line 457 "text_weaver.ipk"
   438:   def cite_url(self,url):
   439:     self.write(url)
   440: 
   441: 


End python section to interscript/weavers/text.py[11]

6.4.5. HTML Weaver
------------------

The most commonly used weaver is sure to be the HTML
weaver. This weaver attempts to generate HTML compliant
with the proposed strict HTML 4.0 DTD. See
http://www.w3.org/TR/REC/-html40

Interscript uses a different markup model to HTML.
While interscript uses begin and end methods like many
HTML tags, HTML paragraph elements are not indicated by
such methods. Instead, text written in many places is
implicitly a paragraph element, and interscript must
detect these places and insert an appropriate start
tag. Similarly, interscript paragraphs are separated by
@p() commands, and terminated by some other block level
construction like a heading or table.

On the other hand, not all text lines are paragraph
content, for example list body content.

Start python section to interscript/weavers/html.py[1]

     1: #line 21 "html_weaver.ipk"
     2: from interscript.weavers.base import weaver_base
     3: import string
     4: try:
     5:   import interscript.core.iscrcopt
     6:   cvt_code = interscript.core.iscrcopt.cvt_code
     7:   cvt_text = interscript.core.iscrcopt.cvt_text
     8:   #print 'Using iscrcopt'
     9: except:
    10:   #print 'iscrcopt NOT AVAILABLE'
    11:   def cvt_code(line):
    12:     l = ''
    13:     for ch in line:
    14:       if ch in '<>':
    15:         l = l + {'<':'&lt;','>':'&gt;'}[ch]
    16:       else:
    17:         l = l + ch
    18:     return l
    19: 
    20:   def cvt_text(line):
    21:     l = ''
    22:     for ch in line:
    23:       if ch in '<>&':
    24:         l = l + {'<':'&lt;','>':'&gt;','&':'&amp;'}[ch]
    25:       else:
    26:         l = l + ch
    27:     return l
    28: 
    29: class html_weaver(weaver_base):
    30: 


End python section to interscript/weavers/html.py[1]

6.4.5.1. Initialisation
-----------------------


Start python section to interscript/weavers/html.py[2]

    31: #line 52 "html_weaver.ipk"
    32:   def __init__(self, pass_frame,writer ,**kwds):
    33:     weaver_base.__init__(self,pass_frame)
    34:     self.verbosity = pass_frame.verbosity
    35:     if self.verbosity>3: print 'initialising html weaver, writer',writer.get_sink_name()
    36:     self.protocol = 'text/html'
    37:     self.sink = writer
    38:     self.acount = 1
    39:     self.hcount = [0]
    40:     self.mode = None
    41:     self.comments = 0
    42:     self.master = pass_frame.master
    43:     self.list = []
    44:     self.name = 'html weaver v1 for '+self.sink.name
    45:     self.heading_level_offset = 0
    46:     self.keywords = kwds
    47:     self.tags = ['html'] # this 'tags' has nothing to do with html tags!
    48:     self.toc = []
    49:     if kwds.has_key('title'):
    50:       self.title=kwds['title']
    51:     else:
    52:       title = self.sink.name
    53: 
    54:     self.prolog()
    55: 


End python section to interscript/weavers/html.py[2]

6.4.5.2. Termination
--------------------


Start python section to interscript/weavers/html.py[3]

    56: #line 78 "html_weaver.ipk"
    57:   def __del__(self):
    58:     self.epilog()
    59:     self.persistent_frame['contents']=self.toc
    60: 


End python section to interscript/weavers/html.py[3]

6.4.5.3. Body Output and Mode Control
-------------------------------------


Start python section to interscript/weavers/html.py[4]

    61: #line 84 "html_weaver.ipk"
    62:   def _setmode(self,mode):
    63:     self._write('\n<'+mode+'>')
    64:     self.mode = mode
    65: 
    66:   def _endmode(self):
    67:     if self.mode:
    68:       self._write('</'+self.mode+'>\n')
    69:       self.mode = None
    70: 
    71:   def _startmode(self,mode):
    72:     self._endmode()
    73:     self._setmode(mode)
    74: 
    75:   def _ensuremode(self,mode):
    76:     if self.mode != mode : self._startmode(mode)
    77: 
    78:   def _writeline(self,line):
    79:     if self.enabled: self.sink.writeline(line)
    80: 
    81:   def _write(self,line):
    82:     if self.enabled: self.sink.write(line)
    83: 
    84:   def writeline(self,line=''):
    85:     self.write(line + '\n')
    86: 
    87:   def write(self,line):
    88:     #hack to correct bug in popular broswers
    89:     #if not self.mode: self._setmode('P')
    90:     if self.translating:
    91:       self._write(cvt_text(line))
    92:     else:
    93:       self._write(line)
    94: 
    95:   def writecode(self,line):
    96:     self._ensuremode('PRE')
    97:     self._writeline(cvt_code(line))
    98: 
    99:   def begin_displayed_text(self):
   100:     self._ensuremode('P')
   101:     # note this is HTML 2, HTML 3 uses BQ instead
   102:     self.write('<BLOCKQUOTE>')
   103: 
   104:   def end_displayed_text(self):
   105:     self.write('</BLOCKQUOTE>')
   106: 
   107:   def begin_displayed_code(self):
   108:     self._write('<PRE>\n')
   109: 
   110:   def end_displayed_code(self):
   111:     self._write('</PRE>')
   112: 
   113:   def line_break(self):
   114:     self._writeline('<BR>')
   115: 
   116:   def page_break(self):
   117:     self.writeline('<BR><HR>')
   118: 
   119:   def write_tagged(self,tag, data):
   120:     self._write('<'+tag+'>')
   121:     self._writeline(data)
   122:     self._write('</'+tag+'>')
   123: 
   124: 
   125:   def code_head(self,tangler, secno):
   126:     if tangler:
   127:       self._endmode()
   128:       filename =tangler.sink.get_sink_name()
   129:       language = tangler.get_language()
   130: 
   131:       self._writeline( '<DIV CLASS="CODE_SECTION_HEAD"><SMALL>Start <EM>'+\
   132:         language+'</EM> section to <STRONG>'+\
   133:         filename+'['+str(secno)+']</STRONG></SMALL></DIV>')
   134:       self._writeline( '<DIV CLASS="CODE">')
   135: 
   136:   def code_foot(self,tangler, secno):
   137:     if tangler:
   138:       self._endmode()
   139:       filename =tangler.sink.get_sink_name()
   140:       language = tangler.get_language()
   141:       self._write( '</DIV><DIV CLASS="CODE_SECTION_FOOT"><SMALL>End <EM>'+\
   142:         language+'</EM> section to <STRONG>'+\
   143:         filename+'['+str(secno)+']</STRONG></SMALL></DIV>')
   144: 
   145:   def script_head(self,language,filename):
   146:       self._endmode()
   147:       self._writeline( '<DIV CLASS="CODE_SECTION_HEAD"><SMALL>Start <EM>'+\
   148:         language+'</EM> section from <STRONG>'+\
   149:         filename+'</STRONG></SMALL></DIV>')
   150:       self._writeline( '<DIV CLASS="CODE">')
   151: 
   152:   def script_foot(self,language,filename):
   153:       self._endmode()
   154:       self._write( '</DIV><DIV CLASS="CODE_SECTION_FOOT"><SMALL>End <EM>'+\
   155:         language+'</EM> section from <STRONG>'+\
   156:         filename+'</STRONG></SMALL></DIV>')
   157: 
   158:   def test_output_head(self,command, status):
   159:     self._endmode()
   160:     self._writeline( '<DIV CLASS="TEST_OUTPUT_SECTION_HEAD"><SMALL>Start <EM>'+\
   161:       'output</EM> section of <STRONG>'+\
   162:       command+'</STRONG></SMALL></DIV>')
   163:     if status:
   164:       self._writeline( '<DIV CLASS="TEST_OUTPUT_RESULT"><BIG>Command returned <STRONG>'+\
   165:         str(status)+'</STRONG></BIG></DIV>')
   166:     if status: div_class = 'BAD_TEST_OUTPUT'
   167:     else: div_class = 'TEST_OUTPUT'
   168:     self._writeline( '<DIV CLASS="'+div_class+'">')
   169: 
   170:   def test_output_foot(self,command,status):
   171:     self._endmode()
   172:     self._writeline( '</DIV><DIV CLASS="TEST_OUTPUT_SECTION_FOOT">')
   173:     self._writeline('<SMALL>End <EM>output</EM> section to <STRONG>'+command+'</STRONG></SMALL></DIV>')
   174: 
   175:   def expected_head(self,command):
   176:     self._endmode()
   177:     self._writeline( '<DIV CLASS="EXPECTED_OUTPUT_SECTION_HEAD"><SMALL>Start <EM>expected</EM> section of <STRONG>'+command+'</STRONG></SMALL></DIV>')
   178:     div_class = 'EXPECTED_OUTPUT'
   179:     self._writeline( '<DIV CLASS="'+div_class+'">')
   180: 
   181:   def expected_foot(self,command):
   182:     self._endmode()
   183:     self._writeline( '</DIV><DIV CLASS="EXPECTED_OUTPUT_SECTION_FOOT">')
   184:     self._writeline('<SMALL>End <EM>expected</EM> section to <STRONG>'+command+'</STRONG></SMALL></DIV>')
   185: 
   186:   def diff_head(self,command):
   187:     self._endmode()
   188:     self._writeline( '<DIV CLASS="DIFF_SECTION_HEAD"><SMALL>Start <EM>diff</EM> section of <STRONG>'+command+'</STRONG></SMALL></DIV>')
   189:     div_class = 'DIFF'
   190:     self._writeline( '<DIV CLASS="'+div_class+'">')
   191: 
   192:   def diff_foot(self,command):
   193:     self._endmode()
   194:     self._writeline( '</DIV><DIV CLASS="DIFF_SECTION_FOOT">')
   195:     self._writeline('<SMALL>End <EM>diff</EM> section to <STRONG>'+command+'</STRONG></SMALL></DIV>')
   196: 


End python section to interscript/weavers/html.py[4]

6.4.5.4. Headings
-----------------


Start python section to interscript/weavers/html.py[5]

   197: #line 221 "html_weaver.ipk"
   198:   def new_heading(self,level):
   199:     while level>len(self.hcount): self.hcount.append(0)
   200:     while level<len(self.hcount): del self.hcount[-1]
   201:     counter = self.hcount[level-1]+1
   202:     self.hcount[level-1] = counter
   203:     return counter
   204: 
   205:   def get_formatted_heading_number(self, sep):
   206:     hnumber = ''
   207:     for i in range(0,len(self.hcount)-1):
   208:       hnumber = hnumber + str(self.hcount[i])+sep
   209:     hnumber = hnumber + str(self.hcount[-1])
   210:     return hnumber
   211: 
   212: 
   213:   def head(self,level, text, atext='', anchor=''):
   214:     self._endmode()
   215:     if anchor == '': anchor = atext
   216:     if anchor == '':
   217:       anchor = 'h'+str(self.acount)
   218:       self.acount = self.acount + 1
   219: 
   220:     self.new_heading(level)
   221:     aname = '<A NAME='+anchor+'></A>'
   222:     hnumber = self.get_formatted_heading_number('.')+'. '
   223:     hprefix = ''
   224:     if self.keywords.has_key('heading_prefix'):
   225:       hprefix = self.keywords['heading_prefix']
   226:     ahref = '<A HREF=#'+anchor+'>'+hprefix+hnumber+str(text)+'</A>'
   227: 
   228:     html_level = level + self.heading_level_offset
   229:     if html_level > 6: html_level = 6
   230:     self._writeline( '<H'+str(html_level)+'>'+aname+hprefix+hnumber+str(text)+'</H'+str(html_level)+'>')
   231: 
   232:     counter = self.hcount[level-1]
   233:     self.toc.append((level,ahref))
   234: 


End python section to interscript/weavers/html.py[5]

6.4.5.5. Lists
--------------

Interscript has a problem here. HTML 4.0 allows the
content of a list item to be either inline text or
block elements.

Interscript has no way to distinguish between inline
text and a paragraph block. Even if we look ahead, a
plain text stream could be intended as either a
paragraph or inline text.

Unfortunately, NS3 and IE4 put blank lines in on </P>
tags, which is wrong: the presence or absence of an
optional tag ought not affect rendering. For this
reason, I've been forced to remove the wrapping of text
in <P> and </P> tags; the @p() command inserts a plain
<P> after the _first_ paragraph. Technically, this is
invalid HTML, the first pargraph is not wrapped in
start and end P tags.

6.4.5.5.1. Numbered Lists
-------------------------


Start python section to interscript/weavers/html.py[6]

   235: #line 277 "html_weaver.ipk"
   236:   def begin_numbered_list(self,start=1, type='1'):
   237:     self._endmode()
   238:     self._write('<OL SEQNUM="'+str(start)+'" TYPE="'+type+'">')
   239: 
   240:   def end_numbered_list(self):
   241:     self._write('</OL>')
   242: 
   243:   def begin_numbered_list_item(self):
   244:     self._write('<LI>')
   245: 
   246:   def end_numbered_list_item(self):
   247:     self._write('</LI>')
   248: 


End python section to interscript/weavers/html.py[6]

6.4.5.5.2. Bullet Lists
-----------------------


Start python section to interscript/weavers/html.py[7]

   249: #line 292 "html_weaver.ipk"
   250:   def begin_bullet_list(self):
   251:     self._endmode()
   252:     self._write('<UL>')
   253: 
   254:   def end_bullet_list(self):
   255:     self._write('</UL>')
   256: 
   257:   def begin_bullet_list_item(self):
   258:     self._write('<LI>')
   259: 
   260:   def end_bullet_list_item(self):
   261:     self._endmode()
   262:     self._write('</LI>')
   263: 


End python section to interscript/weavers/html.py[7]

6.4.5.5.3. Keyed Lists
----------------------


Start python section to interscript/weavers/html.py[8]

   264: #line 308 "html_weaver.ipk"
   265:   def begin_keyed_list(self):
   266:     self._endmode()
   267:     self._write('<DL>')
   268: 
   269:   def end_keyed_list(self):
   270:     self._write('</DL>')
   271: 
   272:   def begin_keyed_list_item(self,key):
   273:     self._write('<DT><B>'+key+'</B></DT><DD>')
   274: 
   275:   def end_keyed_list_item(self):
   276:     self._write('</DD>')
   277: 


End python section to interscript/weavers/html.py[8]

6.4.5.6. Code Output
--------------------


Start python section to interscript/weavers/html.py[9]

   278: #line 323 "html_weaver.ipk"
   279:   # default code line formatting
   280:   def echotangle(self,count,data):
   281:     if self.comments:
   282:       self._writeline(data)
   283:     else:
   284:       self._ensuremode('PRE')
   285:       self._write('<SPAN CLASS="LINENO">%6d: </SPAN>' % count)
   286:       self._writeline(cvt_code(data))
   287: 


End python section to interscript/weavers/html.py[9]

6.4.5.7. Citations
------------------


Start python section to interscript/weavers/html.py[10]

   288: #line 334 "html_weaver.ipk"
   289:   def cite_url(self,url):
   290:     self._write('<A HREF="'+url+'">'+url+'</A>')
   291: 


End python section to interscript/weavers/html.py[10]

6.4.5.8. Paragraphs
-------------------


Start python section to interscript/weavers/html.py[11]

   292: #line 339 "html_weaver.ipk"
   293:   def prose(self): # start of paragraph
   294:     self._ensuremode('P')
   295: 
   296:   def par(self): # paragraph separator
   297:     self._endmode()
   298:     self._ensuremode('P')
   299: 
   300:   def eop(self): # end of paragraph
   301:     self._endmode()
   302: 
   303:   def write_comment(self,v):
   304:     saved_mode = self.mode
   305:     self.write_tagged('SMALL',v)
   306:     self._ensuremode(saved_mode)
   307: 


End python section to interscript/weavers/html.py[11]

6.4.5.9. Fonts
--------------


Start python section to interscript/weavers/html.py[12]

   308: #line 356 "html_weaver.ipk"
   309:   def begin_code(self):
   310:     self._write('<CODE>')
   311: 
   312:   def end_code(self):
   313:     self._write('</CODE>')
   314: 
   315:   def begin_emphasize(self):
   316:     self._write('<EM>')
   317: 
   318:   def end_emphasize(self):
   319:     self._write('</EM>')
   320: 
   321:   def begin_strong(self):
   322:     self._write('<STRONG>')
   323: 
   324:   def end_strong(self):
   325:     self._write('</STRONG>')
   326: 
   327:   def begin_italic(self):
   328:     self._write('<I>')
   329: 
   330:   def end_italic(self):
   331:     self._write('</I>')
   332: 
   333:   def begin_bold(self):
   334:     self._write('<B>')
   335: 
   336:   def end_bold(self):
   337:     self._write('</B>')
   338: 
   339:   def begin_big(self):
   340:     self._write('<BIG>')
   341: 
   342:   def end_big(self):
   343:     self._write('</BIG>')
   344: 
   345:   def begin_small(self):
   346:     self._write('<SMALL>')
   347: 
   348:   def end_small(self):
   349:     self._write('</SMALL>')
   350: 


End python section to interscript/weavers/html.py[12]

6.4.5.10. Identifier Cross Reference Table
------------------------------------------


Start python section to interscript/weavers/html.py[13]

   351: #line 400 "html_weaver.ipk"
   352:   def identifier_reference(self, hlevel=2, *args, **kwds):
   353:     ids = self.master.ids
   354:     if not ids:
   355:       ids = self.pass_frame.ids
   356:     keys = ids.keys()
   357:     keys.sort()
   358:     if hlevel>0:
   359:       self.head(hlevel,'Index of Identifiers')
   360:     self._writeline('<TABLE COLS="2" BORDER="1" CELLPADDING="2">')
   361:     for k in keys:
   362:       refs = ids[k]
   363:       self._write('<TR><TD VALIGN="Top"><CODE> '+k+' </CODE></TD><TD> ')
   364:       old_df = ''
   365:       for sf,sc,df,dc in refs:
   366:         if old_df != '': self._write(', ')
   367:         if old_df != df:
   368:           self._write(df+': <A HREF=#'+sf+':'+str(sc)+'>'+str(dc)+'</A>')
   369:           old_df = df
   370:         else:
   371:           self._write('<A HREF=#'+sf+':'+str(sc)+'>'+str(dc)+'</A>')
   372:       self._write('</TD></TR>')
   373:     self._writeline('</TABLE>')
   374: 


End python section to interscript/weavers/html.py[13]

6.4.5.11. Class Reference Table
-------------------------------


Start python section to interscript/weavers/html.py[14]

   375: #line 425 "html_weaver.ipk"
   376:   def class_reference(self, hlevel=2, *args, **kwds):
   377:     ids = self.master.classes
   378:     if not ids:
   379:       ids = self.pass_frame.classes
   380:     keys = ids.keys()
   381:     keys.sort()
   382:     if hlevel>0:
   383:       self.head(hlevel,'Index of Classes')
   384:     self._writeline('<TABLE COLS="2" BORDER="1" CELLPADDING="2">')
   385:     for k in keys:
   386:       refs = ids[k]
   387:       self._write('<TR><TD VALIGN="Top"><CODE> '+k+' </CODE></TD><TD> ')
   388:       old_df = ''
   389:       for sf,sc,df,dc in refs:
   390:         if old_df != '': self._write(', ')
   391:         if old_df != df:
   392:           self._write(df+': <A HREF=#'+sf+':'+str(sc)+'>'+str(dc)+'</A>')
   393:           old_df = df
   394:         else:
   395:           self._write('<A HREF=#'+sf+':'+str(sc)+'>'+str(dc)+'</A>')
   396:       self._write('</TD></TR>')
   397:     self._writeline('</TABLE>')
   398: 
   399:   # create an HTML anchor
   400:   def set_fc_anchor(self,file,count):
   401:     self._write('<A NAME="'+file+':'+str(count)+'"></A>')
   402: 


End python section to interscript/weavers/html.py[14]

6.4.5.12. Table of Contents
---------------------------


Start python section to interscript/weavers/html.py[15]

   403: #line 454 "html_weaver.ipk"
   404:   def print_contents(self, hlevel=2, maxlev=3, *args, **kwds):
   405:     if hlevel>0:
   406:       self.head(hlevel,'Contents')
   407:     toc = self.persistent_frame.get('contents',[])
   408:     self._write('<PRE>')
   409:     for level, line in toc:
   410:       if level <=maxlev:
   411:         self._writeline(' '*(level*3)+line)
   412:     self._writeline('</PRE>')
   413: 


End python section to interscript/weavers/html.py[15]

6.4.5.13. Code File List
------------------------


Start python section to interscript/weavers/html.py[16]

   414: #line 466 "html_weaver.ipk"
   415:   def print_file_list(self,hlevel=2, *args, **kwds):
   416:     if hlevel>0:
   417:       self.head(hlevel,'File List')
   418:     if self.master.flist:
   419:       for line in self.master.flist:
   420:         self._writeline(line + '<BR>')
   421:     else:
   422:       self._writeline('<P>No data available in pass '+str(self.pass_frame.passno)+'.</P>')
   423: 


End python section to interscript/weavers/html.py[16]

6.4.5.14. Code File Status
--------------------------


Start python section to interscript/weavers/html.py[17]

   424: #line 477 "html_weaver.ipk"
   425:   def print_file_status(self,hlevel=2, *args, **kwds):
   426:     passno = self.pass_frame.passno
   427:     h = 'File Status for pass '+str(passno-1)
   428:     if hlevel>0:
   429:       self.head(hlevel,h)
   430:     if self.master.fdict:
   431:       skeys = self.master.fdict.keys()
   432:       skeys.sort()
   433: 
   434:       h = 'Unchanged Files'
   435:       if hlevel>0:
   436:         self.head(hlevel+1,h)
   437:       else:
   438:         self._writeline('<STRONG>'+h+'<STRONG><BR>')
   439:       for key in skeys:
   440:         status,change_passno = self.master.fdict[key]
   441:         if status == 'unchanged' and change_passno==0:
   442:           self._writeline(key+'<BR>')
   443: 
   444:       h = 'Changed Files'
   445:       if hlevel>0:
   446:         self.head(hlevel+1,h)
   447:       else:
   448:         self._writeline('<STRONG>'+h+'<STRONG><BR>')
   449:       for key in skeys:
   450:         status,change_passno = self.master.fdict[key]
   451:         if status == 'unchanged' and change_passno == 1:
   452:           self._writeline(key+'<BR>')
   453: 
   454:       h = 'Files which required 2 or more passes to converge'
   455:       if hlevel>0:
   456:         self.head(hlevel+1,h)
   457:       else:
   458:         self._writeline('<STRONG>'+h+'<STRONG><BR>')
   459:       for key in skeys:
   460:         status,change_passno = self.master.fdict[key]
   461:         if status == 'unchanged' and change_passno > 1:
   462:           self._writeline(key+' (converged in '+str(change_passno)+' passes)<BR>')
   463: 
   464:       h = 'Unstable Files'
   465:       if hlevel>0:
   466:         self.head(hlevel+1,h)
   467:       else:
   468:         self._writeline('<STRONG>'+h+'<STRONG><BR>')
   469:       for key in skeys:
   470:         status,change_passno = self.master.fdict[key]
   471:         if status == 'changed':
   472:           self._writeline(key+'<BR>')
   473: 
   474:     else:
   475:       self._writeline('<P>No data available in pass '+str(passno)+'.</P>')
   476: 


End python section to interscript/weavers/html.py[17]

6.4.5.15. Input File List
-------------------------


Start python section to interscript/weavers/html.py[18]

   477: #line 531 "html_weaver.ipk"
   478:   def print_source_list(self, hlevel=2, *args, **kwds):
   479:     if hlevel>0:
   480:       self.head(hlevel,'Source List')
   481:     if self.master.iflist:
   482:       for line in self.master.iflist:
   483:         self._writeline(line + '<BR>')
   484:     else:
   485:       self._writeline('<P>No data available in pass '+str(self.pass_frame.passno)+'.</P>')
   486: 


End python section to interscript/weavers/html.py[18]

6.4.5.16. Include List
----------------------


Start python section to interscript/weavers/html.py[19]

   487: #line 542 "html_weaver.ipk"
   488:   def print_include_list(self, hlevel=2, *args, **kwds):
   489:     if hlevel>0:
   490:       self.head(hlevel,'Include List')
   491:     if self.master.include_files:
   492:       for level, type, name in self.master.include_files:
   493:         self._writeline('&nbsp;'*(level*3)+' '+type+': '+name+ '<BR>')
   494:     else:
   495:       self._writeline('<P>No data available in pass '+str(self.pass_frame.passno)+'.</P>')
   496: 


End python section to interscript/weavers/html.py[19]

6.4.5.17. Tables
----------------


Start python section to interscript/weavers/html.py[20]

   497: #line 553 "html_weaver.ipk"
   498:   def begin_table(self, *headings, **kwds):
   499:     border=kwds.get('border',2)
   500:     tbclass = kwds.get('CLASS','DEFAULT_TABLE_CLASS')
   501:     self._writeline('<TABLE CLASS="'+tbclass+'" COLS="'+str(len(headings))+'" BORDER="'+str(border)+'"><TR>')
   502:     for h in headings:
   503:       self._write('<TH>')
   504:       self.write(h)
   505:       self._write('</TH>')
   506:     self._writeline('</TR>')
   507: 
   508:   def table_row(self,data):
   509:     self._write('<TR>')
   510:     for d in data:
   511:       self._write('<TD VALIGN="TOP">')
   512:       lines = string.split(d,'\n')
   513:       for line in lines:
   514:         self.write(line)
   515:         self._write('<BR>')
   516:       self._write('</TD>')
   517:     self._writeline('</TR>')
   518: 
   519:   def end_table(self):
   520:     self._writeline('</TABLE>')
   521: 


End python section to interscript/weavers/html.py[20]

6.4.5.18. HTML Header
---------------------


Start python section to interscript/weavers/html.py[21]

   522: #line 579 "html_weaver.ipk"
   523:   def prolog(self):
   524:     kwds = self.keywords
   525:     w = self._writeline
   526:     w('<DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">')
   527:     w( '<HTML>')
   528:     w( '<HEAD>')
   529: 
   530:     self.write_tagged('TITLE', self.title)
   531: 
   532:     w( '<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">')
   533:     if kwds.has_key('author'):
   534:       author =kwds['author']
   535:       w( '<META NAME="Author" CONTENT="'+author+'">')
   536:     w( '<META NAME="Generator" CONTENT="Interscript">')
   537:     extra = kwds.get('head_extra','');
   538:     w('<LINK REL=STYLESHEET TYPE="text/css" HREF="interscript.css" TITLE="Interscript Standard">')
   539:     w(extra);
   540:     w('<LINK REL=STYLESHEET TYPE="text/css" HREF="user.css" TITLE="User Overrride">')
   541:     w( '</HEAD>')
   542:     w( '<BODY>')
   543:     if kwds.has_key('pagehead'):
   544:       self._write(kwds['pagehead'])
   545:     if kwds.has_key('title'):
   546:       self._write('<H1 ALIGN="CENTER">'+self.title+'</H1>')
   547:     if kwds.has_key('heading_level_offset'):
   548:       self.heading_level_offset = kwds['heading_level_offset']
   549: 
   550:   def epilog(self):
   551:     kwds = self.keywords
   552:     self._endmode()
   553:     if kwds.has_key('pagefoot'):
   554:       self._write(kwds['pagefoot'])
   555:     self._writeline('</BODY>')
   556:     self._writeline('</HTML>')
   557: 


End python section to interscript/weavers/html.py[21]

6.4.5.19. Interscript Style Sheet
---------------------------------

The file 'interscript/doc/interscript.css' is a CSS1
style sheet for interscript documents. It is <LINK>ed
by interscript documentation, but is not currently
copied to the weaver output directory.

HTML documents also <LINK> a non-existant file,
'user.css' which allows the user to override the
interscript standard one.

Start data section to interscript/doc/interscript.css[1]

     1: BODY {
     2:   background-color : #FFFFF0;
     3: }
     4: BODY EM {
     5:   color: #A01010;
     6: }
     7: 
     8: BODY CODE {
     9:   color: #101080;
    10: }
    11: DIV.CODE {
    12:   color : #101080;
    13:   background-color: #E0FFFF;
    14:   margin-top: 0.0 ex;
    15:   padding-top: 0.4 ex;
    16:   padding-bottom: 0.2 ex;
    17:   margin-bottom: 0.0 ex;
    18:   margin-right: 2 ex;
    19:   border: thin solid gray;
    20:   display: block;
    21: }
    22: SPAN.LINENO {
    23:   color : #101010;
    24:   font-size: 80%;
    25: }
    26: 
    27: TABLE.DEFAULT_TABLE_CLASS {
    28:   color: #101010;
    29:   background-color: #F0F0E0;
    30: }
    31: 
    32: TABLE.DIFF {
    33:   color: #204060;
    34:   background-color: #FFE0E0;
    35: }
    36: 
    37: DIV.CODE_SECTION_HEAD {
    38:   margin-top: 1 ex;
    39:   padding-top: 0.2 ex;
    40:   border-top: 0 ex;
    41:   padding-bottom: 0.0 ex;
    42:   border-bottom: 0 ex;
    43:   margin-bottom: -1 ex;
    44:   padding-left: 0.1 em;
    45:   display: block;
    46: }
    47: DIV.CODE_SECTION_FOOT {
    48:   margin-top: -1.0 ex;
    49:   padding-top: 0.0 em;
    50:   border-top-width: 0.0 em;
    51:   border-bottom-width: 0.0 em;
    52:   padding-bottom: 0.0 em;
    53:   margin-bottom: 0.0 em;
    54:   display: block;
    55:   display:none;
    56: }
    57: 
    58: DIV.CODE_SECTION_HEAD SMALL, DIV.CODE_SECTION_FOOT SMALL{
    59:   color : #503020;
    60:   font-size: 80%;
    61:   display: inline;
    62:   color: #101080;
    63: }
    64: DIV.CODE_SECTION_HEAD STRONG, DIV.CODE_SECTION_FOOT STRONG {
    65:   font-family : monospace, courier;
    66:   font-weight: normal;
    67:   font-size: 100%;
    68:   display: inline;
    69:   color : black;
    70: }
    71: DIV.CODE_SECTION_HEAD EM, DIV.CODE_SECTION_FOOT EM {
    72:   display: inline;
    73: }
    74: 
    75: DIV.TEST_OUTPUT {
    76:   color : #101080;
    77:   background-color: #E0FFE0;
    78:   margin-top: 0.0 ex;
    79:   padding-top: 0.4 ex;
    80:   padding-bottom: 0.2 ex;
    81:   margin-bottom: 0.0 ex;
    82:   margin-right: 2 ex;
    83:   border: thin solid gray;
    84:   display: block;
    85: }
    86: DIV.BAD_TEST_OUTPUT {
    87:   color : #101080;
    88:   background-color: #FFE0E0;
    89:   margin-top: 0.0 ex;
    90:   padding-top: 0.4 ex;
    91:   padding-bottom: 0.2 ex;
    92:   margin-bottom: 0.0 ex;
    93:   margin-right: 2 ex;
    94:   border: thin solid gray;
    95:   display: block;
    96: }
    97: DIV.TEST_OUTPUT_RESULT {
    98:   margin-top: 1.5 ex;
    99:   padding-top: 0.2 ex;
   100:   border-top: 0 ex;
   101:   padding-bottom: 0.0 ex;
   102:   border-bottom: 0 ex;
   103:   margin-bottom: -1 ex;
   104:   padding-left: 0.1 em;
   105:   display: block;
   106: }
   107: DIV.TEST_OUTPUT_SECTION_HEAD {
   108:   margin-top: 1 ex;
   109:   padding-top: 0.2 ex;
   110:   border-top: 0 ex;
   111:   padding-bottom: 0.0 ex;
   112:   border-bottom: 0 ex;
   113:   margin-bottom: -1 ex;
   114:   padding-left: 0.1 em;
   115:   display: block;
   116: }
   117: 
   118: DIV.TEST_OUTPUT_SECTION_FOOT {
   119:   margin-top: -1.0 ex;
   120:   padding-top: 0.0 em;
   121:   border-top-width: 0.0 em;
   122:   border-bottom-width: 0.0 em;
   123:   padding-bottom: 0.0 em;
   124:   margin-bottom: 0.0 em;
   125:   display: block;
   126:   display:none;
   127: }
   128: 
   129: DIV.TEST_OUTPUT_SECTION_HEAD SMALL, DIV.TEST_OUTPUT_SECTION_FOOT SMALL{
   130:   color : #503020;
   131:   font-size: 80%;
   132:   display: inline;
   133:   color: #101080;
   134: }
   135: DIV.TEST_OUTPUT_SECTION_HEAD STRONG, DIV.TEST_OUTPUT_SECTION_FOOT STRONG {
   136:   font-family : monospace, courier;
   137:   font-weight: normal;
   138:   font-size: 100%;
   139:   display: inline;
   140:   color : black;
   141: }
   142: DIV.TEST_OUTPUT_SECTION_HEAD EM, DIV.TEST_OUTPUT_SECTION_FOOT EM {
   143:   display: inline;
   144: }
   145: 
   146: DIV.EXPECTED_OUTPUT {
   147:   color : #101080;
   148:   background-color: #E0FFFF;
   149:   margin-top: 0.0 ex;
   150:   padding-top: 0.4 ex;
   151:   padding-bottom: 0.2 ex;
   152:   margin-bottom: 0.0 ex;
   153:   margin-right: 2 ex;
   154:   border: thin solid gray;
   155:   display: block;
   156: }
   157: 
   158: DIV.EXPECTED_OUTPUT_SECTION_HEAD {
   159:   margin-top: 1 ex;
   160:   padding-top: 0.2 ex;
   161:   border-top: 0 ex;
   162:   padding-bottom: 0.0 ex;
   163:   border-bottom: 0 ex;
   164:   margin-bottom: -1 ex;
   165:   padding-left: 0.1 em;
   166:   display: block;
   167: }
   168: 
   169: DIV.EXPECTED_OUTPUT_SECTION_FOOT {
   170:   margin-top: -1.0 ex;
   171:   padding-top: 0.0 em;
   172:   border-top-width: 0.0 em;
   173:   border-bottom-width: 0.0 em;
   174:   padding-bottom: 0.0 em;
   175:   margin-bottom: 0.0 em;
   176:   display: block;
   177:   display:none;
   178: }
   179: 
   180: DIV.EXPECTED_OUTPUT_SECTION_HEAD SMALL, DIV.EXPECTED_OUTPUT_SECTION_FOOT SMALL{
   181:   color : #503020;
   182:   font-size: 80%;
   183:   display: inline;
   184:   color: #101080;
   185: }
   186: DIV.EXPECTED_OUTPUT_SECTION_HEAD STRONG, DIV.EXPECTED_OUTPUT_SECTION_FOOT STRONG {
   187:   font-family : monospace, courier;
   188:   font-weight: normal;
   189:   font-size: 100%;
   190:   display: inline;
   191:   color : black;
   192: }
   193: DIV.EXPECTED_OUTPUT_SECTION_HEAD EM, DIV.EXPECTED_OUTPUT_SECTION_FOOT EM {
   194:   display: inline;
   195: }
   196: 
   197: DIV.DIFF {
   198:   color : #101080;
   199:   background-color: #E0FFE0;
   200:   margin-top: 0.0 ex;
   201:   padding-top: 0.4 ex;
   202:   padding-bottom: 0.2 ex;
   203:   margin-bottom: 0.0 ex;
   204:   margin-right: 2 ex;
   205:   border: thin solid gray;
   206:   display: block;
   207: }
   208: 
   209: DIV.DIFF_SECTION_HEAD {
   210:   margin-top: 1 ex;
   211:   padding-top: 0.2 ex;
   212:   border-top: 0 ex;
   213:   padding-bottom: 0.0 ex;
   214:   border-bottom: 0 ex;
   215:   margin-bottom: -1 ex;
   216:   padding-left: 0.1 em;
   217:   display: block;
   218: }
   219: 
   220: DIV.DIFF_SECTION_FOOT {
   221:   margin-top: -1.0 ex;
   222:   padding-top: 0.0 em;
   223:   border-top-width: 0.0 em;
   224:   border-bottom-width: 0.0 em;
   225:   padding-bottom: 0.0 em;
   226:   margin-bottom: 0.0 em;
   227:   display: block;
   228:   display:none;
   229: }
   230: 
   231: DIV.DIFF_SECTION_HEAD SMALL, DIV.DIFF_SECTION_FOOT SMALL{
   232:   color : #503020;
   233:   font-size: 80%;
   234:   display: inline;
   235:   color: #101080;
   236: }
   237: DIV.DIFF_SECTION_HEAD STRONG, DIV.DIFF_SECTION_FOOT STRONG {
   238:   font-family : monospace, courier;
   239:   font-weight: normal;
   240:   font-size: 100%;
   241:   display: inline;
   242:   color : black;
   243: }
   244: DIV.DIFF_SECTION_HEAD EM, DIV.DIFF_SECTION_FOOT EM {
   245:   display: inline;
   246: }
   247: 


End data section to interscript/doc/interscript.css[1]

6.4.6. Html Auto Stacking Weaver
--------------------------------

This weaver creates a _single_ document consisting of a
heirarchy of HTML pages , using hyperlinks from parent
pages to child pages. The control weaver maintains a
stack of weavers, delegating most commands to the
weaver on the top of the stack.

However, when a heading command at a specified level is
trapped, a new weaver is created and pushed onto the
stack. When a heading at that same level is trapped
again, the weaver on the top of the stack is popped,
and a new weaver pushed in its place.

In this way, all headings at a certain level in a
document will create a new html document.

A stacking weaver requires a list of montonic
increasing integers to determine when to spawn a child
page. When a child is spawned by a level n heading,
that heading becomes the title of the spawned page, and
a heading of level n+m is sent to the child as a level
m heading.

On the other hand, it has to create separate sinks for
each new weaver, except the top level (main page). This
is currently done by supplying a prefix for the child
page filenames, the stacking weaver appends a number
representing the child number, in order the children
were spawned.

Implementation notes. We maintain a document stack.
Each document is triggered by the corresponding heading
level from the supplied trigger list (which had better
be strictly montonic increasing).

When we get a heading level n, we pop off weavers until
n is greater than the trigger level of that weaver.
(The test is performed first).

Then, we push documents onto the stack, until n is
greater than or equal to the document trigger level.
(The test is performed first).

Then, if n is the trigger level of the document, we use
it as a title and hyperlink the document into it's
parent, otherwise we just typeset the heading.

Note that the first loop will destroy a document whose
trigger level is equal to the heading level, and the
second loop will create a new document at this level.

6.4.6.1. Bugs
-------------

Summary: This weaver is not as flexible as it should
be.

Child document filenames are simply the given prefix
plus a sequentially generated count. The child weaver
is always the system html weaver.

Headings in child pages are adjusted to start at level
1. The 'print level' offset is not inherited from the
parent or controllable in any way.

Heading numbers in child documents never reflect the
position in the whole document, only the child page;
that is, the first heading in a child spawned by 2.3.4
will be 1, not 2.3.4.1.

Each page will have it's own table of contents. There's
no overall table for the document. (This will require
links to pages other than 'self').

There are no 'next, previous, home' type links
anywhere, these are necessary.

The spawning weaver assumes HTML. There's no 'in
principle' reason it shouldn't be possible to generate
other documents this way. Although, for example,
'hyperlinks' in printed documents as would be produced
by LaTeX will be of the form, 'see page 99 of such and
such', they're hyperlinks never the less, even if they
have to be followed by the 'manual' browser. :-)

In light of these comments, one can say the current
system confuses production of multi-page web documents,
and production of distinct separate documents (which
happen to be hyperlinked).

6.4.6.2. Code
-------------


Start python section to interscript/weavers/web.py[1]

     1: #line 87 "web_weaver.ipk"
     2: from interscript.weavers.multiplexor import multiplexor
     3: from interscript.weavers.html import html_weaver
     4: from interscript.weavers.html import cvt_code, cvt_text
     5: from interscript.drivers.sinks.bufdisk import named_file_sink
     6: from interscript.drivers.sinks.disk import simple_named_file_sink
     7: import string
     8: import traceback
     9: 
    10: 
    11: class stacking_weaver(multiplexor):
    12:   def mk_head(self, sink):
    13:     w = sink.writeline
    14:     w('<DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">')
    15:     w( '<HTML>')
    16:     w( '<HEAD>')
    17: 
    18:     w('<TITLE>'+self.title+'</TITLE>')
    19: 
    20:     w( '<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">')
    21:     if self.keywords.has_key('author'):
    22:       author =self.keywords['author']
    23:       self.frames_sink.writeline( '<META NAME="Author" CONTENT="'+author+'">')
    24:     version = self.process.global_frame.version
    25: 
    26:     w( '<META NAME="Generator" CONTENT="Interscript '+version+'">')
    27:     w( '</HEAD>')
    28: 
    29:   def mk_frame(self,name, orientation, subframes):
    30:     print 'Making frame',name
    31:     filename = self.pattern % name
    32:     sink = named_file_sink(
    33:       self.pass_frame,
    34:       self.basedir+filename,
    35:       self.master.weaver_prefix,
    36:       eol=self.eol)
    37:     self.mk_head(sink)
    38:     pc = ''
    39:     for frame_name,frame_percent,frame_src in subframes:
    40:       pc = pc + str(frame_percent)+'%,'
    41:     pc = pc[:-1]
    42:     w = sink.writeline
    43:     w( '<FRAMESET '+orientation+'="'+pc+'" scrolling="yes">')
    44:     for frame_name,frame_percent,frame_src in subframes:
    45:       w( '<FRAME NAME="'+frame_name+'" SRC="'+frame_src+'">')
    46:     w( '</FRAMESET>')
    47:     w( '</HTML>')
    48: 
    49:   def mk_frames(self, doc):
    50:     top = self.pattern % 'top' # top level frame
    51:     indexview = self.pattern % 'indexview'
    52:     indexnav = self.pattern % 'indexnav'
    53:     contents = self.pattern % 'toc'
    54:     classref = self.pattern % 'classref'
    55:     funcref = self.pattern % 'funcref'
    56:     identref = self.pattern % 'identref'
    57:     testref = self.pattern % 'testref'
    58:     filestatus = self.pattern % 'filestatus'
    59:     sectionref = self.pattern % 'sectionref'
    60: 
    61:     self.mk_frame('indexview','ROWS', (
    62:       ('indexnav',20,indexnav),
    63:       ('indexdat',80,contents)))
    64:     self.mk_frame('top','COLS',(
    65:       ('indexview',30,indexview),
    66:       ('docview',70,doc)))
    67: 
    68:     nav = '';
    69:     for filename, description in [
    70:       (contents, 'Contents'),
    71:       (classref, 'Classes'),
    72:       (funcref, 'Functions'),
    73:       (identref, 'Identifiers'),
    74:       (testref, 'Tests'),
    75:       (filestatus, 'Status'),
    76:       (sectionref, 'Sections')
    77:       ]:
    78:       nav = nav + '<A HREF="'+filename+'" TARGET="indexdat">'+description+'</A> '
    79:     nav = '<DIV CLASS="NAVIGATION">'+nav+'</DIV>'
    80: 
    81:     filename = self.pattern % 'indexnav'
    82:     sink = named_file_sink(
    83:       self.pass_frame,
    84:       self.basedir+filename,
    85:       self.master.weaver_prefix,
    86:       eol=self.eol)
    87:     self.mk_head(sink)
    88:     w = sink.writeline
    89:     w ('<BODY>')
    90:     w (nav)
    91:     w ('</BODY>')
    92:     w ('</HTML>')
    93: 
    94: 
    95:   def print_table(self,dict,sink):
    96:     keys = dict.keys()
    97:     keys.sort()
    98:     w = sink.writeline
    99:     w('<TABLE COLS="2" BORDER="1" CELLPADDING="2">')
   100:     for k in keys:
   101:       refs = dict[k]
   102:       w('<TR><TD VALIGN="Top"><CODE> '+k+' </CODE></TD><TD> ')
   103:       old_df = ''
   104:       for sf,sc,df,dc in refs:
   105:         key = (sf, sc)
   106:         if self.anchor_file.has_key(key):
   107:           child = self.anchor_file[key]
   108:         else:
   109:           child = ''
   110:         anchor = '<A HREF='+child+'#'+sf+':'+str(sc)+' TARGET="docview">'+str(dc)+'</A>'
   111: 
   112:         if old_df != '': w(', ')
   113:         if old_df != df:
   114:           w(df+': '+ anchor )
   115:           old_df = df
   116:         else:
   117:           w(anchor)
   118:       w('</TD></TR>')
   119:     w('</TABLE>')
   120: 
   121:   def __init__(self,parent_weaver, pattern='', break_list=None, **kwds):
   122:     self.pass_frame = parent_weaver.pass_frame
   123:     self.master = self.pass_frame.master
   124:     self.process = self.pass_frame.process
   125: 
   126:     self.sequence = self.pass_frame.get_new_sequence_number()
   127:     self.persistent_frame = self.master.get_persistent_frame(self.sequence)
   128: 
   129:     self.verbosity = self.pass_frame.verbosity
   130:     self.debug_missing_methods = 0
   131:     self.keywords = kwds
   132: 
   133:     self.original_filename = 'Unknown'
   134: 
   135:     self.toc_depth = self.keywords.get('toc_depth',99)
   136:     self.base = [parent_weaver]
   137:     parent_sink = parent_weaver.sink.name
   138:     self.basedir = string.join(string.split(parent_sink,'/')[:-1],'/')+'/'
   139:     if self.basedir == '/': self.basedir = ''
   140:     print 'Base directory for stacking weaver is',self.basedir
   141:     self.home_file = string.split(parent_sink,'/')[-1]
   142:     if kwds.has_key('title'):
   143:       self.title = kwds['title']
   144:     elif hasattr(parent_weaver,'title'):
   145:       self.title = parent_weaver.title
   146:     else:
   147:       self.title = self.home_file
   148:     self.pattern = pattern
   149:     if string.find(pattern,'%') == -1:
   150:       self.pattern = self.pattern+'_%s.html'
   151:     # break list
   152:     self.break_list = [0]
   153:     if break_list:
   154:       for b in break_list: self.break_list.append(b)
   155:     else:
   156:       for i in range(1,40): self.break_list.append(i)
   157: 
   158:     self.stack = [parent_weaver]
   159:     self.childcount = 0
   160:     self.debug = 0
   161:     self.protocol = 'stacking weaver'
   162:     self.hcount = [1,0]
   163:     self.acount = 0
   164:     self.anchor_file = {}
   165:     self.name = 'stacking weaver'
   166:     self.eol = parent_weaver.sink.eol
   167: 
   168:     # table of contents
   169:     self.toc = []
   170: 
   171:     self.home_anchor = '<A HREF="'+self.home_file+'">Home</A>'
   172:     self.home_nav = '<DIV CLASS="NAVIGATION">'+self.home_anchor+'<BR><HR></DIV>'
   173: 
   174:     self.mk_frames(self.home_file)
   175: 
   176:     # cheat here, guess next exists :-)
   177:     next = self.pattern%('%04d'%(self.childcount+1))
   178:     next = '<A HREF="'+next+'">Next</A>'
   179: 
   180:     nav = ''
   181:     if next : nav = nav + next+' '
   182: 
   183:     hnav = '<HR><DIV CLASS="NAVIGATION">'+nav+'<BR><HR></DIV>'
   184:     parent_weaver._writeline(hnav)
   185: 


End python section to interscript/weavers/web.py[1]

6.4.6.2.1. reference processor
------------------------------


Start python section to interscript/weavers/web.py[2]

   186: #line 273 "web_weaver.ipk"
   187:   def set_original_filename(self, filename):
   188:     self.original_filename = filename
   189: 
   190:   def set_anchor(self, label):
   191:     if not self.persistent_frame.has_key('anchors'):
   192:       self.persistent_frame['anchors']  = {}
   193:     href = self.pattern%('%04d'%(self.childcount))+'#'+label
   194:     self._write('<A NAME="'+label+'"></A>')
   195:     self.persistent_frame['anchors'][label]=href
   196: 
   197:   def get_anchor(self, label):
   198:     href = None
   199:     if self.persistent_frame.has_key('anchors'):
   200:       if self.persistent_frame['anchors'].has_key(label):
   201:         href =self.persistent_frame['anchors'][label]
   202:     return href
   203: 
   204:   def ref_anchor(self, label):
   205:     href = self.get_anchor(label)
   206:     if href:
   207:       self._write('<A HREF="'+href+'">'+label+'</A>')
   208:     else:
   209:       self._write('<EM>Unknown Label:'+label+'</EM>')
   210: 


End python section to interscript/weavers/web.py[2]

6.4.6.2.2. section processor
----------------------------

Process code sections.

Start python section to interscript/weavers/web.py[3]

   211: #line 300 "web_weaver.ipk"
   212:   def code_head(self, tangler, secno):
   213:     dst_filename = tangler.sink.name
   214:     self.set_anchor(dst_filename+'['+str(secno)+']')
   215:     for weaver in self.base:
   216:       weaver.code_head(tangler, secno)
   217: 


End python section to interscript/weavers/web.py[3]

6.4.6.2.3. heading processor
----------------------------

The title of the main document is level 0. The top
level content of that document is level 1. A heading
inserted into the document is level 1. After a level 1
heading is inserted, the immediately following content
is level 2.

The hcount list is a list of current heading numbers at
each level, including 0. The current document level is
the length of this list, minus 1. The last entry in the
list is always 0, except briefly when a heading is
being generated at that level.

Start python section to interscript/weavers/web.py[4]

   218: #line 317 "web_weaver.ipk"
   219:   def _current_level(self): return len(self.hcount)-1
   220:   def _pop_level(self): del self.hcount[-1]
   221:   def _push_level(self): self.hcount.append(0)
   222:   def _next_hnum(self): self.hcount[-1] = self.hcount[-1] + 1
   223: 
   224:   def head(self,level, text, atext='', anchor=''):
   225:     while level > self._current_level():
   226:       self._head(text, atext, anchor)  # synthesised dummy heading
   227:       self._push_level()
   228:     while level < self._current_level():
   229:       self._pop_level()
   230:       self._foot()
   231:     assert level == self._current_level()
   232:     self._next_hnum()
   233:     self._head(text, atext, anchor)
   234:     self._push_level()
   235:     assert self._current_level() == level + 1
   236: 


End python section to interscript/weavers/web.py[4]

6.4.6.2.4. document control
---------------------------

The break level of a page is the level of it title, so
the top level content of that page is 1 more than its
break level.

Start python section to interscript/weavers/web.py[5]

   237: #line 339 "web_weaver.ipk"
   238:   # if the current page has a break level equal to the current level pop it
   239:   # (we'd print a footer first if we used them)
   240:   def _foot(self):
   241:     if self._trig(0) == self._current_level():
   242:       self._popw()
   243: 
   244:   def _head(self,text, atext, anchor):
   245:     level = self._current_level()
   246:     if level == self._trig(1):
   247:       self._pushw(level,text,atext,anchor)
   248:     else:
   249:      self._ins_head(level,text,atext,anchor)
   250: 
   251:   def __del__(self):
   252:     try:
   253:       self.mk_funcref(self.pattern % 'funcref','docview')
   254:       self.mk_classref(self.pattern % 'classref','docview')
   255:       self.mk_identref(self.pattern % 'identref','docview')
   256:       self.mk_contents(self.pattern % 'toc','docview')
   257:       self.mk_testref(self.pattern % 'testref','docview')
   258:       self.mk_sectionref(self.pattern % 'sectionref','docview')
   259:       self.mk_filestatus(self.pattern % 'filestatus')
   260:     except:
   261:       print 'Error in stacking weaver __del__ routine'
   262:       traceback.print_exc()
   263: 
   264:   def get_formatted_heading_number(self, sep):
   265:     hnumber = ''
   266:     for i in range(1,len(self.hcount)-1):
   267:       hnumber = hnumber + str(self.hcount[i])+'.'
   268:     hnumber = hnumber + str(self.hcount[-1])
   269:     return hnumber
   270: 
   271:   def _popw(self):
   272:     if self.debug or self.verbosity>=6 :
   273:       print 'Terminating weaver',self.stack[-1].name
   274:     del self.stack[-1]
   275:     self.base = [self.stack[-1]]
   276: 
   277:   def _new_child(self):
   278:     self.childcount = self.childcount + 1
   279:     last = self.persistent_frame.get('lastchild',-1)
   280:     self.persistent_frame['lastchild']=max(last,self.childcount)
   281:     return self.pattern % ('%04d' % self.childcount)
   282: 
   283:   def _pushw(self,level,text,atext,anchor):
   284:     filename = self._new_child()
   285:     #print 'Spawning Weaver',filename,'for',text
   286:     self.base = [self.stack[-1]]
   287:     hn = self.get_formatted_heading_number('.')+'.'
   288:     hnum = self.get_formatted_heading_number('_')
   289:     h = hn + ' '+text
   290:     self.toc.append([level,hnum,filename,h])
   291:     self._ensuremode('P')
   292:     self._writeline( '<BR><A HREF="'+filename+'">'+h+'</A>')
   293: 
   294:     # create navigation links (kludge!)
   295:     home = self.home_anchor
   296:     up = None
   297:     try:
   298:       up = string.split(self.base[0].sink.name,'/')[-1]
   299:       up_anchor = '<A HREF="'+up+'">Up</A>'
   300:     except: pass
   301: 
   302:     if self.childcount > 1:
   303:       prev = self.pattern % ('%04d'%(self.childcount-1))
   304:       prev_anchor= '<A HREF="'+prev+'">Prev</A>'
   305:     else: prev = None
   306: 
   307:     last = self.persistent_frame.get('lastchild',None)
   308:     if last and self.childcount == last:
   309:       next = None
   310:     else:
   311:       next = self.pattern%('%04d'%(self.childcount+1))
   312:       next_anchor = '<A HREF="'+next+'">Next</A>'
   313: 
   314:     this = self.pattern%('%04d'%(self.childcount))
   315:     this_anchor = '<A HREF="'+this+'">This</A>'
   316: 
   317:     nav = ''
   318:     if next : nav = nav + next_anchor+' '
   319:     if up : nav = nav + up_anchor+' '
   320:     if prev : nav = nav + prev_anchor+' '
   321:     if home: nav = nav + home+' '
   322:     nav = nav + this_anchor+' <SMALL>['+self.original_filename+']</SMALL>'
   323: 
   324:     hnav = '<DIV CLASS="NAVIGATION">'+nav+'<BR><HR></DIV>'
   325:     fnav = '<DIV CLASS="NAVIGATION"><BR><HR>'+nav+'</DIV>'
   326: 
   327:     extra = '<LINK type="text/html" rel="Contents" href="'+self.pattern % 'toc' +'">\r\n'
   328:     extra = '<LINK type="text/html" rel="Identifiers" href="'+self.pattern % 'identref'+'">\r\n'
   329:     extra = '<LINK type="text/html" rel="Classes" href="'+self.pattern % 'classref'+'">\r\n'
   330:     extra = '<LINK type="text/html" rel="Functions" href="'+self.pattern % 'funcref'+'">\r\n'
   331:     extra = extra + '<LINK type="text/html" rel="Start" href="'+self.home_file+'">\r\n'
   332:     if next: extra = extra + '<LINK type="text/html" rel="Next" href="'+next+'">\r\n'
   333:     if prev: extra = extra + '<LINK type="text/html" rel="Prev" href="'+prev+'">\r\n'
   334:     if up: extra = extra + '<LINK type="text/html" rel="Section" href="'+up+'">\r\n'
   335: 
   336:     sink = named_file_sink(self.pass_frame, self.basedir + filename, self.master.weaver_prefix, self.eol)
   337:     child = html_weaver(self.pass_frame,sink,title = h, pagehead = hnav, pagefoot=fnav, heading_prefix=hn, head_extra=extra)
   338:     self.base = [child]
   339:     self.stack.append(child)
   340: 
   341:   def _trig(self,offset=0):
   342:     doc_level = len(self.stack)-1 # 0 origin
   343:     return self.break_list[doc_level+offset]
   344: 
   345:   def _ins_head(self,level,text,atext,anchor):
   346:     adjusted_level = level - self._trig()
   347:     if anchor == '':
   348:       anchor = 'a'+str(self.acount)
   349:       self.acount = self.acount + 1
   350:     hn = self.get_formatted_heading_number('.')+'.'
   351:     hnum = self.get_formatted_heading_number('_')
   352:     h = hn + ' '+text
   353:     filename = self.pattern%('%04d'%self.childcount)
   354:     self.toc.append([level,hnum,filename+'#'+anchor,h])
   355:     for weaver in self.base:
   356:       weaver.head(adjusted_level,text,atext,anchor)
   357: 


End python section to interscript/weavers/web.py[5]

6.4.6.2.5. Table of contents
----------------------------


Start python section to interscript/weavers/web.py[6]

   358: #line 461 "web_weaver.ipk"
   359:   def mk_contents(self,toc_filename,target):
   360:     self.toc_sink = named_file_sink(
   361:       self.pass_frame,
   362:       self.basedir+toc_filename,
   363:       self.master.weaver_prefix,
   364:       eol=self.eol)
   365:     self.mk_head(self.toc_sink)
   366:     w = self.toc_sink.writeline
   367:     w( '<BODY onload="loadHandler()">')
   368:     tocline= """
   369: <SCRIPT type="text/javascript">
   370: <!--
   371: function clickHandler() {
   372:   var targetId, srcElement, targetElement;
   373:   srcElement = window.event.srcElement;
   374:   if (srcElement.className == "Button") {
   375:     targetId = srcElement.id + "d";
   376:     targetElement = document.all(targetId);
   377:     if(targetElement.style.display == "none") {
   378:       targetElement.style.display = "";
   379:       srcElement.checked = true;
   380:     } else {
   381:       targetElement.style.display = "none";
   382:       srcElement.checked = false;
   383:     }
   384:   }
   385: }
   386: 
   387: // this function restores the status of the contents tree
   388: // IE4 preserves the buttons states, but not the visibility
   389: function loadHandler() {
   390:   for (var srcId in document.all) {
   391:     srcElement = document.all(srcId);
   392:     if (srcElement != null && srcElement.className == "Button") {
   393:       var targetId = srcId + "d";
   394:       var targetElement = document.all(targetId);
   395:       if(srcElement.checked == true) {
   396:         targetElement.style.display = "";
   397:       } else {
   398:         targetElement.style.display = "none";
   399:       }
   400:     }
   401:   }
   402: }
   403: document.onclick = clickHandler
   404: document.onload = loadHandler
   405: // -->
   406: </SCRIPT>
   407: """
   408:     if not target:
   409:       nav = '<DIV CLASS="NAVIGATION">'+self.home_anchor+' '+self.frames_anchor+'<BR><HR></DIV>'
   410:       self.toc_sink.writeline(nav)
   411:     w('<H1>Table of Contents</H1>')
   412:     w(tocline)
   413:     last_level = -1
   414:     if self.toc:
   415:       i = 0
   416:       level, hnum, href, text = self.toc[i]
   417:       next_level = 0
   418:       if len(self.toc) > i+1: next_level = self.toc[i+1][0]
   419:       self.emit_contents_line(level, hnum, href, text, level<next_level, target)
   420:       divid = 'h'+hnum+'d'
   421:       w('<DIV ID='+divid+' style="display:">')
   422:       last_level = level
   423:     for i in range(1, len(self.toc)):
   424:       level, hnum, href, text = self.toc[i]
   425:       next_level = 0
   426:       if len(self.toc) > i+1: next_level = self.toc[i+1][0]
   427:       while level <= last_level:
   428:         self.toc_sink.writeline('</DIV>')
   429:         last_level = last_level - 1
   430:       self.emit_contents_line(level, hnum, href, text, level<next_level, target)
   431:       divid = 'h'+hnum+'d'
   432:       w('<DIV ID='+divid+' style="display:">')
   433:       last_level = level
   434:     while 1 <= last_level:
   435:       w('</DIV>')
   436:       last_level = last_level - 1
   437:     w( '</BODY>')
   438:     w( '</HTML>')
   439:     del self.toc_sink
   440:     print 'stacked contents done'
   441: 
   442:   def emit_contents_line(self, level, hnum, href, text, enabled, target):
   443:     self.toc_sink.writeline('&nbsp;'*(3*level))
   444:     headid ='h'+hnum
   445:     flag = 'disabled'
   446:     if enabled: flag = ''
   447:     self.toc_sink.writeline('<INPUT ID='+headid+' CLASS="Button" TYPE="Radio" '+flag+'>')
   448:     if target:
   449:       self.toc_sink.writeline('<A HREF="'+href+'" TARGET="'+target+'">'+text+'</A>')
   450:     else:
   451:       self.toc_sink.writeline('<A HREF="'+href+'">'+text+'</A>')
   452: 


End python section to interscript/weavers/web.py[6]

6.4.6.2.6. Identifier Cross references
--------------------------------------

This is a bit hacky. When a tangler calls
'set_fc_anchor' with a file name and line number, an
anchor with the name

  original_file_name + ':' + str(original_lineno)

is generated. Tanglers which call this function supply
the original source file name and line number (not the
file and line being written to).

Tanglers generate an entry for the identifier occurence
which contains both the original source file and line
number, and the output code file and line number.

The cross reference table the html weaver generates
displays the output code file and line number (not the
orginal file and line number), but it generates a
reference of the form

  '#' + original_file_name + ':' + str(original_lineno)

using the original file name and line number.

The stacking weaver must trap calls to
'identifier_reference' because the hyperlinks the child
weaver charged with generating this table would fail to
refer to the specific html file containing the anchor.
Instead, the stacking weaver must build the table.

To do that, the stacking weaver must know in which html
child document the reference was generated so it can
generate a hyperlink of the form:

  childname + '#' + original_file_name + ':' + str(original_lineno)

Unfortunately, the ids dictionary in the global frame
currently being used to hold identifier cross
references on a document wide basis, does not have this
information available. It wouldn't make sense either,
the information in that table is generated by tanglers,
and has nothing to do with the weaver.

Therefore, the stacking weaver must _also_ intercept
calls to 'set_fc_anchor', and create an entry in a
dictionary keyed by the original filename and original
line number, which tells in which html file the anchor
was set: the stacking weaver knows that!

Start python section to interscript/weavers/web.py[7]

   453: #line 603 "web_weaver.ipk"
   454:   def mk_identref(self,filename,target):
   455:     sink = named_file_sink(
   456:       self.pass_frame,
   457:       self.basedir+filename,
   458:       self.master.weaver_prefix,
   459:       eol=self.eol)
   460:     self.mk_head(sink)
   461:     w = sink.writeline
   462:     w( '<BODY>')
   463: 
   464:     ids = self.master.ids
   465:     if len(ids) == 0:
   466:       ids = self.pass_frame.ids
   467:     w('<H1>Index of Identifiers</H1>')
   468:     self.print_table(ids,sink)
   469: 
   470:     w('</BODY>')
   471:     w('</HTML>')
   472: 
   473:   def mk_sectionref(self,filename,target):
   474:     sink = named_file_sink(
   475:       self.pass_frame,
   476:       self.basedir+filename,
   477:       self.master.weaver_prefix,
   478:       eol=self.eol)
   479:     self.mk_head(sink)
   480:     w = sink.writeline
   481:     w( '<BODY>')
   482: 
   483:     dict = self.pass_frame.section_index
   484:     w('<H1>Index of Sections</H1>')
   485:     keys = dict.keys()
   486:     keys.sort()
   487:     w = sink.writeline
   488:     w('<TABLE COLS="1" BORDER="1" CELLPADDING="2">')
   489:     for k in keys:
   490:       w('<TR><TD VALIGN="Top"><CODE> '+k+' </CODE>: ')
   491:       nsections = len(dict[k])
   492:       for i in range(nsections):
   493:         name = k + '['+str(i+1)+']'
   494:         anchor = '<A HREF='+self.get_anchor(name)+' TARGET="'+target+'">'+str(i+1)+'</A>'
   495:         w(anchor+' ')
   496:       w('</TD></TR>')
   497:     w('</TABLE>')
   498:     w('</BODY>')
   499:     w('</HTML>')
   500: 
   501:   def mk_classref(self,filename,target):
   502:     sink = named_file_sink(
   503:       self.pass_frame,
   504:       self.basedir+filename,
   505:       self.master.weaver_prefix,
   506:       eol=self.eol)
   507:     w = sink.writeline
   508:     self.mk_head(sink)
   509:     w( '<BODY>')
   510: 
   511:     ids = self.master.classes
   512:     if len(ids) == 0:
   513:       ids = self.pass_frame.classes
   514:     w('<H1>Index of Classes</H1>')
   515:     self.print_table(ids,sink)
   516:     w('</BODY>')
   517:     w('</HTML>')
   518: 
   519:   def mk_funcref(self,filename,target):
   520:     sink = named_file_sink(
   521:       self.pass_frame,
   522:       self.basedir+filename,
   523:       self.master.weaver_prefix,
   524:       eol=self.eol)
   525:     w = sink.writeline
   526:     self.mk_head(sink)
   527:     w( '<BODY>')
   528: 
   529:     ids = self.master.functions
   530:     if len(ids) == 0:
   531:       ids = self.pass_frame.functions
   532:     w('<H1>Index of Functions</H1>')
   533:     self.print_table(ids,sink)
   534: 
   535:     w('</BODY>')
   536:     w('</HTML>')
   537: 
   538:   def mk_testref(self,filename,target):
   539:     sink = named_file_sink(
   540:       self.pass_frame,
   541:       self.basedir+filename,
   542:       self.master.weaver_prefix,
   543:       eol=self.eol)
   544:     w = sink.writeline
   545:     self.mk_head(sink)
   546:     w( '<BODY>')
   547: 
   548:     ids = self.master.tests
   549:     if len(ids) == 0:
   550:       ids = self.pass_frame.tests
   551:     w('<H1>Index of Tests</H1>')
   552:     w('<TABLE CLASS="TEST_SUMMARY_TABLE" COLS="4" BORDER="1">')
   553:     w('<TR><TH>No</TH><TH>Description</TH><TH>Kind</TH><TH>Result</TH><TR>')
   554:     keys = ids.keys()
   555:     keys.sort()
   556:     for key in keys:
   557:       descr, label, kind, result = ids[key]
   558:       print 'test entry:',descr,label,kind,result
   559:       href = self.get_anchor(label)
   560:       print 'href=',href
   561:       w('<TR><TD>'+str(key)+'</TD><TD><A TARGET="'+target+'" HREF="'+href+'">'+descr+'</A></TD><TD>'+kind+'</TD><TD>'+result+'</TD></TR>')
   562:     w('</TABLE>')
   563:     w('</BODY>')
   564:     w('</HTML>')
   565: 
   566: 
   567:   def mk_filestatus(self,filename):
   568:     print 'Creating file status file:',filename
   569:     filestatus_output = simple_named_file_sink(
   570:       self.pass_frame,self.basedir+filename, self.master.weaver_prefix,eol='\r\n')
   571:     filestatus_weaver = html_weaver(
   572:       self.pass_frame,
   573:       filestatus_output,title='File Status')
   574:     filestatus_weaver.print_file_status(hlevel=1)
   575: 
   576:   def set_fc_anchor(self,file,count):
   577:     filename = self.base[0].sink.basename
   578:     self.anchor_file[(file,count)]=filename
   579:     for weaver in self.base:
   580:       weaver.set_fc_anchor(file,count)
   581: 
   582:   def heading_reference(self, *args, **kwds): pass # always generated
   583:   def identifier_reference(self, *args, **kwds): pass # always generated
   584:   def class_reference(self, *args, **kwds): pass # always generated
   585:   def function_reference(self, *args, **kwds): pass # always generated
   586:   def test_reference(self, *args, **kwds): pass # always generated
   587: 
   588: 


End python section to interscript/weavers/web.py[7]

6.4.6.3. Test Code
------------------


Start python section to interscript/tests/test_stacking_weaver.pak[1]

     1: #line 741 "web_weaver.ipk"
     2: @mysink = named_file_sink(get_pass_frame(),'interscript/tests/output/test_stacking_weaver.html',eol='\r\n')
     3: @weaver = html_weaver(get_pass_frame(),mysink, title='Stacking Weaver Test')
     4: @weaver = stacking_weaver(weaver,'swt_%s.html',[1,3,99,99,99])
     5: Simple test for stacking weaver : headings are:
     6: @begin_displayed_code()
     7: 1 (link)
     8:   1.1 (internal)
     9:   1.2 (internal)
    10:     1.2.1 (link)
    11:     1.2.2 (link)
    12:           1.2.2.1 (internal)
    13:           1.2.2.2 (internal)
    14:           1.2.2.3 (internal)
    15:     1.2.3
    16: 2
    17: 3.
    18: @end_displayed_code()
    19: Documents should be spawned at level 1 and 3.
    20: @head(1,'Heading 1')
    21: This is a stacking weaver test.
    22: @head(2,'Heading 1.1')
    23: This is a stacking weaver test.
    24: @head(2,'Heading 1.2')
    25: This is a stacking weaver test.
    26: @head(3,'Heading 1.2.1')
    27: This is a stacking weaver test.
    28: @head(3,'Heading 1.2.2')
    29: This is a stacking weaver test.
    30: @head(4,'Heading 1.2.2.1')
    31: This is a stacking weaver test.
    32: @head(4,'Heading 1.2.2.2')
    33: This is a stacking weaver test.
    34: @head(4,'Heading 1.2.2.3')
    35: This is a stacking weaver test.
    36: @head(3,'Heading 1.2.3')
    37: This is a stacking weaver test.
    38: @head(1,'Heading 2')
    39: This is a stacking weaver test.
    40: @head(1,'Heading 3')
    41: This is a stacking weaver test.


End python section to interscript/tests/test_stacking_weaver.pak[1]

Click
interscript/tests/output/test_stacking_weaver.html for
result. See diagnostics from the run below.

Start output section of /usr/local/bin/python interscript/bin/iscr.py --test interscript/tests/test_stacking_weaver.pak

     1: Interscript test mode, loading interscript from current directory
     2: Interscript Package: version 1.0a7 build 1302
     3: thread available
     4: Interscript version 1.0a7 build 1302
     5: Built by root on ruby at Thu Nov 12, 1998 at 12:57 PM (UTC)
     6: Generated by 1.0a7 buildno 1300 host ruby
     7: at Thu Nov 12, 1998 at 12:57 PM (UTC)
     8: cache= /mnt/user1/uestl/py/iscr/interscript/tests/test_stacking_weaver.pak.cache
     9: loaded master frame from cache
    10: Processing interscript/tests/test_stacking_weaver.pak Pass 0
    11: Base directory for stacking weaver is interscript/tests/output/
    12: Making frame indexview
    13: Making frame top
    14: stacked contents done
    15: Creating file status file: swt_filestatus.html
    16: Of 10 files, only 0 were stable on pass 0
    17: There were 10 new files,
    18: and 0 unstable files.
    19: Pickled persistent frames in /mnt/user1/uestl/py/iscr/interscript/tests/test_stacking_weaver.pak.cache

End output section of /usr/local/bin/python interscript/bin/iscr.py --test interscript/tests/test_stacking_weaver.pak

6.4.7. LaTeX Weaver
-------------------

This weaver generates LaTeX2e codes. You'll require a
full LaTeX2e installation to typeset it, along with
common standard packages such as makeidx.

Start python section to interscript/weavers/latex.py[1]

     1: #line 8 "latex_weaver.ipk"
     2: from interscript.weavers.base import weaver_base
     3: import string
     4: 
     5: def cvt_code(line):
     6:   v = '\\verb+'
     7:   for ch in line:
     8:     if ch != '+': v = v + ch
     9:     else: v = v + '+\\verb-+-\\verb+'
    10:   v = v + '+'
    11:   return v
    12: 
    13: def cvt_text(line):
    14:   l = ''
    15:   for ch in line:
    16:     if ch in '$&%#_^{}\\': l = l + '\\'
    17:     l = l + ch
    18:   return l
    19: 
    20: class latex_weaver(weaver_base):


End python section to interscript/weavers/latex.py[1]

6.4.7.1. Initialisation
-----------------------


Start python section to interscript/weavers/latex.py[2]

    21: #line 29 "latex_weaver.ipk"
    22:   def __init__(self, pass_frame, writer ,**kwds):
    23:     weaver_base.__init__(self, pass_frame)
    24:     self.verbosity = pass_frame.verbosity
    25:     self.sink = writer
    26:     if self.verbosity>=3:
    27:       print 'initialising latex weaver, writer',writer.get_sink_name()
    28:     self.protocol = 'LaTeX2e'
    29:     self.acount = 1
    30:     self.tag_stack = []
    31:     self.comments = 0
    32:     self.master = pass_frame.master
    33:     self.list = []
    34:     self.heading_level_offset = 0
    35:     self.name = 'latex2e weaver v1 for '+self.sink.name
    36:     self.verbatim = 0
    37:     self.prolog(kwds)
    38: 


End python section to interscript/weavers/latex.py[2]

6.4.7.2. Termination
--------------------


Start python section to interscript/weavers/latex.py[3]

    39: #line 48 "latex_weaver.ipk"
    40:   def __del__(self):
    41:     self.epilog()
    42: 


End python section to interscript/weavers/latex.py[3]

6.4.7.3. Identifier Cross Reference Table
-----------------------------------------


Start python section to interscript/weavers/latex.py[4]

    43: #line 53 "latex_weaver.ipk"
    44:   def identifier_reference(self, hlevel=2, *args, **kwds):
    45:     ids = self.master.ids
    46:     keys = ids.keys()
    47:     keys.sort()
    48:     if hlevel >0:
    49:       self.head(hlevel,'Index of Identifiers')
    50:     self._writeline('\\begin{tabular}{ll}')
    51:     for k in keys:
    52:       refs = ids[k]
    53:       self._write(cvt_code(k)+'&')
    54:       old_df = ''
    55:       for sf,sc,df,dc in refs:
    56:         if old_df != '': self._write(', ')
    57:         if old_df != df:
    58:           self._writeline(cvt_code(df)+': '+str(dc)+'\\ref{'+sf+':'+str(sc)+'}')
    59:           old_df = df
    60:         else:
    61:           self._write(': '+str(dc)+'\\ref{'+sf+':'+str(sc)+'}')
    62:       self._write('\\\\\n')
    63:     self._writeline('\\end{tabular}\n')
    64: 


End python section to interscript/weavers/latex.py[4]

6.4.7.4. Class Reference Table
------------------------------


Start python section to interscript/weavers/latex.py[5]

    65: #line 76 "latex_weaver.ipk"
    66:   def classs_reference(self, hlevel=2, *args, **kwds):
    67:     ids = self.master.classes
    68:     keys = ids.keys()
    69:     keys.sort()
    70:     if hlevel >0:
    71:       self.head(hlevel,'Index of Classes')
    72:     self._writeline('\\begin{tabular}{ll}')
    73:     for k in keys:
    74:       refs = ids[k]
    75:       self._write('\\verb+'+k+'+&')
    76:       old_df = ''
    77:       for sf,sc,df,dc in refs:
    78:         if old_df != '': self._write(', ')
    79:         if old_df != df:
    80:           self._writeline(cvt_code(df)+': '+str(dc)+'\\ref{'+sf+':'+str(sc)+'}')
    81:           old_df = df
    82:         else:
    83:           self._writeline(': '+str(dc)+'\\ref{'+sf+':'+str(sc)+'}')
    84:       self._write('\\\\\n')
    85:     self._writeline('\\end{tabular}\n')
    86: 
    87:   # create a Latex anchor
    88:   def set_fc_anchor(self,file,count):
    89:     self._write('\\label{'+file+':'+str(count)+'}')
    90: 


End python section to interscript/weavers/latex.py[5]

6.4.7.5. Table of Contents
--------------------------


Start python section to interscript/weavers/latex.py[6]

    91: #line 103 "latex_weaver.ipk"
    92:   def print_contents(self,*args,**kwds):
    93:     self._writeline('\\tableofcontents')
    94: 


End python section to interscript/weavers/latex.py[6]

6.4.7.6. Code File List
-----------------------


Start python section to interscript/weavers/latex.py[7]

    95: #line 108 "latex_weaver.ipk"
    96:   def print_file_list(self, hlevel=2, *args, **kwds):
    97:     self.head(hlevel,'File List')
    98:     for line in self.master.flist:
    99:       self._writeline(cvt_code(line))
   100: 


End python section to interscript/weavers/latex.py[7]

6.4.7.7. Source List
--------------------


Start python section to interscript/weavers/latex.py[8]

   101: #line 115 "latex_weaver.ipk"
   102:   def print_source_list(self, hlevel=2, *args, **kwds):
   103:     self.head(hlevel,'Source List')
   104:     for line in self.master.iflist:
   105:       self._writeline(cvt_code(line))
   106: 


End python section to interscript/weavers/latex.py[8]

6.4.7.8. Latex Preamble
-----------------------

This is a very complex section that provides the latex
preamble and document front matter. The basic command
is like: @w = latex_weaver(sink,documentclass='book',
documentclass_options=['a4paper','11pt'],topmargin='4pt').
[Typeset this as a display .. when the code is done for
that feature]

Start python section to interscript/weavers/latex.py[9]

   107: #line 127 "latex_weaver.ipk"
   108:   def prolog(self,kwds):
   109: 
   110:     # see Kopka pp25-27
   111:     # the default document class is for a book
   112:     # other standard classes include:
   113:     #   article report letter
   114: 
   115:     documentclass = 'book'
   116:     if kwds.has_key('documentclass'):
   117:       documentclass=kwds['documentclass']
   118: 
   119:     # the options are a python list of words
   120:     # for the standard book class they're from the set:
   121:     #   10pt 11pt 12pt
   122:     #   letterpaper legalpaper executivepaper
   123:     #   a4paper a5paper b5paper
   124:     #   landscape
   125:     #   onecolumn twocolumn
   126:     #   oneside twoside
   127:     #   openright openany
   128:     #   notitlepage titlepage
   129: 
   130:     # note: the default paper size Latex uses is
   131:     # american letterpaper. Don't count on this,
   132:     # I intend to make the ISO Standard A4 that everyone
   133:     # else uses the default!
   134: 
   135:     docopts = []
   136:     if kwds.has_key('documentclass_options'):
   137:       docopts =kwds['documentclass']
   138:     docoptstr=''
   139:     if docopts: docoptstr = docopts[0]
   140:     for opt in range(1,len(docopts)):
   141:      docoptstr = dosoptstr + ', ' + opt
   142:     self._writeline('\\documentclass['+docoptstr+']{'+documentclass+'}')
   143: 
   144:     if kwds.has_key('heading_level_offset'):
   145:       self.heading_level_offset = kwds['heading_level_offset']
   146: 
   147:     # page heading control
   148:     pagestyle = 'headings'
   149:     if kwds.has_key('pagestyle'):
   150:       pagestyle=kwds['pagestyle']
   151:     self._writeline('\\pagestyle{'+pagestyle+'}')
   152: 
   153:     pagenumbering= 'arabic'
   154:     if kwds.has_key('pagenumbering'):
   155:       pagenumbering=kwds['pagenumbering']
   156:     self._writeline('\\pagenumbering{'+pagenumbering+'}')
   157: 
   158:     # page layout
   159:     page_format_params = [
   160:       'topmargin','headheight','headsep','topskip','textheight','footskip',
   161:       'oddsidemargin','evensidemargin',
   162:       'textwidth']
   163:     for p in page_format_params:
   164:       if kwds.has_key(p):
   165:         param=kwds[p]
   166:         self._writeline('\\setlength{\\'+p+'}{'+param+'}')
   167: 
   168:     # lines and paragraphs
   169: 
   170:     # Note: we do _not_ permit indented paragraphs AT ALL.
   171:     # Don't even try it. FAR FAR too many things are broken
   172:     # by indentation.
   173: 
   174:     baselinestretch= 1
   175:     if kwds.has_key('baselinestretch'):
   176:       baselinestretch=kwds['baselinestretch']
   177:     self._writeline('\\renewcommand{\\baselinestretch}{'+str(baselinestretch)+'}')
   178: 
   179:     self._writeline('\\setlength{\\parskip 2mm plus 0.5mm minus 1mm}')
   180:     self._writeline('\\setlength{\\parindent 0mm}')
   181: 
   182:     self._writeline( '\\begin{document}')
   183:     if kwds.has_key('title'):
   184:       title=kwds['title']
   185:     else:
   186:       title = self.sink.pass_frame.master.filename
   187:     self._writeline('\\title{'+cvt_text(title)+'}')
   188:     if kwds.has_key('author'):
   189:       author =kwds['author']
   190:       self._writeline('\\author{'+cvt_text(author)+'}')
   191: 
   192:     self._writeline( '\\maketitle')
   193: 
   194:   def epilog(self):
   195:     self._writeline('\\end{document}')
   196: 


End python section to interscript/weavers/latex.py[9]

6.4.7.9. Body Output and Mode Control
-------------------------------------


Start python section to interscript/weavers/latex.py[10]

   197: #line 218 "latex_weaver.ipk"
   198:   def _writeline(self,line=''):
   199:     if self.enabled: self.sink.writeline(line)
   200: 
   201:   def _write(self,line):
   202:     if self.enabled: self.sink.write(line)
   203: 
   204:   def write(self,line):
   205:     if self.translating and not self.verbatim:
   206:       line = string.rstrip(line)
   207:       if line == '': return # don't print any blank lines
   208:       self._write(cvt_text(line))
   209:     else:
   210:       self._write(line)
   211: 
   212:   def writeline(self,line=''):
   213:     self.write(line);
   214:     self._writeline()
   215: 
   216:   def writecode(self,line):
   217:     self._writeline('\\hbox to 0pt{'+cvt_code(line)+'\\hss}\\\\')
   218: 
   219:   def begin_displayed_text(self):
   220:     self._write('\\begin{quote}\n')
   221: 
   222:   def end_displayed_text(self):
   223:     self._write('\\end{quote}\n')
   224: 
   225: 
   226:   def begin_displayed_code(self):
   227:     self._writeline('\\begin{verbatim}')
   228:     self.verbatim = 1
   229: 
   230:   def end_displayed_code(self):
   231:     self._writeline('\\end{verbatim}')
   232:     self.verbatim = 0
   233: 
   234:   def line_break(self):
   235:     self._writeline('\\newline')
   236: 
   237:   def page_break(self):
   238:     self._writeline('\\newpage')
   239: 
   240:   def write_tagged(self,tag, data):
   241:     self._write('{\\'+tag)
   242:     self._write(data)
   243:     self._write('}')
   244: 
   245: 
   246:   def code_head(self,tangler, secno):
   247:     if tangler:
   248:       self._write( '{\\par\\noindent\\small Start section to '+\
   249:         cvt_code(tangler.sink.get_sink_name())+\
   250:         '['+str(secno)+']}\\\\')
   251: 
   252:   def code_foot(self,tangler, secno):
   253:     if tangler:
   254:       self._write( '{\\small End section to '+\
   255:         cvt_code(tangler.sink.get_sink_name())+\
   256:         '['+str(secno)+']}')
   257: 


End python section to interscript/weavers/latex.py[10]

6.4.7.10. Headings
------------------


Start python section to interscript/weavers/latex.py[11]

   258: #line 280 "latex_weaver.ipk"
   259:   def head(self,level, text, atext='', anchor=''):
   260:     cmds = {
   261:       1:'\\part',
   262:       2:'\\chapter',
   263:       3:'\\section',
   264:       4:'\\subsection',
   265:       5:'\\subsubsection',
   266:       6:'\\paragraph',
   267:       7:'\\subparagraph'}
   268:     lev = level+self.heading_level_offset
   269:     if lev>7:lev=7
   270:     cmd = cmds[lev]
   271:     if anchor == '': anchor = atext
   272:     if anchor == '':
   273:       anchor = 'h'+str(self.acount)
   274:       self.acount = self.acount + 1
   275: 
   276:     if atext:
   277:       self._writeline(cmd+'['+atext+']{'+text+'}\\label{'+anchor+'}')
   278:     else:
   279:       self._writeline(cmd+'{'+text+'}\\label{'+anchor+'}')
   280: 


End python section to interscript/weavers/latex.py[11]

6.4.7.11. Tables
----------------


Start python section to interscript/weavers/latex.py[12]

   281: #line 304 "latex_weaver.ipk"
   282:   def begin_table(self, *headings, **kwds):
   283:     self._writeline('\\begin{table}[h]\\begin{tabular}{|'+'l|'*len(headings)+'}\hline')
   284:     self.write(headings[0])
   285:     for h in headings[1:]:
   286:       self._write('&')
   287:       self.write(h)
   288:     self._writeline(r'\\\hline')
   289: 
   290:   def table_row(self,data):
   291:     self.write(data[0])
   292:     for d in data[1:]:
   293:       self._write('&')
   294:       self.write(d)
   295:     self._writeline(r'\\')
   296: 
   297:   def table_rule(self):
   298:     self._writeline(r'\hline')
   299: 
   300:   def end_table(self):
   301:     self._writeline('\\hline\\end{tabular}\\end{table}')
   302: 


End python section to interscript/weavers/latex.py[12]

6.4.7.12. Lists
---------------


6.4.7.12.1. Numbered Lists
--------------------------


Start python section to interscript/weavers/latex.py[13]

   303: #line 328 "latex_weaver.ipk"
   304:   def begin_numbered_list(self,start=1, type='1'):
   305:     self._writeline('\\begin{enumerate}')
   306: 
   307:   def end_numbered_list(self):
   308:     self._writeline('\\end{enumerate}')
   309: 
   310:   def begin_numbered_list_item(self):
   311:     self._writeline('\\item ')
   312: 
   313:   def end_numbered_list_item(self):
   314:     pass
   315: 


End python section to interscript/weavers/latex.py[13]

6.4.7.12.2. Bullet Lists
------------------------


Start python section to interscript/weavers/latex.py[14]

   316: #line 342 "latex_weaver.ipk"
   317:   def begin_bullet_list(self):
   318:     self._writeline('\\begin{itemize}')
   319: 
   320:   def end_bullet_list(self):
   321:     self._writeline('\\end{itemize}')
   322: 
   323:   def begin_bullet_list_item(self):
   324:     self._write('\\item ')
   325: 
   326:   def end_bullet_list_item(self):
   327:     pass
   328: 


End python section to interscript/weavers/latex.py[14]

6.4.7.12.3. Keyed Lists
-----------------------


Start python section to interscript/weavers/latex.py[15]

   329: #line 356 "latex_weaver.ipk"
   330:   def begin_keyed_list(self):
   331:     self._writeline('\\begin{description}')
   332: 
   333:   def end_keyed_list(self):
   334:     self._writeline('\\end{description}')
   335: 
   336:   def begin_keyed_list_item(self,key):
   337:     self._write('\\item[')
   338:     self.write(key)
   339:     self._write(']')
   340: 
   341:   def end_keyed_list_item(self):
   342:     pass
   343: 


End python section to interscript/weavers/latex.py[15]

6.4.7.13. Code Output
---------------------


Start python section to interscript/weavers/latex.py[16]

   344: #line 372 "latex_weaver.ipk"
   345:   # default code line formatting
   346:   def echotangle(self,count,data):
   347:     if self.comments:
   348:       self._writeline(data)
   349:     else:
   350:       self.writecode("%6d: %s" % (count,data))
   351: 


End python section to interscript/weavers/latex.py[16]

6.4.7.14. Paragraphs
--------------------


Start python section to interscript/weavers/latex.py[17]

   352: #line 381 "latex_weaver.ipk"
   353:   def prose(self): # start of paragraph
   354:     self._write('\\noindent ')
   355: 
   356:   def par(self): # paragraph separator
   357:     self._write('\\par\n\\noindent ')
   358: 
   359:   def eop(self): # end of paragraph
   360:     self._write('\\par\n')
   361: 
   362:   def write_comment(self,v):
   363:     self.write_tagged('small',v)
   364: 


End python section to interscript/weavers/latex.py[17]

6.4.7.15. Fonts
---------------


Start python section to interscript/weavers/latex.py[18]

   365: #line 395 "latex_weaver.ipk"
   366:   def begin_code(self):
   367:     self._write('{\\tt ')
   368: 
   369:   def end_code(self):
   370:     self._write('}')
   371: 
   372:   def begin_emphasize(self):
   373:     self._write('{\\em ')
   374: 
   375:   def end_emphasize(self):
   376:     self._write('}')
   377: 
   378:   def begin_strong(self):
   379:     self._write('{\\bfseries ')
   380: 
   381:   def end_strong(self):
   382:     self._write('}')
   383: 
   384:   def begin_italic(self):
   385:     self._write('{\\itshape ')
   386: 
   387:   def end_italic(self):
   388:     self._write('}')
   389: 
   390:   def begin_bold(self):
   391:     self._write('{\\bfseries ')
   392: 
   393:   def end_bold(self):
   394:     self._write('}')
   395: 
   396:   def begin_big(self):
   397:     self._write('{\\large ')
   398: 
   399:   def end_big(self):
   400:     self._write('}')
   401: 
   402:   def begin_small(self):
   403:     self._write('{\\small ')
   404: 
   405:   def end_small(self):
   406:     self._write('}')
   407: 


End python section to interscript/weavers/latex.py[18]

6.4.7.16. Citations
-------------------


Start python section to interscript/weavers/latex.py[19]

   408: #line 439 "latex_weaver.ipk"
   409:   def cite_url(self,url):
   410:     self._write('{\\bfseries ')
   411:     self.write(url)
   412:     self._write('}')
   413: 


End python section to interscript/weavers/latex.py[19]

6.4.8. Unimplemented Weavers
----------------------------

In the following sections, I describe some weavers that
are to be considered standard. Some ought to be
implemented but haven't been yet, including Postscript,
Nroff for man pages, and some Microsoft formats. There
are some typesetting systems I don't think need
support, however. I think texinfo is archaic, as is
troff.

6.4.8.1. microsoft word weaver
------------------------------

isn't implemented yet. produces word files for some
popular version of word. help please!

6.4.8.2. microsoft help weaver
------------------------------

isn't implemented yet. produces micorsoft help files.

6.4.8.3. nroff weaver
---------------------

this unimplemented weaver is useful for producing man
pages on unix systems. man pages are considered
essential base documentation by some people.

6.4.8.4. lout weaver
--------------------

this weaver isn't implemented yet.

6.4.8.5. postscript weaver
--------------------------

this weaver isn't implemented yet. it should output
encapsulated postscript in some sensible format. it
shouldn't be too hard to write, but the results will
probably not be as pleasing as postscript produced from
a proper typesetting system.

6.4.9. auto weaver
------------------

make a weaver loom from the list of descriptors.

Start python section to interscript/weavers/auto.py[1]

     1: #line 193 "weavers.ipk"
     2: from interscript.weavers.multiplexor import multiplexor
     3: from interscript.weavers.text import plain_text_weaver
     4: from interscript.weavers.latex import latex_weaver
     5: from interscript.weavers.html import html_weaver
     6: from interscript.weavers.web import stacking_weaver
     7: from interscript.drivers.sinks.bufdisk import named_file_sink
     8: 
     9: def mk_html(pass_frame,basename, directory, prefix):
    10:  w = named_file_sink(pass_frame,directory+basename+'.html', prefix, eol='\r\n')
    11:  return html_weaver(pass_frame,w,title=basename)
    12: 
    13: def mk_web(pass_frame,basename, directory, prefix):
    14:   w = named_file_sink(pass_frame,directory+basename+'.html', prefix, eol='\r\n')
    15:   w = html_weaver(pass_frame,w,title=basename)
    16:   return stacking_weaver(w,basename+'_%s.html',(1,2,3,4,5,6,7,8,9,10))
    17: 
    18: def mk_latex(pass_frame,basename, directory, prefix):
    19:   w = named_file_sink(pass_frame,directory+basename+'.tex', prefix)
    20:   return latex_weaver(pass_frame,w)
    21: 
    22: def mk_text(pass_frame,basename, directory, prefix):
    23:   w = named_file_sink(pass_frame,directory+basename+'.txt', prefix)
    24:   return plain_text_weaver(pass_frame,w)
    25: 
    26: auto_weavers = {
    27:   'html': mk_html,
    28:   'web': mk_web,
    29:   'latex': mk_latex,
    30:   'text': mk_text
    31:   }
    32: 
    33: def auto_weaver(pass_frame,basename,autoweave):
    34:   prefix = pass_frame.master.weaver_prefix
    35:   directory = pass_frame.master.weaver_directory
    36:   weavers = []
    37:   for w in autoweave:
    38:     if auto_weavers.has_key(w):
    39:       weavers.append( auto_weavers[w](pass_frame, basename, directory, prefix) )
    40:   return multiplexor(pass_frame, weavers)
    41: 


End python section to interscript/weavers/auto.py[1]

6.5. Weaver Filters
-------------------


6.5.1. Markup filtering weaver
------------------------------

This weaver is a multiplexor with the additional
facility to perform search for markup codes in input
lines, and execute appropriate weaver constructions.

Note that only one regexp can be matched: chain filters
together to parse multiple markups shortcuts. Note that
the regexp must not be nullable, that is, it must never
match an empty string.

Note that the start and end method must be weaver
method names, not actual methods.

Start python section to interscript/weavers/filter.py[1]

     1: #line 17 "weaver_filters.ipk"
     2: from interscript.weavers.multiplexor import multiplexor
     3: import re
     4: class markup_filter(multiplexor):
     5:   def __init__(self,pass_frame,regexp,startmethod,endmethod,base=[]):
     6:     multiplexor.__init__(self,pass_frame,base)
     7:     self.regexp = re.compile(regexp)
     8:     self.startmethod = startmethod
     9:     self.endmethod = endmethod
    10:     self.protocol = ('regexp filter',regexp)
    11:     self.name = 'markup filter v1'
    12:     self.translating = 1
    13: 
    14:   def translate(self):
    15:     self.translating = 1
    16:     for weaver in self.base: weaver.translate()
    17: 
    18:   def raw(self):
    19:     self.translating = 0
    20:     for weaver in self.base: weaver.raw()
    21: 
    22:   def rawif(self, tag):
    23:     self.translating = 0
    24:     for weaver in base: weaver.rawif(tag)
    25: 
    26:   def write(self,data):
    27:      if not self.enabled: return
    28:      if not self.translating:
    29:        for weaver in self.base:
    30:          weaver.write(data)
    31:      else:
    32:        match = self.regexp.search(data)
    33:        startpos = 0
    34:        while match:
    35:          midpos = match.start(0)
    36:          endpos = match.end(0)
    37:          if midpos == endpos:
    38:            raise 'nullable regexp '+self.regexp.pattern
    39:          if startpos != midpos:
    40:            for weaver in self.base:
    41:              weaver.write(data[startpos:midpos])
    42:          for weaver in self.base:
    43:            exec 'weaver.'+self.startmethod+'()'
    44:            weaver.write(match.group(1))
    45:            exec 'weaver.'+self.endmethod+'()'
    46:          startpos = endpos
    47:          match = self.regexp.search(data,startpos)
    48:        if startpos != len(data):
    49:          for weaver in self.base:
    50:            weaver.write(data[startpos:])
    51: 
    52:   def writeline(self,data=''):
    53:     if not self.enabled: return
    54:     if data != '': self.write(data)
    55:     for weaver in self.base:
    56:       weaver.writeline()
    57: 
    58:   def begin_displayed_code(self):
    59:     self.translating = 0
    60:     for weaver in self.base:
    61:       try: weaver.begin_displayed_code()
    62:       except KeyboardInterrupt: raise
    63:       except: pass
    64: 
    65:   def end_displayed_code(self):
    66:     self.translating = 0
    67:     for weaver in self.base: weaver.end_displayed_code()
    68: 


End python section to interscript/weavers/filter.py[1]

6.5.2. Convenience commands
---------------------------

These functions are conveniences for creating filter
chains. The function chain_filters chains a sequence of
filters together, the first filter is called first, and
calls the second filter, etc.

the function create filters takes a sequence of triples
as an argument, and returns a sequence of filters: each
triple should consist of a regexp, start method name,
and end method name. The filters are not connected.

The function create_filter_chain combines these two
functions, it takes a sequence of triples and
constructs a filter chain. Such a connected chain is
also known as a pipe.

Start python section to interscript/weavers/filter.py[2]

    69: #line 99 "weaver_filters.ipk"
    70: def chain_filters(filters):
    71:   for i in range(0,len(filters)-1):
    72:     filters[i].base = [filters[i+1]]
    73: 
    74: def create_filters(triples):
    75:   filters = []
    76:   for regexp,startmethod,endmethod in triples:
    77:     filters.append(markup_filter(regexp,startmethod,endmethod))
    78:   return filters
    79: 
    80: def create_filter_chain(triples):
    81:   return chain_filters(create_filters(triples))
    82: 


End python section to interscript/weavers/filter.py[2]

6.6. Tanglers
-------------

A tangler is a driver that accepts lines of code in a
particular language or class of languages, writes it to
a code file, possibly pretty printing it, and displays
it in a weaver fixed at construction time.

Tanglers may also build tables of information about the
code.

The main method of a tangler is writeline, which
requires three arguments: the line of code to be
tangled, and the name of the original file from which
the line came, and the line number within the original
file from which the line came.

If the target language supports original source
references (like c's #line directive), the tangler
should also insert a directive into the code file using
the supplied arguments, unless the file name is the
same as the previous line's original source file, or
the line number is not one more than the previous
line's original line number.

The tangler should attach the original filename and
line number to it's sink device, since more than one
tangler may write to the same sink.

A tangler may supply a write_comment method accepting a
single argument, some text to be comments. The tangler
may assume the text does not contain any end of line
characters. It may write the text into the code file as
comments, not necessarily on one line; however, it may
not write anything which affects the semantics of the
program being generated. The comment should be woven as
ordinary program text.

A tangler may supply a method get_comment_tangler. If
it does, then, when called with no arguments, it shall
return a tangler which can be used to write a single
block of comments to the same sink and weaver as
itself.

A tangler may supply a method get_string_tangler. If it
does, it shall accept two or more arguments, the first
being an end of line string, and the second a width. A
string tangler so returned shall produce a string in
the target language, whose text in the target language
shall have the same natural language interpretation as
the text fed to the string tangler via calls to its
writeline method.

The string tangler may strip trailing, leading, or
interior whitespace from the data supplied via the
writeline method, and it may insert the supplied end of
line string in appropriate places. It may also
translate the encoding of the input to another
encoding, or translate the natural language input to
another natural language.

The string tangler may not insert end of line markers
other than the supplied marker. (The target operating
system for execution of the code is not known to the
string tangler. Also, the caller may supply a blank, to
inhibit multi-line output, or may supply a null string,
to force concatenation of non-blank text.

The width parameter is a hint to the string tangler
indicating approximately how long lines should be if
the string tangler is flowing, justifying, or otherwise
formatting the text as lines of a paragraph.

Note: the purpose of string tanglers is simply to allow
the programmer to write natural language text in
Interscript source files without having to worry about
escape characters, quoting, encodings, or line endings.
Text so provided may be intended to be output for user
help, as a message, or, as part of a run-time
documentation system.

Although Python currently supports only strings of 8
bit characters, a Japanese programmer might construct a
c string tangler that translated some multi-byte
encoding of Japanese characters into a c 'wide' string
literal with a 16 bit encoding.

Start python section to interscript/tanglers/__init__.py[1]

     1: #line 84 "tanglers.ipk"
     2: # tanglers
     3: 


End python section to interscript/tanglers/__init__.py[1]

6.6.1. Tangler Base class
-------------------------


Start python section to interscript/tanglers/base.py[1]

     1: #line 89 "tanglers.ipk"
     2: #---------------------------------------------------------
     3: # tangler base
     4: #---------------------------------------------------------
     5: class tangler_base:
     6:   __class_protocols__ = 'tangler'
     7:   def __init__(self,sink,weaver):
     8:     self.sink = sink
     9:     self.weaver = weaver
    10:     self.inhibit_sref = 0
    11:     self.pass_frame = weaver.pass_frame
    12:     self.master = self.pass_frame.master
    13:     self.process = self.master.process
    14:     self.language = 'data'
    15: 
    16:   def get_language(self): return self.language
    17: 
    18:   def write_comment(self,line):
    19:     pass
    20: 
    21:   def _writeline(self,data):
    22:     self.sink.writeline(data)
    23: 
    24:   def _write_and_echo(self,data):
    25:     self._writeline(data)
    26:     if self.weaver:
    27:       self.weaver.echotangle(self.sink.lines_written,data)
    28: 
    29:   def writeline(self,data,file,count, inhibit_sref=0):
    30:     if not inhibit_sref and not self.inhibit_sref:
    31:       if (file != self.sink.last_source_file or
    32:         count != self.sink.last_source_count+1):
    33:         self.start_section(file,count)
    34:     self.sink.last_source_file = file
    35:     self.sink.last_source_count = count
    36:     self._write_and_echo(data)
    37: 
    38:   def start_section(self,file,count): pass
    39: 
    40: #---------------------------------------------------------
    41: # builtin tanglers: null, data, c, script
    42: #---------------------------------------------------------
    43: 


End python section to interscript/tanglers/base.py[1]

6.6.2. Null Tangler
-------------------

A null tangler eats up input without writing it to
either a sink file or the documentation file. Note that
the start and end section markers are still written
(showing something was skipped over).

Start python section to interscript/tanglers/null.py[1]

     1: #line 138 "tanglers.ipk"
     2: #---------------------------------------------------------
     3: # null tangler
     4: # NOTE: a null tangler is _not_ the same as
     5: # some other tangler with a null sink:
     6: # null tanglers do _not_ write to the weaver!
     7: # Use a null tangler to remove files from the
     8: # source _and_ documentation
     9: 
    10: from interscript.tanglers.base import tangler_base
    11: class null_tangler(tangler_base):
    12:   def __init__(self,weaver=None):
    13:     tangler_base.__init__(self,null_sink(),weaver)
    14:     self.language = 'None'
    15: 
    16:   def get_comment_tangler(self):
    17:     return self
    18: 
    19:   def writeline(self,data,file,count,inhibit_sref=0):
    20:     pass
    21: 


End python section to interscript/tanglers/null.py[1]

6.6.3. Document Tangler
-----------------------

A document tangler writes to the documentation file
without actually writing any source file: it can be
used to present examples or code snippets, for example.

Start python section to interscript/tanglers/doc.py[1]

     1: #line 164 "tanglers.ipk"
     2: #---------------------------------------------------------
     3: # document tangler: writes text as doco,
     4: # doesn't generate a file. Not the same as 'no' tangler
     5: from interscript.tanglers.base import tangler_base
     6: 
     7: class doc_tangler(tangler_base):
     8:   def __init__(self,weaver):
     9:     tangler_base.__init__(self,null_sink(),weaver)
    10:     self.language = 'document'
    11: 
    12:   def get_comment_tangler(self):
    13:     return self
    14: 
    15:   def writeline(self,data,file,count,inhibit_sref=0):
    16:     self.weaver.writeline(data,file,count)
    17: 


End python section to interscript/tanglers/doc.py[1]

6.6.4. Data Tangler
-------------------

The data tangler simply write output to the attached
sink file and weaver, it does not parse the input in
any way or emit any line numbers. There is no
associated comment tangler either.

Start python section to interscript/tanglers/data.py[1]

     1: #line 186 "tanglers.ipk"
     2: #--------------------------------------------------
     3: # data tangler: write to a file
     4: # can not emit source line cross references
     5: from interscript.tanglers.base import tangler_base
     6: 
     7: class data_tangler(tangler_base):
     8:   pass
     9: 


End python section to interscript/tanglers/data.py[1]

6.6.5. c Tangler
----------------

The c tangler is used to write c code. It supports
#line directives refering to the original input.

The write_comment method can be used to insert comments
into a code file without them appearing in the
documentation file.

There is an associated comment tangler which writes
text as block of comments to the same sink: the whole
block is enclosed in a single /* */ pair and nicely
formatted. (The c comment tangler cannot interleave
comments).

Design note: The idea of comment tanglers was to get
rid of a 'mode' for tanglers. But the idea seems to
have problems.

There is also an associated string tangler which
generates native strings.

Start python section to interscript/tanglers/c.py[1]

     1: #line 214 "tanglers.ipk"
     2: #---------------------------------------------------------
     3: # c tangler: write to a file, insert source line numbers
     4: # using '#line' pre-processor directives
     5: from interscript.tanglers.base import tangler_base
     6: import re
     7: import string
     8: class c_tangler(tangler_base):
     9:   def __init__(self,sink,weaver):
    10:     tangler_base.__init__(self,sink,weaver)
    11:     self.matchid = re.compile('^[^A-Za-z_]*([A-Za-z_][A-Za-z_0-9]*)(.*)$')
    12:     self.language = 'C'
    13: 
    14:   def write_comment(self,line,file,count):
    15:     self.writeline('/* '+line+'*/')
    16: 
    17:   def start_section(self, file, count):
    18:     data = '#line '+str(count)+' '+'"'+file+'"'
    19:     self._writeline(data)
    20:     if self.weaver:
    21:       self.weaver.echotangle(self.sink.lines_written,data)
    22: 
    23:   def get_comment_tangler(self):
    24:     return c_comment_tangler(self.sink)
    25: 
    26:   def get_string_tangler(self,eol='\\n',width=0):
    27:     return c_string_tangler(self.sink,self.weaver,eol,width)
    28: 
    29:   def writeline(self,data,file,count,inhibit_sref=0):
    30:     tangler_base.writeline(self,data,file,count,inhibit_sref)
    31: 


End python section to interscript/tanglers/c.py[1]

6.6.5.1. c comment tangler
--------------------------


Start python section to interscript/tanglers/c.py[2]

    32: #line 246 "tanglers.ipk"
    33: #---------------------------------------------------------
    34: class c_comment_tangler(tangler_base):
    35:   def __init__(self,sink,weaver):
    36:     tangler_base.__init__(self,sink,weaver)
    37: 
    38:   def writeline(self,data,file,count,inhibit_sref=0):
    39:     if self.count == 0:
    40:       self._writeline('/* '+data)
    41:     else:
    42:       self._writeline(' * '+data)
    43:     self.weaver.writeline(data)
    44: 
    45:   def __del__(self):
    46:     self._writeline(' */')
    47: 


End python section to interscript/tanglers/c.py[2]

6.6.5.2. c string tangler
-------------------------


Start python section to interscript/tanglers/c.py[3]

    48: #line 263 "tanglers.ipk"
    49: #---------------------------------------------------------
    50: class c_string_tangler(tangler_base):
    51:   def __init__(self,sink,weaver,eol,width):
    52:     print 'Initialising c string tangler, eol=',eol,'width=',width
    53:     tangler_base.__init__(self,sink,weaver)
    54:     self.eol=eol
    55:     self.width=width
    56:     self.language = 'C string'
    57: 
    58:   def writeline(self,data,file,count,inhibit_sref=0):
    59:     data = string.rstrip(data) # remove trailing spaces
    60:     if self.width > 0: data = string.ljust(data, self.width)
    61:     line = '"'
    62:     for ch in data:
    63:       if ch in '\\"': line = line + '\\' + ch
    64:       else: line = line + ch
    65:     line = line + self.eol + '"'
    66:     self._writeline(line)
    67:     self.weaver.writeline(data)
    68: 


End python section to interscript/tanglers/c.py[3]

6.6.6. c++ Tangler
------------------

The c++ tangler is the same as the c tangler except it
writes c++ style comments and is associated with a
corresponding c++ comment tangler.

Start python section to interscript/tanglers/cpp.py[1]

     1: #line 290 "tanglers.ipk"
     2: #---------------------------------------------------------
     3: # c++ tangler: write to a file, insert source line numbers
     4: # using '#line' pre-processor directives
     5: from interscript.tanglers.base import tangler_base
     6: import re
     7: 
     8: class cpp_tangler(tangler_base):
     9:   def __init__(self,sink,weaver):
    10:     tangler_base.__init__(self,sink,weaver)
    11:     self.matchid = re.compile('^[^A-Za-z_]*([A-Za-z_][A-Za-z_0-9]*)(.*)$')
    12:     self.language = 'C++'
    13: 
    14:   def writeline(self,data,file,count,inhibit_sref=0):
    15:     tangler_base.writeline(self,data,file,count,inhibit_sref)
    16: 
    17:   def write_comment(self,line):
    18:     self._writeline('// '+line)
    19: 
    20:   def start_section(self, file, count):
    21:     data = '#line '+str(count)+' '+'"'+file+'"'
    22:     self._writeline(data)
    23:     if self.weaver:
    24:       self.weaver.echotangle(self.sink.lines_written,data)
    25: 
    26:   def get_comment_tangler(self):
    27:     return cpp_comment_tangler(self.sink, weaver)
    28: 
    29:   def get_string_tangler(self,eol,width):
    30:     return c_string_tangler(self.sink,self.weaver,eol,width)
    31: 
    32: 


End python section to interscript/tanglers/cpp.py[1]

6.6.6.1. Hash Tangler
---------------------


Start python section to interscript/tanglers/cpp.py[2]

    33: #line 323 "tanglers.ipk"
    34: class hash_comment_tangler(tangler_base):
    35:   def __init__(self, writer, weaver, prefix):
    36:     tangler_base.__init__(self,writer, weaver)
    37:     self.prefix = prefix
    38:     self.language = prefix+' comment'
    39: 
    40:   def writeline(self,data,file,count,inhibit_sref=0):
    41:     self.weaver.writeline(data)
    42:     self._writeline(self.prefix+line)
    43: 


End python section to interscript/tanglers/cpp.py[2]

6.6.6.2. c++ comment tangler
----------------------------


Start python section to interscript/tanglers/cpp.py[3]

    44: #line 335 "tanglers.ipk"
    45: class cpp_comment_tangler(hash_comment_tangler):
    46:   def __init__(self, writer,weaver):
    47:     hash_comment_tangler.__init__(self,writer,weaver,'// ')
    48: 


End python section to interscript/tanglers/cpp.py[3]

6.6.7. Java Tangler
-------------------

The Java tangler is the same as the c++ tangler, except
there are no #line directives. This is because I don't
know enough about Java to write a Java tangler. Help!

Start python section to interscript/tanglers/java.py[1]

     1: #line 345 "tanglers.ipk"
     2: from interscript.tanglers.base import tangler_base
     3: from interscript.tanglers.cpp import cpp_comment_tangler
     4: from interscript.tanglers.c import c_string_tangler
     5: import re
     6: class java_tangler(tangler_base):
     7:   def __init__(self,sink,weaver):
     8:     tangler_base.__init__(self,sink,weaver)
     9:     self.matchid = re.compile('^[^A-Za-z_]*([A-Za-z_][A-Za-z_0-9]*)(.*)$')
    10:     self.language = 'java'
    11: 
    12:   def writeline(self,data,file,count,inhibit_sref=0):
    13:     tangler_base.writeline(self,data,file,count,inhibit_sref)
    14: 
    15:   def write_comment(self,line):
    16:     self._writeline('// '+line)
    17: 
    18:   def start_section(self, file, count):
    19:     data = '//#line '+str(count)+' '+'"'+file+'"'
    20:     self._writeline(data)
    21:     if self.weaver:
    22:       self.weaver.echotangle(self.sink.lines_written,data)
    23: 
    24:   def get_comment_tangler(self):
    25:     return cpp_comment_tangler(self.sink)
    26: 
    27:   def get_string_tangler(self,eol,width):
    28:     return c_string_tangler(self.sink,self.weaver,eol,width)
    29: 


End python section to interscript/tanglers/java.py[1]

6.6.8. Tcl Tangler
------------------


Start python section to interscript/tanglers/tcl.py[1]

     1: #line 376 "tanglers.ipk"
     2: from interscript.tanglers.base import tangler_base
     3: from interscript.tanglers.cpp import hash_comment_tangler


End python section to interscript/tanglers/tcl.py[1]

Start python section to interscript/tanglers/tcl.py[2]

     4: #line 379 "tanglers.ipk"
     5: 
     6: class tcl_tangler(tangler_base):
     7:   def __init__(self,sink,weaver):
     8:     tangler_base.__init__(self,sink,weaver)
     9:     self.language = 'tcl'
    10: 
    11:   def writeline(self,data,file,count,inhibit_sref=0):
    12:     tangler_base.writeline(self,data,file,count,inhibit_sref)
    13: 
    14:   def write_comment(self,line):
    15:     self._writeline('# '+line)
    16: 
    17:   def start_section(self, file, count):
    18:     data = 'line '+str(count)+' '+'"'+file+'"'
    19:     self._writeline(data)
    20:     if self.weaver:
    21:       self.weaver.echotangle(self.sink.lines_written,data)
    22: 
    23:   def get_comment_tangler(self):
    24:     return hash_comment_tangler(self.sink,weaver, '# ')
    25: 
    26:   def get_string_tangler(self,eol,width=0):
    27:     # This is _wrong_ and needs to be fixed!
    28:     return c_string_tangler(self.sink,self.weaver,eol,width)
    29: 
    30: 


End python section to interscript/tanglers/tcl.py[2]

6.6.8.1. Python Tangler
-----------------------

The Python tangler is particularly important for
Interscript, because it is the native script language
in which Interscript is written, in which commands are
issued in source files, and with which it can be
extended by the user.

This very document is itself the Python source for
Interscript, and that leads to interesting
bootstrappping problems in development.

Start python section to interscript/tanglers/python.py[1]

     1: #line 13 "python_tangler.ipk"
     2: #---------------------------------------------------------
     3: # python tangler: write to a file, insert source line numbers
     4: # using '#line ' comments
     5: # works for Python
     6: from interscript.tanglers.base import tangler_base
     7: import re
     8: import string
     9: from interscript.tokenisers.python import python_tokeniser
    10: import keyword
    11: import token
    12: 
    13: class python_tangler(tangler_base):
    14:   def __init__(self,sink,weaver):
    15:     tangler_base.__init__(self,sink,weaver)
    16:     self.matchPOD = re.compile('^ *#@(.*)$')
    17:     self.matchcomment = re.compile('^([^#]*)#.*$')
    18:     self.excludeid = []
    19:     self.userdict = {}
    20:     self.tokeniser = python_tokeniser()
    21:     self.language = 'python'
    22: 
    23:   def __del__(self):
    24:     try:
    25:       tokens = self.tokeniser.close()
    26:     except:
    27:         print 'Tokeniser error'
    28:         print 'closing tokeniser for',self.sink.name
    29: 
    30:   def writeline(self,data,file,count,inhibit_sref=0):
    31:     match = self.matchPOD.match(data)
    32:     if match:
    33:       command = match.group(1)
    34:       py_exec(command,file,count,globals(),self.userdict)
    35:     else:
    36:       self.weaver.set_fc_anchor(file,count)
    37:       # special hack to preserve leading #! line
    38:       if self.sink.lines_written == 0 and len(data)>2:
    39:         inhibit_sref = data[:2]=='#!'
    40:       tangler_base.writeline(self,data,file,count, inhibit_sref)
    41: 
    42:       try:
    43:         tokens = self.tokeniser.tokenize(data+'\n')
    44:       except TokenError, e:
    45:         print 'Tokeniser error',e
    46:         print 'in file',file,'line',line
    47:         print 'data['+data+']'
    48: 
    49: 
    50:       dst_count = self.sink.lines_written
    51:       dst_file = self.sink.name
    52:       class_name = 0
    53:       function_name = 0
    54:       level = 0
    55:       for kind,id,lstart,lend,dummy in tokens:
    56:         if kind == token.INDENT:
    57:           level = level + 1
    58:         elif kind == token.DEDENT:
    59:           level = level - 1
    60:         if kind is token.NAME:
    61:           if not (keyword.iskeyword(id) or id in self.excludeid):
    62:             if not self.pass_frame.ids.has_key(id): self.pass_frame.ids[id]=[]
    63:             self.pass_frame.ids[id].append((file,count,dst_file,dst_count))
    64:             if class_name:
    65:               #print 'class',id
    66:               if not self.pass_frame.classes.has_key(id): self.pass_frame.classes[id]=[]
    67:               self.pass_frame.classes[id].append((file,count,dst_file,dst_count))
    68:               class_name = 0
    69:             elif function_name:
    70:               if not self.pass_frame.functions.has_key(id): self.pass_frame.functions[id]=[]
    71:               self.pass_frame.functions[id].append((file,count,dst_file,dst_count))
    72:               function_name = 0
    73:           elif id == 'class':
    74:             class_name = 1
    75:           elif id == 'def':
    76:             function_name = 1
    77: 
    78:   def write_comment(self,line,file,count):
    79:     self.writeline('# '+line,file,count)
    80: 
    81:   def start_section(self, file, count):
    82:     data = '#line '+str(count)+' '+'"'+file+'"'
    83:     self._writeline(data)
    84:     if self.weaver:
    85:       self.weaver.echotangle(self.sink.lines_written,data)
    86: 
    87:   def get_comment_tangler(self):
    88:     return script_comment_tangler(self.sink)
    89: 
    90:   def get_string_tangler(self,eol,width):
    91:     return c_string_tangler(self.sink,self.get_weaver(),eol,width)
    92: 


End python section to interscript/tanglers/python.py[1]

6.6.8.2. Python comment tangler
-------------------------------


Start python section to interscript/tanglers/python.py[2]

    93: #line 106 "python_tangler.ipk"
    94: class script_comment_tangler(tangler_base):
    95:   def writeline(self,data,file,count,inhibit_sref=0):
    96:     if self.weaver:
    97:       self.weaver.writeline(data)
    98:     self._writeline('# '+line)
    99: 


End python section to interscript/tanglers/python.py[2]

6.6.8.3. Perl Tangler
---------------------

This tangler attempts to support Perl's POD
constructions. It implements the commands pod, cut,
head1, head2, over, back, item, for, begin, and end.

Contrary to the perlpod manpage, Interscript pod
commands are terminated at the end of a line, not the
end of a 'paragraph'. It shouldn't make the slightest
bit of difference, however, since weavers ignore blank
lines anyhow.

When the tangler is in paragraph mode, blank lines are
translated to end of paragraph commands. Paragraph mode
is triggered by any non-command non-blank data lines,
so you won't get an end of paragraph after a command
before the first text.

Currently, support for the for/begin/end pod
constructions is a hack. Interscript uses a different
(better!) mechanism.

Start python section to interscript/tanglers/perl.py[1]

     1: #line 22 "perl_tangler.ipk"
     2: from interscript.tanglers.base import tangler_base
     3: from interscript.tanglers.c import c_string_tangler
     4: import re
     5: import string
     6: 
     7: class perl_tangler(tangler_base):
     8:   def __init__(self,sink,weaver, heading_level_offset = 2):
     9:     tangler_base.__init__(self,sink,weaver)
    10:     self.language = 'perl'
    11: 
    12:     self.mode = 'code'
    13:     self.list_type = []
    14:     self.pod_re = re.compile('^=([A-Za-z][A-Za-z0-9_]*) *(.*)$')
    15:     self.heading_level_offset = heading_level_offset
    16:     self.esc_re = re.compile('^(.*?)(>|[IBSCLFXZE]<)(.*)$')
    17:     self.digits_re = re.compile('^([0-9]+)>(.*)$')
    18:     self.entity_re = re.compile('^([A-Za-z]+)>(.*)$')
    19:     # this is not a full list, we should in fact call a weaver routine.
    20:     self.html_entity = {
    21:       'amp':'&',
    22:       'lt':'<',
    23:       'gt':'>',
    24:       'quot':'"',
    25:       'copy':'C',
    26:       'trade':'T',
    27:       'reg':'R'}
    28: 
    29:   def __del__(self):
    30:     self.flow_escape()
    31:     self.end_lists()
    32: 
    33:   def flow_escape(self):
    34:     line = self.flow_text
    35:     if not line: return
    36:     self.flow_text = ''
    37:     # process balanced text,
    38:     # if there is an unbalanced >, the text after it is returned
    39:     # write a >, and then try again.
    40:     tail = self.flow_parse(line)
    41:     while tail:
    42:       if verbosity >=4: print 'Unbalanced > in perl POD text'
    43:       self.weaver.write('>')
    44:       tail = self.flow_parse(tail)
    45: 
    46:   # recursive descent parser
    47:   def flow_parse(self,tail):
    48:     if not tail: return ''
    49:     weaver = self.weaver
    50: 
    51:     match = self.esc_re.match(tail)
    52:     while match:
    53:       pre, cmd, tail = match.group(1,2,3)
    54:       if pre: weaver.write(pre)
    55:       if cmd=='>': return tail
    56: 
    57:       assert len(cmd)==2 and cmd[1]=='<'
    58:       cmd = cmd[0]
    59:       if cmd == 'I':
    60:         weaver.begin_italic()
    61:         tail = self.flow_parse(tail)
    62:         weaver.end_italic()
    63:       elif cmd == 'B':
    64:         weaver.begin_bold()
    65:         tail = self.flow_parse(tail)
    66:         weaver.end_bold()
    67:       elif cmd == 'S':
    68:         # should be non-breaking spaces, but interscript
    69:         # doesn't implement that
    70:         tail = self.flow_parse(tail)
    71:       elif cmd == 'C':
    72:         weaver.begin_code()
    73:         tail = self.flow_parse(tail)
    74:         weaver.end_code()
    75:       elif cmd == 'L':
    76:         # a link: we just hack it for now
    77:         weaver.write('[')
    78:         tail = self.flow_parse(tail)
    79:         weaver.write(']')
    80:       elif cmd == 'F':
    81:         # filename
    82:         weaver.begin_code()
    83:         tail = self.flow_parse(tail)
    84:         weaver.end_code()
    85:       elif cmd == 'X':
    86:         # index entry??  (Does this mean print it, or index it?)
    87:         # I'll just print it as code :-)
    88:         weaver.begin_code()
    89:         tail = self.flow_parse(tail)
    90:         weaver.end_code()
    91:       elif cmd == 'Z':
    92:         # zero width character? What's that mean?
    93:         tail = self.flow_parse(tail)
    94:       elif cmd == 'E':
    95:         match = self.digits_re.match(tail)
    96:         if match:
    97:           digits, tail = match.group(1,2)
    98:           n = chr(int(digits))
    99:           weaver.write(n)
   100:         else:
   101:           match = self.entity_re.match(tail)
   102:           if match:
   103:             entity, tail = match.group(1,2)
   104:             data = self.html_entity.get(entity,'E<'+entity+'>')
   105:             weaver.write(data)
   106:           else:
   107:             # nothing we recognize, print literally
   108:             weaver.write('E<')
   109:             tail = self.flow_parse(tail)
   110:             weaver.write('>')
   111: 
   112:       match = self.esc_re.match(tail)
   113: 
   114:     # no (more) matches, so just weave the tail
   115:     self.weaver.writeline(tail)
   116:     return ''
   117: 
   118: 
   119:   def end_list_item(self):
   120:     kind = self.list_type[-1]
   121:     weaver = self.weaver
   122:     if kind == 'keyed': weaver.end_keyed_list_item()
   123:     elif kind == 'bullet': weaver.end_bullet_list_item()
   124:     elif kind == 'numbered': weaver.end_numbered_list_item()
   125: 
   126:   def end_list(self):
   127:     kind = self.list_type[-1]
   128:     weaver = self.weaver
   129:     if kind == 'keyed': weaver.end_keyed_list()
   130:     elif kind == 'bullet': weaver.end_bullet_list()
   131:     elif kind == 'numbered': weaver.end_numbered_list()
   132:     del self.list_type[-1]
   133: 
   134:   def end_lists(self):
   135:     while self.list_type: self.end_list()
   136: 
   137:   def begin_list(self,kind):
   138:     # print '** list type:',kind
   139:     self.list_type.append(kind)
   140:     weaver = self.weaver
   141:     if kind == 'keyed': weaver.begin_keyed_list()
   142:     elif kind == 'bullet': weaver.begin_bullet_list()
   143:     elif kind == 'numbered': weaver.begin_numbered_list()
   144: 
   145:   def begin_list_item(self,key=None):
   146:     kind = self.list_type[-1]
   147:     weaver = self.weaver
   148:     if kind == 'keyed': weaver.begin_keyed_list_item(key)
   149:     elif kind == 'bullet': weaver.begin_bullet_list_item()
   150:     elif kind == 'numbered': weaver.begin_numbered_list_item()
   151: 
   152:   def writeline(self,data,file,count,inhibit_sref=0):
   153:     if not inhibit_sref and not self.inhibit_sref:
   154:       if (file != self.sink.last_source_file or
   155:         count != self.sink.last_source_count+1):
   156:         self.start_section(file,count)
   157:     self.sink.last_source_file = file
   158:     self.sink.last_source_count = count
   159:     tangler_base._writeline(self,data)
   160: 
   161:     # try to find a pod command
   162:     pod = self.pod_re.match(data)
   163: 
   164:     # if we're in code mode, and we didn't
   165:     # get a pod command, just echotangle as code
   166:     # otherwise, switch to pod mode
   167: 
   168:     if self.mode == 'code':
   169:       if pod: self.mode = 'pod'
   170:       else:
   171:         self.weaver.echotangle(self.sink.lines_written,data)
   172:         return
   173: 
   174:     # now we're in pod mode, if we didn't get a pod command,
   175:     # strip the line to see if it's blank.
   176:     # if not, weave it and switching pod end of para detection on
   177:     # otherwise, emit an end of paragraph if detection is on
   178:     # unless we're in litpar mode, in which case we have to
   179:     # emulate an 'end' cmd
   180:     # pod_par means: 0 - begin of para, 1 - flowing text, 2 - literal text
   181:     assert self.mode == 'pod'
   182:     if not pod:
   183:       line = string.rstrip(data)
   184:       if line:
   185:         if not self.pod_par:
   186:           self.pod_par = (line[0] in ' \t')+1
   187:           if self.pod_par == 1: self.flow_text = ''
   188:         if self.pod_par-1:
   189:           self.weaver.writecode(line)
   190:         else:
   191:           # we have to search for escapes here!
   192:           self.flow_text = self.flow_text + line + ' '
   193:       elif self.pod_par:
   194:         self.flow_escape()
   195:         self.weaver.par()
   196:         self.pod_par = 0 # beginning of paragraph
   197:       return
   198: 
   199:     # we've got a pod command, so turn para detection off
   200:     assert pod
   201:     self.pod_par = 0
   202:     cmd = pod.group(1)
   203: 
   204:     # if we're cuttiung back to code, terminate lists and list
   205:     # items correctly if nececcary and switch back to code mode
   206: 
   207:     if cmd == 'cut':
   208:       self.end_lists()
   209:       if hasattr(self,'pod_mode'):
   210:         if self.pod_mode in ['lit','litpar']:
   211:           self.weaver.enable() # disable rawmode
   212:           self.weaver.translate() # disable rawmode
   213:         del self.pod_mode
   214:       self.mode = 'code'
   215:       return
   216: 
   217:     # Otherwise, just process the command
   218: 
   219:     if cmd == 'head1':
   220:       self.end_lists()
   221:       self.weaver.head(1+self.heading_level_offset, pod.group(2))
   222: 
   223:     elif cmd == 'head2':
   224:       self.end_lists()
   225:       self.weaver.head(2+self.heading_level_offset, pod.group(2))
   226: 
   227:     elif cmd == 'over':
   228:       # list of unknown type pending, wait for =item
   229:       self.pod_mode = 'list'
   230: 
   231:     elif cmd == 'back':
   232:       self.end_list_item()
   233:       self.end_list()
   234: 
   235:     elif cmd == 'item':
   236:       if not hasattr(self,'pod_mode'):
   237:         if verbosity >=2: print 'POD: item before over'
   238:         self.pod_mode = 'list'
   239:       key = pod.group(2)
   240:       key = string.strip(key)
   241:       if self.pod_mode == 'item':
   242:         self.end_list_item()
   243:       else:
   244:         self.pod_mode = 'item'
   245:         list_type = 'keyed'
   246:         if len(key)==1:
   247:           if key in '*+.-':
   248:             list_type = 'bullet'
   249:         self.begin_list(list_type)
   250:       if self.list_type[-1] == 'keyed':
   251:         # interscript doesn't support formatting of any kind
   252:         # in keyed list keys (because LaTeX doesn't)
   253:         # we need another kind of list (LaTeX can be given one)
   254:         # For now, we remove any X<...> stuff
   255:         stripkey = ''
   256:         tail = key
   257:         match = self.esc_re.match(tail)
   258:         while match:
   259:           pre, cmd, tail = match.group(1,2,3)
   260:           stripkey = stripkey + pre
   261:           match = self.esc_re.match(tail)
   262:         if tail: stripkey = stripkey + tail
   263:         key = stripkey
   264: 
   265:       self.begin_list_item(key)
   266: 
   267:     elif cmd == 'for':
   268:       self.weaver.rawif(pod.group(2))
   269:       self.pod_mode = 'litpar'
   270:     elif cmd == 'begin':
   271:       self.weaver.rawif(pod.group(2))
   272:       self.pod_mode = 'lit'
   273:     elif cmd == 'end':
   274:       self.weaver.enable()
   275:       self.weaver.translate()
   276:       self.weaver.pod_mode = ''
   277: 
   278:   def write_comment(self,line):
   279:     self._writeline('# '+line)
   280: 
   281:   def start_section(self, file, count):
   282:     data = '#line '+str(count)+' '+'"'+file+'"'
   283:     self._writeline(data)
   284:     self.weaver.echotangle(self.sink.lines_written,data)
   285: 
   286:   def get_comment_tangler(self):
   287:     return hash_comment_tangler(self.sink,weaver, '# ')
   288: 
   289:   def get_string_tangler(self,eol,width):
   290:     # This is _wrong_ and needs to be fixed!
   291:     return c_string_tangler(self.sink,self.get_weaver(),eol,width)
   292: 
   293: 


End python section to interscript/tanglers/perl.py[1]

6.7. Tokenisers
---------------

Tokenise various languages.

Start python section to interscript/tokenisers/__init__.py[1]

     1: #line 4 "tokenizers.ipk"
     2: # tokenisers package
     3: 


End python section to interscript/tokenisers/__init__.py[1]

6.7.1. Python Tokeniser
-----------------------

This module was modified from tokenize.py of the
standard library marked

  __version__ = "Ka-Ping Yee, 26 October 1997; patched, GvR 3/30/98'

The module provides tokenisation of python source code.

The module provides a class 'python_tokenize' and a
function 'tokenize'.

The function tokenize is provided for compatibility
with the original tokenize.py. It accepts up to four
arguments. The first argument, readline, is required
and is a callback function which fetches a line for
tokenisation. It should return a line with a trailing
newline character, or an empty string to indicate end
of input. The second argument, tokeneater, is a
callback which is called with each token as an
argument. If omitted, it defaults to a pretty-printing
routine which writes a formatted display of the token
to sys.stdout.

The class constructor and function accept two optional
arguments. The argument squashop defaults to 0 for the
class and 1 for the function. If set, all special
tokens are reported as token OP. The argument
report_comments defaults to 0 for the constructor and 1
for the function. If set, comments are reported as
COMMENTS, and blanks lines and mid-statement end of
lines are reported as NL.

If squashop and report_comments are both zero, the
result is 'pure' token stream suitable for parsing, if
both are set the result is more suitable for pretty
printing.

The class provides the following methods. The method
reset() resets the tokenizer state. The method write
accepts arbitrary text data. The method writeline shall
be called with a single line including a trailing
newline character, or with an empty string, indicating
end of input. The method get_tokens is called to fetch
tokens which have been produced and clears the token
queue. The method close signals end of input and
returns any trailing tokens. The method tokenize
accepts any text data and returns the tokens from the
queue. Tokens which span lines are report after the
line in which they are terminated is processed.

The format of a token consists of an integer token
index corresponding to python tokens as listed in the
file token.py, the lexeme which the token represents,
the starting and ending positions of the lexeme as
(line, column) pairs, and the source containing the
lexeme. Lines are numbered from 1.

Start python section to interscript/tokenisers/python.py[1]

     1: #line 49 "iscrtkpy.ipk"
     2: __version__ = "Ka-Ping Yee 1997/10/26; GvR 1998/3/20, Skaller 1998/7/24"
     3: 
     4: import string, re
     5: from token import *
     6: 
     7: COMMENT = N_TOKENS
     8: tok_name[COMMENT] = 'COMMENT'
     9: NL = N_TOKENS + 1
    10: tok_name[NL] = 'NL'
    11: WHITESPACE = N_TOKENS+2
    12: tok_name[WHITESPACE] = 'WHITESPACE'
    13: 
    14: 
    15: # Changes from 1.3:
    16: #     Ignore now accepts \f as whitespace.  Operator now includes '**'.
    17: #     Ignore and Special now accept \n or \r\n at the end of a line.
    18: #     Imagnumber is new.  Expfloat is corrected to reject '0e4'.
    19: # Note: to quote a backslash in a regex, it must be doubled in a r'aw' string.
    20: 
    21: def group(*choices): return '(' + string.join(choices, '|') + ')'
    22: def any(*choices): return apply(group, choices) + '*'
    23: def maybe(*choices): return apply(group, choices) + '?'
    24: 
    25: Whitespace = r'[ \f\t]*'
    26: Comment = r'#[^\r\n]*'
    27: Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment)
    28: Name = r'[a-zA-Z_]\w*'
    29: 
    30: Hexnumber = r'0[xX][\da-fA-F]*[lL]?'
    31: Octnumber = r'0[0-7]*[lL]?'
    32: Decnumber = r'[1-9]\d*[lL]?'
    33: Intnumber = group(Hexnumber, Octnumber, Decnumber)
    34: Exponent = r'[eE][-+]?\d+'
    35: Pointfloat = group(r'\d+\.\d*', r'\.\d+') + maybe(Exponent)
    36: Expfloat = r'[1-9]\d*' + Exponent
    37: Floatnumber = group(Pointfloat, Expfloat)
    38: Imagnumber = group(r'0[jJ]', r'[1-9]\d*[jJ]', Floatnumber + r'[jJ]')
    39: Number = group(Imagnumber, Floatnumber, Intnumber)
    40: 
    41: Single = any(r"[^'\\]", r'\\.') + "'"
    42: Double = any(r'[^"\\]', r'\\.') + '"'
    43: Single3 = any(r"[^'\\]",r'\\.',r"'[^'\\]",r"'\\.",r"''[^'\\]",r"''\\.") + "'''"
    44: Double3 = any(r'[^"\\]',r'\\.',r'"[^"\\]',r'"\\.',r'""[^"\\]',r'""\\.') + '"""'
    45: Triple = group("[rR]?'''", '[rR]?"""')
    46: String = group("[rR]?'" + any(r"[^\n'\\]", r'\\.') + "'",
    47:                '[rR]?"' + any(r'[^\n"\\]', r'\\.') + '"')
    48: 
    49: Operator = group('\+', '\-', '\*\*', '\*', '\^', '~', '/', '%', '&', '\|',
    50:                  '<<', '>>', '==', '<=', '<>', '!=', '>=', '=', '<', '>')
    51: Bracket = '[][(){}]'
    52: Special = group(r'\r?\n', r'[:;.,`]')
    53: Funny = group(Operator, Bracket, Special)
    54: 
    55: PlainToken = group(Number, Funny, String, Name)
    56: Token = Ignore + PlainToken
    57: 
    58: ContStr = group("[rR]?'" + any(r'\\.', r"[^\n'\\]") + group("'", r'\\\r?\n'),
    59:                 '[rR]?"' + any(r'\\.', r'[^\n"\\]') + group('"', r'\\\r?\n'))
    60: PseudoExtras = group(r'\\\r?\n', Comment, Triple)
    61: PseudoToken = Whitespace + group(PseudoExtras, Number, Funny, ContStr, Name)
    62: 
    63: tokenprog, pseudoprog, single3prog, double3prog = map(
    64:     re.compile, (Token, PseudoToken, Single3, Double3))
    65: endprogs = {"'": re.compile(Single), '"': re.compile(Double),
    66:             "'''": single3prog, '"""': double3prog,
    67:             "r'''": single3prog, 'r"""': double3prog,
    68:             "R'''": single3prog, 'R"""': double3prog, 'r': None, 'R': None}
    69: 
    70: opdict = {
    71:   '(':LPAR,
    72:   ')':RPAR,
    73:   '[':LSQB,
    74:   ']':RSQB,
    75:   ':':COLON,
    76:   ',':COMMA,
    77:   ';':SEMI,
    78:   '+':PLUS,
    79:   '-':MINUS,
    80:   '*':STAR,
    81:   '/':SLASH,
    82:   '|':VBAR,
    83:   '&':AMPER,
    84:   '<':LESS,
    85:   '>':GREATER,
    86:   '=':EQUAL,
    87:   '.':DOT,
    88:   '%':PERCENT,
    89:   '`':BACKQUOTE,
    90:   '{':LBRACE,
    91:   '}':RBRACE,
    92:   '==':EQEQUAL,
    93:   '!=':NOTEQUAL,
    94:   '<>':NOTEQUAL,
    95:   '<=':LESSEQUAL,
    96:   '>=':GREATEREQUAL,
    97:   '~':TILDE,
    98:   '^':CIRCUMFLEX,
    99:   '<<':LEFTSHIFT,
   100:   '>>':RIGHTSHIFT,
   101:   '**':DOUBLESTAR
   102:   }
   103: 
   104: tabsize = 8
   105: TokenError = 'TokenError'
   106: def printtoken(type, token, (srow, scol), (erow, ecol), line): # for testing
   107:     print "%d,%d-%d,%d:\t%s\t%s" % \
   108:         (srow, scol, erow, ecol, tok_name[type], repr(token))
   109: 
   110: def tokenise(readline, tokeneater=printtoken,squashop=1, report_comments=1):
   111:   t = python_tokeniser(squashop, report_comments)
   112:   line = readline()
   113:   while line:
   114:     t.writeline(line)
   115:     for token in t.tokens:
   116:       apply(tokeneater,token)
   117:     t.tokens = []
   118:     line = readline()
   119:   t.writeline('')
   120:   for token in t.tokens:
   121:     apply(tokeneater,token)
   122:   t.tokens = []
   123: 
   124: namechars, numchars = string.letters + '_', string.digits
   125: 
   126: class python_tokeniser:
   127:   def __init__(self, squashop=0, report_comments=0):
   128:     self.squashop = squashop
   129:     self.report_comments = report_comments
   130:     self.reset()
   131: 
   132:   def reset(self):
   133:     self.lnum = self.parenlev = self.continued = 0
   134:     self.contstr, self.needcont = '', 0
   135:     self.contline = None
   136:     self.indents = [0]
   137:     self.tokens = []
   138:     self.buffer = ''
   139: 
   140:   def get_tokens(self):
   141:     tmp = self.tokens
   142:     self.tokens = []
   143:     return tmp
   144: 
   145:   def tokenize(self,data):
   146:     self.write(data)
   147:     return self.get_tokens()
   148: 
   149:   def tokeneater(self,*args):
   150:     self.tokens.append(args)
   151: 
   152:   def close(self):
   153:     if self.buffer:
   154:       self.writeline(self.buffer)
   155:       self.buffer = ''
   156:     self.writeline('')
   157:     return self.get_tokens()
   158: 
   159:   def write(self,data):
   160:     lines = string.split(data,'\n')
   161:     if lines:
   162:       lines[0]=lines[0]+self.buffer
   163:       self.buffer = ''
   164:     for line in lines[:-1]:
   165:       self.writeline(line+'\n')
   166:     self.buffer = lines[-1]
   167: 
   168:   def writeline(self,line):
   169:     lnum = self.lnum = self.lnum + 1
   170:     pos, max = 0, len(line)
   171:     tokeneater = self.tokeneater
   172: 
   173:     if self.contstr:                                   # continued string
   174:         if not line:
   175:             raise TokenError, ("EOF in multi-line string", self.strstart)
   176:         endmatch = self.endprog.match(line)
   177:         if endmatch:
   178:             pos = end = endmatch.end(0)
   179:             tokeneater(STRING, self.contstr + line[:end],
   180:                        self.strstart, (lnum, end), self.contline + line)
   181:             self.contstr, self.needcont = '', 0
   182:             self.contline = None
   183:         elif self.needcont and line[-2:] != '\\\n' and line[-3:] != '\\\r\n':
   184:             tokeneater(ERRORTOKEN, self.contstr + line,
   185:                        self.strstart, (lnum, len(line)), self.contline)
   186:             self.contstr = ''
   187:             self.contline = None
   188:             return
   189:         else:
   190:             self.contstr = self.contstr + line
   191:             self.contline = self.contline + line
   192:             return
   193: 
   194:     elif self.parenlev == 0 and not self.continued:    # new statement
   195:         if not line: self._close(); return
   196: 
   197:         column = 0
   198:         while pos < max:                               # measure leading whitespace
   199:             if line[pos] == ' ': column = column + 1
   200:             elif line[pos] == '\t': column = (column/tabsize + 1)*tabsize
   201:             elif line[pos] == '\f': column = 0
   202:             else: break
   203:             pos = pos + 1
   204:         if pos == max: self._close(); return           # omitted newline
   205: 
   206:         if line[pos] in '#\r\n':                       # skip comments or blank lines
   207:             if self.report_comments:
   208:               tokeneater((NL, COMMENT)[line[pos] == '#'], line[pos:],
   209:                        (lnum, pos), (lnum, len(line)), line)
   210:             return
   211: 
   212:         if column > self.indents[-1]:                  # count indents or dedents
   213:             self.indents.append(column)
   214:             tokeneater(INDENT, line[:pos], (lnum, 0), (lnum, pos), line)
   215:         while column < self.indents[-1]:
   216:             self.indents = self.indents[:-1]
   217:             tokeneater(DEDENT, '', (lnum, pos), (lnum, pos), line)
   218: 
   219:     else:                                              # continued statement
   220:         if not line:
   221:             raise TokenError, ("EOF in multi-line statement", (lnum, 0))
   222:         self.continued = 0
   223: 
   224:     while pos < max:
   225:         pseudomatch = pseudoprog.match(line, pos)
   226:         if pseudomatch:                                # scan for tokens
   227:             start, end = pseudomatch.span(1)
   228:             spos, epos, pos = (lnum, start), (lnum, end), end
   229:             token, initial = line[start:end], line[start]
   230: 
   231:             if initial in numchars \
   232:                 or (initial == '.' and token != '.'):  # ordinary number
   233:                 tokeneater(NUMBER, token, spos, epos, line)
   234:             elif initial in '\r\n':
   235:                 if self.parenlev == 0:
   236:                   tokeneater(NEWLINE, token, spos, epos, line)
   237:                 elif self.report_comments:
   238:                   tokeneater(NL, token, spos, epos, line)
   239: 
   240:             elif initial == '#':
   241:                 if self.report_comments:
   242:                   tokeneater(COMMENT, token, spos, epos, line)
   243:             elif token in ("'''", '"""',               # triple-quoted
   244:                            "r'''", 'r"""', "R'''", 'R"""'):
   245:                 self.endprog = endprogs[token]
   246:                 endmatch = self.endprog.match(line, pos)
   247:                 if endmatch:                           # all on one line
   248:                     pos = endmatch.end(0)
   249:                     token = line[start:pos]
   250:                     tokeneater(STRING, token, spos, (lnum, pos), line)
   251:                 else:
   252:                     self.strstart = (lnum, start)      # multiple lines
   253:                     self.contstr = line[start:]
   254:                     self.contline = line
   255:                     break
   256:             elif initial in ("'", '"') or \
   257:                 token[:2] in ("r'", 'r"', "R'", 'R"'):
   258:                 if token[-1] == '\n':                  # continued string
   259:                     self.strstart = (lnum, start)
   260:                     self.endprog = endprogs[initial] or endprogs[token[1]]
   261:                     self.contstr, self.needcont = line[start:], 1
   262:                     self.contline = line
   263:                     break
   264:                 else:                                  # ordinary string
   265:                     tokeneater(STRING, token, spos, epos, line)
   266:             elif initial in namechars:                 # ordinary name
   267:                 tokeneater(NAME, token, spos, epos, line)
   268:             elif initial == '\\':                      # continued stmt
   269:                 self.continued = 1
   270:             else:
   271:                 if initial in '([{': self.parenlev = self.parenlev + 1
   272:                 elif initial in ')]}': self.parenlev = self.parenlev - 1
   273:                 if self.squashop:
   274:                   tokeneater(OP, token, spos, epos, line)
   275:                 else:
   276:                   op = opdict[token]
   277:                   tokeneater(op, token, spos, epos, line)
   278:         else:
   279:             tokeneater(ERRORTOKEN, line[pos],
   280:                        (lnum, pos), (lnum, pos+1), line)
   281:             pos = pos + 1
   282: 
   283: 
   284:   def _close(self):
   285:       for indent in self.indents[1:]:          # pop remaining indent levels
   286:           self.tokeneater(DEDENT, '', (self.lnum, 0), (self.lnum, 0), '')
   287:       self.tokeneater(ENDMARKER, '', (self.lnum, 0), (self.lnum, 0), '')
   288: 
   289: if __name__ == '__main__':                     # testing
   290:     import sys
   291:     if len(sys.argv) > 1: tokenize(open(sys.argv[1]).readline)
   292:     else: tokenize(sys.stdin.readline)
   293: 


End python section to interscript/tokenisers/python.py[1]

6.7.2. Parsers
--------------

While a tangler is used to 'pretty print' code in some
language, a parser, or 'meta-tangler' is used to
actually execute it. A meta-tangler has a writeline
method just like any other tangler object, however
instead of writing output to a code file and weaver, it
translates the input and executes it.

Start python section to interscript/parsers/__init__.py[1]

     1: #line 12 "parsers.ipk"
     2: # input parsers


End python section to interscript/parsers/__init__.py[1]

6.7.2.1. HTML Parser
--------------------

We provide an HTML meta tangler. The sgml_wrapper class
maps writeline calls from the control algorithm to feed
calls of on the parser.

Construction of an html_filter object feeds the parser
with an initial <HTML> tag. Termination semantics are
as follows: if the external data source becomes
exhausted, processing of buffered data should be forced
by calling the close method of the sgml_wrapper or
html_filter object.

If a </HTML> ending tag is detected, an eoi exception
is thrown.

In either case, reset() me be called to reinitialise
the object state, or the object can be destroyed. Note
that the current implementation dispatches tags to
global methods, rather than to a weaver bound to the
object. This is to permit bindings to Interscript
operations other than weaving.

Embedded Python and Tcl (if supported) can be executed
in the Interscript environment using the <SCRIPT> tag
as follows:

  <SCRIPT LANGUAGE="Python"><!--
    print "Hello World"
  #-->
  </SCRIPT>
  <SCRIPT LANGUAGE="Tcl"><!--
    puts "Hello World"
  #-->

Note that the used of comments is _not_ optional. If an
error is detected during execution, a diagnostic will
be printed but will not terminate continued processing
of the document beyond the ending SCRIPT tag.

Start python section to interscript/parsers/html.py[1]

     1: #line 39 "html_parser.ipk"
     2: from interscript.drivers.sources.base import eoi
     3: import string
     4: import traceback
     5: class sgml_wrapper:
     6:   def __init__(self, sgml):
     7:     self.sgml = sgml
     8: 
     9:   def writeline(self,data,file,count):
    10:     self.sgml.feed(data)
    11: 
    12:   def close():
    13:     self.sgml.close(self)
    14: 
    15:   def reset(self):
    16:     self.sgml.reset()
    17: 
    18: # this is a hack: sgmllib needs to be imported here
    19: # so the class SGMLParser defined in it can be used as a base
    20: import sgmllib
    21: 
    22: class html_filter(sgmllib.SGMLParser):
    23:   def __init__(self, input_frame):
    24:     sgmllib.SGMLParser.__init__(self)
    25:     self.save_data = 0
    26:     self.script_language = ''
    27:     self.input_frame = input_frame
    28:     self.weaver = input_frame.get_weaver()
    29:     self.verbosity = input_frame.verbosity
    30: 
    31:     # feeding <HTML> in here is a hack to get around a bug in sgmllib,
    32:     # which fails to process unbalanced end tags correctly
    33:     self.feed('<HTML>')
    34: 
    35:   def _save(self):
    36:     self.save_data = 1
    37:     self.saved_data = ''
    38:   def _saved(self):
    39:     self.save_data = 0
    40:     return self.saved_data
    41: 
    42:   def handle_data(self,data):
    43:     new_data = ''
    44:     for ch in data:
    45:       if ch == '\n': ch = ' \n'
    46:       new_data = new_data + ch
    47:     if self.save_data:
    48:       self.saved_data = self.saved_data + new_data
    49:     else:
    50:       self.weaver.write(new_data)
    51: 
    52:   def handle_comment(self,data):
    53:     if self.verbosity>=5: print 'SGML comment',data
    54:     if self.script_language != '':
    55:       self.saved_comments = self.saved_comments + data
    56: 
    57:   def start_html(self, attributes): pass
    58:   def start_head(self, attributes): pass
    59:   def end_head(self): pass
    60:   def start_body(self, attributes): pass
    61:   def end_body(self): pass
    62:   def end_html(self):
    63:     del self.input_frame
    64:     del self.weaver
    65:     raise eoi
    66: 
    67: # fonts
    68:   def start_b(self,attributes): self.weaver.begin_bold()
    69:   def end_b(self): self.weaver.end_bold()
    70: 
    71:   def start_i(self,attributes): self.weaver.begin_italic()
    72:   def end_i(self): self.weaver.end_italic()
    73: 
    74:   def start_em(self,attributes): self.weaver.begin_emphasize()
    75:   def end_em(self): self.weaver.end_emphasize()
    76: 
    77:   def start_strong(self,attributes): self.weaver.begin_strong()
    78:   def end_strong(self): self.weaver.end_strong()
    79: 
    80:   def start_small(self,attributes): self.weaver.begin_small()
    81:   def end_small(self): self.weaver.end_small()
    82: 
    83:   def start_big(self,attributes): self.weaver.begin_big()
    84:   def end_big(self): self.weaver.end_big()
    85: 
    86:   def start_code(self,attributes): self.weaver.begin_code()
    87:   def end_code(self): self.weaver.end_code()
    88: 
    89: # paragraphs
    90:   def start_p(self,attributes): self.weaver.prose()
    91:   def end_p(self): self.weaver.eop()
    92: 
    93: # displays
    94:   def start_pre(self,attributes): self.weaver.begin_displayed_code()
    95:   def end_pre(self): self.weaver.end_displayed_code()
    96: 
    97: #lists
    98:   def start_ol(self,attributes):
    99:     self.weaver.begin_numbered_list()
   100:     self.list_kind = 'ol'
   101:   def end_ol(self):
   102:     self.weaver.end_numbered_list()
   103: 
   104:   def start_dl(self,attributes):
   105:     self.weaver.begin_keyed_list()
   106:     self.list_kind = 'dl'
   107:   def end_dl(self):
   108:     self.weaver.end_keyed_list()
   109: 
   110:   def start_ul(self,attributes):
   111:     self.weaver.begin_bullet_list()
   112:     self.list_kind = 'ul'
   113:   def end_ul(self):
   114:     self.weaver.end_bullet_list()
   115: 
   116: #list items
   117:   def start_li(self,attributes):
   118:     if self.list_kind == 'ol':
   119:       self.weaver.begin_numbered_list_item()
   120:     else:
   121:       self.weaver.begin_bullet_list_item()
   122: 
   123:   def end_li(self):
   124:     if self.list_kind == 'ol':
   125:       self.weaver.end_numbered_list_item()
   126:     else:
   127:       self.weaver.end_bullet_list_item()
   128: 
   129:   def start_dt(self,attributes): self._save()
   130:   def end_dt(self):
   131:     self.weaver.begin_keyed_list_item(self._saved())
   132: 
   133:   def start_dd(self,attributes): pass
   134:   def end_dd(self): self.weaver.end_keyed_list_item()
   135: 
   136: #headings
   137:   def start_h1(self,attributes): self._save()
   138:   def end_h1(self): self.weaver.head(1,self._saved())
   139: 
   140:   def start_h2(self,attributes): self._save()
   141:   def end_h2(self): self.weaver.head(2,self._saved())
   142: 
   143:   def start_h3(self,attributes): self._save()
   144:   def end_h3(self): self.weaver.head(3,self._saved())
   145: 
   146:   def start_h4(self,attributes): self._save()
   147:   def end_h4(self): self.weaver.head(4,self._saved())
   148: 
   149:   def start_h5(self,attributes): self._save()
   150:   def end_h5(self): self.weaver.head(5,self._saved())
   151: 
   152:   def start_h6(self,attributes): self._save()
   153:   def end_h6(self): self.weaver.head(6,self._saved())
   154: 
   155:   def unknown_starttag(self,tag,attributes):
   156:     print 'UNKNOWN START TAG',tag,attributes
   157: 
   158:   def unknown_endtag(self,tag):
   159:     print 'UNKNOWN END TAG',tag
   160: 
   161:   def unknown_charref(self,ref):
   162:     print 'BAD CHAR REF',ref
   163: 
   164:   def unknown_entityref(self,ref):
   165:     print 'UNKNOWN ENTITY REF',ref
   166: 
   167:   # due to a bug in sgmllib, this routine will
   168:   # never be called
   169:   def report_unbalanced(self,tag):
   170:     print 'LONELY ENDTAG',tag
   171: 
   172:   def start_script(self,attributes):
   173:     if self.verbosity>=6: print 'start of script'
   174:     for param, value in attributes:
   175:       if string.lower(param) == 'language':
   176:         self.script_language = string.lower(value)
   177:         self.saved_comments = ''
   178: 
   179:   def end_script(self):
   180:     if self.verbosity>=6: print 'end of script'
   181:     if self.script_language == 'python':
   182:       try:
   183:         exec self.saved_comments in globals(),self.input_frame.userdict
   184:       except:
   185:         print "Error executing python <SCRIPT>"
   186:         traceback.print_exc()
   187:     else:
   188:       print 'Sorry',self.script_language,'not available'


End python section to interscript/parsers/html.py[1]

6.8. Html parser test
---------------------

    This is an Html test for bold and italicsand
    STRONG, _emphasised_ and code.Sizes too: big and
    small.We can also do lists: an ordered list

 1.  an ordered list item

 2.  another ordered list item

    and an unordered list

*    an unordered list item

*    another unordered list item

    and a description list:

dl key 1 
     description 1

dl key 2 
     description 2

Here's a code display, using PRE tag:

A code example.With PRE tags.

Now for some script.print "Python script"weave("Some
Python script made this.")

6.9. LALR(1) Parser table generator
-----------------------------------


6.9.1. Credits
--------------

This code is heavily adapted from the original by

  mailto:scott@chronis.icgroup.com
__version__ = "$Id: Grammar.py,v 0.2 1997/12/13 03:02:13 scott Exp scott $"

Scott has all the credit for implementing the very
complex LALR parser table generator. The algorithms are
from the Dragon Book, 'Compilers: Principles,
Techniques and Tools' by Aho Sethi and Ullman, Addison
Wesley. Mine has ISBN 0-201-10194-7, but is an old
(1987) edition. Page numbers herein refer to that.

6.9.1.1. Dependencies
---------------------

This module requires the Maxtal 'sets' module which
contains the class 'set', used to represent a set. See
'sets.pak'.

Start python section to interscript/parsers/lalr1.py[1]

     1: #line 21 "lalr1_parser.ipk"
     2: import sets
     3: import stacks
     4: set = sets.set
     5: gstack = stacks.stack
     6: 


End python section to interscript/parsers/lalr1.py[1]

6.9.1.2. Reserved Symbols
-------------------------

The following symbols are reserved by the system. EPS
is a nonterminal which derives nothing, commonly called
epsilon. It can be used in the client grammar, but must
not be the LHS of a production.

EOF is a terminal, marking the end of the input stream.
It must not be used in the client grammar. Some parsers
may prohibit the inclusion of this symbol in the input
stream, others may require it, and others may require
an infinite stream of them at the end of the input.

In addition, the empty string and -1 are reserved as
symbols by the system.

Note: this is ugly. We should use a class, but the
repr() of a class is ugly.

Start python section to interscript/parsers/lalr1.py[2]

     7: #line 42 "lalr1_parser.ipk"
     8: EPS = "<EPS>"
     9: EOF = "<EOF>"
    10: 


End python section to interscript/parsers/lalr1.py[2]

6.9.1.3. Grammar Productions
----------------------------

A production of a grammar is any object with two
attributes, 'LHS' and 'RHS' where LHS is a nonterminal
of the grammar, and the RHS is a sequence of grammar
symbols, possibly empty.

There is no restriction on what kind of sequence is
used, nor on what kind of objects the grammar symbols
are, except that the string "<EPS>" may not be the LHS
symbol of a production, and "<EOF>" and None may not be
used at all.

It is recommended that interned strings or integers be
used as grammar symbols. Strings make debugging easy
because they can be read. Integers are commonly
produced by other generating software.

Note that a production may have other attributes. In
particular, the attribute 'func' may be used by parsers
as a function to be invoked when deriving a
nonterminal, thus implementing syntax directed parsing
for S-attributed grammars.

Note: There may be a prohibition against using -1 as a
grammar symbol. This should be fixed, it should be
permitted.

Start python section to interscript/parsers/lalr1.py[3]

    11: #line 67 "lalr1_parser.ipk"
    12: class Production:
    13:   def __init__(self, LHS, RHS, **kwds):
    14:     self.LHS = LHS
    15:     self.RHS = RHS
    16:     for k in kwds.keys():
    17:       setattr(self,k,kwds[k])
    18: 
    19:   def __len__(self):
    20:     return len(self.RHS)
    21: 
    22:   def __repr__(self):
    23:     d = self.__dict__.copy()
    24:     del d['LHS']
    25:     del d['RHS']
    26:     return `self.LHS` + " -> " + `self.RHS`+' '+`d`
    27: 


End python section to interscript/parsers/lalr1.py[3]

6.9.1.4. Grammar
----------------

A grammar object is initialised by a nonempty sequence
of productions, and a start symbol. The start symbol is
required and must equal the LHS of at least one
production.

An optional argument with keyword 'verbosity' defaults
to 0, and may be used to control the amount of
debugging information output during the initialisation
process.

The sequence of productions is copied, but the
productions objects are not.

The constructor builds some tables of auxilliary
information about the grammar. After construction the
following attributes are defined:

symbols 
     The set of symbols used in the client grammar,
    excluding "<EPS>". These are called the client
    symbols.

nonterms 
     The set of symbols found on the LHS of some
    production.

terms 
     The set of client symbols of the grammar excluding
    those found on the LHS of a production.

epslhs 
     The subset of client nonterminals which directly
    derive either nothing or "<EPS>".

lhsdereps 
     The subset of client nonterminals which derive
    nothing. This is a (possibly improper) superset of
    'epslhs'.

lhsprods 
     A dictionary of keyed by client nonterminals,
    returning the set of productions of the
    nonterminal.

productions 
     A list of productions in the same order as the
    sequence passed to the constructor.

start 
     The nonterminal which the client denoted as the
    start symbol of the grammar.

firstmap 
     A dictionary keyed by grammar symbols, defining
    the FIRST set of that symbol. The dictionary
    includes entried for "<EOF>", "<EPS>", and a
    special entry for -1 (which is used internally as
    an error state).

    The elements of the FIRST sets will be either
    client terminals or "<EOF>".

    The FIRST set of a symbol is defined as follows:
    for a terminal, the singleton set containing just
    that terminal, for a nonterminal, the set of
    terminals which could be the first symbol of a
    derivation of the nonterminal, and including
    "<EPS>" if the nonterminal can derive nothing. See
    the Dragon book [4.4 p188].

followmap 
     A dictionary keyed by client nonterminals defining
    the FOLLOW set of that nonterminal.

    The FOLLOW set is defined as the set of terminals
    which can follow the nonterminal in some sentential
    form of the grammar, and including "<EOF>" if the
    nonterminal can appear at the right of a sentential
    form. See the Dragon book [4.4 p189].

Start python section to interscript/parsers/lalr1.py[4]

    28: #line 154 "lalr1_parser.ipk"
    29: class Grammar:
    30:   DummyLA = -1
    31: 


End python section to interscript/parsers/lalr1.py[4]

6.9.1.4.1. Constructor
----------------------


Start python section to interscript/parsers/lalr1.py[5]

    32: #line 159 "lalr1_parser.ipk"
    33:   def __init__(self, prods, start,verbosity=0):
    34:     self.verbosity = verbosity
    35:     self.start = start
    36:     self.productions = prods[:]
    37: 
    38:     if verbosity>1:
    39:       for i in range(len(self.productions)):
    40:         print i,':',self.productions[i]
    41: 
    42:     # calculate set of symbols, nonterminals, terminals
    43:     # and non-terminals directly deriving epsilon
    44:     self.symbols = set()
    45:     self.nonterms = set()
    46:     self.epslhs = set()
    47:     self.lhsprods = {}
    48:     for p in self.productions:
    49:       if not self.lhsprods.has_key(p.LHS): self.lhsprods[p.LHS]=set()
    50:       self.lhsprods[p.LHS].insert(p)
    51:       self.nonterms.insert(p.LHS)
    52:       if len(p.RHS)==1:
    53:         if p.RHS[0] is EPS: self.epslhs.insert(p.LHS)
    54:       elif len(p.RHS)==0: self.epslhs.insert(p.LHS)
    55: 
    56:       for sym in p.RHS:
    57:         if sym != EPS: self.symbols.insert(sym)
    58: 
    59:     self.terms = self.symbols - self.nonterms
    60: 
    61:     if verbosity>1:
    62:       print 'Symbols',self.symbols
    63:       print 'Terminals',self.terms
    64:       print 'NonTerminals',self.nonterms
    65:       print 'Directly Derive epsilon',self.epslhs
    66: 
    67:     self.calc_lhsdereps()
    68:     if verbosity>1:
    69:       print 'Derive epsilon',self.lhsdereps
    70: 
    71:     self.calc_firstmap()
    72:     if verbosity>1:
    73:       print 'First Sets:'
    74:       for nt in self.firstmap.keys():
    75:         print nt,'->',self.firstmap[nt]
    76: 
    77:     self.calc_followmap()
    78:     if verbosity>1:
    79:       print 'Follow Sets:'
    80:       for nt in self.followmap.keys():
    81:         print nt,'->',self.followmap[nt]
    82: 
    83: 


End python section to interscript/parsers/lalr1.py[5]

6.9.1.4.2. Find Nullable NonTerminals
-------------------------------------

Non-optimal!

Start python section to interscript/parsers/lalr1.py[6]

    84: #line 213 "lalr1_parser.ipk"
    85:   def calc_lhsdereps(self):
    86:     res = self.epslhs.copy()
    87:     wnts = self.nonterms - res
    88: 
    89:     converged = 0
    90:     while not converged:
    91:       converged = 1
    92:       for nt in wnts:
    93:         for p in self.lhsprods[nt]:
    94:           p_nullable = 1
    95:           for sym in p.RHS:
    96:             if not res.contains(sym):
    97:               p_nullable = 0
    98:               break
    99:           if p_nullable:
   100:             res.insert(nt)
   101:             wnts = wnts.remove(nt)
   102:             converged = 0
   103:             break
   104:         if not converged: break
   105:     self.lhsdereps = res
   106: 
   107: 


End python section to interscript/parsers/lalr1.py[6]

6.9.1.4.3. Find FIRST sets
--------------------------

Calculcate FIRST(sym) for each symbol in the grammar,
including epsilon, EOF, and DummyLA (whatever that is).

Start python section to interscript/parsers/lalr1.py[7]

   108: #line 240 "lalr1_parser.ipk"
   109:   def calc_firstmap(self):
   110:       res = {}
   111:       for sym in self.terms + [EPS, EOF, Grammar.DummyLA]:
   112:           res[sym] = set(sym)
   113:       while 1:
   114:           added = 0
   115:           for nt in self.nonterms:
   116:               firsts = res.get(nt, set())
   117:               for p in self.lhsprods[nt]:
   118:                   if not p.RHS:
   119:                       if not firsts.contains(EPS):
   120:                           added = 1
   121:                           firsts.insert(EPS)
   122:                       continue
   123:                   i = 0
   124:                   while i < len(p.RHS):
   125:                       f = res.get(p.RHS[i], set())
   126:                       for t in f:
   127:                           if not firsts.contains(t):
   128:                               added = 1
   129:                               firsts.insert(t)
   130:                       if self.lhsdereps.contains(p.RHS[i]):
   131:                           i = i + 1
   132:                       else: break
   133:               res[nt] = firsts
   134:           if not added:
   135:               break
   136:       self.firstmap = res
   137: 
   138: 
   139:   #
   140:   # these function are used as the grammar produces the tables (or writes them
   141:   # to a file)
   142:   #
   143:   def firstofstring(self, gs_list):
   144:     tmpres = {}
   145:     allhaveeps = 1
   146:     for x in range(len(gs_list)):
   147:       tmp = self.firstmap[gs_list[x]]
   148:       for s in tmp: tmpres[s] = 1
   149:       if EPS in tmp: del tmpres[EPS]
   150:       else:
   151:           allhaveeps = 0
   152:           break
   153:     if allhaveeps: tmpres[EPS] = 1
   154:     return tmpres.keys()
   155: 
   156: 


End python section to interscript/parsers/lalr1.py[7]

6.9.1.4.4. Augment grammar
--------------------------

This function adds a production S' -> S to the grammar
where S was the start symbol.

Start python section to interscript/parsers/lalr1.py[8]

   157: #line 292 "lalr1_parser.ipk"
   158:   def augment(self):
   159:     lhss = map(lambda x: x.LHS, self.productions)
   160:     newsym = self.start
   161:     while newsym in lhss: newsym = newsym + "'"
   162:     self.productions.insert(0, Production(newsym, [self.start]))
   163: 
   164:   def unaugment(self):
   165:     del self.productions[0]
   166: 


End python section to interscript/parsers/lalr1.py[8]

6.9.1.4.5. Find FOLLOW sets
---------------------------

Calculcate FOLLOW(sym) for each symbol in the grammar,
including epsilon,

Start python section to interscript/parsers/lalr1.py[9]

   167: #line 304 "lalr1_parser.ipk"
   168:   def calc_followmap(self):
   169:       eof = EOF
   170:       follow = {}
   171:       startsym = self.productions[0].LHS
   172:       follow[startsym] = set(eof)
   173:       nts = self.nonterms
   174:       for p in self.productions:
   175:           cutoff = range(len(p.RHS))
   176:           cutoff.reverse()
   177:           for c in cutoff[:-1]:  # all but the first of the RHS elements
   178:               f = self.firstmap[p.RHS[c]].copy()
   179:               f.excise(EPS)
   180:               if follow.has_key(p.RHS[c - 1]):
   181:                   if p.RHS[c -1] in nts:
   182:                       follow[p.RHS[c -1]] = follow[p.RHS[c - 1]] + f[:]
   183:               else:
   184:                   if p.RHS[c -1] in nts:
   185:                       follow[p.RHS[c - 1]] = f[:]
   186:       for p in self.productions:
   187:           if not p.RHS: continue
   188:           cutoff = range(len(p.RHS))
   189:           cutoff.reverse()
   190:           if p.RHS[-1] in nts:
   191:               if follow.has_key(p.LHS):
   192:                   add = follow[p.LHS]
   193:               else:
   194:                   add = []
   195: 
   196:               if follow.has_key(p.RHS[-1]):
   197:                   follow[p.RHS[-1]] = follow[p.RHS[-1]] + add
   198:               else:
   199:                   follow[p.RHS[-1]] = add
   200:           for c in cutoff[:-1]:
   201:               f = self.firstmap[p.RHS[c]].copy()
   202:               if EPS in f:
   203:                   if follow.has_key(p.LHS):
   204:                       add = follow[p.LHS]
   205:                   else:
   206:                       add = set()
   207:                   if follow.has_key(p.RHS[c-1]):
   208:                       follow[p.RHS[c-1]] = follow[p.RHS[c-1]] + add
   209:                   elif add:
   210:                       follow[p.RHS[c - 1]] = add
   211:       for k in follow.keys():
   212:           d = set()
   213:           for i in follow[k]: d.insert(i)
   214:           follow[k] = d
   215:       self.followmap = follow
   216: 


End python section to interscript/parsers/lalr1.py[9]

6.9.1.4.6. LALR Closure
-----------------------

Compute the LALR Closure. An 'item' is logically a pair
consisting of a production and an indictor of a
position in the RHS. It is represented here as a pair
of integers: an index into the sequence of productions,
and an integer between 0 and the length of the RHS of
the production, inclusive.

Start python section to interscript/parsers/lalr1.py[10]

   217: #line 360 "lalr1_parser.ipk"
   218:   def closure(self, items):
   219:       res = items[:]
   220:       todo = items[:]
   221:       while 1:
   222:           more = []
   223:           for (prodind, rhsind), term in todo:
   224:               if rhsind >= len(self.productions[prodind].RHS):
   225:                   continue
   226:               for p in self.lhsprods.get(self.productions[prodind].RHS[rhsind], []):
   227:                   try:
   228:                       newpart = self.productions[prodind].RHS[rhsind + 1]
   229:                   except IndexError:
   230:                       newpart = EPS
   231:                   stringofsyms = [newpart, term]
   232:                   for t in self.firstofstring(stringofsyms):
   233:                       if ((self.productions.index(p), 0), t) not in res:
   234:                           more.append(((self.productions.index(p), 0), t))
   235:                   if term == EOF and newpart == EPS:
   236:                       if ((self.productions.index(p), 0), EOF) not in res:
   237:                           more.append(((self.productions.index(p), 0), EOF))
   238:           if more:
   239:               res = res + more
   240:               todo = more
   241:           else:
   242:               break
   243:       return res
   244: 
   245: 
   246: #  def goto(self, items, sym):
   247: #      itemset = []
   248: #      for (prodind, rhsind), term in items:
   249: #          try:
   250: #              if self.productions[prodind].RHS[rhsind] == sym and ((prodind, rhsind+1), term) not in itemset:
   251: #                  itemset.append( ((prodind, rhsind +1), term))
   252: #          except IndexError:
   253: #              pass
   254: #      return self.closure(itemset)
   255: 


End python section to interscript/parsers/lalr1.py[10]

6.9.2. LALR Parser Table generator
----------------------------------

This is the efficient LALR parser generator described
in the Dragon book.

Start python section to interscript/parsers/lalr1.py[11]

   256: #line 401 "lalr1_parser.ipk"
   257: class LALRGrammar(Grammar):
   258: 
   259:   def __init__(self, prods, start, verbosity=0):
   260:     Grammar.__init__(self, prods, start, verbosity)
   261:     self.calc_ntfirstmap()
   262:     self.calc_tfirstmap()
   263:     self.augment()
   264:     self.calc_LALR1items()
   265:     self.calc_action_table()
   266:     self.calc_goto_table()
   267:     self.unaugment()
   268: 


End python section to interscript/parsers/lalr1.py[11]

6.9.3. mkntfirstmap
-------------------

Computes all nonterms A, first of (strings n) such that
some nonterminal B derives [A, n] in zero or more steps
of (rightmost) derivation. used to help make epsilon
productions quickly calculable. (B may == A)

This is to help mak epsilon productions work with
kernel items and to compute goto transitions from
kernel.

Start python section to interscript/parsers/lalr1.py[12]

   269: #line 422 "lalr1_parser.ipk"
   270:   def calc_ntfirstmap(self):
   271:     res = {}
   272:     for p in self.productions:
   273:       if p.RHS and p.RHS[0] in self.nonterms:
   274:         fos = self.firstofstring(p.RHS[1:])
   275:         fos.sort()
   276:         if not res.has_key(p.LHS):
   277:           res[p.LHS] = {}
   278:         if not res[p.LHS].has_key(p.RHS[0]):
   279:           res[p.LHS][p.RHS[0]] = []
   280:         for i in fos:
   281:           if i not in res[p.LHS].get(p.RHS[0], []):
   282:             res[p.LHS][p.RHS[0]] = fos
   283: 
   284:     while 1:
   285:       foundmore = 0
   286:       reskeys = res.keys()
   287:       for nt in reskeys:
   288:         rhsdict = res[nt]
   289:         for rnt in rhsdict.keys():
   290:           if rnt in reskeys:
   291:             d = res[rnt]
   292:             for k in d.keys():
   293:               if not res[nt].has_key(k):
   294:                 fos = self.firstofstring(d[k]+ res[nt][rnt])
   295:                 foundmore = 1
   296:                 fos.sort()
   297:                 res[nt][k] = fos
   298:               else:
   299:                 fos = self.firstofstring(d[k] + res[nt][rnt])
   300:                 fos.sort()
   301:                 if fos != res[nt][k]:  # then res[nt][k] is contained in fos
   302:                   foundmore = 1
   303:                   res[nt][k] = fos
   304:       if not foundmore: break
   305:     #
   306:     # this part accounts for the fact that a nonterminal will
   307:     # produce exactly itself in zero steps
   308:     #
   309:     for p in self.productions:
   310:       if res.has_key(p.LHS):
   311:         res[p.LHS][p.LHS] = [EPS]
   312:       else:
   313:         res[p.LHS] = {p.LHS: [EPS]}
   314:     self.ntfirstmap = res
   315: 


End python section to interscript/parsers/lalr1.py[12]

6.9.3.1. newmkntfirstmap
------------------------

computes all nonterms A, first of (strings n) such that
some nonterminal B derives [A, n] in zero or more steps
of (rightmost) derivation. used to help make epsilon
productions quickly calculable. (B may == A)

Start python section to interscript/parsers/lalr1.py[13]

   316: #line 473 "lalr1_parser.ipk"
   317:   def newmkntfirstmap(self):
   318:     res = {}
   319:     pi = 0
   320:     for p in self.productions:
   321:       if p.RHS and p.RHS[0] in self.nonterms:
   322:         if not res.has_key(p.LHS):
   323:           res[p.LHS] = {}
   324:         if not res[p.LHS].has_key(p.RHS[0]):
   325:           res[p.LHS][p.RHS[0]] = 1
   326: 
   327:     while 1:
   328:       foundmore = 0
   329:       reskeys = res.keys()
   330:       for nt in reskeys:
   331:         rhsdict = res[nt]
   332:         for rnt in rhsdict.keys():
   333:           if rnt in reskeys:
   334:             d = res[rnt]
   335:             for k in d.keys():
   336:               if not res[nt].has_key(k):
   337:                 foundmore = 1
   338:                 res[nt][k] = 1
   339:       if not foundmore:
   340:         break
   341:     #
   342:     # this part accounts for the fact that a nonterminal will
   343:     # produce exactly itself in zero steps
   344:     #
   345:     for p in self.productions:
   346:       if res.has_key(p.LHS):
   347:         res[p.LHS][p.LHS] = 1
   348:       else:
   349:         res[p.LHS] = {p.LHS: 1}
   350:     self.ntfirstmap = res
   351: 


End python section to interscript/parsers/lalr1.py[13]

6.9.3.2. mktfirstmap
--------------------

This is to help make shifts work with only kernel
items.

For each nonterminal C, compute the set of all
terminals a, such that C derives ax in zero or more
steps of (rightmost) derivation where the last
derivation is not an epsilon (empty) production.

assumes .mkfirstntmap() has been run and has already
produced self.ntfirstmap

Start python section to interscript/parsers/lalr1.py[14]

   352: #line 517 "lalr1_parser.ipk"
   353:   def calc_tfirstmap(self):
   354:     res = {}
   355:     for p in self.productions:
   356:       if not res.has_key(p.LHS):
   357:         res[p.LHS] = []
   358:       if p.RHS and p.RHS[0] in self.terms:
   359:         res[p.LHS].append(p.RHS[0])
   360:     while 1:
   361:       foundmore = 0
   362:       reskeys = res.keys()
   363:       for nt in self.ntfirstmap.keys():
   364:         arrows = self.ntfirstmap[nt]
   365:         for k in arrows.keys():
   366:           for t in res[k]:
   367:             if t not in res[nt]:
   368:               foundmore = 1
   369:               res[nt].append(t)
   370:       if not foundmore: break
   371:     self.tfirstmap = res
   372: 
   373: 


End python section to interscript/parsers/lalr1.py[14]

6.9.3.3. goto
-------------


Start python section to interscript/parsers/lalr1.py[15]

   374: #line 540 "lalr1_parser.ipk"
   375:   def goto(self, itemset, sym):
   376:     res = []
   377:     for (pi, ri) in itemset:
   378:       if ri == len(self.productions[pi].RHS):
   379:         continue
   380:       s = self.productions[pi].RHS[ri]
   381:       if s == sym:
   382:         res.append((pi, ri+1))
   383:       d = self.ntfirstmap.get(s, {})
   384:       for k in d.keys():
   385:         for p in self.lhsprods[k]:
   386:           if p.RHS and p.RHS[0] == sym:
   387:             i = self.productions.index(p)
   388:             if (i, 1) not in res: res.append((i, 1))
   389:     res.sort()
   390:     return res
   391: 


End python section to interscript/parsers/lalr1.py[15]

6.9.3.4. lookaheads
-------------------


Start python section to interscript/parsers/lalr1.py[16]

   392: #line 559 "lalr1_parser.ipk"
   393:   def lookaheads(self, itemset):
   394:     setsofitems = kernels = self.kernelitems
   395:     spontaneous = []
   396:     propagates = {}
   397:     gotomap = {}
   398:     for (kpi, kri) in itemset:
   399:       C = self.closure([((kpi, kri), Grammar.DummyLA)])
   400:       for (cpi, cri), t in C:
   401:         if (cri) == len(self.productions[cpi].RHS):
   402:           continue
   403:         s = self.productions[cpi].RHS[cri]
   404:         if gotomap.has_key(s):
   405:           newstate = gotomap[s]
   406:         else:
   407:           newstate = setsofitems.index(self.goto(itemset, s))
   408:           gotomap[s] = newstate
   409:         if t != Grammar.DummyLA:
   410:           spontaneous.append((newstate, (cpi, cri+1), t))
   411:         else:
   412:           if propagates.has_key((kpi, kri)):
   413:             propagates[(kpi, kri)].append((newstate, (cpi, cri+1)))
   414:           else:
   415:             propagates[(kpi, kri)]=[(newstate, (cpi, cri+1))]
   416:     return spontaneous, propagates
   417: 
   418: 


End python section to interscript/parsers/lalr1.py[16]

6.9.3.5. kernelsoflalr1items
----------------------------


Start python section to interscript/parsers/lalr1.py[17]

   419: #line 587 "lalr1_parser.ipk"
   420:   def kernelsoflalr1items(self):
   421:     res = [[(0, 0)]]
   422:     todo = [[(0, 0)]]
   423:     while 1:
   424:       newtodo = []
   425:       for items in todo:
   426:         for s in self.terms + self.nonterms + [EOF]:
   427:           g = self.goto(items, s)
   428:           if g and g not in res:
   429:             newtodo.append(g)
   430:       if not newtodo:
   431:         break
   432:       else:
   433:         if self.verbosity>1:
   434:           print "found %d more kernels" % (len(newtodo))
   435:         res = res + newtodo
   436:         todo = newtodo
   437:     res.sort()
   438:     return res
   439: 
   440: 


End python section to interscript/parsers/lalr1.py[17]

6.9.3.6. Calculate LALR1items
-----------------------------


Start python section to interscript/parsers/lalr1.py[18]

   441: #line 610 "lalr1_parser.ipk"
   442:   def initLALR1items(self):
   443:       self.kernelitems = kernels = self.kernelsoflalr1items()
   444:       props = {}
   445:       la_table = []
   446:       for x in range(len(kernels)):
   447:           la_table.append([])
   448:           for y in range(len(kernels[x])):
   449:               la_table[x].append([])
   450:       la_table[0][0] = [EOF]
   451:       if self.verbosity>1:
   452:           print "initLALR1items, kernels done, calculating propagations and spontaneous lookaheads"
   453:       state_i = 0
   454:       for itemset in kernels:
   455:           if self.verbosity>1:
   456:               print ".",
   457:           sp, pr = self.lookaheads(itemset)
   458:           for ns, (pi, ri), t in sp:
   459:               inner = kernels[ns].index((pi, ri))
   460:               la_table[ns][inner].append(t)
   461:           props[state_i] = pr
   462:           state_i = state_i + 1
   463:       return la_table, props
   464: 
   465: 
   466:   def calc_LALR1items(self):
   467:       la_table, props = self.initLALR1items()
   468:       if self.verbosity>1:
   469:           print "done init LALR1items"
   470:       soi = self.kernelitems
   471:       while 1:
   472:           added_la = 0
   473:           state_i = 0
   474:           for state in la_table:
   475:               ii = 0
   476:               for propterms in state:
   477:                   if not propterms:
   478:                       ii = ii + 1
   479:                       continue
   480:                   item = soi[state_i][ii]
   481:                   ii = ii + 1
   482:                   try:
   483:                       proplist = props[state_i][item]
   484:                   except KeyError:
   485:                       continue
   486:                   for pstate, pitem in proplist:
   487:                       inner = soi[pstate].index(pitem)
   488:                       for pt in propterms:
   489:                           if pt not in la_table[pstate][inner]:
   490:                               added_la = 1
   491:                               la_table[pstate][inner].append(pt)
   492:               state_i = state_i + 1
   493:           if not added_la:
   494:               break
   495:       #
   496:       # this section just reorganizes the above data
   497:       # to the state it's used in later...
   498:       #
   499:       if self.verbosity>1:
   500:           print "done with lalr1items, reorganizing the data"
   501:       res = []
   502:       state_i = 0
   503:       for state in soi:
   504:           item_i = 0
   505:           inner = []
   506:           for item in state:
   507:               for term in la_table[state_i][item_i]:
   508:                   if (item, term) not in inner:
   509:                       inner.append((item, term))
   510:               item_i = item_i + 1
   511:           inner.sort() # keeps productions in order!
   512:           res.append(inner)
   513:           state_i = state_i + 1
   514:       self.LALRitems = res
   515: 


End python section to interscript/parsers/lalr1.py[18]

6.9.3.7. Calculate Action Table
-------------------------------


Start python section to interscript/parsers/lalr1.py[19]

   516: #line 686 "lalr1_parser.ipk"
   517:   def pr_conflict(self,state,sym,rej,acc):
   518:     if rej[0]=='r' and acc[0]=='s':
   519:       print "Shift/Reduce Conflict, Use Shift [%d,%s]:" % (state, repr(sym)), rej, "->", acc
   520:     elif rej[0]=='s' and acc[0]=='r':
   521:       print "WARNING! Shift/Reduce Conflict, Use Reduce [%d,%s]:" % (state, repr(sym)), rej, "->", acc
   522:     elif rej[0]=='s' and acc[0]=='s':
   523:       print "WARNING! Shift/Shift Conflict[%d,%s]:" % (state, repr(sym)), rej, "->", acc
   524:     elif rej[0]=='r' and acc[0]=='r':
   525:       print "WARNING! Reduce/Reduce Conflict[%d,%s]:" % (state, repr(sym)), rej, "->", acc
   526:     else:
   527:       print "WARNING! WEIRD Conflict[%d,%s]:" % (state, repr(sym)), rej, "->", acc
   528: 
   529:   def resolve(self,at,sym,new,state_i):
   530:     old = at.get(sym)
   531:     # new entry
   532:     if not old:
   533:       at[sym] = new
   534:       return
   535: 
   536:     # same entry
   537:     if old == new: return
   538: 
   539:     # resolve shift/reduce conflict in favour of shift
   540:     if old[0]=='s' and new[0] =='r':
   541:       self.pr_conflict(state_i,sym,new,old)
   542:       return
   543:     if old[0]=='r' and new[0] =='s':
   544:       self.pr_conflict(state_i,sym,old,new)
   545:       at[sym] = new
   546:       return
   547: 
   548:     if old[0]=='r' and new[0]=='r':
   549:       oldp = self.productions[old[1]]
   550:       newp = self.productions[new[1]]
   551:       oldpri = None
   552:       newpri = None
   553:       if hasattr(oldp,'priority'): oldpri = oldp.priority
   554:       if hasattr(newp,'priority'): newpri = newp.priority
   555:       if oldpri > newpri: return
   556:       elif newpri > oldpri:
   557:         at[sym]=new
   558:         return
   559: 
   560:       # resolve reduce/reduce conflict in favour of earlier production
   561:       if old[1] > new[1]:
   562:         at[sym]=new
   563:         self.pr_conflict(state_i,sym,old,new)
   564:       else:
   565:         self.pr_conflict(state_i,sym,new,old)
   566:       return
   567: 
   568:     self.pr_conflict(state_i,sym,old,new)
   569:     at[sym] = new
   570: 
   571:   def calc_action_table(self):
   572:     items = self.LALRitems
   573:     res = []
   574:     state_i = 0
   575:     terms = self.terms.list()
   576:     terms.append(EOF)
   577:     for state in items:
   578:       at = {}
   579:       res.append(at)
   580:       for (prodind, rhsind), term in state:
   581:         if (rhsind ) == len(self.productions[prodind].RHS):
   582:             if prodind != 0: new = ("r", prodind)
   583:             else: new = ("a", None)
   584:             self.resolve(at,term,new,state_i)
   585:         # calculate reduction by epsilon productions
   586:         #
   587:         elif self.productions[prodind].RHS[rhsind] in self.nonterms:
   588:           nt = self.productions[prodind].RHS[rhsind]
   589:           ntfirst = self.firstmap[nt]
   590:           ntfirsts = self.ntfirstmap.get(nt, {})
   591:           for k in ntfirsts.keys():
   592:             if self.epslhs.contains(k):
   593:               reduceterms = self.followmap[k]
   594:               print `((prodind, rhsind), term)`, reduceterms
   595:               for r in reduceterms:
   596:                 new = ("r", self.epslhs[k])
   597:                 self.resolve(at,r,new,state_i)
   598:           #
   599:           # calculate the shifts that occur but whose normal items aren't in the kernel
   600:           #
   601:           tfirsts = self.tfirstmap[nt]
   602:           for t in tfirsts:
   603:             g = self.goto(self.kernelitems[state_i], t)
   604:             try:
   605:               news = self.kernelitems.index(g)
   606:             except ValueError:
   607:               continue
   608:             new = ("s", news)
   609:             self.resolve(at,t,new,state_i)
   610:         #
   611:         # compute the rest of the shifts that occur 'normally' in the kernel
   612:         #
   613:         else:
   614:           t = self.productions[prodind].RHS[rhsind]
   615:           gt = self.goto(self.kernelitems[state_i], t)
   616:           if gt in self.kernelitems:
   617:             news = self.kernelitems.index(gt)
   618:             new = ("s", news)
   619:             self.resolve(at,t,new,state_i)
   620:       state_i = state_i + 1
   621:     self.action_table = res
   622: 


End python section to interscript/parsers/lalr1.py[19]

6.9.3.8. Calculate goto table
-----------------------------


Start python section to interscript/parsers/lalr1.py[20]

   623: #line 794 "lalr1_parser.ipk"
   624:   def calc_goto_table(self):
   625:       items = self.kernelitems
   626:       res = []
   627:       nonterms = self.nonterms.list()
   628:       for state in items:
   629:         gt = {}
   630:         for nt in nonterms:
   631:           goto = self.goto(state, nt)
   632:           if goto in items: gt[nt] = items.index(goto)
   633:         res.append(gt)
   634:       self.goto_table = res
   635: 
   636: 


End python section to interscript/parsers/lalr1.py[20]

6.9.3.9. Parser
---------------

As described in the Dragon book, 4.7 Fig 4.30. p219,
but with the addition of action functions on reduce
corresponding to evaluation of S-attributes.

We hope to upgrade this to L-attributes later.

Start python section to interscript/parsers/lalr1.py[21]

   637: #line 814 "lalr1_parser.ipk"
   638:   def parse(self, data, reduce='func'):
   639:     act = self.action_table
   640:     go = self.goto_table
   641:     stack = gstack()
   642: 
   643:     stack.push(0)
   644:     n = len(data)
   645:     ip = 0
   646: 
   647:     while 1:
   648:       s = stack.top
   649:       if ip < len(data): token = data[ip]
   650:       else: token = (EOF,None)
   651:       a = token[0]
   652:       action = act[s][a]
   653:       #print 'Symbol',a,'Value',token[1],'State',s,'Action',action
   654:       if action[0]=='s':
   655:         s = action[1]
   656:         ip = ip + 1
   657:         stack.push(token)
   658:         stack.push(s)
   659:       elif action[0]=='r':
   660:         prodn = action[1]-1
   661:         prod = self.productions[prodn]
   662:         nt = prod.LHS
   663:         #print 'Reduce ',nt,'-->',
   664:         #for sym in prod.RHS: print sym,
   665:         #print
   666:         n = len(prod.RHS)
   667:         vals = []
   668:         while n:
   669:           stack.pop()
   670:           vals.insert(0,stack.pop())
   671:           n = n -1
   672:         vals.insert(0,prod)
   673:         args = tuple(vals)
   674:         #print 'args=',args
   675:         res = None
   676:         if hasattr(prod,reduce):
   677:           if callable(getattr(prod,reduce)):
   678:             res = apply(getattr(prod,reduce),args)
   679:         s = stack.top
   680:         stack.push((nt,res))
   681:         stack.push(go[s][nt])
   682:       elif act[s][a][0]=='a':
   683:         stack.pop()
   684:         res = stack.pop()
   685:         stack.pop()
   686:         assert not stack.s
   687:         print 'Accept'
   688:         return res
   689:       else:
   690:         raise 'Parse Error'
   691: 


End python section to interscript/parsers/lalr1.py[21]

6.9.4. LR parser engine
-----------------------

The 'parse' function is a temporary test parser to
checkout the tables produced by LR parser generators.

data 
     A sequence of pairs. The first item in each pair
    should usually be a terminal symbol of the grammar,
    and the second item some value or other data
    associated with it.

gram 
     A sequence of productions.

tab  A pair (action, goto) containing LR parsing tables
    as follows: each table consists of a sequence of
    rows, one row for each non-error state of the
    parser. Each row is a single dictionary, keyed by a
    grammar symbol.

    For the action table, the keys are terminals of the
    grammar, and the values are pairs consisting of an
    action indicator and an integer. The action
    indicator must be an empty string for an error, the
    letter 'r' for a reduce operation, the letter 's'
    for a shift operation, or the letter 'a' for an
    accept operation.

    For a reduce, the integer is the index of the
    production in the grammar to be reduced, plus one.
    (The plus one is an artefact of the use of an
    augmented grammar in the LALR1 parser generator.
    This needs to be fixed.)

    For a shift, the integer is a state to be shifted.

    For the goto table, the keys are nonterminals of
    the grammar, the value is simply an integer
    denoting the state to be shifted.

start 
     The initial state of the parser. Defaults to 0.

reduce 
     The reduction functor to use.

    This is the name of the attribute of a production
    which will be used to locate a function to be
    executed when the production is reduced, and
    defaults to 'func'. This facility is useful to
    allow a single grammar to have multiple action
    categories. For example, a grammar for expressions
    could build a parse tree, emit the input in reverse
    polish order, evaluate the expression, or generate
    code to evaluate the expression.

6.9.5. Test function
--------------------

This is a very simple test. Much more is needed.

Start python section to interscript/parsers/lalr1.py[22]

   692: #line 921 "lalr1_parser.ipk"
   693: 
   694: def pr_tab(p):
   695:   at = p.action_table
   696:   gt = p.goto_table
   697:   print 'actions'
   698:   for i in range(len(at)):
   699:     print i,':',at[i]
   700:   print 'gotos'
   701:   for i in range(len(gt)):
   702:     print i,':',gt[i]
   703: 
   704: def _test1():
   705:     # the token interface will be defined in
   706:     # a wrapper module and in the lexer.
   707:     # this is the format the information will be stored in
   708:     #
   709:     # first, define the tokens
   710:     #
   711:     #(id, plus, times, lparen, rparen, eof) = range(6)
   712:     #
   713:     # then, define how you want them to appear
   714:     # in the documentation of the output file
   715:     #
   716:     toks = ["id", "+", "*", "(", ")","$"]
   717:     (id, plus, times, lparen, rparen, eof) = toks
   718: 
   719:     #
   720:     # define the productions (LHS=left hand side, RHS=right hand side)
   721:     #
   722:     prods = map(lambda x: Production(x[0], x[1]), [("E", ["E", plus, "T"]),
   723:                                                    ("E", ["T"]),
   724:                                                    ("T", ["T", times, "F"]),
   725:                                                    ("T", ["F"]),
   726:                                                    ("F", [lparen, "E", rparen]),
   727:                                                    ("F", [id])])
   728:     g = LALRGrammar(prods, "E")
   729:     #
   730: 
   731:     #
   732:     # define functions for the parser to use
   733:     #
   734:     def fadd(prod,*args):
   735:         print "adding %d with %d" % (args[0][1], args[2][1])
   736:         return args[0][1] + args[2][1]
   737: 
   738:     def fdummy(prod,*args):
   739:         print "calling fdummy with args %s" % `args`
   740:         return args[0][1]
   741: 
   742:     def ftimes(prod,*args):
   743:         print "multiplying %d with %d" % (args[0][1], args[2][1])
   744:         return args[0][1] * args[2][1]
   745: 
   746:     def fparens(prod,*args):
   747:         print "handling parens, returning whats in between"
   748:         return args[1][1]
   749: 
   750: 
   751:     #
   752:     # register the functions
   753:     #
   754:     for i in range(len(g.productions)):
   755:         if len(g.productions[i].RHS) == 1:
   756:             g.productions[i].func=fdummy
   757:     g.productions[0].func=fadd
   758:     g.productions[2].func=ftimes
   759:     g.productions[4].func=fparens
   760:     #
   761:     # produce the parser
   762:     #
   763:     pr_tab(g)
   764: 
   765:     #
   766:     # this is the input as would be returned by the Lexer
   767:     # (3+(4*2))*2*5 =110
   768:     input = [(lparen, "("), (id, 3), (plus, "+"),
   769:              (lparen, "("), (id, 4), (times, "*"), (id, 2),  (rparen, ")"),
   770:              (rparen, ")"), (times, "*"),
   771:              (id, 2), (times, "*"), (id, 5) ]
   772: 
   773:     res = g.parse(input)
   774:     print 'RESULT=',res
   775: 


End python section to interscript/parsers/lalr1.py[22]

6.9.6. Bootstrapping
--------------------

The function below is used to bootstrap the parser. We
use the standard Python tokeniser, and contruct a
grammar by hand to recognize grammars. The reduction
functor is used to build a list of productions, the
functions named in the grammar are attached to these
productions. The grammar we will use is:

  G -> Plist
  Plist -> Plist newline P
  Plist -> P
  P -> LHS = RHS { func }
  RHS -> RHS sym
  RHS -> sym

We will immediately test the generated parser to parse
a string representing a grammar, namely, the same
grammar, which should generate an equivalent parser.

  G = Plist { build_grammar }
  Plist = Plist newline P { add_production_to_list }
  Plist = P { new_production_list }
  P = LHS = RHS "{" func "}" { build_production }
  RHS = RHS sym { add_sym_to_list }
  RHS = sym { new_RHS_list}

6.9.6.1. Test Tokeniser
-----------------------


Start python section to interscript/parsers/lalr1.py[23]

   776: #line 1032 "lalr1_parser.ipk"
   777: 
   778: import string
   779: def tokenise(s):
   780:   i = 0
   781:   s = s + '\0'
   782:   toks = []
   783:   while 1:
   784:     while s[i] == ' ': i = i + 1
   785:     if s[i] in '#\0': return toks
   786:     lexeme = ''
   787:     if s[i] in string.letters+'_':
   788:       while s[i] in string.letters+'_'+string.digits:
   789:         lexeme = lexeme+s[i]
   790:         i = i + 1
   791:       toks.append(('NAME',lexeme))
   792:       continue
   793:     if s[i] == '"':
   794:       i = i + 1
   795:       while s[i] not in '"\0':
   796:         lexeme = lexeme + s[i]
   797:         i = i + 1
   798:       toks.append(('STRING',lexeme))
   799:       if s[i]=='\0': return toks
   800:       i = i + 1
   801:       continue
   802:     if s[i] == "'":
   803:       i = i + 1
   804:       while s[i] not in "'\0":
   805:         lexeme = lexeme + s[i]
   806:         i = i + 1
   807:       toks.append(('STRING',lexeme))
   808:       if s[i]=='\0': return toks
   809:       i = i + 1
   810:       continue
   811:     if s[i] in '{}=':
   812:       toks.append((s[i],s[i]))
   813:       i = i + 1
   814:       continue
   815:     if s[i] in '#\n':
   816:       toks.append(('NEWLINE','\n'))
   817:       i = i + 1
   818:       continue
   819:     if s[i]=='\\':
   820:       i = i + 1
   821:       if s[i] == '\0': return toks
   822:       toks.append(('STRING',s[i]))
   823:       i = i + 1
   824:       continue
   825:     toks.append(('STRING',s[i]))
   826:     i = i + 1
   827:     continue
   828: 


End python section to interscript/parsers/lalr1.py[23]

6.9.6.2. Bootstrapping test
---------------------------


Start python section to interscript/parsers/lalr1.py[24]

   829: #line 1086 "lalr1_parser.ipk"
   830: def _test2():
   831:     def dummy(prod,*args):
   832:       print 'dummy:reduce',prod,'args=',args
   833:       return None
   834: 
   835:     def new_production_list(prod, *args):
   836:       return [args[0][1]]
   837: 
   838:     def add_production_to_list(prod, *args):
   839:       return args[0][1]+[args[1][1]]
   840: 
   841:     def build_production(prod, *args):
   842:       return Production(args[0][1],args[2][1],func=args[4][1])
   843: 
   844:     def new_symbol_list(prod, *args):
   845:       return [args[0][1]]
   846: 
   847:     def add_symbol_to_list(prod, *args):
   848:       return args[0][1]+[args[1][1]]
   849: 
   850:     def name_as_symbol(prod, *args):
   851:       return args[0][1]
   852: 
   853:     def string_as_symbol(prod, *args):
   854:       return args[0][1]
   855: 
   856:     def just_copy(prod, *args):
   857:       return args[0][1]
   858: 
   859:     bootgram = [
   860:       ('G',    ['Plist'], just_copy),
   861:       ('Plist', ['Plist','P'], add_production_to_list),
   862:       ('Plist', ['P'], new_production_list),
   863:       ('P', ['NAME','=','RHS','{','NAME','}','NEWLINE'],build_production),
   864:       ('RHS',['RHS','SYM'],add_symbol_to_list),
   865:       ('RHS',['SYM'],new_symbol_list),
   866:       ('SYM',['NAME'],name_as_symbol),
   867:       ('SYM',['STRING'],string_as_symbol)
   868:     ]
   869:     print 'bootgram=',bootgram
   870:     prods = map(lambda x: Production(x[0], x[1], func=x[2]), bootgram)
   871: 
   872:     g = LALRGrammar(prods, "G", verbosity=1)
   873:     pr_tab(g)
   874: 
   875:     print
   876:     print '---------------------------------------------------'
   877:     print '  TABLES FOR MANUALLY CONSTRUCTED GRAMMAR GENERATED'
   878:     print '---------------------------------------------------'
   879:     print
   880: 
   881:     input = """G       = Plist { just_copy }
   882: Plist   = Plist P { add_production_to_list }
   883: Plist   = P { new_production_list }
   884: P       = NAME \= RHS \{ NAME \} NEWLINE { build_production }
   885: RHS     = RHS SYM { add_symbol_to_list }
   886: RHS     = SYM { new_symbol_list }
   887: SYM     = NAME { name_as_symbol }
   888: SYM     = STRING { string_as_symbol }
   889: """
   890:     tokens = tokenise(input)
   891:     print 'tokens=',tokens
   892:     for t in tokens: print t[1],
   893:     print
   894: 
   895:     res = g.parse(tokens)
   896:     prods = res[1]
   897:     print 'RESULT=',prods
   898: 
   899:     for p in prods: p.func = eval(p.func)
   900:     print 'BOUND=',prods
   901: 
   902:     g = LALRGrammar(prods, "G", verbosity=2)
   903:     pr_tab(g)
   904: 
   905:     print
   906:     print '--------------------------------------------------------'
   907:     print '  TABLES FOR AUTOMATICALLY CONSTRUCTED GRAMMAR GENERATED'
   908:     print '--------------------------------------------------------'
   909:     print
   910: 
   911:     res = g.parse(tokens)
   912:     prods = res[1]
   913:     print 'RESULT=',prods
   914: 
   915:     print '--------------------------------------------------------'
   916:     print ' IF THAT WORKED, WE HAVE A BOOTSTRAP                    '
   917:     print '--------------------------------------------------------'
   918: 
   919:     input = """G = S {dummy}
   920: S = if E then S else S {dummy}
   921: S = if E then S {dummy}
   922: S = X {dummy}
   923: E = X {dummy}
   924: """
   925:     tokens = tokenise(input)
   926:     res = g.parse(tokens)
   927:     prods = res[1]
   928:     print 'RESULT=',prods
   929: 
   930:     for p in prods: p.func = eval(p.func)
   931:     print 'BOUND=',prods
   932: 
   933:     ifthenelse = LALRGrammar(prods, "G", verbosity=2)
   934:     pr_tab(ifthenelse)
   935:     input = """if X then X else X"""
   936:     tokens = tokenise(input)
   937:     toks = []
   938:     for tok in tokens[:]:
   939:       toks.append((tok[1],tok[1]))
   940:     print toks
   941:     res = ifthenelse.parse(toks)
   942:     prods = res[1]
   943:     print 'RESULT=',prods
   944: 
   945: 


End python section to interscript/parsers/lalr1.py[24]

Start python section to interscript/tests/tgram.py[1]

     1: #line 1205 "lalr1_parser.ipk"
     2: import interscript.parsers.lalr1
     3: interscript.parsers.lalr1._test1()
     4: interscript.parsers.lalr1._test2()


End python section to interscript/tests/tgram.py[1]

Start output section of /usr/local/bin/python interscript/tests/tgram.py

     1: Interscript Package: version 1.0a7 build 1253
     2: Using iscrcopt
     3: thread available
     4: Interscript version 1.0a7 build 1253
     5: Built by root on ruby at Mon Nov 09, 1998 at 07:08 PM (UTC)
     6: Generated by 1.0a7 buildno 1252 host ruby
     7: at Mon Nov 09, 1998 at 07:08 PM (UTC)
     8: actions
     9: 0 : {'id': ('s', 11), '(': ('s', 9)}
    10: 1 : {'+': ('s', 3), '<EOF>': ('a', None)}
    11: 2 : {'+': ('s', 3), ')': ('s', 10)}
    12: 3 : {'id': ('s', 11), '(': ('s', 9)}
    13: 4 : {'<EOF>': ('r', 1), '*': ('s', 6), '+': ('r', 1), ')': ('r', 1)}
    14: 5 : {'<EOF>': ('r', 2), '*': ('s', 6), '+': ('r', 2), ')': ('r', 2)}
    15: 6 : {'id': ('s', 11), '(': ('s', 9)}
    16: 7 : {'<EOF>': ('r', 3), '*': ('r', 3), '+': ('r', 3), ')': ('r', 3)}
    17: 8 : {'<EOF>': ('r', 4), '*': ('r', 4), '+': ('r', 4), ')': ('r', 4)}
    18: 9 : {'id': ('s', 11), '(': ('s', 9)}
    19: 10 : {'<EOF>': ('r', 5), '*': ('r', 5), '+': ('r', 5), ')': ('r', 5)}
    20: 11 : {'<EOF>': ('r', 6), '*': ('r', 6), '+': ('r', 6), ')': ('r', 6)}
    21: gotos
    22: 0 : {'F': 8, 'T': 5, 'E': 1}
    23: 1 : {}
    24: 2 : {}
    25: 3 : {'F': 8, 'T': 4}
    26: 4 : {}
    27: 5 : {}
    28: 6 : {'F': 7}
    29: 7 : {}
    30: 8 : {}
    31: 9 : {'F': 8, 'T': 5, 'E': 2}
    32: 10 : {}
    33: 11 : {}
    34: calling fdummy with args (('id', 3),)
    35: calling fdummy with args (('F', 3),)
    36: calling fdummy with args (('T', 3),)
    37: calling fdummy with args (('id', 4),)
    38: calling fdummy with args (('F', 4),)
    39: calling fdummy with args (('id', 2),)
    40: multiplying 4 with 2
    41: calling fdummy with args (('T', 8),)
    42: handling parens, returning whats in between
    43: calling fdummy with args (('F', 8),)
    44: adding 3 with 8
    45: handling parens, returning whats in between
    46: calling fdummy with args (('F', 11),)
    47: calling fdummy with args (('id', 2),)
    48: multiplying 11 with 2
    49: calling fdummy with args (('id', 5),)
    50: multiplying 22 with 5
    51: calling fdummy with args (('T', 110),)
    52: Accept
    53: RESULT= ('E', 110)
    54: bootgram= [('G', ['Plist'], <function just_copy at 80aef08>), ('Plist', ['Plist', 'P'], <function add_production_to_list at 8156368>), ('Plist', ['P'], <function new_production_list at 8163be8>), ('P', ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'], <function build_production at 815a2a8>), ('RHS', ['RHS', 'SYM'], <function add_symbol_to_list at 80ba860>), ('RHS', ['SYM'], <function new_symbol_list at 8156128>), ('SYM', ['NAME'], <function name_as_symbol at 8155e60>), ('SYM', ['STRING'], <function string_as_symbol at 80aeee8>)]
    55: actions
    56: 0 : {'NAME': ('s', 5)}
    57: 1 : {'<EOF>': ('a', None)}
    58: 2 : {'<EOF>': ('r', 1), 'NAME': ('s', 5)}
    59: 3 : {'<EOF>': ('r', 2), 'NAME': ('r', 2)}
    60: 4 : {'<EOF>': ('r', 3), 'NAME': ('r', 3)}
    61: 5 : {'=': ('s', 6)}
    62: 6 : {'NAME': ('s', 14), 'STRING': ('s', 15)}
    63: 7 : {'{': ('s', 8), 'NAME': ('s', 14), 'STRING': ('s', 15)}
    64: 8 : {'NAME': ('s', 9)}
    65: 9 : {'}': ('s', 10)}
    66: 10 : {'NEWLINE': ('s', 11)}
    67: 11 : {'<EOF>': ('r', 4), 'NAME': ('r', 4)}
    68: 12 : {'{': ('r', 5), 'NAME': ('r', 5), 'STRING': ('r', 5)}
    69: 13 : {'{': ('r', 6), 'NAME': ('r', 6), 'STRING': ('r', 6)}
    70: 14 : {'{': ('r', 7), 'NAME': ('r', 7), 'STRING': ('r', 7)}
    71: 15 : {'{': ('r', 8), 'NAME': ('r', 8), 'STRING': ('r', 8)}
    72: gotos
    73: 0 : {'G': 1, 'Plist': 2, 'P': 4}
    74: 1 : {}
    75: 2 : {'P': 3}
    76: 3 : {}
    77: 4 : {}
    78: 5 : {}
    79: 6 : {'RHS': 7, 'SYM': 13}
    80: 7 : {'SYM': 12}
    81: 8 : {}
    82: 9 : {}
    83: 10 : {}
    84: 11 : {}
    85: 12 : {}
    86: 13 : {}
    87: 14 : {}
    88: 15 : {}
    89: 
    90: ---------------------------------------------------
    91:   TABLES FOR MANUALLY CONSTRUCTED GRAMMAR GENERATED
    92: ---------------------------------------------------
    93: 
    94: tokens= [('NAME', 'G'), ('=', '='), ('NAME', 'Plist'), ('{', '{'), ('NAME', 'just_copy'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'Plist'), ('=', '='), ('NAME', 'Plist'), ('NAME', 'P'), ('{', '{'), ('NAME', 'add_production_to_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'Plist'), ('=', '='), ('NAME', 'P'), ('{', '{'), ('NAME', 'new_production_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'P'), ('=', '='), ('NAME', 'NAME'), ('STRING', '='), ('NAME', 'RHS'), ('STRING', '{'), ('NAME', 'NAME'), ('STRING', '}'), ('NAME', 'NEWLINE'), ('{', '{'), ('NAME', 'build_production'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'RHS'), ('=', '='), ('NAME', 'RHS'), ('NAME', 'SYM'), ('{', '{'), ('NAME', 'add_symbol_to_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'RHS'), ('=', '='), ('NAME', 'SYM'), ('{', '{'), ('NAME', 'new_symbol_list'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'SYM'), ('=', '='), ('NAME', 'NAME'), ('{', '{'), ('NAME', 'name_as_symbol'), ('}', '}'), ('NEWLINE', '\012'), ('NAME', 'SYM'), ('=', '='), ('NAME', 'STRING'), ('{', '{'), ('NAME', 'string_as_symbol'), ('}', '}'), ('NEWLINE', '\012')]
    95: G = Plist { just_copy }
    96: Plist = Plist P { add_production_to_list }
    97: Plist = P { new_production_list }
    98: P = NAME = RHS { NAME } NEWLINE { build_production }
    99: RHS = RHS SYM { add_symbol_to_list }
   100: RHS = SYM { new_symbol_list }
   101: SYM = NAME { name_as_symbol }
   102: SYM = STRING { string_as_symbol }
   103: 
   104: Accept
   105: RESULT= ['G' -> ['Plist'] {'func': 'just_copy'}, 'Plist' -> ['Plist', 'P'] {'func': 'add_production_to_list'}, 'Plist' -> ['P'] {'func': 'new_production_list'}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': 'build_production'}, 'RHS' -> ['RHS', 'SYM'] {'func': 'add_symbol_to_list'}, 'RHS' -> ['SYM'] {'func': 'new_symbol_list'}, 'SYM' -> ['NAME'] {'func': 'name_as_symbol'}, 'SYM' -> ['STRING'] {'func': 'string_as_symbol'}]
   106: BOUND= ['G' -> ['Plist'] {'func': <function just_copy at 80aef08>}, 'Plist' -> ['Plist', 'P'] {'func': <function add_production_to_list at 8156368>}, 'Plist' -> ['P'] {'func': <function new_production_list at 8163be8>}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': <function build_production at 815a2a8>}, 'RHS' -> ['RHS', 'SYM'] {'func': <function add_symbol_to_list at 80ba860>}, 'RHS' -> ['SYM'] {'func': <function new_symbol_list at 8156128>}, 'SYM' -> ['NAME'] {'func': <function name_as_symbol at 8155e60>}, 'SYM' -> ['STRING'] {'func': <function string_as_symbol at 80aeee8>}]
   107: 0 : 'G' -> ['Plist'] {'func': <function just_copy at 80aef08>}
   108: 1 : 'Plist' -> ['Plist', 'P'] {'func': <function add_production_to_list at 8156368>}
   109: 2 : 'Plist' -> ['P'] {'func': <function new_production_list at 8163be8>}
   110: 3 : 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': <function build_production at 815a2a8>}
   111: 4 : 'RHS' -> ['RHS', 'SYM'] {'func': <function add_symbol_to_list at 80ba860>}
   112: 5 : 'RHS' -> ['SYM'] {'func': <function new_symbol_list at 8156128>}
   113: 6 : 'SYM' -> ['NAME'] {'func': <function name_as_symbol at 8155e60>}
   114: 7 : 'SYM' -> ['STRING'] {'func': <function string_as_symbol at 80aeee8>}
   115: Symbols set('SYM', 'Plist', '=', 'NEWLINE', 'RHS', '}', 'STRING', 'NAME', 'P', '{')
   116: Terminals set('STRING', '}', 'NEWLINE', 'NAME', '=', '{')
   117: NonTerminals set('G', 'Plist', 'SYM', 'RHS', 'P')
   118: Directly Derive epsilon set()
   119: Derive epsilon set()
   120: First Sets:
   121: -1 -> set(-1)
   122: Plist -> set('NAME')
   123: } -> set('}')
   124: { -> set('{')
   125: P -> set('NAME')
   126: <EPS> -> set('<EPS>')
   127: NAME -> set('NAME')
   128: = -> set('=')
   129: SYM -> set('STRING', 'NAME')
   130: <EOF> -> set('<EOF>')
   131: NEWLINE -> set('NEWLINE')
   132: RHS -> set('STRING', 'NAME')
   133: G -> set('NAME')
   134: STRING -> set('STRING')
   135: Follow Sets:
   136: G -> set('<EOF>')
   137: Plist -> set('<EOF>', 'NAME')
   138: SYM -> set('{', 'STRING', 'NAME')
   139: RHS -> set('{', 'STRING', 'NAME')
   140: P -> set('<EOF>', 'NAME')
   141: found 4 more kernels
   142: found 2 more kernels
   143: found 4 more kernels
   144: found 2 more kernels
   145: found 1 more kernels
   146: found 1 more kernels
   147: found 1 more kernels
   148: initLALR1items, kernels done, calculating propagations and spontaneous lookaheads
   149: . . . . . . . . . . . . . . . . done init LALR1items
   150: done with lalr1items, reorganizing the data
   151: actions
   152: 0 : {'NAME': ('s', 5)}
   153: 1 : {'<EOF>': ('a', None)}
   154: 2 : {'<EOF>': ('r', 1), 'NAME': ('s', 5)}
   155: 3 : {'<EOF>': ('r', 2), 'NAME': ('r', 2)}
   156: 4 : {'<EOF>': ('r', 3), 'NAME': ('r', 3)}
   157: 5 : {'=': ('s', 6)}
   158: 6 : {'NAME': ('s', 14), 'STRING': ('s', 15)}
   159: 7 : {'{': ('s', 8), 'NAME': ('s', 14), 'STRING': ('s', 15)}
   160: 8 : {'NAME': ('s', 9)}
   161: 9 : {'}': ('s', 10)}
   162: 10 : {'NEWLINE': ('s', 11)}
   163: 11 : {'<EOF>': ('r', 4), 'NAME': ('r', 4)}
   164: 12 : {'{': ('r', 5), 'NAME': ('r', 5), 'STRING': ('r', 5)}
   165: 13 : {'{': ('r', 6), 'NAME': ('r', 6), 'STRING': ('r', 6)}
   166: 14 : {'{': ('r', 7), 'NAME': ('r', 7), 'STRING': ('r', 7)}
   167: 15 : {'{': ('r', 8), 'NAME': ('r', 8), 'STRING': ('r', 8)}
   168: gotos
   169: 0 : {'G': 1, 'Plist': 2, 'P': 4}
   170: 1 : {}
   171: 2 : {'P': 3}
   172: 3 : {}
   173: 4 : {}
   174: 5 : {}
   175: 6 : {'RHS': 7, 'SYM': 13}
   176: 7 : {'SYM': 12}
   177: 8 : {}
   178: 9 : {}
   179: 10 : {}
   180: 11 : {}
   181: 12 : {}
   182: 13 : {}
   183: 14 : {}
   184: 15 : {}
   185: 
   186: --------------------------------------------------------
   187:   TABLES FOR AUTOMATICALLY CONSTRUCTED GRAMMAR GENERATED
   188: --------------------------------------------------------
   189: 
   190: Accept
   191: RESULT= ['G' -> ['Plist'] {'func': 'just_copy'}, 'Plist' -> ['Plist', 'P'] {'func': 'add_production_to_list'}, 'Plist' -> ['P'] {'func': 'new_production_list'}, 'P' -> ['NAME', '=', 'RHS', '{', 'NAME', '}', 'NEWLINE'] {'func': 'build_production'}, 'RHS' -> ['RHS', 'SYM'] {'func': 'add_symbol_to_list'}, 'RHS' -> ['SYM'] {'func': 'new_symbol_list'}, 'SYM' -> ['NAME'] {'func': 'name_as_symbol'}, 'SYM' -> ['STRING'] {'func': 'string_as_symbol'}]
   192: --------------------------------------------------------
   193:  IF THAT WORKED, WE HAVE A BOOTSTRAP
   194: --------------------------------------------------------
   195: Accept
   196: RESULT= ['G' -> ['S'] {'func': 'dummy'}, 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': 'dummy'}, 'S' -> ['if', 'E', 'then', 'S'] {'func': 'dummy'}, 'S' -> ['X'] {'func': 'dummy'}, 'E' -> ['X'] {'func': 'dummy'}]
   197: BOUND= ['G' -> ['S'] {'func': <function dummy at 81563a8>}, 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 81563a8>}, 'S' -> ['if', 'E', 'then', 'S'] {'func': <function dummy at 81563a8>}, 'S' -> ['X'] {'func': <function dummy at 81563a8>}, 'E' -> ['X'] {'func': <function dummy at 81563a8>}]
   198: 0 : 'G' -> ['S'] {'func': <function dummy at 81563a8>}
   199: 1 : 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 81563a8>}
   200: 2 : 'S' -> ['if', 'E', 'then', 'S'] {'func': <function dummy at 81563a8>}
   201: 3 : 'S' -> ['X'] {'func': <function dummy at 81563a8>}
   202: 4 : 'E' -> ['X'] {'func': <function dummy at 81563a8>}
   203: Symbols set('if', 'X', 'E', 'else', 'S', 'then')
   204: Terminals set('if', 'then', 'X', 'else')
   205: NonTerminals set('S', 'G', 'E')
   206: Directly Derive epsilon set()
   207: Derive epsilon set()
   208: First Sets:
   209: if -> set('if')
   210: -1 -> set(-1)
   211: <EOF> -> set('<EOF>')
   212: X -> set('X')
   213: G -> set('if', 'X')
   214: <EPS> -> set('<EPS>')
   215: E -> set('X')
   216: else -> set('else')
   217: S -> set('if', 'X')
   218: then -> set('then')
   219: Follow Sets:
   220: G -> set('<EOF>')
   221: E -> set('then')
   222: S -> set('else', '<EOF>')
   223: found 4 more kernels
   224: found 2 more kernels
   225: found 1 more kernels
   226: found 1 more kernels
   227: found 1 more kernels
   228: found 1 more kernels
   229: initLALR1items, kernels done, calculating propagations and spontaneous lookaheads
   230: . . . . . . . . . . . done init LALR1items
   231: done with lalr1items, reorganizing the data
   232: Shift/Reduce Conflict, Use Shift [6,'else']: ('r', 3) -> ('s', 7)
   233: actions
   234: 0 : {'if': ('s', 3), 'X': ('s', 9)}
   235: 1 : {'<EOF>': ('a', None)}
   236: 2 : {'<EOF>': ('r', 1)}
   237: 3 : {'X': ('s', 10)}
   238: 4 : {'then': ('s', 5)}
   239: 5 : {'if': ('s', 3), 'X': ('s', 9)}
   240: 6 : {'else': ('s', 7), '<EOF>': ('r', 3)}
   241: 7 : {'if': ('s', 3), 'X': ('s', 9)}
   242: 8 : {'else': ('r', 2), '<EOF>': ('r', 2)}
   243: 9 : {'else': ('r', 4), '<EOF>': ('r', 4)}
   244: 10 : {'then': ('r', 5)}
   245: gotos
   246: 0 : {'S': 2, 'G': 1}
   247: 1 : {}
   248: 2 : {}
   249: 3 : {'E': 4}
   250: 4 : {}
   251: 5 : {'S': 6}
   252: 6 : {}
   253: 7 : {'S': 8}
   254: 8 : {}
   255: 9 : {}
   256: 10 : {}
   257: [('if', 'if'), ('X', 'X'), ('then', 'then'), ('X', 'X'), ('else', 'else'), ('X', 'X')]
   258: dummy:reduce 'E' -> ['X'] {'func': <function dummy at 81563a8>} args= (('X', 'X'),)
   259: dummy:reduce 'S' -> ['X'] {'func': <function dummy at 81563a8>} args= (('X', 'X'),)
   260: dummy:reduce 'S' -> ['X'] {'func': <function dummy at 81563a8>} args= (('X', 'X'),)
   261: dummy:reduce 'S' -> ['if', 'E', 'then', 'S', 'else', 'S'] {'func': <function dummy at 81563a8>} args= (('if', 'if'), ('E', None), ('then', 'then'), ('S', None), ('else', 'else'), ('S', None))
   262: dummy:reduce 'G' -> ['S'] {'func': <function dummy at 81563a8>} args= (('S', None),)
   263: Accept
   264: RESULT= None

End output section of /usr/local/bin/python interscript/tests/tgram.py

6.10. Architecture
------------------

This part discusses the architectural design of
Interscript. Here is a simplified picture of which
objects refer to which others. Interscript maintains
several objects that act as _frames_ (or namespaces)
for symbol lookup.

  FRAMES
    process_frame  --> global_frame
    master_frame   --> process_frame
    pass_frame     --> master_frame
    input_frame    --> pass_frame
                   --> weaver_filter
                   --> stack of tangler

    tangler        --> multiplexor
                   --> sink

    weaver_filter  --> multiplexor
    multiplexor    --> weaver
    weaver         --> sink

6.10.1. The universal frame
---------------------------

The most global frame is the _universal frame_, and
contains fixed information such as interscript code,
constant limits of the implementation, and options
which are hardwired into the software. There's no
actual universal frame object, but the Python and
Interscript 'standards' may be considered universals.

6.10.2. Install Frame
---------------------

The _install frame_ is used to control how generated
files are installed.

An _installation point_ is a directory which will
contain a certain class of files. An _installation
mode_ is an indication of the permissions an installed
file should have.

The install frame contains the default places where
development occurs, and where code and documentation
should be installed depending on the designation
ascribed to the file in interscript sources.

Installation is controlled in several ways. The _kind_
of file is identified in the interscript source, this
tells the language used for script, the kind of script,
or whether the final target is an executable of
dynamically loadable binary. (This information is
generally gleaned from the tangler used, plus options
if required).

The _stability_ of the version is also identifed as
experimental, test, alpha, beta, or production in the
interscript source. This information must be supplied
by the vendor.

The _domain of applicability_ is identified in the
interscript source, this tells if the code is
univerally useful, platform specific, should be
tailored per site, or tailored per user-group or user.
This information must be supplied by the vendor.

Start python section to interscript/frames/install.py[1]

     1: #line 62 "frames.ipk"
     2: class install_frame:
     3:   def __init__(self):
     4:     py = 'python'+self.python_version
     5:     self.platform_independent_install_point = sys.prefix
     6:     self.python_header_install_point = sys.prefix + '/include/'+py
     7:     self.python_module_install_point = sys.prefix + '/lib/'+py
     8:     self.python_platform_install_point = sys.prefix + '/lib/'+py+'/plat-'+platform.python_plat
     9: 
    10:     self.platform_dependent_install_point = sys.exec_prefix
    11:     self.python_config_install_point = sys.exec_prefix + '/lib/'+self.python_version+'/config'
    12:     self.python_dynload_install_point = sys.exec_prefix + '/lib/'+self.python_version+'/lib-dynload'
    13: 
    14:     # this is where stuff goes during development
    15:     d = self.development_point = {}
    16:     d['documentation']=self.platform.get_working_directory()
    17:     d['python module']=self.platform.get_working_directory()
    18:     d['python package']=self.platform.get_working_directory()
    19:     d['python cmodule']=self.platform.get_working_directory()
    20:     d['python script']=self.platform.get_working_directory()
    21:     d['executable']=self.platform.get_working_directory()
    22: 
    23:     # this is where standard stuff gets installed for general use
    24:     d = self.standard_install_point = {}
    25:     d['documentation']=sys.prefix+'/doc'
    26:     d['python module']=self.python_module_install_point
    27:     d['python package']=self.python_module_install_point
    28:     d['python cmodule']=self.python_dynload_install_point
    29:     d['python script']=sys.exec_prefix+'/bin'
    30:     d['executable']=sys.exec_prefix+'/bin'
    31: 
    32:     # this is where platform dependent stuff gets installed for general use
    33:     d = self.platform_install_point = {}
    34:     d['documentation']=sys.prefix+'/doc'
    35:     d['python module']=self.python_module_install_point
    36:     d['python package']=self.python_module_install_point
    37:     d['python cmodule']=self.python_dynload_install_point
    38:     d['python script']=sys.exec_prefix+'/bin'
    39:     d['executable']=sys.exec_prefix+'/bin'
    40: 
    41:     # this is where site dependent stuff gets installed for general use
    42:     d = self.site_install_point = {}
    43:     d['documentation']=sys.prefix+'/doc'
    44:     d['python module']=self.python_module_install_point+'/site-packages'
    45:     d['python package']=self.python_module_install_point+'/site-packages'
    46:     d['python cmodule']=self.python_dynload_install_point+'/site-packages'
    47:     d['python script']=sys.exec_prefix+'/bin'
    48:     d['executable']=sys.exec_prefix+'/bin'
    49: 
    50:     # this is where user-group dependent stuff gets installed for use
    51:     d = self.usergrp_install_point = {}
    52:     d['documentation']=sys.prefix+'/doc'
    53:     d['python module']=self.python_module_install_point+'/site-packages'
    54:     d['python package']=self.python_module_install_point+'/site-packages'
    55:     d['python cmodule']=self.python_dynload_install_point+'/site-packages'
    56:     d['python script']=sys.exec_prefix+'/bin'
    57:     d['executable']=sys.exec_prefix+'/bin'
    58: 
    59:     # this is where user dependent stuff gets installed for use
    60:     d = self.user_install_point = {}
    61:     d['documentation']=sys.prefix+'/doc'
    62:     d['python module']=self.python_module_install_point+'/site-packages'
    63:     d['python package']=self.python_module_install_point+'/site-packages'
    64:     d['python cmodule']=self.python_dynload_install_point+'/site-packages'
    65:     d['python script']=sys.exec_prefix+'/bin'
    66:     d['executable']=sys.exec_prefix+'/bin'
    67: 
    68:     self.install_point = {
    69:       'dev': self.development_point,
    70:       'std': self.standard_install_point,
    71:       'plat': self.platform_install_point,
    72:       'site': self.site_install_point,
    73:       'grp': self.usergrp_install_point,
    74:       'user': self.user_install_point
    75:       }
    76: 
    77:     self.trust_map = {
    78:       'experimental':'dev',
    79:       'test':'dev',
    80:       'alpha':'dev',
    81:       'beta':'user',
    82:       'production':'user'
    83:       }
    84: 
    85:   def print_install(self):
    86:     print 'INSTALL POINTS'
    87:     print '----------------------------'
    88:     for k in self.install_point.keys():
    89:       print 'Install Point',k
    90:       d = self.install_point[k]
    91:       for t in d.keys():
    92:         print ' ',t,'-->',d[t]
    93:       print '----------------------------'
    94:       print
    95:     print 'TRUST MAP'
    96:     print '----------------------------'
    97:     for k in self.trust_map.keys():
    98:       print ' ',k,'-->',self.trust_map[k]
    99:     print '----------------------------'
   100:     print
   101: 
   102: 


End python section to interscript/frames/install.py[1]

6.10.3. The user frame
----------------------

The _user frame_ contains data and functions related to
a particular user, such as the users email address. In
addition, Interscript imports the module 'user', which
executes the file 'pythonrc.py' in the user's home
directory under Unix.

Start python section to interscript/frames/user.py[1]

     1: #line 169 "frames.ipk"
     2: class user_frame:
     3:   pass
     4: 


End python section to interscript/frames/user.py[1]

6.10.4. The project frame
-------------------------

The _project frame_ contains data and functions related
to a specific project. A project may consist of a
number of users, and it may contain configuration
information such as the style of page headings used,

that relate to all the software of a given project.
There is currently no project frame, because the
architecture of projects has not yet been designed,
however comments from users are solicited.

Start python section to interscript/frames/project.py[1]

     1: #line 182 "frames.ipk"
     2: class project_frame:
     3:   pass
     4: 
     5: 


End python section to interscript/frames/project.py[1]

Start python section to interscript/frames/__init__.py[1]

     1: #line 188 "frames.ipk"
     2: # frames subpackage


End python section to interscript/frames/__init__.py[1]

6.10.5. The platform frame
--------------------------

The _platform frame_ contains platform specific data
and functions, such as the name of the operating
system. These include build options of Python itself,
and the platform dependent modules such as 'posix'
representing platform specific operating system
services and access paths.

Start python section to interscript/frames/platform.py[1]

     1: #line 10 "platform_frame.ipk"
     2: import sys
     3: import os
     4: 
     5: class platform_frame:
     6:   def __init__(self):
     7:     self.python_plat = sys.platform
     8:     self.uname = ['unknown','unknown','unknown']
     9:     try:
    10:       f = os.popen('uname -s','r')
    11:       self.uname[0] = f.read()
    12:       f.close()
    13:       del f
    14:       f = os.popen('uname -v','r')
    15:       self.uname[1] = f.read()
    16:       f.close()
    17:       del f
    18:       f = os.popen('uname -r','r')
    19:       self.uname[2] = f.read()
    20:       f.close()
    21:       del f
    22:     except:
    23:       pass
    24:     self.python_os = os.name
    25:       # one of 'nt', 'posix','dos','mac'
    26: 
    27:   def map_filename(self,path,base,extension):
    28:     return string.join(path,os.sep)+os.sep+base+'.'+extension
    29: 
    30:   def get_working_directory(self):
    31:     return os.getcwd()
    32: 


End python section to interscript/frames/platform.py[1]

6.10.6. The site frame
----------------------

The _site frame_ contains data and functions which
configure Interscript to a specific installation. For
example, the availability of various tanglers and
weavers is likely to be site specific. The specific
location of the Python and Interscript systems is part
of the conceptual site frame.

Start python section to interscript/frames/site.py[1]

     1: #line 10 "site_frame.ipk"
     2: import sys
     3: class site_frame:
     4:   def __init__(self, platform):
     5:     self.platform = platform
     6:     self.python_version = sys.version[:3]
     7:     self.builtin_module_names = sys.builtin_module_names
     8:     self.python_module_search_path = sys.path
     9: 


End python section to interscript/frames/site.py[1]

6.10.7. The process frame
-------------------------

The _process frame_ contains data configuring a
particular processing run. That data is gleaned from
command line switches, the environment, defaults from
other frames, and statistics gathered during
processing, and also some data configured by user
commands in input files.

User commands may read symbols from this dictionary,
and may modify the objects to which the symbol names
refer, but are protected from rebinding the names to
other objects or values.

The organisation of the process frame is currently
haphazard and may change from version to version.

Make an object to hold global variables that user
objects can hang onto, to ensure these variables live
during the execution of the object destructors.

Start python section to interscript/frames/processf.py[1]

     1: #line 22 "process_frame.ipk"
     2: from interscript.frames.site import site_frame
     3: from interscript.frames.platform import platform_frame
     4: from interscript.frames.masterf import master_frame
     5: from interscript.drivers.sources.base import eoi
     6: 
     7: import traceback
     8: 
     9: class process_frame:
    10:   def __init__(self, global_frame, process_options, argument_frames):
    11:     self.global_frame = global_frame
    12:     self.process_options = process_options
    13:     self.argument_frames = argument_frames
    14: 
    15:     self.break_on_error = 0
    16:     self.debug_constructors = 0
    17:     self.debug_destructors = 0
    18:     self.verbosity = process_options.verbosity
    19:     self.update_files = 1
    20: 
    21:     plat = platform_frame()
    22:     self.site_frame = site_frame(plat)
    23:     # self.site_frame.print_install()
    24: 
    25:   def run(self):
    26:     for argument_frame in self.argument_frames:
    27:       master_frame(self,argument_frame)
    28: 
    29:   def get_process_frame(self): return self
    30: 


End python section to interscript/frames/processf.py[1]

6.10.7.1. Execute Python Script
-------------------------------


Start python section to interscript/frames/processf.py[2]

    31: #line 53 "process_frame.ipk"
    32:   def py_exec(self,py,file,count,glb,user):
    33:     # get a lock here, release it later
    34:     try:
    35:       if self.verbosity>4: print 'Executing',py
    36:       exec py in glb,user
    37:     except KeyboardInterrupt:
    38:       self.update_files = 0
    39:       raise KeyboardInterrupt
    40:     except eoi:
    41:       raise eoi
    42:     except:
    43:       print 'error executing python:',file,count,':',py
    44:       traceback.print_exc()
    45:       if self.break_on_error:
    46:         self.update_files = 0
    47:         sys.exit()
    48: 


End python section to interscript/frames/processf.py[2]

6.10.8. Master Frame
--------------------

There is a _master frame_ for each master document file
read by Interscript. It contains data for the document
which persists between passes on the source. For
example the master table of contents is part of the
master frame. Information in the master frame is often
collected during pass and set into the frame at the end
of the pass so it can be used by the next pass: the
table of contents is such an example.

Conceptually, if multiple master documents are
processed by interscript, elaboration of each document
is performed in a separate thread. The master frame
contains thread local data, whereas the process frame
is shared by all threads.

At the 'document' level, the failure to generate a
document correctly is at most fatal to that thread of
control because documents are in some senses
independent. However, when multiple documents form a
project, inter-document dependencies may dictate
abortion of the whole processes if one of the master
threads fails.

Start python section to interscript/frames/masterf.py[1]

     1: #line 23 "master_frame.ipk"
     2: from interscript.frames.passf import pass_frame
     3: from interscript.drivers.sources.disk import parse_source_filename
     4: import pickle
     5: 
     6: class master_frame:
     7:   def __init__(self,process,argument_frame):
     8:     self.process = process
     9:     for k in argument_frame.__dict__.keys():
    10:       if process.verbosity > 4:
    11:         print 'setting MASTER',k,'as',argument_frame.__dict__[k]
    12:       setattr(self,k,argument_frame.__dict__[k])
    13:     self.ids = {}
    14:     self.iflist = []
    15:     self.flist = []
    16:     self.fdict = {}
    17:     self.toc = []
    18:     self.include_files = []
    19:     self.classes = {}
    20:     self.functions = {}
    21:     self.sequence_limit = -1
    22:     self.tests = {}
    23:     self.persistent_frames = {}
    24:     self.cache_name =parse_source_filename(
    25:         self.filename+'.cache', self.source_prefix) [3]
    26:     print 'cache=',self.cache_name
    27:     try:
    28:       cache = open(self.cache_name,'r')
    29:       self.persistent_frames = pickle.load(cache)
    30:       cache.close()
    31:       del cache
    32:       print 'loaded master frame from cache'
    33:     except: pass
    34: 


End python section to interscript/frames/masterf.py[1]

6.10.8.1. Run the passes
------------------------

At present, we just run the number of passes specified
as the limit. This will change when convergence testing
is implemented.

The pass loop moves the identifier list, input and
output file lists, and table of contents from the pass
frame to the master frame after each pass. This permits
this data to be read by the next pass (while it is also
regenerating the data for the pass after that).

Conceptually, the loop is said to converge when this
data is the same twice in a row. However we do not test
this yet because the state information is not complete.

To be complete, it must determine whether the input and
output files have changed between passes. This will be
done by comparing the file lists, and comparing the
last modification dates on the files between passes.

Finally, output to simple files will be ignored. It is
necessary to also add user controlled hooks into the
convergence tests, because user script can have
arbitrary side effects.

Start python section to interscript/frames/masterf.py[2]

    35: #line 80 "master_frame.ipk"
    36:     for passno in range(self.passes):
    37:       curpass = pass_frame(self, passno)
    38:       self.ids = curpass.ids        # idlist
    39:       self.flist = curpass.flist    # output file list
    40:       self.iflist = curpass.iflist  # input file list
    41:       self.toc = curpass.toc        # table of contents
    42:       self.include_files = curpass.include_files # include files
    43:       self.classes = curpass.classes # classes
    44:       self.functions = curpass.functions # functions
    45:       self.tests = curpass.tests # functions
    46: 
    47:       if self.sequence_limit == -1:
    48:         self.sequence_limit = curpass.sequence
    49:       elif self.sequence_limit != curpass.sequence:
    50:         print 'WARNING: SEQUENCE COUNTER DISPARITY BETWEEN PASSES'
    51:       ds = curpass.fdict
    52:       dd = self.fdict
    53: 
    54:       file_count = 0
    55:       stable_file_count = 0
    56:       unstable_file_count = 0
    57:       new_file_count = 0
    58:       for k in ds.keys():
    59:         file_count = file_count + 1
    60:         if not dd.has_key(k):
    61:           dd[k]=(ds[k],passno)
    62:           new_file_count = new_file_count + 1
    63:         else:
    64:           if ds[k]=='unchanged':
    65:             stable_file_count = stable_file_count + 1
    66:           else:
    67:             unstable_file_count = unstable_file_count + 1
    68:           if ds[k]!='unchanged' or dd[k][0]!='unchanged':
    69:             dd[k]=(ds[k],passno)
    70:       converged = file_count == stable_file_count
    71:       if converged:
    72:         print 'All',file_count,'output files stable on pass',passno,' -- breaking'
    73:         break
    74:       else:
    75:         print 'Of',file_count,'files, only',stable_file_count,'were stable on pass',passno
    76:         print 'There were',new_file_count,'new files,'
    77:         print 'and',unstable_file_count,'unstable files.'
    78:     try:
    79:       cache = open(self.cache_name,'w')
    80:       pickle.dump(self.persistent_frames, cache)
    81:       cache.close()
    82:       del cache
    83:       print 'Pickled persistent frames in', self.cache_name
    84:     except:
    85:       print 'Pickle FAILURE'
    86: 
    87:   def get_master_frame(self): return self
    88: 
    89:   def get_persistent_frame(self, seq):
    90:     if not self.persistent_frames.has_key(seq):
    91:       self.persistent_frames[seq]={}
    92:     return self.persistent_frames[seq]
    93: 


End python section to interscript/frames/masterf.py[2]

6.10.9. The pass frame
----------------------

The _pass frame_ contains data gleaned during a single
pass over a master document. The pass frame is
reinitialised for each pass, so that defaults are
re-established.

For each master document thread, multiple passes on a
document are performed sequentially: a pass may depend
on previous passes.

The thread generating a document is said to converge if
an successive passes reach a fixed point; that is, if
all the outputs from one pass are the same as from the
previous pass.

Conceptually, interscript repeats passes indefinitely
until the passes converge; in practice, a limit is
placed on the number of passes for two reasons: the
first is to prevent unstable definitions locking up the
computer, and the second is to allow for the fact that
detection of convergence is, in general, difficult and
not fully implemented.

For example, although comparison of output files is a
useful tool, minor variations between passes, such as
time stamps, may lead to failed comparisons based on
representation rather than semantic content.

Start python section to interscript/frames/passf.py[1]

     1: #line 26 "pass_frame.ipk"
     2: import string
     3: import traceback
     4: import sys
     5: 
     6: from interscript.drivers.sources import source_open_error
     7: from interscript.drivers.sources.disk import named_file_source
     8: from interscript.drivers.sources.base import eoi
     9: from interscript.frames.inputf import input_frame
    10: from interscript.weavers.auto import auto_weaver
    11: from interscript.drivers.sources.base import eoi
    12: from interscript.parsers.html import sgml_wrapper, html_filter
    13: 
    14: class pass_frame:
    15:   def __init__(self,master, passno):
    16:     # the display
    17:     self.master = master
    18:     self.process = master.process
    19: 
    20:     self.passno = passno
    21: 
    22:     self.verbosity = master.verbosity
    23:     self.echo_input = master.echo_input
    24:     self.autoweave = master.autoweave
    25: 
    26:     self.ids = {}
    27:     self.flist = []
    28:     self.fdict = {}
    29:     self.iflist = []
    30:     self.toc = []
    31:     self.include_files = []
    32:     self.classes = {}
    33:     self.functions = {}
    34:     self.testno = 0
    35:     self.sequence = 0
    36:     self.tests = {}
    37:     self.section_index = {}
    38: 
    39:     if self.verbosity>=3:
    40:       print 'Autoweave',self.autoweave
    41: 
    42:     file = self.master.filename
    43:     if self.verbosity>=2:
    44:       print 'Processing',file,'Pass',passno
    45: 


End python section to interscript/frames/passf.py[1]

6.10.9.1. Construct input object
--------------------------------

We construct an disk source driver by taking the first
command line argument as a disk file name. We construct
an input frame, and attach the source driver to it.

Start python section to interscript/frames/passf.py[2]

    46: #line 76 "pass_frame.ipk"
    47: 
    48:     basename = file
    49:     if string.find(file,'.') != -1:
    50:       basename = string.join(string.split(file,'.')[:-1],'.')
    51: 
    52:     weaver = auto_weaver(self, basename, self.autoweave)
    53:     userdict = {
    54:       'echo_input':self.echo_input,
    55:       'verbosity':self.verbosity
    56:       }
    57: 
    58:     try:
    59:       input_file =named_file_source(self,file, self.master.source_prefix)
    60: 
    61:     except source_open_error, filename:
    62:       if self.verbosity>1:
    63:         print 'Cannot Open File',filename,'for input (ignored)'
    64:       raise
    65:     except KeyboardError:
    66:       raise
    67:     except:
    68:       print "Program error opening",file
    69:       traceback.print_exc()
    70:       raise
    71: 
    72:     self.include_files.append((1,'interscript',file))
    73:     inpt = input_frame(
    74:       self,
    75:       input_file,
    76:       [],
    77:       weaver,
    78:       userdict,
    79:       1)
    80:     inpt.set_warning_character(python='@')
    81:     if self.verbosity>=3:
    82:       print 'input from',inpt.source.get_source_name()
    83: 
    84:     inpt.file_pass()
    85:     # at this point, inpt, weaver, userdict, input_file
    86:     # should all be released (even if 'pass_frame' is held onto,
    87:     # these symbols are defined in the __init__ function frame)
    88: 
    89:   def get_pass_frame(self):
    90:     return self
    91: 
    92:   def get_new_test_number(self):
    93:     self.testno = self.testno + 1
    94:     return self.testno
    95: 
    96:   def get_new_sequence_number(self):
    97:     self.sequence = self.sequence + 1
    98:     return self.sequence
    99: 


End python section to interscript/frames/passf.py[2]

6.10.10. Input Frame
--------------------

An _input frame_ is created for every document and
subdocument. Input frames are stacked, typically by
@include_file() commands. Input frames implement
lexical scoping and help to isolate lexical information
pertaining to a single file from containing files. For
example, the main parser table, weaver, and tangler
stack are part of an input frame.

When the input frame is stacked, its symbol dictionary
is copied to the new input frame so that included files
inherit the names from the including file. Any
rebindings of those name in the new input file will be
lost when the file ends, however.

For example, if the weaver is set to an HTML weaver,
and an included file sets the weaver to a plain text
weaver, when the included files ends, documentation
will be written to the HTML weaver. The plain text
weaver will have been closed when the last reference to
it disappeared.

You should note, however, that while you cannot change
the bindings of name in parent documents, you certainly
can change the attributes of the objects named by them,
and these changes are persistent. For example, if a
child document changes an attribute of weaver inherited
from the parent, the attribute will remain changed even
after the child is exhausted. It's not that this
behaviour is 'deliberate', so much as it's the way
Python works :-)

Start python section to interscript/frames/inputf.py[1]

     1: #line 34 "input_frame.ipk"
     2: import string
     3: import re
     4: import traceback
     5: import sys
     6: import os
     7: import tempfile
     8: 
     9: # these imports _should_ come from the global frame!
    10: from interscript.drivers.sources.base import eof, eoi
    11: from interscript.drivers.sources.disk import named_file_source
    12: from interscript.drivers.sources.disk import parse_source_filename
    13: from interscript.drivers.sources.disk import loadfile
    14: from interscript.drivers.sinks.bufdisk import named_file_sink
    15: from interscript.drivers.sinks.disk import simple_named_file_sink
    16: 
    17: from interscript.tanglers.c import c_tangler
    18: from interscript.tanglers.cpp import cpp_tangler
    19: from interscript.tanglers.python import python_tangler
    20: from interscript.tanglers.perl import perl_tangler
    21: from interscript.tanglers.java import java_tangler
    22: from interscript.tanglers.tcl import tcl_tangler
    23: from interscript.tanglers.doc import doc_tangler
    24: from interscript.tanglers.null import null_tangler
    25: from interscript.tanglers.data import data_tangler
    26: 
    27: from interscript.parsers.html import sgml_wrapper, html_filter
    28: try:
    29:   import interscript.utilities.diff
    30: except:
    31:   pass
    32: 
    33: def compile_parse_tab(res):
    34:   return map(lambda x: [re.compile(x[0]), x[1]], res)
    35: 
    36: 


End python section to interscript/frames/inputf.py[1]

6.10.10.1. The input frame
--------------------------

Note the horrid hack: the reg_list is a list of pairs,
the second entry is a method which should be bound to
this frame, but the frame isn't built yet. So the
caller must pass an empty list, and populate it after
the frame is constructed.

MUCH WORSE: doing this creates a circular reference,
preventing the frame being destroyed!

This has to be fixed.

Start python section to interscript/frames/inputf.py[2]

    37: #line 80 "input_frame.ipk"
    38: class input_frame:
    39: 
    40:   def __init__(self, pass_frame, src, reg_list, weaver, userdict, depth):
    41:     # the display
    42:     self.pass_frame = pass_frame
    43:     self.master = pass_frame.master
    44:     self.process = self.master.process
    45:     self.global_frame = self.process.global_frame
    46: 
    47:     self.weaver = weaver
    48:     self.weaver_stack = []
    49: 
    50:     self.depth = depth
    51:     self.source = src
    52:     self.userdict = userdict
    53:     self.reg_list = reg_list
    54:     self.read_buffer = []
    55: 
    56:     self.tangler_stack = []
    57:     self.tangler = None
    58:     self.line_offset = 0
    59:     self.original_filename = src.get_source_name()
    60:     self.weaver.set_original_filename(self.original_filename)
    61:     self.head_offset = 0
    62:     self.verbosity = pass_frame.verbosity
    63:     self.tabwidth = self.master.tabwidth
    64: 
    65:     self.cont_re = re.compile('^$|^ (.*)$')
    66:     self.any_line_re = re.compile('^(.*)$')
    67: 
    68:     if self.verbosity>=6:
    69:       print 'initialising input frame',src.get_source_name()
    70:     self.post_methods()
    71: 
    72:   def __del__(self):
    73:     if self.verbosity>=6:
    74:       print 'frame',self.source.name,'deleting'
    75: 


End python section to interscript/frames/inputf.py[2]

6.10.10.1.1. Post user methods
------------------------------

This bit is very hacky! We grab bound methods from
input frame and put them in the user dictionary.

Start python section to interscript/frames/inputf.py[3]

    76: #line 123 "input_frame.ipk"
    77:   def post_methods(self):
    78:     # input frame methods
    79:     method_names = self.__class__.__dict__.keys()
    80:     is_begin_or_end_method = lambda x: x[:3]=='end' or x[:5]=='begin'
    81:     method_names = filter(is_begin_or_end_method, method_names)
    82:     method_names = method_names + [
    83:       'head','heading','push_head','pop_head','set_head',
    84:       'set_warning_character',
    85:       'doc','p','eop','cite_url',
    86:       'output','c_output','cpp_output',
    87:       'python_output','perl_output',
    88:       'interscript_output',
    89:       'push','pop','select','comment','resume_code',
    90:       'tangler_push','tangler_pop','tangler_set', # temporarily!
    91:       'untangle',
    92:       'weave','weave_line','tangle',
    93:       'print_identifier_cross_reference',
    94:       'print_contents',
    95:       'print_file_list',
    96:       'print_source_list',
    97:       'print_include_list',
    98:       'print_class_reference',
    99:       'print_file_status',
   100:       'get_weaver',
   101:       'table_row', 'table_rule',
   102:       'item',
   103:       'include_file','include_source',
   104:       'include_code','insert_code','display_code',
   105:       'include_html','html',
   106:       'capture_output','print_output',
   107:       'capture_python_output','print_python_output','print_python_test_output',
   108:       'set_weaver','get_weaver', 'push_weaver','pop_weaver',
   109:       'get_attribute',
   110:       'interscript_from_options',
   111:       'python','test_python',
   112:       'set_anchor','ref_anchor'
   113:       ]
   114:     for m in method_names:
   115:       exec 'self.userdict[m]=self.'+m
   116: 
   117:     # pass frame methods
   118:     method_names = [
   119:       'get_pass_frame'
   120:     ]
   121:     for m in method_names:
   122:       exec 'self.userdict[m]=self.pass_frame.'+m
   123: 
   124:     # processs frame methods
   125:     method_names = ['get_process_frame']
   126:     for m in method_names:
   127:       exec 'self.userdict[m]=self.process.'+m
   128: 
   129:     #master frame methods
   130:     method_names = ['get_master_frame']
   131:     for m in method_names:
   132:       exec 'self.userdict[m]=self.master.'+m
   133: 


End python section to interscript/frames/inputf.py[3]

6.10.10.1.2. close
------------------

Remove circular references.

Start python section to interscript/frames/inputf.py[4]

   134: #line 183 "input_frame.ipk"
   135:   def close(self):
   136:     if self.verbosity>=6:
   137:       print 'closing frame',self.source.name
   138:     del self.userdict
   139:     del self.tangler
   140:     del self.weaver
   141:     del self.reg_list
   142:     while self.tangler_stack: del self.tangler_stack[-1]
   143:     while self.weaver_stack: del self.weaver_stack[-1]
   144:     try: raise eoi
   145:     except: pass
   146: 


End python section to interscript/frames/inputf.py[4]

6.10.10.1.3. Process file data
------------------------------


Start python section to interscript/frames/inputf.py[5]

   147: #line 197 "input_frame.ipk"
   148:   def file_pass(self):
   149:     while 1:
   150:       try:
   151:         file,count,line = self.readline()
   152: 
   153:         echo = 0
   154:         if self.userdict.has_key('echo_input'):
   155:           echo = self.userdict['echo_input']
   156:         if self.verbosity>=6 or echo and self.verbosity!=0:
   157:           print '%s %6s: %s' % (file,count,line)
   158:         for r in self.reg_list:
   159:           match = r[0].match(line)
   160:           if match:
   161:             r[1](match,file,count,self.process.global_frame.__dict__,self.userdict)
   162:             break
   163:       except eoi:
   164:         if self.verbosity>=3: print 'EOI detected'
   165:         if self.verbosity>=4: print 'Poping input stack'
   166:         if self.tangler:
   167:           self.select(None)
   168:         self.close()
   169:         return
   170:       except KeyboardInterrupt:
   171:         print '!!!!!!!!! KEYBOARD INTERRUPT !!!!!!!!!'
   172:         self.process.update_files = 0
   173:         self.close()
   174:         raise KeyboardInterrupt
   175:       except SystemExit,value:
   176:         print '!!!!!!!!! SYSTEM EXIT !!!!!!!!!'
   177:         self.process.update_files = 0
   178:         self.close()
   179:         raise SystemExit,value
   180:       except:
   181:         if self.verbosity>=1:
   182:           print '!!!!!!!!! PROGRAM ERROR !!!!!!!!!'
   183:           traceback.print_exc()
   184:         self.process.update_files = 0
   185:         self.close()
   186:         sys.exit(1)
   187: 


End python section to interscript/frames/inputf.py[5]

6.10.10.2. Commands
-------------------


6.10.10.2.1. Invoke interscript
-------------------------------

This is a blocking call. Beware of side-effects. [Bug
in current implementation].

6.10.10.2.1.1. Implementation
-----------------------------


Start python section to interscript/frames/inputf.py[6]

   188: #line 244 "input_frame.ipk"
   189:   def interscript_from_options(self,*args):
   190:     from interscript  import run_from_options
   191:     svdout = sys.stdout
   192:     svdin  = sys.stdin
   193:     svderr = sys.stderr
   194:     run_from_options(args)
   195:     sys.stdout = svdout
   196:     sys.stdin  = svdin
   197:     sys.stderr = svderr
   198: 


End python section to interscript/frames/inputf.py[6]

6.10.10.2.1.2. Test
-------------------


Start data section to interscript/tests/call_test.tpk[1]

     1: @print 'running call test'
     2: @head(1,'Call test')
     3: This is a call test.


End data section to interscript/tests/call_test.tpk[1]

Output should be at:
interscript/tests/output/call_test.html.

6.10.10.2.2. Misc: move these
-----------------------------


Start python section to interscript/frames/inputf.py[7]

   199: #line 271 "input_frame.ipk"
   200:   def get_attribute(self,name,default=None):
   201:     if self.userdict.has_key(name):
   202:       return self.userdict[name]
   203:     else:
   204:       return default
   205: 


End python section to interscript/frames/inputf.py[7]

6.10.10.2.3. begin/end blocks
-----------------------------

The @begin() command creates a new scope, by pushing a
new input_frame onto the input stack. The @end()
command is used to end a @begin() block. An unbalanced
@end() command can be used to terminate input from a
file before the physical end of file.

Use of begin/end blocks is important to limit the
lifetime of various objects. In particular, if a
tangler is writing code to a named_file_sink, then the
file will not be closed properly until it is destroyed.

Start python section to interscript/frames/inputf.py[8]

   206: #line 289 "input_frame.ipk"
   207:   def begin(self):
   208:     ho = self.head_offset
   209:     self.select(None)
   210:     inpt = input_frame(
   211:       self.pass_frame,
   212:       self.source,
   213:       [],
   214:       self.weaver,
   215:       self.userdict.copy(),
   216:       self.depth)
   217:     inpt.head_offset = ho
   218:     inpt.set_warning_character(python=self.python_warn)
   219:     inpt.file_pass()
   220: 
   221:   def end(self):
   222:     self.select(None)
   223:     raise eoi
   224: 


End python section to interscript/frames/inputf.py[8]

6.10.10.2.3.1. Begin/end test
-----------------------------


Start data section to interscript/tests/test_beginend.pak[1]

     1: @head(1,'Begin/End test')
     2: We test the @begin and @end commands with a simple scoping test.
     3: First, a level is set to 1, and we create a class with
     4: a destructor and an object thereof.
     5: @class X:
     6:    def __init__(self, level, weaver):
     7:      self.level = level
     8:      self.weaver = weaver
     9:    def __del__(self):
    10:      self.weaver.writeline('X object of level '+str(self.level)+' destroyed.')
    11:  level=1
    12:  x = X(level, get_weaver())
    13: @weave('Level is '+str(level)+'. ')
    14: @p()
    15: Now, a new block.
    16: @begin()
    17: Check old value of level.
    18: @weave('Level is '+str(level)+'. ')
    19: Now set level to 2, and create another class object.
    20: @level = 2
    21: @x = X(level,get_weaver())
    22: @weave('Level is '+str(level)+'. ')
    23: @p()
    24: Finally we end the block here. This should kill the inner class object.
    25: @end()
    26: @p()
    27: Back in the old block we test the level.
    28: @weave('Level is '+str(level)+'. ')
    29: That's the simple test. Now, the outer class object should die.


End data section to interscript/tests/test_beginend.pak[1]

We can look at the html results at
../tests/test_beginend.html

6.10.10.2.4. Include file/source
--------------------------------

The include file command is a subroutine call. It
pushes a new input frame associated with the file,
causing input to be read from the file. The new input
frame has a copy of the user dictionary from the
calling frame. When the file ends, the frame is popped
which destroys the dictionary; all newly created
symbols are lost.

The new input frame does not inherit the parent frames
lexicology. In particular, the warning character is
restablished as @. This command reasserts document
mode, both in the included file and the current one.

The command include_source is a generalisation of
include_file which takes any source driver as the
input.

Start python section to interscript/frames/inputf.py[9]

   225: #line 360 "input_frame.ipk"
   226:   def include_file(self,name):
   227:     if self.verbosity>=2:
   228:       print 'input from',name
   229:     self.pass_frame.include_files.append((self.depth+1,'interscript',name))
   230:     self.include_source(named_file_source(self.pass_frame,name, self.source.directory))
   231: 
   232:   def include_source(self,source):
   233:     self.select(None)
   234:     ho = self.head_offset
   235:     inpt = input_frame(
   236:       self.pass_frame,
   237:       source,
   238:       [],
   239:       self.weaver,
   240:       self.userdict.copy(),
   241:       self.depth+1)
   242:     inpt.head_offset = ho
   243:     inpt.set_warning_character(python='@')
   244:     inpt.file_pass()
   245:     self.weaver.set_original_filename (self.original_filename)
   246: 


End python section to interscript/frames/inputf.py[9]

6.10.10.2.5. Include/insert code
--------------------------------

The @insert_code command inserts a code file into the
input stream of the current tangler. It is used to yank
code fragments from external files.

The external file can contain any data, all of which is
interpreted as literal program code: lines beginning
with @, for example, will not be processed as commands.

Note that the file is not written directly to the
tangler's sink driver, it is written to the tangler.
Therefore, if the code contains embedded documentation
constructions in a form the tangler understands,
documentation can still be woven. For example if a Perl
module containing POD constructions is imported, a
perl_tangler will still recognize these constructions
and send appropriate documentation commens to its
associated weaver.

The @include_code command is the same as the
@insert_code command except that the tangler is passed
as an argument, rather than the current tangler being
used.

Start python section to interscript/frames/inputf.py[10]

   247: #line 403 "input_frame.ipk"
   248:   def insert_code(self,name):
   249:     self.tangler_push(self.tangler)
   250:     self.pass_frame.include_files.append((self.depth+1,'code: '+self.tangler.language,name))
   251:     inpt = input_frame(
   252:       self.pass_frame,
   253:       named_file_source(self,name, self.source.directory),
   254:       [(self.any_line_re,self.process.do_web)],
   255:       self.weaver,
   256:       self.userdict.copy(),
   257:       self.depth+1)
   258:     inpt.file_pass()
   259: 
   260:   def include_code(self,name,tangler):
   261:     self.tangler_set(None)
   262:     self.pass_frame.include_files.append((self.depth+1,'code: '+self.tangler.language,name))
   263:     inpt = input_frame(
   264:       self.pass_frame,
   265:       named_file_source(self,name, self.source.directory),
   266:       [(self.any_line_re,self.process.do_web)],
   267:       self.weaver,
   268:       self.userdict.copy(),
   269:       self.depth+1)
   270:     inpt.select(tangler)
   271:     inpt.file_pass()
   272: 


End python section to interscript/frames/inputf.py[10]

6.10.10.2.6. Include html
-------------------------

The @html and @include_html commands switch the input
parser from raw interscript to a small subset of HTML.
The parser maps tags to calls to interscript commands.
For example, <B> is translated to call the bold()
command.

Python script can be included in the html using the
tags <SCRIPT LANGUAGE="python"> .. </SCRIPT> At
present, the translator has even less features than
interscript: it's only a stub for a more full scale
translator.

The @html() command is similar, except it takes html
data from the current input source.

In both cases, </HTML> terminates HTML parsing.

Start python section to interscript/frames/inputf.py[11]

   273: #line 444 "input_frame.ipk"
   274:   def include_html(source):
   275:     self.select(None)
   276:     r = []
   277:     self.pass_frame.include_files.append((self.depth+1,'html: '+self.tangler.language,name))
   278:     inpt = input_frame(
   279:       self.pass_frame,
   280:       source,
   281:       r,
   282:       self.weaver,
   283:       self.userdict.copy(),
   284:       self.depth+1)
   285:     inpt.html_parser = sgml_wrapper(html_filter(inpt))
   286:     r.append((inpt.any_line_re,inpt.do_html))
   287:     inpt.file_pass()
   288: 
   289:   def html(self):
   290:     self.select(None)
   291:     r = []
   292:     inpt = input_frame(
   293:       self.pass_frame,
   294:       self.source,
   295:       r,
   296:       self.weaver,
   297:       self.userdict.copy(),
   298:       self.depth)
   299:     inpt.html_parser = sgml_wrapper(html_filter(inpt))
   300:     r.append((inpt.any_line_re,inpt.do_html))
   301:     inpt.file_pass()
   302: 
   303: 
   304: 


End python section to interscript/frames/inputf.py[11]

6.10.10.2.7. Verbosity
----------------------


Start python section to interscript/frames/inputf.py[12]

   305: #line 477 "input_frame.ipk"
   306:   def get_verbosity(self):
   307:     if self.userdict.has_key('verbosity'):
   308:       return self.userdict['verbosity']
   309:     return self.verbosity
   310: 


End python section to interscript/frames/inputf.py[12]

6.10.10.2.8. Weaver Control
---------------------------


Start python section to interscript/frames/inputf.py[13]

   311: #line 484 "input_frame.ipk"
   312:   def get_weaver(self):
   313:     return self.weaver
   314: 
   315:   def set_weaver(self,w):
   316:     tmp = self.weaver
   317:     self.weaver = w
   318:     return tmp
   319: 
   320:   def push_weaver(self,w):
   321:     self.weaver_stack.append(self.weaver)
   322:     self.weaver = w
   323: 
   324:   def pop_weaver(self):
   325:     self.weaver = self.weaver_stack[-1]
   326:     del self.weaver_stack[-1]
   327: 


End python section to interscript/frames/inputf.py[13]

6.10.10.2.9. Some Commands
--------------------------


Start python section to interscript/frames/inputf.py[14]

   328: #line 502 "input_frame.ipk"
   329:   def set_anchor(self,label):
   330:     if self.tangler:
   331:       self.tangler.weaver.set_anchor(label)
   332:     else:
   333:       self.weaver.set_anchor(label)
   334: 
   335:   def ref_anchor(self,label):
   336:     self.weaver.ref_anchor(label)
   337: 
   338:   def set_warning_character(self,python=None):
   339:     res = self.make_parse_tab(pywarn=python)
   340:     res = compile_parse_tab(res)
   341:     self.reg_list = res
   342:     self.python_warn = python
   343: 
   344:   def normal_line(self,data,file,count):
   345:     weaver = self.get_weaver()
   346:     if self.tangler:
   347:       self.tangler.writeline(data,file,count)
   348:     else:
   349:       weaver.writeline(data)
   350: 


End python section to interscript/frames/inputf.py[14]

6.10.10.2.10. Input functions
-----------------------------

These are private functions used to actually perform
input from the input_frame source driver.

Start python section to interscript/frames/inputf.py[15]

   351: #line 528 "input_frame.ipk"
   352:   def enqueue_input(self,file, count, line):
   353:     self.read_buffer.append((file,count,line))
   354: 
   355:   def dequeue_input(self):
   356:     data = self.read_buffer[0]
   357:     del self.read_buffer[0]
   358:     return data
   359: 
   360:   def line(self, number, filename):
   361:     self.inpt.original_file = filename
   362:     self.inpt.line_offset = number - inpt.src.get_lines_read()
   363: 
   364:   def readline(self):
   365:     while 1:
   366:       if self.read_buffer:
   367:         return self.dequeue_input()
   368:       try:
   369:         line = self.source.readline()
   370:         self.real_filename = self.source.get_source_name()
   371:         self.real_count = self.source.get_lines_read()
   372:         self.original_count = self.real_count + self.line_offset
   373:         line = string.rstrip(line)
   374:         self.line = string.expandtabs(line,self.tabwidth)
   375:         return (self.original_filename,self.original_count,self.line)
   376:       except KeyboardInterrupt:
   377:         # should inhibit output for process, not globally
   378:         self.process.update_files = 0
   379:         raise KeyboardInterrupt
   380:       except eof:
   381:         if self.verbosity>=4:
   382:           print 'readline: EOF'
   383:         self.line = None
   384:         raise eoi
   385:       else:
   386:         if self.process.verbosity>=1:
   387:           print 'program error in readline:',sys.exc_info()
   388:         self.process.update_files = 0
   389: 


End python section to interscript/frames/inputf.py[15]

6.10.10.2.11. Untangle
----------------------

This command is used inside an interscript tangler to
put an external file into a .pak archive. It writes out
a 'select' (seeline 13721 ) command, and then the file,
with leading @ characters converted to two @
characters. Use it like this:

  @select(archive)
  @untangle('fred.pak')
  @untangle('joe.pak')

This mechanism is required to support include files,
since a top level document is incomplete otherwise.

Note that the inserted code is not woven! In fact, the
tangler never sees it, it simply supplies the sink
object.

Start python section to interscript/frames/inputf.py[16]

   390: #line 586 "input_frame.ipk"
   391:   def untangle(self,name):
   392:     if not self.tangler:
   393:       raise 'untangle without active tangler'
   394:     f = open(name)
   395:     data = f.readlines()
   396:     f.close()
   397:     self.tangler.sink.writeline('@select(output("'+name+'"))')
   398:     for line in data:
   399:       l = string.rstrip(line)
   400:       if len(l):
   401:         if l[0]=='@': l = '@'+l
   402:       self.inpt.tangler.sink.writeline(l)
   403:     self.tangler.sink.writeline('@select(None)')
   404:     self.tangler.weaver.begin_small()
   405:     self.tangler.weaver.writeline('Included '+name+', '+str(len(data))+' lines.')
   406:     self.tangler.weaver.end_small()
   407:     self.tangler.weaver.line_break()
   408: 


End python section to interscript/frames/inputf.py[16]

6.10.10.2.12. Parser functions
------------------------------

These function actually execute script.

For example, these methods control the disposition of
the log. This requires switching the sys.stdout and/or
sys.stdlog object.

Similarly, these routines determine the disposition of
keyboard interrupts, which normally terminate the whole
process. It may be necessary to terminate master
documents threads.

Finally, these routines will permits synchronisation of
delayed file updates: in a 'commit/rollback' scenario,
only the process frame has enough information to permit
or deny updates which may depend on all threads
elaborating their client documents correctly.

Note that the verbosity level used here is not the same
as the master or pass frame verbosity: even if these
values are changed by the user document, debugging
script execution is determined by the process level
verbosity setting. This is necessary so user level
document debugging does not interfere with interscript
developer debugging.

Other features controlled at the process level include
the use of threads, the availability of a GUI, etc.

6.10.10.2.13. Utilities
-----------------------

Python is announced by a single @. If the line ends in
a : (or some others) the script continues to the first
line with a non-space in column 1. The first space is
remove from each continuation line.

Tcl is announced by a single !. If the line ends in {,
or with a \, then the script is continued as for
Python.

In both cases, the head line can have spaces or a #
initiated comment afterwards and still be detected: for
Tcl, neither is permitted after a slosh and will
generate a Tcl error.

Start python section to interscript/frames/inputf.py[17]

   409: #line 646 "input_frame.ipk"
   410: # regexp's for the main functions
   411: 
   412:   def make_parse_tab(self, pywarn = None):
   413:     res = []
   414:     if pywarn:
   415:       res = res + [
   416:         ['^'+pywarn+'('+pywarn+')(.*)$',self.do_quote_at],
   417:         ['^'+pywarn+'(.*[-+*/%:,\([{]) *(#.*)?$', self.do_exec_suite],
   418:         ['^'+pywarn+'(.*)$',self.do_exec_line]
   419:         ]
   420: 
   421: 
   422:     res = res + [
   423:       ['^(.*)$',self.do_web]
   424:       ]
   425:     return res
   426: 


End python section to interscript/frames/inputf.py[17]

6.10.10.2.14. Collect stuff
---------------------------

This command reads ahead in the input until a line is
found that does not match the supplied continuation
pattern. It then returns all the input lines as a
single string. It can be used to drive user defined
parser objects, which process the returned string.

For example, the built-in python suite processes uses
this function to collect a multiline python suite
before executing it.

Start python section to interscript/frames/inputf.py[18]

   427: #line 673 "input_frame.ipk"
   428:   def collect_stuff(self,prefix, cont_re, echo):
   429:     saved = prefix
   430:     try:
   431:       file2,count2,line = self.readline()
   432:       match = cont_re.match(line)
   433:       while match:
   434:         if echo:
   435:           print '%s %6s: %s' % (file2,count2,line)
   436:         body = match.group(1)
   437:         if not body: body = ''
   438:         saved = saved+'\n'+body
   439:         file2,count2,line = self.readline()
   440:         match = cont_re.match(line)
   441:       self.enqueue_input(file2,count2,line)
   442:     except eoi:
   443:       pass
   444:     saved = saved + '\n'
   445:     return saved
   446: 
   447:   def collect_lines_upto(self,terminal):
   448:     term_re = re.compile('^'+terminal+'$')
   449:     saved = []
   450:     file,count,line = self.readline()
   451:     match = term_re.match(line)
   452:     while not match:
   453:       saved.append(line)
   454:       file,count,line = self.readline()
   455:       match = term_re.match(line)
   456:     return saved
   457: 
   458:   def collect_upto(self,terminal):
   459:     return string.join(self.collect_lines_upto(terminal), '\n')+'\n'
   460: 
   461:   def python(self, terminal='@end_python'):
   462:     file = self.original_filename
   463:     count = self.original_count
   464:     glb = self.global_frame.__dict__
   465:     user = self.userdict
   466:     data = self.collect_upto(terminal)
   467:     self.process.py_exec(data,file,count,glb,user)
   468: 


End python section to interscript/frames/inputf.py[18]

6.10.10.2.14.1. Test python function
------------------------------------

Woven from embedded python script.

6.10.10.2.14.2. Print diff table
--------------------------------

Print out a diff table with side by side comparison of
files. Requires the diff module. No table is generated
if the files compare equal.

Start python section to interscript/frames/inputf.py[19]

   469: #line 726 "input_frame.ipk"
   470:   def print_diff_table(self, comparison,
   471:     actual_heading='Actual', expected_heading='Expected',
   472:     ok_message='Data compared equal.',
   473:     diff_message='Differential follows.'):
   474: 
   475:     equal = len(comparison) == 0
   476:     our_weaver = self.get_weaver()
   477:     if not equal:
   478:       if diff_message:
   479:         our_weaver.writeline(diff_message)
   480:       our_weaver.begin_table('Actual','Expected', CLASS='DIFF')
   481:       for section in comparison:
   482:         left = section[0][1:]
   483:         right = section[1][1:]
   484:         left = string.join(left,'\n')
   485:         right = string.join(right,'\n')
   486:         our_weaver.table_row([left, right])
   487:       our_weaver.end_table()
   488:     else:
   489:       if ok_message:
   490:         our_weaver.writeline(ok_message)
   491: 


End python section to interscript/frames/inputf.py[19]

6.10.10.2.15. Python Test
-------------------------


Start python section to interscript/frames/inputf.py[20]

   492: #line 750 "input_frame.ipk"
   493:   def test_python(self,
   494:     hlevel=None,
   495:     descr=None,
   496:     source_filename=None,
   497:     source_terminator=None,
   498:     expect_filename=None,
   499:     expect_terminator=None,
   500:     diff_context=0):
   501: 
   502:     testno = self.pass_frame.get_new_test_number()
   503:     testlabel = 'test_'+str(testno)
   504:     test_record = self.pass_frame.tests[testno]=\
   505:       [descr,testlabel,'python','Aborted']
   506:     expect = expect_filename or expect_terminator
   507: 
   508:     # print heading
   509:     if hlevel: our_hlevel = hlevel
   510:     else: our_hlevel = self.last_head+1
   511:     if descr == None: descr = 'Test'
   512:     self.head(our_hlevel,'Test '+str(testno)+': '+descr)
   513:     self.set_anchor(testlabel)
   514: 
   515:     our_weaver = self.get_weaver()
   516: 
   517:     if source_terminator:
   518:       our_weaver.writeline('On-the-fly python test script follows.')
   519:       source_origin_line = self.original_count
   520:       source_origin_file = self.original_filename
   521:       test_code = self.collect_lines_upto(source_terminator)
   522:       our_weaver.script_head('python',source_origin_file)
   523:       for i in range(len(test_code)):
   524:         our_weaver.echotangle(source_origin_line+i+1,test_code[i])
   525:       our_weaver.script_foot('python',source_origin_file)
   526:     elif source_filename:
   527:       our_weaver.writeline('Python test script from file '+source_filename+'.')
   528: 
   529:     if expect_terminator:
   530:       expected_origin_line = self.original_count
   531:       expected_origin_file = self.original_filename
   532:       expected_output = self.collect_lines_upto(expect_terminator)
   533:     elif expect_filename:
   534:       # FIX to make document relative
   535:       our_weaver.writeline('Expected output from file '+expected_filename+'.')
   536:       expected_lines = loadfile(expect_filename)
   537: 
   538:     # execute the test code
   539: 
   540:     if source_filename:
   541:       our_source_filename = source_filename
   542:     else:
   543:       our_source_filebase = tempfile.mktemp()
   544:       our_source_filename = our_source_filebase + '_test.py'
   545:       f = open(our_source_filename,'w')
   546:       f.write(string.join(test_code,'\n')+'\n')
   547:       f.close()
   548: 
   549:     our_weaver.writeline('Actual output follows.')
   550:     status, actual_output = self.print_python_output(our_source_filename)
   551:     cmd_ok = status == 0
   552: 
   553:     # delete the file if it was created anonymously
   554:     if not source_filename:
   555:       os.remove(our_source_filename)
   556: 
   557:     if expect:
   558:       try:
   559:         diff_lines = interscript.utilities.diff.diff_lines
   560:         comparison = diff_lines(actual_output, expected_output, context=diff_context)
   561:         equal = len(comparison)==0
   562:         self.pass_frame.tests[testno][2]= 'diff'
   563:         self.pass_frame.tests[testno][3]= ('Fail','Ok')[equal]
   564:         if not equal:
   565:           our_weaver.writeline('On-the-fly expected output follows.')
   566:           our_weaver.expected_head(expected_origin_file)
   567:           for i in range(len(expected_output)):
   568:             our_weaver.echotangle(expected_origin_line+i+1,expected_output[i])
   569:           our_weaver.expected_foot(expected_origin_file)
   570:           self.print_diff_table(comparison)
   571:       except ImportError:
   572:         our_weaver.writeline('Unable to import diff to perform comparison.')
   573:       except KeyboardInterrupt: raise
   574:       except SystemExit: raise
   575:       except:
   576:         traceback.print_exc()
   577:     else:
   578:       self.pass_frame.tests[testno][3]='Inspect'
   579: 


End python section to interscript/frames/inputf.py[20]

6.10.10.2.15.1. Test 2: Python test test
----------------------------------------

On-the-fly python test script follows.   838: print 'Hello from python test function test.'
   839: print 'Hello again from python test function test.'
Actual output follows.

Start output section of /usr/local/bin/python /usr/tmp/@13003.1508_test.py

     1: Hello from python test function test.
     2: Hello again from python test function test.

End output section of /usr/local/bin/python /usr/tmp/@13003.1508_test.py

6.10.10.2.15.2. Test 3: Python diff test
----------------------------------------

On-the-fly python test script follows.   843: print 'Hello from python diff function test.'
   844: print 'Hello again from python diff function test.'
Actual output follows.

Start output section of /usr/local/bin/python /usr/tmp/@13003.1510_test.py

     1: Hello from python diff function test.
     2: Hello again from python diff function test.

End output section of /usr/local/bin/python /usr/tmp/@13003.1510_test.py

On-the-fly expected output follows.

Start expected section of input_frame.ipk

   846: Hello from python diff function test.
   847: Hello error from python diff function test.

End expected section of input_frame.ipk

Differential follows.
+---------------------------------------------------+---------------------------------------------------+
| Actual                                            | Expected                                          |
+---------------------------------------------------+---------------------------------------------------+
|   2:! Hello again from python diff function test. |   2:! Hello error from python diff function test. |
+---------------------------------------------------+---------------------------------------------------+

6.10.10.2.16. Match Processing functions
----------------------------------------

These commands process match objects which the parser
has construted while scanning the parser table trying
to match the input string.

6.10.10.2.16.1. Single Line Python Script
-----------------------------------------


Start python section to interscript/frames/inputf.py[21]

   580: #line 856 "input_frame.ipk"
   581:   def do_exec_line(self,match, file,count,glb,user):
   582:     self.process.py_exec(match.group(1),file,count,glb,user)
   583: 


End python section to interscript/frames/inputf.py[21]

6.10.10.2.16.2. Multi-Line Python Script
----------------------------------------


Start python section to interscript/frames/inputf.py[22]

   584: #line 861 "input_frame.ipk"
   585:   def do_exec_suite(self,match,file,count,glb,user):
   586:     saved = self.collect_stuff(match.group(1), self.cont_re, user['echo_input'])
   587:     self.process.py_exec(saved,file,count,glb,user)
   588: 


End python section to interscript/frames/inputf.py[22]

6.10.10.2.16.3. Ordinary Text
-----------------------------


Start python section to interscript/frames/inputf.py[23]

   589: #line 867 "input_frame.ipk"
   590:   def do_web(self,match,file,count,glb,user):
   591:     self.normal_line(match.group(1),file,count)
   592: 


End python section to interscript/frames/inputf.py[23]

6.10.10.2.16.4. Doubled Warning character
-----------------------------------------


Start python section to interscript/frames/inputf.py[24]

   593: #line 872 "input_frame.ipk"
   594:   def do_quote_at(self,match,file,count,glb,user):
   595:     self.normal_line(match.group(1)+match.group(2),file,count)
   596: 


End python section to interscript/frames/inputf.py[24]

6.10.10.2.16.5. Html input
--------------------------


Start python section to interscript/frames/inputf.py[25]

   597: #line 877 "input_frame.ipk"
   598:   def do_html(self,match,file,count,glb,user):
   599:     self.html_parser.writeline(match.group(1),file,count)
   600: 


End python section to interscript/frames/inputf.py[25]

6.10.10.2.17. The Tangler Stack
-------------------------------

Interscript supports stacking of tanglers. The internal
commands tangler_push, and tangler_pop push and pop the
stack, while tangler_set changes the top of the stack.

These operations call the start and end section methods
of the tangler objects as they become, or cease to
become, the current tangler, this ensures correct
source file references are inserted into the code
files.

Start python section to interscript/frames/inputf.py[26]

   601: #line 890 "input_frame.ipk"
   602:   def tangler_push(self,f):
   603:     self.tangler_stack.append(self.tangler)
   604:     self.tangler = f
   605: 
   606:   def tangler_pop(self):
   607:     self.tangler = self.tangler_stack[-1]
   608:     del self.tangler_stack[-1]
   609: 
   610:   def tangler_set(self,f):
   611:     self.tangler = f
   612: 


End python section to interscript/frames/inputf.py[26]

6.10.11. Commands
-----------------

These functions are members of the class input_frame!.

6.10.11.1. User Commands
------------------------

The following global functions are intended to be
invoked by the user in the Interscript source file.

Start python section to interscript/frames/inputf.py[27]

   613: #line 910 "input_frame.ipk"
   614: #---------------------------------------------------------
   615: # USER COMMANDS
   616: 


End python section to interscript/frames/inputf.py[27]

6.10.11.1.1. Tangler Constructors
---------------------------------

These are convenience functions that construct tanglers
and associated sinks. The argument is a filename which
is used to construct a sink to which the tangler is
linked. The tanglers are linked to the current weaver.

Start python section to interscript/frames/inputf.py[28]

   617: #line 920 "input_frame.ipk"
   618:   def output(self,f):
   619:     filename = self.master.tangler_directory+f
   620:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   621:     return data_tangler(sink,self.get_weaver())
   622: 
   623:   def c_output(self,f):
   624:     filename = self.master.tangler_directory+f
   625:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   626:     return c_tangler(sink,self.get_weaver())
   627: 
   628:   def cpp_output(self,f):
   629:     filename = self.master.tangler_directory+f
   630:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   631:     return cpp_tangler(sink,self.get_weaver())
   632: 
   633:   def python_output(self,f):
   634:     filename = self.master.tangler_directory+f
   635:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   636:     return python_tangler(sink,self.get_weaver())
   637: 
   638:   def perl_output(self,f):
   639:     filename = self.master.tangler_directory+f
   640:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   641:     return perl_tangler(sink,self.get_weaver())
   642: 
   643:   # temporarily, we'll use a data tangler
   644:   def interscript_output(self,f):
   645:     filename = self.master.tangler_directory+f
   646:     sink = named_file_sink(self.pass_frame,filename,self.master.tangler_prefix)
   647:     return data_tangler(sink,self.get_weaver())
   648: 


End python section to interscript/frames/inputf.py[28]

6.10.11.1.2. Managing the tangler stack
---------------------------------------

These commands also push, pop, and set the tangler
stack, but they also generate start and end section
messages in the documentation file. They are intended
for use by the end user.

Pushing and popping the stack is useful, for example,
to temporarily write a C function declaration to the
header file just before it is defined. (In fact,
Interscript has some special case functionality to
support this while avoiding writing the function
signature twice.)

The begin and end comments functions push and pop the
tangler stack, the push comments function pushes the
comment tangler associated withe the current tangler
(if there is a current tangler, otherwise it pushes
None. The comment tangler associated with the current
tangler could also be, and is by default, None.)

Note that the begin and end comments functions invoke
the internal push and pop functions; that is, they do
not write begin and end section messages to the
documentation file. That's because the comments are
actually written to the code file, they're just
formatted by the weaver like documentation rather than
code.

Design note: There is always a current tangler object,
possibly None. Interscript knows it is writing code,
except when the current tangler is None, in which case
it is writing documentation. I'm not so sure this is a
wonderful scheme :-(

The begin and end string functions write the enclosed
lines to the code file formatted as a (single) string.
The resultant output is echoed to the documentation
file.

The begin and end string functions reformat the
enclosed lines as a single string in the target
programming language. In C, for example, special
character such as slosh are replaced by two sloshes.
(Note that a leading @ can be included only by writing
@@).

The begin string function supports two optional
arguments. The eol argument determines what to insert
at the end of a line, it will usually be either an
empty string, a newline character sequence, or a single
space.

Trailing whitespace is removed from each line. It is
not possible to disable this feature, and that is
deliberate. However, a second argument, width, if
positive, will then pad the line to the specified width
with spaces. This feature is designed to support two
dimensional character arrays.

Design note: there is as yet not support for
international character sets.

Start python section to interscript/frames/inputf.py[29]

   649: #line 1002 "input_frame.ipk"
   650:   def push(self,f):
   651:     if self.tangler: self.code_foot()
   652:     self.tangler_push(f)
   653:     if self.tangler: self.code_head()
   654: 
   655:   def pop(self):
   656:     if self.tangler: self.code_foot()
   657:     self.tangler_pop()
   658:     if self.tangler: self.code_head()
   659: 
   660: #line 1013 "input_frame.ipk"
   661:   def select(self,f):
   662:     if self.tangler: self.code_foot()
   663:     self.tangler_set(f)
   664:     if self.tangler: self.code_head()
   665: 
   666:   def code_head(self):
   667:     dst_filename = self.tangler.sink.name
   668:     dst_lineno = self.tangler.sink.lines_written
   669:     src_filename = self.original_filename
   670:     src_lineno = self.original_count
   671: 
   672:     index = self.pass_frame.section_index
   673:     list = index.get(dst_filename, [])
   674:     list.append((dst_lineno, src_filename, src_lineno))
   675:     index[dst_filename]=list
   676:     secno = len(list)
   677:     self.weaver.code_head(self.tangler, secno)
   678: 
   679:   def code_foot(self):
   680:     dst_filename = self.tangler.sink.name
   681:     index = self.pass_frame.section_index
   682:     list = index.get(dst_filename, [])
   683:     secno = len(list)
   684:     self.weaver.code_foot(self.tangler, secno)
   685: 
   686:   def begin_comments(self):
   687:     if self.tangler:
   688:       self.tangler_push(self.tangler.get_comment_tangler())
   689:     else:
   690:       self.tangler_push(None)
   691: 
   692:   def end_comments(self):
   693:     self.tangler_pop()
   694: 
   695:   def resume_code(self):
   696:     self.tangler_pop()
   697: 
   698:   def comment(self,v):
   699:     self.get_weaver().write_comment(v)
   700: 
   701:   def begin_string(self,eol = ' ', width = 0):
   702:     if self.tangler:
   703:       self.tangler_push(self.tangler.get_string_tangler(eol,width))
   704:     else:
   705:       self.tangler_push(None)
   706: 
   707:   def end_string(self):
   708:     tangler_pop()
   709: 
   710:   def weave(self,s):
   711:     weaver = self.get_weaver()
   712:     weaver.write(s)
   713: 
   714:   def weave_line(self,s):
   715:     weaver = self.get_weaver()
   716:     weaver.writeline(s)
   717: 
   718:   def tangle(self,s, inhibit_sref=0):
   719:     if self.tangler:
   720:       line = self.original_count
   721:       file = self.original_filename
   722:       self.tangler.writeline(s,file,line,inhibit_sref)
   723:     else:
   724:       print "tangle: No tangler for",s
   725: 


End python section to interscript/frames/inputf.py[29]

6.10.11.1.3. Miscellaneous
--------------------------

The usual stuff that is hard to classify.

Start python section to interscript/frames/inputf.py[30]

   726: #line 1081 "input_frame.ipk"
   727:   def print_contents(self,*args, **kwds):
   728:     self.select(None)
   729:     weaver = self.get_weaver()
   730:     apply(weaver.print_contents, args, kwds)
   731: 
   732:   def print_file_list(self,*args,**kwds):
   733:     self.select(None)
   734:     weaver = self.get_weaver()
   735:     apply(weaver.print_file_list, args, kwds)
   736: 
   737:   def print_file_status(self,*args,**kwds):
   738:     self.select(None)
   739:     weaver = self.get_weaver()
   740:     apply(weaver.print_file_status, args, kwds)
   741: 
   742:   def print_source_list(self, *args, **kwds):
   743:     self.select(None)
   744:     weaver = self.get_weaver()
   745:     apply(weaver.print_source_list, args, kwds)
   746: 
   747:   def print_include_list(self, *args, **kwds):
   748:     self.select(None)
   749:     weaver = self.get_weaver()
   750:     apply(weaver.print_include_list, args, kwds)
   751: 
   752:   def macro(self,name):
   753:     self.select(None)
   754:     weaver = self.get_weaver()
   755:     return data_tangler(memory(name),weaver)
   756: 
   757:   def print_identifier_cross_reference(self, *args, **kwds):
   758:     self.select(None)
   759:     weaver = self.get_weaver()
   760:     apply(weaver.identifier_reference, args, kwds)
   761: 
   762:   def print_class_reference(self, *args, **kwds):
   763:     self.select(None)
   764:     weaver = self.get_weaver()
   765:     apply(weaver.class_reference, args, kwds)
   766: 


End python section to interscript/frames/inputf.py[30]

6.10.11.1.4. Capture command output
-----------------------------------

This is unix dependent at present.

Start python section to interscript/frames/inputf.py[31]

   767: #line 1124 "input_frame.ipk"
   768:   def capture_output(self,command):
   769:     commands = self.global_frame.commands
   770:     status, output = commands.getstatusoutput(command)
   771:     data = string.split(output,'\n')
   772:     return (status,data)
   773: 
   774:   def print_output(self,command):
   775:     status, data = self.capture_output(command)
   776:     weaver = self.get_weaver()
   777:     weaver.test_output_head(command, status)
   778:     for i in range(len(data)):
   779:       line = data[i]
   780:       l = string.rstrip(line)
   781:       weaver.echotangle(i+1,l)
   782:     weaver.test_output_foot(command, status)
   783:     return (status, data)
   784: 
   785:   def capture_python_output(self,script):
   786:     return self.capture_output(sys.executable+' '+script)
   787: 
   788:   def print_python_output(self,script):
   789:     return self.print_output(sys.executable+' '+script)
   790: 
   791:   def print_python_test_output(self,script, descr):
   792:     testno = self.pass_frame.get_new_test_number()
   793:     testlabel = 'test_'+str(testno)
   794:     self.pass_frame.tests[testno]=[descr,testlabel,'python','Unknown']
   795:     self.set_anchor(testlabel)
   796:     return self.print_python_output(script)
   797: 


End python section to interscript/frames/inputf.py[31]

6.10.11.1.5. Documentation Constructions
----------------------------------------

The following constructions affect the documentation
file only. They are used to designate headings,
separate paragraphs, print tables, etc.

These functions are members of class input_frame.

6.10.11.1.5.1. Headings
-----------------------


Start python section to interscript/frames/inputf.py[32]

   798: #line 1162 "input_frame.ipk"
   799:   def head(self, level, text, atext='', anchor=''):
   800:     level = int(level)
   801:     level = level + self.head_offset
   802:     self.last_head = level
   803:     if self.get_verbosity()>=3: print ('  '*(level-1))+'"'+text+'"'
   804:     self.pass_frame.toc.append((level,text))
   805:     weaver = self.get_weaver()
   806:     if self.tangler: self.code_foot()
   807:     self.tangler_set(None)
   808:     weaver.head(level,text,atext,anchor)
   809: 
   810:   # like heading, but to be used in code as well:
   811:   # doesn't switch to document mode, doesn't do
   812:   # code headings and footings
   813: 
   814:   def heading(self, level, text, atext='', anchor=''):
   815:     level = int(level)
   816:     level = level + self.head_offset
   817:     self.last_head = level
   818:     if self.get_verbosity()>=3: print ('  '*(level-1))+'"'+text+'"'
   819:     self.pass_frame.toc.append((level,text))
   820:     weaver = self.get_weaver()
   821:     weaver.head(level,text,atext,anchor)
   822: 
   823:   def push_head(self, amt=1):
   824:     self.head_offset = self.head_offset + amt
   825: 
   826:   def pop_head(self, amt=1):
   827:     self.push_head(-amt)
   828: 
   829:   def set_head(self, amt=None):
   830:     if amt != None:
   831:       self.head_offset = amt - 1
   832:     else:
   833:       self.head_offset = self.last_head - 1
   834: 


End python section to interscript/frames/inputf.py[32]

6.10.11.1.5.2. Paragraphs
-------------------------


Start python section to interscript/frames/inputf.py[33]

   835: #line 1200 "input_frame.ipk"
   836:   def doc(self):
   837:     weaver = self.get_weaver()
   838:     if self.tangler: self.code_foot()
   839:     self.tangler_set(None)
   840: 
   841:   def p(self): # end a paragraph and start a new one
   842:     weaver = self.get_weaver()
   843:     weaver.par()
   844: 
   845:   def eop(self): # end a paragraph without starting a new one
   846:     weaver.eop()
   847: 


End python section to interscript/frames/inputf.py[33]

6.10.11.1.5.3. Citations
------------------------


Start python section to interscript/frames/inputf.py[34]

   848: #line 1214 "input_frame.ipk"
   849:   def cite_url(self, url):
   850:     weaver = self.get_weaver()
   851:     weaver.cite_url(url)
   852: 


End python section to interscript/frames/inputf.py[34]

6.10.11.1.5.4. Tables
---------------------


Start python section to interscript/frames/inputf.py[35]

   853: #line 1220 "input_frame.ipk"
   854:   def begin_table(self, *headings, **kwds):
   855:     apply(self.get_weaver().begin_table, headings, kwds)
   856: 
   857:   def table_row(self, *data):
   858:     self.get_weaver().table_row(data)
   859: 
   860:   def end_table(self):
   861:     self.get_weaver().end_table()
   862: 
   863:   def table_rule(self):
   864:     self.get_weaver().table_rule()
   865: 


End python section to interscript/frames/inputf.py[35]

6.10.11.1.5.5. Lists
--------------------


Start python section to interscript/frames/inputf.py[36]

   866: #line 1234 "input_frame.ipk"
   867:   def begin_list(self, style):
   868:     weaver = self.get_weaver()
   869:     weaver.begin_list(style)
   870: 
   871:   def end_list(self):
   872:     weaver = self.get_weaver()
   873:     weaver.end_list()
   874: 
   875:   def item(self,*args, **kwds):
   876:     weaver = self.get_weaver()
   877:     apply(weaver.item, args, kwds)
   878: 
   879:   def begin_numbered_list(self, start=1):
   880:     weaver = self.get_weaver()
   881:     weaver.begin_numbered_list(start)
   882: 
   883:   def end_numbered_list(self):
   884:     weaver = self.get_weaver()
   885:     weaver.end_numbered_list()
   886: 
   887:   def begin_numbered_list_item(self):
   888:     weaver = self.get_weaver()
   889:     weaver.begin_numbered_list_item()
   890: 
   891:   def end_numbered_list_item(self):
   892:     weaver = self.get_weaver()
   893:     weaver.end_numbered_list_item()
   894: 
   895:   def begin_bullet_list(self):
   896:     weaver = self.get_weaver()
   897:     weaver.begin_bullet_list()
   898: 
   899:   def end_bullet_list(self):
   900:     weaver = self.get_weaver()
   901:     weaver.end_bullet_list()
   902: 
   903:   def begin_bullet_list_item(self):
   904:     weaver = self.get_weaver()
   905:     weaver.begin_bullet_list_item()
   906: 
   907:   def end_bullet_list_item(self):
   908:     weaver = self.get_weaver()
   909:     weaver.end_bullet_list_item()
   910: 
   911:   def begin_keyed_list(self):
   912:     weaver = self.get_weaver()
   913:     weaver.begin_keyed_list()
   914: 
   915:   def end_keyed_list(self):
   916:     weaver = self.get_weaver()
   917:     weaver.end_keyed_list()
   918: 
   919:   def begin_keyed_list_item(self, key):
   920:     weaver = self.get_weaver()
   921:     weaver.begin_keyed_list_item(key)
   922: 
   923:   def end_keyed_list_item(self):
   924:     weaver = self.get_weaver()
   925:     weaver.end_keyed_list_item()
   926: 


End python section to interscript/frames/inputf.py[36]

6.10.11.1.5.6. Fonts
--------------------


Start python section to interscript/frames/inputf.py[37]

   927: #line 1296 "input_frame.ipk"
   928:   def begin_emphasize(self):
   929:     weaver = self.get_weaver()
   930:     weaver.begin_emphasize()
   931: 
   932:   def end_emphasize(self):
   933:     weaver = self.get_weaver()
   934:     weaver.end_emphasize()
   935: 
   936:   def begin_strong(self):
   937:     weaver = self.get_weaver()
   938:     weaver.begin_strong()
   939: 
   940:   def end_strong(self):
   941:     weaver = self.get_weaver()
   942:     weaver.end_strong()
   943: 
   944:   def begin_italic(self):
   945:     weaver = self.get_weaver()
   946:     weaver.begin_italic()
   947: 
   948:   def end_italic(self):
   949:     weaver = self.get_weaver()
   950:     weaver.end_italic()
   951: 
   952:   def begin_bold(self):
   953:     weaver = self.get_weaver()
   954:     weaver.begin_bold()
   955: 
   956:   def end_bold(self):
   957:     weaver = self.get_weaver()
   958:     weaver.end_bold()
   959: 
   960:   def begin_big(self):
   961:     weaver = self.get_weaver()
   962:     weaver.begin_big()
   963: 
   964:   def end_big(self):
   965:     weaver = self.get_weaver()
   966:     weaver.end_big()
   967: 
   968:   def begin_small(self):
   969:     weaver = self.get_weaver()
   970:     weaver.begin_small()
   971: 
   972:   def end_small(self):
   973:     weaver = self.get_weaver()
   974:     weaver.end_small()
   975: 
   976:   def begin_code(self):
   977:     weaver = self.get_weaver()
   978:     weaver.begin_code()
   979: 
   980:   def end_code(self):
   981:     weaver = self.get_weaver()
   982:     weaver.end_code()
   983: 


End python section to interscript/frames/inputf.py[37]

6.10.11.1.5.7. Displays
-----------------------


Start python section to interscript/frames/inputf.py[38]

   984: #line 1354 "input_frame.ipk"
   985:   def begin_displayed_code(self):
   986:     weaver = self.get_weaver()
   987:     weaver.begin_displayed_code()
   988: 
   989:   def end_displayed_code(self):
   990:     weaver = self.get_weaver()
   991:     weaver.end_displayed_code()
   992: 
   993:   # this command is used to print out a code file 'verbatim'
   994:   def display_code(self,name):
   995:     self.pass_frame.include_files.append((self.depth+1,'code',name))
   996:     self.begin_displayed_code()
   997:     filename = parse_source_filename(name, self.source.directory)[3]
   998:     f = open(filename)
   999:     data = f.readlines()
  1000:     f.close()
  1001:     weaver = self.get_weaver()
  1002:     for line in data:
  1003:       l = string.rstrip(line)
  1004:       weaver.writeline(l)
  1005:     self.end_displayed_code()
  1006: 
  1007: #line 1377 "input_frame.ipk"
  1008: 


End python section to interscript/frames/inputf.py[38]

6.11. Module getoptions
-----------------------

Provides generic command line processing. Input: a list
of words, each of which is an option or filename.
Output: a list of pairs, each of which is a list of
option value pairs, and a filename. The format for
options is --longoption, --longoption=value, -abcd,
-abcdef=value.

Start python section to interscript/getoptions.py[1]

     1: #line 10 "options.ipk"
     2: import re
     3: 
     4: longopts=re.compile('^--([A-Za-z][-A-Za-z_0-9]*)(?:=(.*))?$')
     5: shortopts=re.compile('^-([A-Za-z]+)(?:=(.*))?$')
     6: 
     7: 
     8: def getopt(args):
     9:   wordno = 0
    10:   result = []
    11:   opts=[]
    12:   while wordno<len(args):
    13:     filename = ''
    14:     word = args[wordno]
    15:     #print 'word',word,
    16:     match = longopts.match(word)
    17:     if match:
    18:       opts.append((match.group(1),match.group(2)))
    19:       #print ':longopt'
    20:     else:
    21:       match = shortopts.match(word)
    22:       if match:
    23:         #print ':shortopt'
    24:         for letter in match.group(1)[:-1]:
    25:           opts.append((letter,None))
    26:         opts.append((match.group(1)[-1],match.group(2)))
    27:       else:
    28:         #print ':filename'
    29:         filename = args[wordno]
    30:         result.append((opts,filename))
    31:         opts=[]
    32:     wordno = wordno + 1
    33: 
    34:   if opts:
    35:     result.append((opts,filename))
    36:   return result


End python section to interscript/getoptions.py[1]

6.11.1. Test 4: generic getoptions module
-----------------------------------------

On-the-fly python test script follows.    47: from interscript import getoptions
    48: args = ['--longopt', '--longopt-value=value', '-short', '-shortval=val2','filename']
    49: result = getoptions.getopt(args)
    50: print result
Actual output follows.

Start output section of /usr/local/bin/python /usr/tmp/@13003.1537_test.py

     1: Interscript Package: version 1.0a7 build 1253
     2: Using iscrcopt
     3: thread available
     4: Interscript version 1.0a7 build 1253
     5: Built by root on ruby at Mon Nov 09, 1998 at 07:08 PM (UTC)
     6: Generated by 1.0a7 buildno 1252 host ruby
     7: at Mon Nov 09, 1998 at 07:08 PM (UTC)
     8: [([('longopt', None), ('longopt-value', 'value'), ('s', None), ('h', None), ('o', None), ('r', None), ('t', None), ('s', None), ('h', None), ('o', None), ('r', None), ('t', None), ('v', None), ('a', None), ('l', 'val2')], 'filename')]

End output section of /usr/local/bin/python /usr/tmp/@13003.1537_test.py

6.12. Get Options
-----------------

This module gets interscript options from the command
line.

The verbosity option provides coarse generic control
over the amount of information Interscript writes. The
default is 2, which only prints 'error' messages. Level
1 is useful when you're fairly sure there are no
errors, for example, you have a trustworthy readonly
document, but have deleted the output files during a
cleanup. Level 0 is very dangerous, since even
trustworthy documents can unexpectedly cause errors:
perhaps you installed a new version of Python, or
perhaps you forgot that by default downloaded files are
reloaded from the network automatically once a month?

Start python section to interscript/getframes.py[1]

     1: #line 16 "interscript_options.ipk"
     2: import sys
     3: import traceback
     4: 
     5: from interscript.getoptions import getopt
     6: 


End python section to interscript/getframes.py[1]

6.12.1. Help Dictionary
-----------------------


Start python section to interscript/getframes.py[2]

     7: #line 23 "interscript_options.ipk"
     8: #option help dictionary
     9: shortoptdict = { 'v':'verbose (verbosity=6)' }
    10: 
    11: longoptdict = {
    12:   'echo_input=':{0:"Don't echo input (default)",1:'Echo input'},
    13:   'verbosity=': {
    14:     0:'no messages', # none _at all_, not even serious errors
    15:     1:'fatal messages', # _only_ serious errors
    16:     2:'warnings',       # all errors
    17:     3:'brief progress', # open and close weavers, tanglers, files
    18:     4:'progress and information', # headings, 'watch it run'
    19:     5:'full progress and information', # full user information
    20:     6:'user source debugging', # full debugging of _user_ script
    21:     7:'interscript debugging', # basic control flow debugging
    22:     8:'full interscript debugging', # enough to debug interscript itself
    23:     9:'eveything' # for unintelligible output :-)
    24:   },
    25:   'weaver=': {
    26:     'html': 'flat html',
    27:     'tex': 'plain tex (not implemented)',
    28:     'latex': 'latex2e',
    29:     'text':'plain text',
    30:     'ps': 'postscript (not implemented)',
    31:     'eps': 'encapsulated postscript (not implemented)',
    32:     'rtf': 'Rich Text (not implemented)',
    33:     'word': 'microsoft word (not implemented)',
    34:     'none': 'disable built-in weaving'
    35:   },
    36:   'tangler-prefix=':'absolute native os prefix prepended to tangled code filenames',
    37:   'weaver-prefix=':'absolute native os prefix prepended to woven documentation filenames',
    38:   'tangler-directory=':'interscript filename prefix prepended to tangled code filenames',
    39:   'weaver-directory=':'interscript filename prefix prepended to woven documentation filenames',
    40:   'source-prefix=':'absolute native prefix prepended to input filename',
    41:   'python=':'execute python script',
    42:   'update=':{
    43:     0:'Allow buffered file write (default)',
    44:     1:'Inhibit buffered file write'},
    45:   'download=':{
    46:     0:'only download by ftp or http when necessary',
    47:     1:'force download by ftp or http'},
    48:   'refresh_interval=':
    49:     'download when local file is older than this number of days (default 28)',
    50:   'tabwidth=':'column width for tab expansion (default 8)',
    51:   'passes=':'passs on each file (default 1)',
    52:   'logfile=':'<filename> for messages',
    53:   'help':'this help',
    54:   'usage':'this help' }
    55: 
    56: 
    57: def print_help():
    58:   print 'Usage: python iscr.py [options] <filename>'
    59:   print '  options:'
    60:   for k in shortoptdict.keys(): print_help1(k)
    61:   for k in longoptdict.keys(): print_help1(k)
    62: 
    63: def print_help1(k):
    64:   if longoptdict.has_key(k):
    65:     k = '--'+ k + '='
    66:     values = longoptdict[k]
    67:   elif shortoptdict.has_key(k+'='):
    68:     k = '-' + k + '='
    69:     values = shortoptdict[k]
    70:   else:
    71:     values = 'Unknown option'
    72: 
    73:   print '  '+k,
    74:   if values is None:
    75:     print
    76:   elif type(values) is type({}):
    77:     print
    78:     for value in values.keys():
    79:       print '   '+str(value)+':',values[value]
    80:   else:
    81:     print values
    82: 


End python section to interscript/getframes.py[2]

6.12.2. Argument Frame
----------------------

Just an empty class to hook attributes to.

Start python section to interscript/getframes.py[3]

    83: #line 101 "interscript_options.ipk"
    84: class argument_frame:
    85:   def copy(self):
    86:      other = argument_frame()
    87:      other.__dict__ = self.__dict__.copy()
    88:      return other
    89: 
    90: def getoption_frames(args): # note: has side effects!
    91:   parsed =  getopt(args)
    92:   process_options = argument_frame()
    93:   process_options.verbosity = 2
    94:   master_frames = []
    95: 
    96:   frame = argument_frame()
    97:   frame.echo_input = 0
    98:   frame.update_files = 1
    99:   frame.tabwidth = 8
   100:   frame.verbosity = 2
   101:   frame.download = 'regularly'
   102:   frame.refresh_interval = 28
   103:   frame.passes = 1
   104:   frame.weaver_prefix = ''
   105:   frame.tangler_prefix = ''
   106:   frame.weaver_directory= ''
   107:   frame.tangler_directory = ''
   108:   frame.source_prefix = ''
   109:   frame.autoweave = []
   110:   frame.useropt = {}
   111: 
   112:   for opts,filename in parsed:
   113:     for opt,value in opts:
   114:       try:
   115:         if opt == 'verbosity': process_options.verbosity = frame.verbosity = int(value)
   116:         elif opt == 'echo_input': frame.echo_input = int(value)
   117:         elif opt == 'v': process_options.verbosity = frame.verbosity = 6
   118:         elif opt == 'noupdate': frame.update_files = 0
   119:         elif opt == 'nodownload': frame.download = 'never'
   120:         elif opt == 'download': frame.download = 'always'
   121:         elif opt == 'tabwidth': frame.tabwidth = int(value)
   122:         elif opt == 'passes': frame.passes = int(value)
   123:         elif opt == 'weaver': frame.autoweave.append(value)
   124:         elif opt == 'weaver-prefix': frame.weaver_prefix = value
   125:         elif opt == 'tangler-prefix': frame.tangler_prefix = value
   126:         elif opt == 'weaver-directory': frame.weaver_directory = value
   127:         elif opt == 'tangler-directory': frame.tangler_directory = value
   128:         elif opt == 'source-prefix': frame.source_prefix = value
   129:         elif opt == 'python':
   130:           try:
   131:             if process_options.verbosity>=3:
   132:               print 'Executing python:'
   133:               print value
   134:             exec value
   135:           except:
   136:             print 'Error in python option'
   137:             traceback.print_exc()
   138:         elif opt == 'logfile':
   139:           oldsysout = sys.stdout
   140:           sys.stdout = open(value,'a')
   141:         elif opt in ['help', 'usage']:
   142:           print_help()
   143:           print
   144:         else:
   145:           # FIX: all options should be OK (user options?)
   146:           print 'Nonstandard option',opt,'value',value,'accepted as user option'
   147:           frame.useropt[opt]=value
   148:         if process_options.verbosity>=4: print 'Option:',opt,value
   149:       except:
   150:         print 'Warning: Option',opt,'has bad value',value
   151:         prefix = ''
   152:         while opt[0]=='-': prefix = prefix + '-'; opt=opt[1:]
   153:         print_help1(opt)
   154: 
   155:     frame.filename = filename
   156:     master_frames.append(frame.copy())
   157:   return process_options, master_frames
   158: 
   159: 


End python section to interscript/getframes.py[3]

6.13. Utility Modules
---------------------


Start python section to interscript/utilities/__init__.py[1]

     1: #line 428 "interscript/src/iscr.pak"
     2: # interscript utilities
     3: 


End python section to interscript/utilities/__init__.py[1]

6.13.1. compiler package
------------------------

This package

Start python section to interscript/compilers/c.py[1]

     1: #line 13 "compilers.ipk"
     2: import os
     3: import sys
     4: import string
     5: import interscript.compilers.cconfig
     6: 
     7: 
     8: class python_module:
     9:   def __init__(self,**kwds):
    10:     self.config = interscript.compilers.cconfig.config()
    11:     self.config.append_dict(kwds)
    12: 
    13:   def configure(self,**kwds):
    14:     self.config.append_dict(kwds)
    15: 
    16:   def compile(self,filename, **kwds):
    17:     config = self.config.copy()
    18:     config.append_dict(kwds)
    19: 
    20:     base = string.join(string.split(filename,'.')[:-1],'.')
    21:     obj = base+'.o'
    22:     cc ='gcc -g -O2 -fpic -fPIC -pedantic '
    23:     inc = '-I' + sys.prefix + '/include/python1.5 '
    24:     if sys.prefix != sys.exec_prefix:
    25:       inc = inc + '-I' + sys.exec_prefix + '/include/python1.5 '
    26:     cstr = str(config)+' '
    27:     arg = cc + cstr +  inc + '-c '+filename + ' -o '+ obj
    28:     print 'system',repr(arg)
    29:     os.system(arg)
    30:     return obj
    31: 
    32:   def link(self,modname, filenames, **kwds):
    33:     config = self.config.copy()
    34:     config.append_dict(kwds)
    35: 
    36:     dll = modname +'.so'
    37:     cc ='gcc -shared -Xlinker -export-dynamic '
    38:     cstr = str(config) + ' '
    39:     lib = '-L' + sys.exec_prefix + '/lib/python1.5 '
    40:     files = string.join(filenames) + ' '
    41:     arg = cc + cstr + lib + files + '-o ' + dll
    42: 
    43:     print 'system',repr(arg)
    44:     os.system(arg)
    45:     return dll
    46: 
    47: class application:
    48:   def __init__(self,**kwds):
    49:     self.config = interscript.compilers.cconfig.config()
    50:     self.config.append_dict(kwds)
    51: 
    52:   def configure(self,**kwds):
    53:     self.config.append_dict(kwds)
    54: 
    55:   def compile(self,filename, **kwds):
    56:     config = self.config.copy()
    57:     config.append_dict(kwds)
    58: 
    59:     base = string.join(string.split(filename,'.')[:-1],'.')
    60:     obj = base+'.o'
    61:     cc ='gcc -g -O2 -fpic -fPIC -pedantic '
    62:     inc = '-I' + sys.prefix + '/include/python1.5 '
    63:     if sys.prefix != sys.exec_prefix:
    64:       inc = inc + '-I' + sys.exec_prefix + '/include/python1.5 '
    65:     cstr = str(config)+' '
    66:     arg = cc + cstr +  inc + '-c '+filename + ' -o '+ obj
    67:     print 'system',repr(arg)
    68:     os.system(arg)
    69:     return obj
    70: 
    71:   def link(self,appname, filenames, **kwds):
    72:     config = self.config.copy()
    73:     config.append_dict(kwds)
    74: 
    75:     cc ='gcc '
    76:     cstr = str(config) + ' '
    77:     lib = '-L' + sys.exec_prefix + '/lib/python1.5 '
    78:     files = string.join(filenames) + ' '
    79:     arg = cc + cstr + lib + files + '-o ' + appname
    80: 
    81:     print 'system',repr(arg)
    82:     os.system(arg)
    83:     return appname
    84: 


End python section to interscript/compilers/c.py[1]

Start python section to interscript/compilers/cpp.py[1]

     1: #line 97 "compilers.ipk"
     2: import os
     3: import sys
     4: import string
     5: import interscript.compilers.cconfig
     6: 
     7: class python_module:
     8:   def __init__(self,**kwds):
     9:     self.config = compilers.cconfig.config()
    10:     self.config.append_dict(kwds)
    11: 
    12:   def configure(self,**kwds):
    13:     self.config.append_dict(kwds)
    14: 
    15:   def compile(self,filename, **kwds):
    16:     config = self.config.copy()
    17:     config.append_dict(kwds)
    18: 
    19:     base = string.join(string.split(filename,'.')[:-1],'.')
    20:     obj = base+'.o'
    21:     cc ='g++ -g -O2 -fhandle-exceptions -fpic -fPIC -pedantic '
    22:     inc = '-I' + sys.prefix + '/include/python1.5 '
    23:     if sys.prefix != sys.exec_prefix:
    24:       inc = inc + '-I' + sys.exec_prefix + '/include/python1.5 '
    25:     cstr = str(config)+' '
    26:     arg = cc + cstr +  inc + '-c '+filename + ' -o '+ obj
    27:     print 'system',repr(arg)
    28:     result = os.system(arg)
    29:     if result != 0:
    30:       raise 'Compiler Error'
    31:     return obj
    32: 
    33:   def link(self,modname, filenames, **kwds):
    34:     config = self.config.copy()
    35:     config.append_dict(kwds)
    36: 
    37:     cc ='g++ -shared -Xlinker -export-dynamic '
    38:     cstr = str(config) + ' '
    39:     lib = '-L' + sys.exec_prefix + '/lib/python1.5 '
    40:     dll = modname +'.so'
    41:     files = string.join(filenames) + ' '
    42:     arg = cc + cstr + lib + files + '-o '+dll
    43: 
    44:     print 'system',repr(arg)
    45:     result = os.system(arg)
    46:     if result != 0:
    47:       raise 'Linker Error'
    48:     return dll
    49: 
    50: class application:
    51:   def __init__(self,**kwds):
    52:     self.config = interscript.compilers.cconfig.config()
    53:     self.config.append_dict(kwds)
    54: 
    55:   def configure(self,**kwds):
    56:     self.config.append_dict(kwds)
    57: 
    58:   def compile(self,filename, **kwds):
    59:     config = self.config.copy()
    60:     config.append_dict(kwds)
    61: 
    62:     base = string.join(string.split(filename,'.')[:-1],'.')
    63:     obj = base+'.o'
    64:     cc ='g++ -g -O2 -fhandle-exceptions -fpic -fPIC -pedantic '
    65:     inc = '-I' + sys.prefix + '/include/python1.5 '
    66:     if sys.prefix != sys.exec_prefix:
    67:       inc = inc + '-I' + sys.exec_prefix + '/include/python1.5 '
    68:     cstr = str(config)+' '
    69:     arg = cc + cstr +  inc + '-c '+filename + ' -o '+ obj
    70:     print 'system',repr(arg)
    71:     result = os.system(arg)
    72:     if result != 0:
    73:       raise 'Compiler Error'
    74:     return obj
    75: 
    76:   def link(self,appname, filenames, **kwds):
    77:     config = self.config.copy()
    78:     config.append_dict(kwds)
    79: 
    80:     cc ='g++ '
    81:     cstr = str(config) + ' '
    82:     lib = '-L' + sys.exec_prefix + '/lib/python1.5 '
    83:     files = string.join(filenames) + ' '
    84:     arg = cc + cstr + lib + files + '-o '+appname
    85: 
    86:     print 'system',repr(arg)
    87:     result = os.system(arg)
    88:     if result != 0:
    89:       raise 'Linker Error'
    90:     return appname
    91: 


End python section to interscript/compilers/cpp.py[1]

Start python section to interscript/compilers/cconfig.py[1]

     1: #line 188 "compilers.ipk"
     2: import os
     3: import sys
     4: import string
     5: 
     6: class config:
     7:   def __init__(self,**kwds):
     8:     self.libdirs = []
     9:     self.incdirs = []
    10:     self.libs = []
    11:     self.macros = {}
    12:     self.switches = {}
    13:     self.extra = ''
    14:     self.append_dict(kwds)
    15: 
    16:   def copy(self):
    17:     c = config()
    18:     c.libdirs = self.libdirs[:]
    19:     c.incdirs = self.incdirs[:]
    20:     c.libs = self.libs[:]
    21:     c.macros = self.macros.copy()
    22:     c.switches = self.switches.copy()
    23:     c.extra = self.extra
    24:     return c
    25: 
    26:   def append_kwds(self,**kwds):
    27:     self.append_dict(kwds)
    28: 
    29:   def append_dict(self,kwds):
    30:     if kwds.has_key('libdirs'):
    31:       self.libdirs[-1:-1]=kwds['libdirs']
    32:     if kwds.has_key('incdirs'):
    33:       self.incdirs[-1:-1]=kwds['incdirs']
    34:     if kwds.has_key('libs'):
    35:       self.libs[-1:-1]=kwds['libs']
    36:     if kwds.has_key('extra'):
    37:       self.extra = self.extra + ' ' + kwds['extra']
    38:     if kwds.has_key('macros'):
    39:       macros = kwds['macros']
    40:       for macro in macros:
    41:         self.macros[macro] = macros[macro]
    42:     if kwds.has_key('switches'):
    43:       switches = kwds['switches']
    44:       for switch in switches:
    45:         self.switches[switch] = switches[switch]
    46: 
    47:   def __str__(self):
    48:     s = self.extra
    49:     for x in self.libdirs: s = s + ' -L' + x
    50:     for x in self.incdirs : s = s + ' -I' + x
    51:     for x in self.libs: s = s + ' -l' + x
    52:     for x in self.macros.keys():
    53:       s = s + ' -D' + x
    54:       if self.macros[x]: s = s + '=' + self.macros[x]
    55:     for x in self.switches.keys():
    56:       s = s + ' -' + x + self.switches[x]
    57:     return s
    58: 


End python section to interscript/compilers/cconfig.py[1]

Start python section to interscript/compilers/__init__.py[1]

     1: #line 246 "compilers.ipk"
     2: # compiler package


End python section to interscript/compilers/__init__.py[1]

Start python section to interscript/tests/test_compilers.py[1]

     1: #line 248 "compilers.ipk"
     2: import os
     3: import sys
     4: sys.path = ['']+sys.path
     5: import interscript.compilers.c
     6: cc = interscript.compilers.c.application()
     7: obj = cc.compile('interscript/tests/example.c')
     8: print 'Object file',obj,'generated'
     9: exe = cc.link('interscript/tests/example.exe',[obj])
    10: print 'Executable',exe,'generated'
    11: # sorry... this is unix dependent :-(
    12: os.system('interscript/tests/example.exe')
    13: print 'Executable executed'
    14: 


End python section to interscript/tests/test_compilers.py[1]

Start C section to interscript/tests/example.c[1]

     1: #line 262 "compilers.ipk"
     2: #include <stdio.h>
     3: void main() {
     4:   printf("Hello World from compiler test\n");
     5: }
     6: 


End C section to interscript/tests/example.c[1]

6.13.1.1. Test
--------------

Test it.

Start output section of /usr/local/bin/python interscript/tests/test_compilers.py

     1: Hello World from compiler test
     2: Interscript Package: version 1.0a7 build 1302
     3: thread available
     4: Interscript version 1.0a7 build 1302
     5: Built by root on ruby at Thu Nov 12, 1998 at 12:57 PM (UTC)
     6: Generated by 1.0a7 buildno 1300 host ruby
     7: at Thu Nov 12, 1998 at 12:57 PM (UTC)
     8: system 'gcc -g -O2 -fpic -fPIC -pedantic  -I/usr/local/include/python1.5 -c interscript/tests/example.c -o interscript/tests/example.o'
     9: Object file interscript/tests/example.o generated
    10: system 'gcc  -L/usr/local/lib/python1.5 interscript/tests/example.o -o interscript/tests/example.exe'
    11: Executable interscript/tests/example.exe generated
    12: Executable executed

End output section of /usr/local/bin/python interscript/tests/test_compilers.py

6.13.2. Interscript Standard module diff for Python
---------------------------------------------------

This site depedent module implements diff and patch for
Python.

6.13.2.1. Description
---------------------

The diff module is an optional Interscript component.
however it is required by the version control
subsystem, and may be used by other functions if
present. The module provides several functions.

6.13.2.2. Requirements
----------------------

The function 'compare_files' shall return 1 if two text
files have the same contents, and 0 otherwise. The
comparison may ignore whitespace at the end of lines
but is not required to. The compare files function
shall return 1 if the files are exactly equal, in
particular if the same file is given as both arguments.

The function 'compare_code_files' is the same as
'compare_files' except it may accept keyword arguments
specifying the language of the source code. If such
arguments are supplied the function may perform a
semantic comparison of the code, and return 1 if the
codes are equivalent. The purpose of the function is
primarily to avoid rebuilding files differing only in
comments, blank lines, etc.

The function 'diff_files' compares two text files, the
original file and an upgraded file, and returns a
string describing the differences. The string may be in
any format, but it must be possible for the patch_file
function to modify the original file using this string,
and produce an upgraded file which compares equal using
the compare files function.

The function 'diff_strings' compares two strings,
returning a context diff as a string.

The function 'diff_lines' compares two sequences of
strings, returning a diff descriptor as follows (code,
left, right). Code may be 'c' for change or 's' for
common, left and right are line counts.

The function 'patch_file' takes a string and an
original file, and the name of an upgraded file, and
produces an upgraded file which has been modified by
the patch string in such a way that the compare_files
function will report the files are equal. In addition,
the function shall accept an empty patch string
signifying no differences between the original and
upgraded files.

This function is not required to operate correctly if
the orginal file has been modified since the patch
string was produced by diff_files.

The names of the argument files shall not denote the
same file.

The function shall return a result

The function 'posix_patch' is optional. If this
function is present, it will take a patch string
produced by the standard posix context diff and patch
the file. The purpose of this function is to facilitate
use of posix context diff as an Internet standard
method of transporting source code patches.

The specification of this function is defined as the
result of calling diff with the -C option on a system
compliant with the ISO Posix standard.

6.13.2.3. Posix Implementation
------------------------------

The following implementation utilises the os.popen and
os.system function to dispatch requests to the
operating system diff and patch commands, and is
intended to work on posix compliant systems. Well, it
works on my Linux box :-)

Start python section to interscript/utilities/diff.py[1]

     1: #line 75 "diff.ipk"
     2: import os
     3: import tempfile
     4: import string
     5: import re
     6: 
     7: def compare_files(o,m):
     8:   cmd = 'diff -q '+o+' '+m
     9:   f = os.popen(cmd,'r')
    10:   output = f.read()
    11:   result = f.close()
    12:   return len(output)==0
    13: 
    14: def compare_code_files(o,m,**kwds):
    15:   # slack implementation
    16:   return compare_files(o,m)
    17: 
    18: def diff_files(o,m,patch=None, context=10):
    19:   cmd = 'diff -C'+str(context)+' '+o+' '+m
    20:   f = os.popen(cmd,'r')
    21:   output = f.read()
    22:   result = f.close()
    23:   if patch:
    24:     f = open(patch,'w')
    25:     f.write(output)
    26:     f.close()
    27:   return output
    28: 
    29: def diff_strings(o,m,context=0):
    30:   foname = tempfile.mktemp()
    31:   fmname = tempfile.mktemp()
    32:   fo = open(foname,'w')
    33:   fm = open(fmname,'w')
    34:   fo.write(o)
    35:   fm.write(m)
    36:   fo.close()
    37:   fm.close()
    38:   result = diff_files(foname, fmname,context=context)
    39:   os.unlink(foname)
    40:   os.unlink(fmname)
    41:   return result
    42: 
    43: def diff_lines(o,m,context=0):
    44:   os = string.join(o,'\n')+'\n'
    45:   om = string.join(m,'\n')+'\n'
    46:   result = diff_strings(os,om,context=context)
    47:   del os
    48:   del om
    49:   data = string.split(result,'\n')[:-1]
    50:   cs = data[0][0]
    51:   cm = data[1][0]
    52:   sep = data[2]
    53:   lth = len(data)
    54:   sections = []
    55:   for i in range(2,lth):
    56:     if data[i] == sep:
    57:       sections.append([])
    58:     else:
    59:       sections[-1].append(data[i])
    60:   del data
    61:   del lth
    62:   del sep
    63: 
    64:   for i in range(len(sections)):
    65:     section = sections[i]
    66:     sections[i] = []
    67:     for j in range(len(section)):
    68:       line = section[j]
    69:       code = line[0]+line[1]
    70:       if code == cs*2 or code == cm*2:
    71:         k = 0
    72:         first = 0
    73:         count = 0
    74:         while line[k] not in '0123456789': k = k + 1
    75:         while line[k] in '0123456789':
    76:           first = first * 10 +ord(line[k])-ord('0')
    77:           k = k + 1
    78:         first = first - 1
    79:         sections[i].append([[first,0]])
    80:       else:
    81:         lineno = first + count
    82:         count = count + 1
    83:         sections[i][-1][0][1] = count
    84:         sections[i][-1].append(('%3d'%(lineno+1))+':'+line)
    85:   return sections
    86: 
    87: def patch_file(o,diff,m):
    88:   cmd = 'patch -s -c -o ' + m + ' ' + o + ' -'
    89:   print cmd
    90:   f = os.popen(cmd,'w')
    91:   f.write(diff)
    92:   result = f.close()
    93: 
    94: def posix_patch(o,diff,m):
    95:   patch_file(o,diff,m)
    96: 


End python section to interscript/utilities/diff.py[1]

6.13.2.4. Test routine 1
------------------------

First, create some files to compare!

Start data section to interscript/tests/diff1.dat[1]

     1: A file with several lines of text.
     2: A line to be deleted.
     3: To be used by the diff module test.
     4: This line will be changed.
     5: The diff module is used to compare and patch test files.
     6: 


End data section to interscript/tests/diff1.dat[1]

Start data section to interscript/tests/diff2.dat[1]

     1: A file with several lines of text.
     2: To be used by the diff module test.
     3: This line is changed.
     4: A NEW LINE INSERTED.
     5: The diff module is used to compare and patch test files.
     6: 


End data section to interscript/tests/diff2.dat[1]

Start python section to interscript/tests/test_diff.py[1]

     1: #line 191 "diff.ipk"
     2: import sys
     3: import string
     4: sys.path = [''] + sys.path
     5: import interscript.utilities.diff
     6: diff = interscript.utilities.diff
     7: cmp = diff.compare_files('diff1.dat', 'diff1.dat')
     8: assert cmp
     9: cmp = diff.compare_files('diff1.dat', 'diff2.dat')
    10: assert not cmp
    11: patch = diff.diff_files('diff1.dat','diff2.dat')
    12: print 'patch below'
    13: print patch
    14: print
    15: diff.patch_file('diff1.dat',patch,'diff3.dat')
    16: cmp = diff.compare_files('diff2.dat','diff3.dat')
    17: assert cmp
    18: f = open('diff1.dat')
    19: d1 = f.readlines()
    20: f.close()
    21: for i in range(len(d1)): d1[i] = string.rstrip(d1[i])
    22: f = open('diff2.dat')
    23: d2 = f.readlines()
    24: for i in range(len(d2)): d2[i] = string.rstrip(d2[i])
    25: f.close()
    26: 
    27: d = diff.diff_lines(d1,d2)
    28: for section in d:
    29:   print '*****'
    30:   for part in section:
    31:     print part
    32: 


End python section to interscript/tests/test_diff.py[1]

Now run it.

Start output section of /usr/local/bin/python interscript/tests/test_diff.py

     1: Interscript Package: version 1.0a7 build 1302
     2: thread available
     3: Interscript version 1.0a7 build 1302
     4: Built by root on ruby at Thu Nov 12, 1998 at 12:57 PM (UTC)
     5: Generated by 1.0a7 buildno 1300 host ruby
     6: at Thu Nov 12, 1998 at 12:57 PM (UTC)
     7: patch below
     8: *** diff1.dat	Tue Aug 11 04:10:58 1998
     9: --- diff2.dat	Tue Aug 11 04:10:58 1998
    10: ***************
    11: *** 1,6 ****
    12:   A file with several lines of text.
    13: - A line to be deleted.
    14:   To be used by the diff module test.
    15: ! This line will be changed.
    16:   The diff module is used to compare and patch test files.
    17: 
    18: --- 1,6 ----
    19:   A file with several lines of text.
    20:   To be used by the diff module test.
    21: ! This line is changed.
    22: ! A NEW LINE INSERTED.
    23:   The diff module is used to compare and patch test files.
    24: 
    25: 
    26: 
    27: patch -s -c -o diff3.dat diff1.dat -
    28: *****
    29: [[1, 1], '  2:- A line to be deleted.']
    30: [[0, 0]]
    31: *****
    32: [[3, 1], '  4:! This line will be changed.']
    33: [[2, 2], '  3:! This line is changed.', '  4:! A NEW LINE INSERTED.']

End output section of /usr/local/bin/python interscript/tests/test_diff.py

6.13.2.5. Test Routine 2
------------------------

Use "test_python" function.

6.13.2.5.1. Test 6: Python diff test
------------------------------------

On-the-fly python test script follows.   230: print 'Hello from python diff function test.'
   231: print 'Hello again from python diff function test.'
   232: print 'Hello third from python diff function test.'
   233: print 'Hello fourth from python diff function test.'
Actual output follows.

Start output section of /usr/local/bin/python /usr/tmp/@13003.1563_test.py

     1: Hello from python diff function test.
     2: Hello again from python diff function test.
     3: Hello third from python diff function test.
     4: Hello fourth from python diff function test.

End output section of /usr/local/bin/python /usr/tmp/@13003.1563_test.py

On-the-fly expected output follows.

Start expected section of diff.ipk

   235: Hello from python diff function test.
   236: Hello error from python diff function test.
   237: Hello fourth from python diff function test.
   238: Hello fifth from python diff function test.

End expected section of diff.ipk

Differential follows.
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
| Actual                                                                                              | Expected                                          |
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
|   2:! Hello again from python diff function test.
  3:! Hello third from python diff function test. |   2:! Hello error from python diff function test. |
|                                                                                                     |   4:+ Hello fifth from python diff function test. |
+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
Hmmm. There should be a difference table here.

6.14. Application and tool directory
------------------------------------

This is where the interscript mainline and any other
shell commands go.

Start python section to interscript/bin/__init__.py[1]

     1: #line 439 "interscript/src/iscr.pak"
     2: # dummy interscript.bin
     3: 


End python section to interscript/bin/__init__.py[1]

6.14.1. Stand alone unix/nt mainline
------------------------------------

Command line stub.

Start python section to interscript/bin/iscr.py[1]

     1: #!/usr/bin/env python
     2: import sys
     3: args = sys.argv[1:]
     4: if len(args)>0 and args[0]=='--test':
     5:   print 'Interscript test mode, loading interscript from current directory'
     6:   if sys.path[0]!='':
     7:     sys.path = ['']+ sys.path
     8:   args = args[1:]
     9: import interscript
    10: interscript.run_from_options(args)
    11: 


End python section to interscript/bin/iscr.py[1]

6.14.2. Windows launcher
------------------------

A batch file to launch interscript. You will need to
edit the file!

Start python section to interscript/bin/iscr.bat[1]

     1: #line 461 "interscript/src/iscr.pak"
     2: python iscr.py %1 %2 %3 %4 %5 %6 %7 %8 %9
     3: 


End python section to interscript/bin/iscr.bat[1]

6.15. Test package
------------------


Start python section to interscript/tests/__init__.py[1]

     1: #line 465 "interscript/src/iscr.pak"
     2: # dummy interscript/test
     3: 


End python section to interscript/tests/__init__.py[1]

7. Appendicies
--------------


7.1. File List
--------------

interscript/doc/iscr.txt
interscript/doc/iscrp.html
interscript/doc/iscrs.html
interscript/doc/iscr_indexview.html
interscript/doc/iscr_top.html
interscript/doc/iscr_indexnav.html
interscript/doc/iscr.tex
interscript/doc/iscr_0001.html
interscript/doc/iscr_0002.html
interscript/doc/iscr_0003.html
interscript/doc/iscr_0004.html
interscript/doc/iscr_0005.html
interscript/doc/iscr_0006.html
interscript/doc/iscr_0007.html
interscript/doc/iscr_0008.html
interscript/doc/iscr_0009.html
interscript/doc/iscr_0010.html
interscript/doc/iscr_0011.html
interscript/doc/iscr_0012.html
interscript/doc/iscr_0013.html
interscript/doc/iscr_0014.html
interscript/doc/iscr_0015.html
interscript/doc/iscr_0016.html
interscript/doc/iscr_0017.html
interscript/doc/iscr_0018.html
interscript/doc/iscr_0019.html
interscript/doc/iscr_0020.html
interscript/doc/iscr_0021.html
interscript/doc/iscr_0022.html
interscript/doc/iscr_0023.html
interscript/doc/iscr_0024.html
interscript/doc/iscr_0025.html
interscript/doc/iscr_0026.html
interscript/doc/iscr_0027.html
interscript/doc/iscr_0028.html
interscript/doc/iscr_0029.html
interscript/doc/iscr_0030.html
interscript/doc/iscr_0031.html
interscript/doc/iscr_0032.html
interscript/doc/iscr_0033.html
interscript/doc/iscr_0034.html
interscript/doc/iscr_0035.html
interscript/doc/iscr_0036.html
interscript/doc/iscr_0037.html
interscript/doc/iscr_0038.html
interscript/doc/iscr_0039.html
interscript/doc/iscr_0040.html
interscript/doc/iscr_0041.html
interscript/doc/iscr_0042.html
interscript/doc/iscr_0043.html
interscript/doc/iscr_0044.html
interscript/doc/iscr_0045.html
interscript/doc/iscr_0046.html
interscript/doc/iscr_0047.html
interscript/doc/iscr_0048.html
interscript/doc/iscr_0049.html
interscript/doc/iscr_0050.html
interscript/doc/iscr_0051.html
interscript/doc/iscr_0052.html
interscript/doc/iscr_0053.html
interscript/doc/iscr_0054.html
interscript/doc/iscr_0055.html
interscript/doc/iscr_0056.html
interscript/doc/iscr_0057.html
interscript/doc/iscr_0058.html
interscript/doc/iscr_0059.html
interscript/doc/iscr_0060.html
interscript/doc/iscr_0061.html
interscript/doc/iscr_0062.html
interscript/doc/iscr_0063.html
interscript/doc/iscr_0064.html
interscript/doc/iscr_0065.html
interscript/doc/iscr_0066.html
interscript/doc/iscr_0067.html
interscript/doc/iscr_0068.html
interscript/doc/iscr_0069.html
interscript/doc/iscr_0070.html
interscript/doc/iscr_0071.html
interscript/doc/iscr_0072.html
interscript/doc/iscr_0073.html
interscript/doc/iscr_0074.html
interscript/doc/iscr_0075.html
mymodule.py
interscript/doc/iscr_0076.html
interscript/doc/iscr_0077.html
interscript/doc/iscr_0078.html
interscript/doc/iscr_0079.html
interscript/doc/iscr_0080.html
interscript/doc/iscr_0081.html
interscript/doc/iscr_0082.html
interscript/doc/iscr_0083.html
interscript/doc/iscr_0084.html
interscript/doc/iscr_0085.html
interscript/doc/iscr_0086.html
interscript/doc/iscr_0087.html
interscript/doc/iscr_0088.html
interscript/doc/iscr_0089.html
interscript/doc/iscr_0090.html
interscript/doc/iscr_0091.html
interscript/doc/iscr_0092.html
interscript/doc/iscr_0093.html
interscript/doc/iscr_0094.html
interscript/doc/iscr_0095.html
interscript/doc/iscr_0096.html
interscript/doc/iscr_0097.html
interscript/doc/iscr_0098.html
interscript/doc/iscr_0099.html
interscript/doc/iscr_0100.html
interscript/doc/iscr_0101.html
interscript/doc/iscr_0102.html
interscript/doc/iscr_0103.html
interscript/doc/iscr_0104.html
interscript/doc/iscr_0105.html
interscript/doc/iscr_0106.html
interscript/doc/iscr_0107.html
interscript/doc/iscr_0108.html
interscript/doc/iscr_0109.html
interscript/doc/iscr_0110.html
interscript/doc/iscr_0111.html
interscript/doc/iscr_0112.html
interscript/doc/iscr_0113.html
interscript/doc/iscr_0114.html
interscript/doc/iscr_0115.html
interscript/doc/iscr_0116.html
interscript/doc/iscr_0117.html
interscript/doc/iscr_0118.html
interscript/doc/iscr_0119.html
interscript/doc/iscr_0120.html
interscript/doc/iscr_0121.html
interscript/doc/iscr_0122.html
interscript.README
interscript.pth
interscript/__init__.py
interscript/doc/iscr_0123.html
interscript/doc/iscr_0124.html
interscript/doc/iscr_0125.html
interscript/doc/iscr_0126.html
interscript/core/__init__.py
interscript/doc/iscr_0127.html
interscript/core/sets.py
interscript/doc/iscr_0128.html
interscript/tests/test_sets.py
interscript/doc/iscr_0129.html
interscript/doc/iscr_0130.html
interscript/core/stacks.py
interscript/doc/iscr_0131.html
interscript/doc/iscr_0132.html
interscript/doc/iscr_0133.html
interscript/core/protocols.py
interscript/doc/iscr_0134.html
interscript/tests/test_protocol.py
interscript/doc/iscr_0135.html
interscript/drivers/__init__.py
interscript/doc/iscr_0136.html
interscript/doc/iscr_0137.html
interscript/drivers/sources/__init__.py
interscript/doc/iscr_0138.html
interscript/drivers/sources/base.py
interscript/doc/iscr_0139.html
interscript/doc/iscr_0140.html
interscript/doc/iscr_0141.html
interscript/drivers/sources/disk.py
interscript/doc/iscr_0142.html
interscript/drivers/sources/url.py
interscript/doc/iscr_0143.html
interscript/drivers/sources/ftp.py
interscript/doc/iscr_0144.html
interscript/drivers/sources/http.py
interscript/doc/iscr_0145.html
interscript/drivers/sources/stdin.py
interscript/doc/iscr_0146.html
interscript/drivers/sinks/__init__.py
interscript/drivers/sinks/util.py
interscript/doc/iscr_0147.html
interscript/drivers/sinks/base.py
interscript/doc/iscr_0148.html
interscript/drivers/sinks/null.py
interscript/doc/iscr_0149.html
interscript/drivers/sinks/disk.py
interscript/doc/iscr_0150.html
interscript/drivers/sinks/bufdisk.py
interscript/doc/iscr_0151.html
interscript/drivers/sinks/stdout.py
interscript/doc/iscr_0152.html
interscript/drivers/storage/__init__.py
interscript/doc/iscr_0153.html
interscript/drivers/storage/memory.py
interscript/doc/iscr_0154.html
interscript/drivers/storage/disk.py
interscript/doc/iscr_0155.html
interscript/weavers/__init__.py
interscript/doc/iscr_0156.html
interscript/weavers/base.py
interscript/doc/iscr_0157.html
interscript/weavers/raw.py
interscript/doc/iscr_0158.html
interscript/weavers/multiplexor.py
interscript/doc/iscr_0159.html
interscript/weavers/text.py
interscript/doc/iscr_0160.html
interscript/doc/iscr_0161.html
interscript/doc/iscr_0162.html
interscript/doc/iscr_0163.html
interscript/doc/iscr_0164.html
interscript/doc/iscr_0165.html
interscript/doc/iscr_0166.html
interscript/doc/iscr_0167.html
interscript/doc/iscr_0168.html
interscript/doc/iscr_0169.html
interscript/doc/iscr_0170.html
interscript/weavers/html.py
interscript/doc/iscr_0171.html
interscript/doc/iscr_0172.html
interscript/doc/iscr_0173.html
interscript/doc/iscr_0174.html
interscript/doc/iscr_0175.html
interscript/doc/iscr_0176.html
interscript/doc/iscr_0177.html
interscript/doc/iscr_0178.html
interscript/doc/iscr_0179.html
interscript/doc/iscr_0180.html
interscript/doc/iscr_0181.html
interscript/doc/iscr_0182.html
interscript/doc/iscr_0183.html
interscript/doc/iscr_0184.html
interscript/doc/iscr_0185.html
interscript/doc/iscr_0186.html
interscript/doc/iscr_0187.html
interscript/doc/iscr_0188.html
interscript/doc/iscr_0189.html
interscript/doc/iscr_0190.html
interscript/doc/iscr_0191.html
interscript/doc/iscr_0192.html
interscript/doc/interscript.css
interscript/doc/iscr_0193.html
interscript/doc/iscr_0194.html
interscript/doc/iscr_0195.html
interscript/weavers/web.py
interscript/doc/iscr_0196.html
interscript/doc/iscr_0197.html
interscript/doc/iscr_0198.html
interscript/doc/iscr_0199.html
interscript/doc/iscr_0200.html
interscript/doc/iscr_0201.html
interscript/doc/iscr_0202.html
interscript/tests/test_stacking_weaver.pak
interscript/doc/iscr_0203.html
interscript/weavers/latex.py
interscript/doc/iscr_0204.html
interscript/doc/iscr_0205.html
interscript/doc/iscr_0206.html
interscript/doc/iscr_0207.html
interscript/doc/iscr_0208.html
interscript/doc/iscr_0209.html
interscript/doc/iscr_0210.html
interscript/doc/iscr_0211.html
interscript/doc/iscr_0212.html
interscript/doc/iscr_0213.html
interscript/doc/iscr_0214.html
interscript/doc/iscr_0215.html
interscript/doc/iscr_0216.html
interscript/doc/iscr_0217.html
interscript/doc/iscr_0218.html
interscript/doc/iscr_0219.html
interscript/doc/iscr_0220.html
interscript/doc/iscr_0221.html
interscript/doc/iscr_0222.html
interscript/doc/iscr_0223.html
interscript/doc/iscr_0224.html
interscript/doc/iscr_0225.html
interscript/doc/iscr_0226.html
interscript/doc/iscr_0227.html
interscript/doc/iscr_0228.html
interscript/doc/iscr_0229.html
interscript/weavers/auto.py
interscript/doc/iscr_0230.html
interscript/doc/iscr_0231.html
interscript/weavers/filter.py
interscript/doc/iscr_0232.html
interscript/doc/iscr_0233.html
interscript/tanglers/__init__.py
interscript/tanglers/base.py
interscript/doc/iscr_0234.html
interscript/doc/iscr_0235.html
interscript/tanglers/null.py
interscript/doc/iscr_0236.html
interscript/tanglers/doc.py
interscript/doc/iscr_0237.html
interscript/tanglers/data.py
interscript/doc/iscr_0238.html
interscript/tanglers/c.py
interscript/doc/iscr_0239.html
interscript/doc/iscr_0240.html
interscript/doc/iscr_0241.html
interscript/tanglers/cpp.py
interscript/doc/iscr_0242.html
interscript/doc/iscr_0243.html
interscript/doc/iscr_0244.html
interscript/tanglers/java.py
interscript/doc/iscr_0245.html
interscript/tanglers/tcl.py
interscript/doc/iscr_0246.html
interscript/tanglers/python.py
interscript/doc/iscr_0247.html
interscript/doc/iscr_0248.html
interscript/tanglers/perl.py
interscript/doc/iscr_0249.html
interscript/tokenisers/__init__.py
interscript/doc/iscr_0250.html
interscript/tokenisers/python.py
interscript/doc/iscr_0251.html
interscript/parsers/__init__.py
interscript/doc/iscr_0252.html
interscript/parsers/html.py
interscript/doc/iscr_0253.html
interscript/doc/iscr_0254.html
interscript/doc/iscr_0255.html
interscript/parsers/lalr1.py
interscript/doc/iscr_0256.html
interscript/doc/iscr_0257.html
interscript/doc/iscr_0258.html
interscript/doc/iscr_0259.html
interscript/doc/iscr_0260.html
interscript/doc/iscr_0261.html
interscript/doc/iscr_0262.html
interscript/doc/iscr_0263.html
interscript/doc/iscr_0264.html
interscript/doc/iscr_0265.html
interscript/doc/iscr_0266.html
interscript/doc/iscr_0267.html
interscript/doc/iscr_0268.html
interscript/doc/iscr_0269.html
interscript/doc/iscr_0270.html
interscript/doc/iscr_0271.html
interscript/doc/iscr_0272.html
interscript/doc/iscr_0273.html
interscript/doc/iscr_0274.html
interscript/doc/iscr_0275.html
interscript/doc/iscr_0276.html
interscript/doc/iscr_0277.html
interscript/doc/iscr_0278.html
interscript/doc/iscr_0279.html
interscript/doc/iscr_0280.html
interscript/doc/iscr_0281.html
interscript/tests/tgram.py
interscript/doc/iscr_0282.html
interscript/doc/iscr_0283.html
interscript/doc/iscr_0284.html
interscript/frames/install.py
interscript/doc/iscr_0285.html
interscript/frames/user.py
interscript/doc/iscr_0286.html
interscript/frames/project.py
interscript/frames/__init__.py
interscript/doc/iscr_0287.html
interscript/frames/platform.py
interscript/doc/iscr_0288.html
interscript/frames/site.py
interscript/doc/iscr_0289.html
interscript/frames/processf.py
interscript/doc/iscr_0290.html
interscript/doc/iscr_0291.html
interscript/frames/masterf.py
interscript/doc/iscr_0292.html
interscript/doc/iscr_0293.html
interscript/frames/passf.py
interscript/doc/iscr_0294.html
interscript/doc/iscr_0295.html
interscript/frames/inputf.py
interscript/doc/iscr_0296.html
interscript/doc/iscr_0297.html
interscript/doc/iscr_0298.html
interscript/doc/iscr_0299.html
interscript/doc/iscr_0300.html
interscript/doc/iscr_0301.html
interscript/doc/iscr_0302.html
interscript/doc/iscr_0303.html
interscript/tests/call_test.tpk
interscript/doc/iscr_0304.html
interscript/doc/iscr_0305.html
interscript/doc/iscr_0306.html
interscript/tests/test_beginend.pak
interscript/doc/iscr_0307.html
interscript/doc/iscr_0308.html
interscript/doc/iscr_0309.html
interscript/doc/iscr_0310.html
interscript/doc/iscr_0311.html
interscript/doc/iscr_0312.html
interscript/doc/iscr_0313.html
interscript/doc/iscr_0314.html
interscript/doc/iscr_0315.html
interscript/doc/iscr_0316.html
interscript/doc/iscr_0317.html
interscript/doc/iscr_0318.html
interscript/doc/iscr_0319.html
interscript/doc/iscr_0320.html
interscript/doc/iscr_0321.html
interscript/doc/iscr_0322.html
interscript/doc/iscr_0323.html
interscript/doc/iscr_0324.html
interscript/doc/iscr_0325.html
interscript/doc/iscr_0326.html
interscript/doc/iscr_0327.html
interscript/doc/iscr_0328.html
interscript/doc/iscr_0329.html
interscript/doc/iscr_0330.html
interscript/doc/iscr_0331.html
interscript/doc/iscr_0332.html
interscript/doc/iscr_0333.html
interscript/doc/iscr_0334.html
interscript/doc/iscr_0335.html
interscript/doc/iscr_0336.html
interscript/doc/iscr_0337.html
interscript/doc/iscr_0338.html
interscript/doc/iscr_0339.html
interscript/doc/iscr_0340.html
interscript/doc/iscr_0341.html
interscript/doc/iscr_0342.html
interscript/doc/iscr_0343.html
interscript/doc/iscr_0344.html
interscript/getoptions.py
interscript/doc/iscr_0345.html
interscript/doc/iscr_0346.html
interscript/getframes.py
interscript/doc/iscr_0347.html
interscript/doc/iscr_0348.html
interscript/doc/iscr_0349.html
interscript/utilities/__init__.py
interscript/tests/test_compilers.py
interscript/tests/example.c
interscript/compilers/c.py
interscript/compilers/cpp.py
interscript/compilers/cconfig.py
interscript/compilers/__init__.py
interscript/doc/iscr_0350.html
interscript/doc/iscr_0351.html
interscript/doc/iscr_0352.html
interscript/doc/iscr_0353.html
interscript/doc/iscr_0354.html
interscript/doc/iscr_0355.html
interscript/utilities/diff.py
interscript/doc/iscr_0356.html
interscript/tests/diff1.dat
interscript/tests/diff2.dat
interscript/tests/test_diff.py
interscript/doc/iscr_0357.html
interscript/doc/iscr_0358.html
interscript/doc/iscr_0359.html
interscript/bin/__init__.py
interscript/doc/iscr_0360.html
interscript/bin/iscr.py
interscript/doc/iscr_0361.html
interscript/bin/iscr.bat
interscript/doc/iscr_0362.html
interscript/tests/__init__.py
interscript/doc/iscr_0363.html
interscript/doc/iscr_0364.html
interscript/doc/iscr_0365.html
interscript/doc/iscr_0366.html
interscript/doc/iscr_0367.html
interscript/doc/iscr_0368.html
interscript/doc/iscr_0369.html
interscript/doc/iscr_0370.html
interscript/doc/iscr_0371.html
interscript/doc/iscr_0372.html
interscript/doc/iscr_0373.html
interscript/doc/iscr_0374.html
interscript/doc/iscr_0375.html
interscript/doc/iscr_0376.html
interscript/doc/iscr_0377.html
interscript/doc/iscr_0378.html
interscript/doc/iscr_0379.html
interscript/doc/iscr_0380.html
interscript/doc/iscr_0381.html
interscript/doc/iscr_0382.html
interscript/doc/iscr_0383.html
interscript/doc/iscr_0384.html
interscript/doc/iscr_0385.html
interscript/doc/iscr_0386.html
interscript/doc/iscr_0387.html
interscript/doc/iscr_0388.html
interscript/doc/iscr_0389.html
interscript/tests/example.pak
interscript/doc/iscr_0390.html
interscript/doc/iscr_0391.html
interscript/doc/iscr_0392.html
interscript/doc/iscr_0393.html
interscript/doc/iscr_0394.html
interscript/doc/iscr_0395.html
interscript/doc/iscr_0396.html
interscript/doc/iscr_funcref.html
interscript/doc/iscr_classref.html
interscript/doc/iscr_identref.html
interscript/doc/iscr_toc.html
interscript/doc/iscr_testref.html
interscript/doc/iscr_sectionref.html
/mnt/user1/uestl/py/iscr/interscript/doc/iscr_filestatus.html

7.2. Source List
----------------

interscript/src/iscr.pak
introduction.ipk
requirements.ipk
design.ipk
tutorial.pak
iscrqa.ipk
sets.ipk
stacks.ipk
protocols.ipk
drivers.ipk
source_drivers.ipk
sink_drivers.ipk
storage_drivers.ipk
weavers.ipk
text_weaver.ipk
html_weaver.ipk
web_weaver.ipk
latex_weaver.ipk
weaver_filters.ipk
tanglers.ipk
python_tangler.ipk
perl_tangler.ipk
tokenizers.ipk
iscrtkpy.ipk
parsers.ipk
html_parser.ipk
lalr1_parser.ipk
frames.ipk
platform_frame.ipk
site_frame.ipk
process_frame.ipk
master_frame.ipk
pass_frame.ipk
input_frame.ipk
options.ipk
interscript_options.ipk
compilers.ipk
diff.ipk
bugs.ipk
installation.ipk

7.3. Include List
-----------------

    interscript: interscript/src/iscr.pak
       interscript: introduction.ipk
       interscript: requirements.ipk
       interscript: design.ipk
       interscript: tutorial.pak
          interscript: iscrqa.ipk
       code: iscr.pak
       interscript: sets.ipk
       interscript: stacks.ipk
       interscript: protocols.ipk
       interscript: drivers.ipk
          interscript: source_drivers.ipk
          interscript: sink_drivers.ipk
          interscript: storage_drivers.ipk
       interscript: weavers.ipk
          interscript: text_weaver.ipk
          interscript: html_weaver.ipk
          interscript: web_weaver.ipk
          interscript: latex_weaver.ipk
       interscript: weaver_filters.ipk
       interscript: tanglers.ipk
          interscript: python_tangler.ipk
          interscript: perl_tangler.ipk
       interscript: tokenizers.ipk
          interscript: iscrtkpy.ipk
       interscript: parsers.ipk
          interscript: html_parser.ipk
          interscript: lalr1_parser.ipk
       interscript: frames.ipk
          interscript: platform_frame.ipk
          interscript: site_frame.ipk
          interscript: process_frame.ipk
          interscript: master_frame.ipk
          interscript: pass_frame.ipk
          interscript: input_frame.ipk
       interscript: options.ipk
       interscript: interscript_options.ipk
       interscript: compilers.ipk
       interscript: diff.ipk
       interscript: bugs.ipk
       interscript: installation.ipk

7.4. Bugs (etc) for Version 1a7
-------------------------------

As a new development, Interscript has a lot of bugs,
design flaws, and omissions. Please send comments to

  skaller@maxtal.com.au

7.4.1. These debugging notes
----------------------------

These notes reflect current status. They're not date
stamped, and there's no indication if a bug has been
fixed (the comment, hopefully, just gets removed). This
is hopeless for bug tracking.

What we need is more like a database with time and
versions stamped comments, organised by topic so fixes
can be noted.

These notes are written separately from the program and
documentation. For a list generated by cummulating
comments made by me throughout the source, see
iscrbuglist.html for this information.

7.4.2. Control System
---------------------

Keyboard interrupts are still dubious. This has been
improved over the 1a1 release.

The frames described in the doco are now implemented,
more or less. Still needs some work.

No threads yet, but the frame implementation is good
preparation for supporting them.

The SGML input filter is too primitive.

There's no error handling system. There's error
handling, but i wouldn't call it a 'system' :-)

The markup filters are in the wrong place.

7.4.3. Tangler Bugs
-------------------

String tanglers add an extra space on the last line.
It's not easy to stop this without buffering. Also, the
Python string tangler is all wrong. Python doesn't
concatenate a sequence of string literals like C does.

The python tangler currently tokenises the generated
code file (as it is being generated). The tokeniser
used is the standard one, tokeniser.py. Unfortunately,
it has the wrong control interface, and we have to map
it to one the tangler can drive. This requires
coroutines, but Python doesn't have them, so we are
forced to emulate them using threads.

The python tokeniser is now fully functional.

The control system has been almost completely redone.
It still uses the lists of regular expressions paired
with functions to execute, but the functions are now
methods bound to frames, usually the current input
frame.

The predefined methods are replaced for every input
frame constructed. This means users cannot override
them properly, since they're not inherited. Really, we
should use the Guido hack: store functions in the user
dictionary, and turn them into bound methods at the
point of lookup.

Correct implementation of the scoping mechanism is
severely hampered by Python's lack of suitable scoping.

7.4.4. Weaver Bugs
------------------

The table of identifiers gets wide under Latex, the
items don't wrap.

7.4.5. Misc Bugs
----------------

Table of contents and file lists are now stored in
memory and preserved between passes, so it is necessary
to run two passes to get them to print out correctly.
Because the storage is weaver independent, being
generated by the multiplexor, there are no references
in the table of contents. This needs to be fixed by
allowing each weaver private storage persistent across
passes. At this time this is not possible because there
is no way to uniquely identifier a weaver instance (a
new object is constructed each pass).

Heading numbers should have user controlled formats.

7.4.6. Design Issues
--------------------

The interactions in this system are VERY complex. For
example, consider an Interscript source which generates
a Interscript source, or a shell archive. In this case
multiple tanglers may be needed to generate a single
file.

It should be possible to perform 'categorical'
constructions on tanglers: chaining tanglers, writing
to multiple outputs, writing to selected outputs, etc.

7.4.7. Omissions
----------------


7.4.7.1. Building
-----------------

There's no build environment.

7.4.7.2. Scripting
------------------

The Tcl interface doesn't map variables.

There's no Perl interface.

7.4.7.3. Parsing
----------------

No support for Internationalisation.

to invoke Interscript this way. More generally, it
would be nice if Interscript could 'automatically
recognize' various input files.

It's tricky to change the warning character (default
@). Perhaps the first character on the first line
should be taken as the warning character (unless the
first line starts with #!, in which case the second
line is used. Curently, Interscript can't work unless
some executable code is at the start of the primary
source file (at least to set up the weaver).

Perl users will hate having @ as the escape character
because it is the first character of all array
identifiers. (Is that right?)

The parser is not modularised properly.

There's no XML filter.

There's no LaTeX filter (that's hard!)

7.4.7.4. Weaving
----------------

There's no weaver for plain TeX, Postscript, or Word,
nroff, Lout.

The support for hypertext is primitive.

The identifier index uses HTML hyperlinks. What should
be displayed as the anchor text: the input source
reference or the output code reference? Or both? The
input source is where editing would be done, but the
output tells where the identifier ended up, which is
probably more useful.

No pictures or drawings.

7.4.7.5. Tangling
-----------------

There's no support for 'chunking' (noweb) or 'macros'
(FunnelWeb): i.e. the ability to define code fragments
to be included inside others, building the code
heirarchical. This is an absolutely fundamental
requirement, and it is not omitted by accident. There
is some primitive macro support: you can tangle a file
and read it in on the next pass, or you can tangle a
memory sink and then use it as a source. It's easy to
write commands that simplify use of these mechanisms,
and I had some, but they've been removed.

The reason there's no chunking is that I'm not happy
with any of the mechanisms yet, nor certain of the
requirements. FunnelWeb, for example, tangles code
after scanning the whole document, and so can include a
macro A in a macro B, even if B is defined _before_ A.
This isn't possible with a simple one pass dynamic
system. On the other hand, it seems obvious that
tanglers that generate chunks should perform language
specific processing, such as formatting comments and
introducing the #line directives, while the generated
output should be included raw (since it has already
been processed).

Unfortunately, this is a gross oversimplification of
the requirements of a meta-programmable system. For
example, a lot of programming would benefit from chunks
with arguments, which FunnelWeb supports (and which is
why they're called macros). As another example,
FunnelWeb tracks usage of chunks, and requires every
chunk be used exactly once, unless otherwise declared
in chunk definition. Yet another example: templates, or
skeleton. This is conceptually the same as a macro with
arguments, but that is a lousy way to actually have to
write the call. The correct method is to use
co-routines (interleaving), diversions, continuations
(etc).

So we see that generating code by expanding with
substitutions, even the simplest kind of parameterless
chunk substitutions, seems to be the thin end of the
wedge leading to the development of a complete
programming language --- and Interscript is already
equipped with at least one of those.

Here's a final 'killer' example. One of the languages
we are sure to want to tangle is ... Interscript.

If a tangler supports source cross references such as
the #line directive, then there will always be such a
directive as the first line of a code file.
Unfortunately, some languages and systems have special
requirements on what's in the first line. Cases I know
of include:

 1.  the Unix #! convention, which could be used by
    Python and Tcl programs,

 2.  vi: set tabwidth type lines in text files
    (including programs),

 3.  and quote a lot of typesetting other systems
    (including compilers) which require global flags to
    be set by pragmas and the like, right on the first
    line.

The solution is to have a special command to generate
the first lines of a code file, which doesn't trigger
generating the #line directive.

There's no tangler for many languages: Modula, Pascal,
Cobol, Scheme, etc etc.

There's no support for bolding keywords.

There's no Tcl string tangler. The Perl string tangler
is bogus.

7.4.8. New in 1a5
-----------------

By far the most important improvement is the
implementing of the frame architecture, and the
repackaging of interscript as a python package.

The bulk of interscript is now built as a package in a
tree structured directory. Using the frames
architecture, the command line interface has been
reduced to few lines, which will allow interscript to
be embedded in other front end programs.

The web (stacking html) weaver now generates a table of
contents which uses javascript to produce a dynamic
tree under Internet Explorer.

There are also two home pages for a document, and two
tables of contents. The alternate home page creates two
framnes, and loads an alternate table of contents into
the first frame, which targets the second frame.

The web weaver now generates <LINK> tags similar
exhibiting the relationship between the documents.

Tcl script has been discarded completely.

The command line launcher now accepts an option
'--test' which pushes the current directory onto the
system path to force package loading to look in the
current directory. This is the place interscript and
user packages will often be built from. The purpose of
this hack is to allow documents containing both code
and tests for the code, to allow the test code to
actually find the newly created modules before they're
installed in standard places. By default, sys.path is
not modified, so unix '#!/usr/local/bin/python'
launched scripts will not search the current directory.
This hopefully improves separation between reliable
installed software and that under development.

If the first line of a Python output starts with '#!',
line numbers are not generated until the next line.

The autoweaver for html and web now makes sinks which
use carriage return linefeed pairs as end of line
characters. This is to fix bugs in stupid Microsoft
software such as Notepad and Wordpad, which refuses to
recognize them as end of line. This is important for
viewing HTML sources from Internet Explorer.

Sink drivers have been rearchitected to support
standard python output file protocols so they can be
used, for example, as the sys.stdout device. The Python
line handling protocol sucks, stupidly returning the
end of line characters at the end of lines on input,
and requiring them to be there already on output.
Interscript's own protocol returns lines without line
terminators, adds them on output, and recognizes
linefeed in strings (only) as it's internal end of line
terminator.

Option processing now uses an interscript module
(getopt wasn't up to it).

There are now four options, '--weaver_prefix',
'--weaver_directory', '--tangler_prefix',
'--tangler_directory', which allow output to be
directed somewhere other than the current directory
from the command line. See next item.

Interscript now fixes file names to use a subset of the
Unix filename convention. All interscript filenames
must use a sensible set of characters for component
names, and separate them with '/'. Filename must be
relative.

A filename given for a tangler or weaver constructed by
the builtin tangler functions or autoweave is prefixed
by the directory option before being split into
components. Then the prefix is prepended to the first
component.

Directories are built automatically (and somewhat
prematurely).

The effect is that the prefix_* should usually denote
an existing directory in native operating system
format, with a directory separator at the end, and the
directory_* should be a unix pathname with a '/' at the
end.

One of the benefits of this arrangement is that Windows
clients can specify output onto another drive, for
example using the option:

  --weaver_prefix="d:\project\document files\"

Note the quotes!

7.4.8.1. Separate compilation
-----------------------------

Interscript is now constructed in such a way that it is
possible to tangle subdocuments to extract individual
modules. For example, if a weaver is changed, it is
possible to update that component by processing the ipk
(included package) file without rebuilding all of
interscript. If weaving is enabled, a separate document
for that component will be generated, it will not be
related to the pages for the same component generated
as part of a full build. How would we get the right
heading numbers? Or predict the filenames to be
generated?

This needs to be fixed. The problem is one of design
rather than implementation: the web weaver might be
adapted to use filenames derived from the current input
source, locking the structure of the generated
documents to the input source structure. At present,
these are independent except that breaks occur on
headings.

Probably, 'include_file' is not enough to support what
is required.

7.4.9. Under development and planned
------------------------------------

Under development is a syntax driven LALR translator
which I hope to use to replace the 'micky mouse'
control system.

The patch/diff module is working. I still haven't
figured out how to use it to implement version control.

A simple Tkinter front end is under development for
launching interscript runs, as an alternative to the
command line launcher 'iscr.py'.

The current prefix and directory options for tanglers
and weavers is effective but too primitive. The
interscript source contains a currently unused
installation frame which illustrates the mechanism
intended to replace this. Interscript filenames will be
extended to full URI syntax, with the protocol 'iscr'
being used to designate vectoring of the output to a
directory which is the value of a dictionary keyed by
the 'machine' component of the URI, together with a
trust level assigned by the client.

This mechanism allows the author to identify an
arbitrary set of logical file systems into which output
is put, and the client to map these logical file
systems onto actual output points.

For example, the client will output stuff to a project
development structure during testing. When happy, the
output will be remapped to the installation structure.
For example, html documentation could be automatically
posted to the live Internet web site.

The main impediment to actually implementing this, is
that to use the mechanism, the user will have to set up
the installation frame to use interscript. It is very
difficult to provide sensible defaults (other than
'current directory'). Doing this requires non-trivial
programming work by the client, constructing or
modifying the installation frame. This is not so bad if
it is done from a GUI interface.

Move to Java. Because advanced interscript use requires
a GUI based architecture to manage whole projects (as
opposed to just processing a single document at a
time), interscript is likely to move to JPython. This
is advance warning to users: try to get JPyton running!
It is also going to use a lot more advanced HTML
features like javascript, style sheets, and other stuff
which is currently a nightmare due to the lack of a
standard Document Object Model for browsers.

Interscript current supports Internet Explorer DOM, in
lieu of a standard. Although it is possible to
incorporate some Netscape DOM features, the principal
feature needed -- outlining -- requires dynamically
setting the display style of an element, something
Netscape doesn't support at all (at the time of
writing, and as far as I can tell). Therefore, Netscape
is treated like a dumb browser, and enhanced DHTML
features will only work on Internet Explorer.

The emphasis on GUI interfaces cannot be
overemphasised. The move to dynamic, fully featured,
control over browsing the generated documents, together
with full scale meta data generation to facilitate
clients such as search engines and web crawlers
indexing the documents, is considered crucial to
getting a viable literate programmed system running.
Use of Java AWT for the input side seems inevitable, as
this appears to be the only GUI which is likely to be
widely supported, standardised, and which also allows
interaction between output and input by way of browser
applet plugins.

7.5. Installation Guide
-----------------------

How to install interscript on your computer.

7.5.1. What you need
--------------------

You need a modern computer running a vaguely posix like
operating system, C, and CPython 1.5.1. Interscript
should run out of the box on a Linux system, and with
minimal effort on most Unix boxes, and without too much
effort on 32 bit MS-Windows systems. Some enhancements
may require site dependent configuration. Please see
below for more details.

7.5.2. Where to get it
----------------------

You can ftp the interscript tar ball anonymously from
ftp://ftp.triode.net.au/ftp/skaller/interscript.tgz
which is a tarred, gziped, archive. WinZip should
handle this format.

7.5.3. Instant install
----------------------

Here are the quick installation instructions.

7.5.3.1. Unpack the archive
---------------------------

Unpack the tar ball somewhere Python will find it. The
result will be

*    a directory "interscript"

*    a file "interscript.pth"

On my linux box, I use the directory:

  /usr/local/lib/python1.5/site-packages/

7.5.3.2. Unix: Make the command line launch script executable
-------------------------------------------------------------

Optional but recommended. Copy the file
"interscript/bin/iscr.py" to someplace on you execution
path, renaming it "iscr" and making it executable. For
example, I use:

  cp /usr/local/lib/python1.5/site-packages/interscript/bin/iscr.py ~/bin/iscr
  chmod a+x iscr

You may need to edit the first line of this script,
which is

  #!/usr/bin/env python

7.5.3.3. Windows: Make the command line launch script executable
----------------------------------------------------------------

Copy the file interscript/bin/iscr.bat to someplace on
your execution path. You will (definitely!) need to
edit this file, replacing "iscr.py" with the absolute
pathname of the file "interscript/bin/iscr.py". You may
also need to change the name of the python application
executable. Note that this file only supports 9
arguments.

7.5.3.4. Mac: write a launch script
-----------------------------------

I am told the Mac does not have a shell tool.
Interscript can be launched from any Python script by
simply importing the module "interscript" and calling
the function

  interscript.run_from_arguments(args)

where "args" is a python list of strings, each of which
is a command line argument as would be parsed by a Unix
system. You will need to write a Python scipt that
gleans this information from somewhere such as a a
popup edit window, or by reading a response file. I
would be interested in including such a script with
interscript if someone wanted to send it to me at
mailto:skaller@maxtal.com.au

It is very likely that the current version of the
interscript package will also require some changes to
work on the Mac. Please email any advice to me at
mailto:skaller@maxtal.com.au

I will be working to isolate platform dependencies
better in the next version, and to try to provide
pre-configured multi-platform support.

7.5.3.5. Install doco
---------------------

If necessary copy, or move, the HTML documentation in
interscript/doc to someplace on your local WWW server.
You will need to copy all the ".html" files, and the
file "interscript.css" if you want to use the
interscript CSS1 cascading style sheet.

7.5.3.6. Test It
----------------

You can check interscript is installed from the command
line like:

  iscr --weaver=html --passes=3 example.pak

where "example.pak" is any interscript master source
file; for example, the file
"interscript/tests/example.pak", which is displayed
below.

Start python section to interscript/tests/example.pak[1]

     1: #line 101 "installation.ipk"
     2: 
     3: @head(1,'An Interscript Example')
     4: This is a minimal example of interscript at work.
     5: @head(2,'Tangling a litte script')
     6: @select(python_output('interscript/tests/example.py'))
     7: print 'Hello from an example python program'
     8: @head(2,'Executing the script')
     9: @print_python_output('interscript/tests/example.py')
    10: 


End python section to interscript/tests/example.pak[1]

7.5.4. Configuration
--------------------

Interscript supports some options which may need
configuration.

WARNING: There is currently no standard way to maintain
the interscript package. Any changes you make may get
wiped out by the next interscript installation. Please
keep a copy somewhere to reconfigure interscript after
the next installation! [I'm working on this.]

7.5.4.1. Compilers sub-package
------------------------------

This sub-package provides interscript with the ability
to compile dynmically loadable python modules written
in C or C++, and with the ability to build stand alone
command lines tools using a command line C and C++
compiler.

The files "c.py", "cpp.py", and "cconfig.py" in
"interscript/compilers/" are set up for my Linux box,
and invoke "gcc" and "g++" with appropriate options for
building python modules and command line executables.
You may need to modify (or even rewrite) these modules
to make them work at your site.

These modules are not required by interscript, but they
do allow it to build some speedup code automatically,
they're generally nice to have, and they will be used
by me when distributing C or C++ code hosted by
interscript (instead of makefiles).

These files are generated by interscript from the
source "interscript/src/compilers.ipk".

7.5.4.2. Diff sub-package
-------------------------

This package provides the ability to diff and patch
files. It is not required but is very strongly
recommended! The script is
"interscript/utilities/diff.py", and is sourced from
"interscript/src/diff.ipk" and requires GNU diff and
GNU patch.

7.5.4.3. Command line tool execution
------------------------------------

The python module 'commands' is used to execute client
commands, including python script, and capture its
output. This module is theoretically optional ...

Note that when python is invoked, the string
"sys.executable" is used to name it. This attribute is
not documented in the python documentation.

7.5.4.4. Speedups
-----------------

The module "interscript.core.iscrcopt" is an optional
dynamically loadable C module which speeds up some of
the character by character lexical processing that
interscript has to do. If the compilers module works, a
build of interscript, and in particular the source
"interscript/src/iscrcopt.ipk", will generate the
binary module. This module is already built for Redhat
Linux 5. This file is completely platform independent
(but the binary isn't, of course).

7.5.4.5. Other
--------------

There are currently a few other platform dependencies.
I'm working on fixing this. That is, I will at least
try to isolate such code in such a way that
re-installation or upgrading of interscript will not
clobber client modifications. This WILL happen at
present!

7.5.5. Building Interscript
---------------------------

If you are keen, you can try to rebuild interscript.
The whole of the source is in "interscript/src". I
recommend you create a development directory separate
from the initial (pre-built) installation. Copy the
sources into this directory and invoke the pre-built
interscript command line tool 'iscr' on the file
'iscr.pak' like this:

  iscr iscr.pak

This should build a directory 'interscript' in the
current directory. Now issue the command

  iscr --test "--weavers='all'" --passes=4 iscr.pak

This should rebuild interscript, this time using the
package built by the previous run, and also generating
the documentation. If you have problems, try using the
"-v" switch:

  iscr --test -v "--weavers='all'" --passes=4 iscr.pak

To capture the output, use the logfile option:

  iscr --test --logfile=iscr.log -v "--weavers='all'" --passes=4 iscr.pak

Note that the "--test" switch must come first, it is a
command line tool option, not an interscript option.
This option adds the current directory to the python
path before trying to import the interscript module. If
your build of interscript fails, you can edit the
sources and rebuild without the "--test" option. You
can't build with it (unless the build failed utterly--
or you delete the interscript directory or
interscript.pth file first).

To install your new version of interscript, copy the
interscript directory to your standard installation
point. Don't do this unless you're sure it works! (If
it doesn't, you may not be able to rebuild interscript
without re-installing from the tar ball .. because you
need a working copy of interscript to build
interscript).

You can rebuild any code file by just running
interscript on the ".ipk" file defining it. This won't
rebuild the documentation correctly! But it is very
much faster than rebuilding the whole package.
