Implementation notes: OCxMON versions of the NeTraMet meter

Nevil Brownlee, 1 Oct 98

This note documents changes to the meter following Nevil's visit 
to Reston in August 98.

Memory Allocation
-----------------

OCxMON runs using a minimal amount of DPMI memory, preferring to
allocate all of its buffer space in flat memory at system startup.
But NeTraMet assumed it could always get memory by calling malloc().

New version allocates all its space in flat memory at startup.
EARLY_ALLOC define tells compiler to do it, this is enabled by default.
There was no difficulty doing this for the flow table and flow index but
it was harder for rulesets, ruleset hashtables, and flow hashtables,
since these were allocated 'one per ruleset'. 

We now have only a single hash table; this required putting the ruleset
number into the flow key when computing the hash.  (Since we now have an
efficient hash algorithm there's no advantage in using multiple tables.)

Rulesets and their hashtables use space allocated at startup.  Since we
can't tell how much space is required, we use a command-line option to
tell us.

  -u nnnn     means 'allocate space for a maximum of nnnn rules'
  
The ruleset space is used to hold all rulesets on the meter; default is
2000 rules.  This space is allocated at the top of the rulset area when
a ruleset is downloaded and opened.  It is compacted when a ruleset is
destroyed.

The Flow Index
--------------

FLOW_INDEX define tells compiler it should use a flow index, this is
disabled by default.  The flow index was necessary for the 16-bit meter,
being kept in near memory to access the flows in far memory.  It also
allowed finding flows by looking them up in the flow index, rather than
computing the index.  Now that we have flat memory and fast processors
there is no advantage in keeping the flow index.

Ruleset Chains
--------------

RS_CHAINS define tells compiler it should maintain chains of flows for
each ruleset, this is enabled by default.  This allows the meter to find
the next flow for a ruleset without having to do linear searches of the
flow table, which reduces the amount of work when several rulesets are
active.  If only one ruleset is running, however, there is no
improvement in searching.  

However, RS_CHAINS also maintains a chain of idle flow numbers.  When
create() requests a flow number it has only to look at the idle chain
to see whether it must ask the garbage collector to find one.  Before,
number_flow() did a serial search of the whole flow table before
realising it had to garbage-collect!


