Building Flat-File Databases For Your Palm Pilot on the Desktop
---------------------------------------------------------------

This package includes several utilities that let you convert text
files in CSV (comma-separated value) format to DB and MobileDB
database formats and back. This document will show you how to use
these conversion utilities.

When you convert from text files to a flat-file Palm database, you
need two files. The first file is a text file in CSV format which
contains the actual data records. The second file is a text file which
we will call the "info file" because it contains information on what
fields the database will contain. This information is sometimes called
"metadata". The info file is not required if you use command-line
options to pass the necessary metadata to the conversion utility.


CSV FORMAT TEXT FILES

The data file is in the common CSV format, or comma-separated values
format. Every line of the file contains a single record. The fields
for the record are separated by commas. For all of the example below,
let us suppose we are building a database which will hold names and
ages. The following lines might appear in the data file:

  Some Person,58
  Another Person,90

Now you might notice a problem with this example. What if you have a
field that contains a comma? Without some sort of protection, a comma
which is part of a field will be interpreted as the separator between
fields. The solution is to place double quotes around the
field. Inside double quotes, commas have no special significance. Here
is an example:

  "Last, First",80
  "Look it is a comma, And another, one",45

The quotes do not end up as a part of the field. If you want to have
quotes as a part of the field, then you need to place two quote marks
next to each other like this:

  "This field has one quote mark "" in it",90

The conversion utilities also support a slightly different method of
protecting commas and quotes which we call "extended CSV mode". This
mode was added so that you can have fields which contain return
characters which lets a field span multiple lines.

The extended CSV mode is similar to the normal CSV mode in that you
separate each field using commas and you can use double quotes to
protect commas from being interpreted as the field separator. However,
the difference comes in when you want to add a double quote into the
field. Under the extended CSV mode, you need to preface the quote mark
with a backslash (\) like this:

  "This field has one quote mark \" in it",90

If you want to have a backslash in the field, then you need to double
it like this:

  "Look! It is a backslash -> \\",56

You can use \n to substitute a newline into the field. A \t
substitutes a tab character into the field. You can also use \
followed by a number to substitute the ASCII character which has the
number as its values. Here is an example:

  "Line 1\nLine 2 with a tab\tHA",05

The extended CSV mode may be a bit confusing at first. You should only
really use it if you want newline characters in a field. Most people
should use the normal CSV mode since it is simpler.


INFO FILE FORMAT

The "metadata" for a flat-file database is stored in an info file
which is a normal text file containing commands describing the
metadata. Metadata includes the names and types of the fields, the
database title, and some other information that would make no sense
being stored in the data file.

Each line of the info file contains one of the directives described
below. Each parameter is separated by one or more spaces. You can use
double quotes to protect spaces that are part of a parameter.

The "field" directive tells the conversion utility about what fields
to expect in the data file. There should be one field directives for
each field in the data file. The directives should be in the correct
order. It has the following format:

  field NAME TYPE [WIDTH]

The items in capitals are parameters which you should replace with
actual values. Replace NAME with the name of the field. The TYPE of
the field should be "string", "boolean", or "integer" if the type of
the field is a string, checkbox (true/false), or a number
respectively. The WIDTH parameter tells the database program how wide
the column holding this field should be on the main database
screen. It is optional and defaults to 80 pixels. It can only take on
values between 10 and 160.

The "title" directive sets the title of the database as you would see
it in the database selection screen and at the top of the database
screen when you open the database on your Palm Pilot. It has the
following format:

  title TITLE

Replace TITLE with the title that you want.

The "backup" directive determines if the backup bit is set for the PDB
file that the conversion utility. If this bit is set, then the
database will be copied to your desktop PC when you HotSync next. It
has the following format:

  backup SETTING

Replace SETTING with "on" or "off" if you want the backup bit on or
off, respectively. The default is "off".

The "find" directive determines if the resulting database will be
searched when you use the Pilot's Find button. It has the following
format:

  find SETTING

Replace SETTING with "on" or "off" if you want the database to be
searched when the find button is pressed. The default is "on". This
setting is supported by the DB database program.

The "extended" directive determines if the data file will be
interpreted using the extended CSV mode discussed in the section on
CSV files. It has the following format:

  extended SETTING

Replace SETTING with "on" or "off" if you want the data file to read
using extended CSV mode. The default is "off".

Here is an example data file that might be used for the database of
names and ages discussed in the CSV section:

  title "Names and Ages"
  field "Name" string 100
  field "Age" integer 60
  backup on


CONVERTING TEXT FILES TO PALM PILOT FORMAT

The csv2pdb utility converts a text file in CSV mode into a Pilot
database you can HotSync to your Palm Pilot. The metadata describing
the database can be specified using an info file and/or command-line
options.

The csv2pdb takes the following command-line parameters:

  csv2pdb [options] DATA_FILE OUTPUT_FILE

Replace DATA_FILE with the name of the text file which contains the
data that you want in the database. This data file should be in one of
two CSV formats described above.

Replace OUTPUT_FILE with the name of the PDB file which will become a
DB-format Palm Pilot database. This is the file that you will HotSync
to your Palm Pilot to install the database.

The metadata needs to be specified using one of the options listed below:

  -i FILE    Read the info FILE for the metadata.
  -b         Set the backup bit. (Same as "backup on" in info file.)
  -e         Use extended CSV mode. (Same as "extended on" in info file.)
  -t TITLE   Set database title to TITLE. (Same as "title TITLE" in info file.)
  -f SPEC    Add a field described by SPEC.
             SPEC consists of the field name, field type, and optional
             field width separated by commas.
  -d         Output a DB-format database. (This is the default.)
  -m         Output a MobileDB-format database. This option must be
             appear before anything else on the command-line.

You can also pass the "-h" option to csv2pdb to see a quick help
screen describing all of the options above. Here are a few examples:

Suppose you have a data file called foods.txt and an info file called
foods.ifo. The following command will build a MobileDB database file
called foods.pdb that you can HotSync to your Pilot:

  csv2pdb -m -i foods.ifo foods.txt foods.pdb

Suppose we wanted to build the database with two fields that we
described in CSV section. If the data file is "z.txt", we can use
the following command to build a database file called "z.pdb":

  csv2pdb -f Name,string -f Age,integer -b -t "Names and Ages" z.txt z.pdb

The other options should be straight-forward.


CONVERTING PALM PILOT FORMAT DATABASES BACK TO TEXT FILES

The pdb2csv utility lets you takes a flat-file Palm Pilot database
file and convert it back into a data file and info file. You can find
your DB databases in the "Backup" directory on your PC if you are
using 3COM's Palm Desktop software. It can convert DB-format and
MobileDB-format databases.

The pdb2csv command has the following parameters:

  pdb2csv [-d|-m] [-e] PDB_FILE DATA_FILE INFO_FILE

Replace PDB_FILE with the name of the PDB file that was HotSync'ed off
your Pilot.

Replace DATA_FILE with the name of the file that you want to receive
all of the data records. This file will be in the normal CSV format by
default. You can pass the -e option to pdb2csv to have the file be in
the extended CSV format.

Replace INFO_FILE with the name of the file that you want to receive
the metadata. This file will be in the info file format that has been
described previously.

Normally, pdb2csv will figure out the format of the database based on
some information in the header of the PDB file. If you want to force
pdb2csv to interpret the PDB file as a MobileDB database, then pass
the -m option. Similarly, if you want the PDB file interpreted as a
DB-format database, pass the -d option.

Here are some examples:

Suppose you wanted to convert a database called drinks.pdb back into
text files. Just run this command and you're done:

  pdb2csv drinks.pdb drinks.txt drinks.ifo

If you wanted drinks.txt to use the extended CSV format, then you
would just run this command:

  pdb2csv -e drinks.pdb drinks.txt drinks.ifo

These files are in the same format that the csv2pdb discussed above
expects. Thus, you can rebuild the original Palm Pilot database very
easily by running:

  csv2pdb -i drinks.ifo drinks.txt new-drinks.pdb
