Wednesday, July 15, 2009

My Latest Post

This is just some random text I yanked from some text file:
- Flat category structure: all category directories under single
  nexus -> short path to files, category list easily derived from
  nexus directory contents.
- OR two-level system with alphabetic directories in nexus and
  category directories sorted under initial -> compact directory
  listing for shell.
- OR support both of the above, with one dynamicaly generated from
  the other -> advantages of both at cost of developing and/or
  operating generator program and additional data space.

Posted: 08:45 Category: tir Permaink Comments: 1

Deletion Protocol

The search for an easy method to delete files across TIR mirror sets.

TIR data is stored in a master data set on a central server and a local mirror data set on each client platform. Operations (create, modify, delete) on files in the dataset should be supported on the central server and all clients, and operation results propagated to the master and all mirror data sets.

Assume single user who operates on only one mirror set or the master set at one time.

The cavenet synchronization script (based on rsync) provides adequate support for file creation and modification. (Weaknesses include lack of support for simultaneous modifications to the same file on different mirror sets.) However, deletions must be performed and repeated manually on the server and all clients to be effective.

Basic function of rsync is to compare source and target data sets and copy from source to target files that are older or non-existent in the target data set. The --delete option causes rsync to delete files from the target data set that do not exist in the source.

The difficulty arises in that when a file exits in one mirror set and not in another the cases of a file having been deleted from the set where it doesn't exist and of a file having been newly created in the set where exists are indistiguishable, but call for very different synchronization processing. (Deletion of obsolete file in the former case and copying of the new file in the latter.)

Alternatives:

Provide get/put synchronization protocol. User tracks mirror states.

User rsync --delete in both directions. Usage similar to existing cavenet (though cavenet doesn't support deletion). User tracking would consist of:

Unsynchronized changes on at most one client or the server at a time.
Execute sync get on client before making any changes.
Execute sync put on client at end of change session.

NOTE: Creating a file in a client data set then executing sync get before a sync put will result in the loss of the new file.

—> Will implement this solution as it is simplest and requires little modification of the existing cavenet script.

Registration procedure for files created on clients that would add the new files to the master set outside of the synchronization process. Synchronize with rsync --delete both directions.

Could use single sync procedure that executes get and put in sequence. Files non-existent on either client or server would be deleted since files created on the client during the change session would already exist on server by end of session due to rregistration.

Register deletions to server data set during change session. (Opposite of create registration process.)
Save synchronization status in hidden files in master and mirror data sets; program get or put execution. (How is deletion/creation ambiguity handled?)

Posted: 08:45 Category: tir Permaink Comments: 1