#!/usr/bin/perl -w

# Kenneth Howlett's unix archive program
# see the comments at the end of this file

use integer;

# These make the source code easier to understand:
$Success = 1;
$Failure = 0;
$No = 0;
$Yes = 1;
$Ask = 2;
$Exclude = 0;
$Include = 1;


# user configuration: you can change these next things if you want

$CompressionCommand = 'gzip -9 --no-name';
$UncompressionCommand = 'gzip -d';
# you might think that changing the compression commands would make
# your version of kxarc incompatible with other versions, but no.
# kxarc puts the uncompression command into the archive, so it
# knows how to uncompress the archive.
# All that matters is that the compression and uncompression commands
# go together.
# If you do change the compression and uncompression programs, note that
# kxarc expects both to compress/uncompress standard input to standard output.
# Also kxarc assumes that if the uncompression program produces no output,
# then the compressed data is truncated, that more compressed data is needed;
# and kxarc assumes that if the uncompression program produces output and
# returns a nonzero exit code, then the compressed data is corrupted.

$CompressionInputFile = "/tmp/kxarcCompIn$$";
$CompressionOutputFile = "/tmp/kxarcCompOut$$";

$MaxSizeUnReadArchiveData = 1000000;
#$MaxSizeUnReadArchiveData = 4194304;
#$MaxSizeUnReadArchiveData = 1048576;
# $MaxSizeUnReadArchiveData controls how much memory kxarc can use for
# sorting out errors in corrupted archives. In some situations, kxarc
# needs a lot of memory for this; and the bigger you make
# $MaxSizeUnReadArchiveData, the more files kxarc can unarchive from a
# corrupted archive, and the more memory kxarc will use. But in most
# situatations it does not matter what you put for $MaxSizeUnReadArchiveData.
# kxarc does not use the memory if it does not need it, and the situations
# where kxarc needs the memory are very rare.
# Try 1/4 to 1/2 the amount of memory in your computer.
# If your version of perl has large integer bugs, you should not use a
# number larger than 2147483647. That is 2 gigabytes; you probably do not
# want to use a number that large anyway because you probably do not have
# that much memory. If kxarc tries to use more memory than you have in
# your computer, you will get a core dump.

$SizeIfTrueSizeUnknown = 16777216;
# If the uncompression program makes no output, kxarc assumes that means
# that $SizeOfArchiveData is incorrect and is smaller than it should be.
# Then kxarc resets $SizeOfArchiveData to $SizeIfTrueSizeUnknown and tries
# again.
# If $SizeOfArchiveData is too small, gzip will not uncompress any data.
# If $SizeOfArchiveData is too big, gzip ignores the extra data. Therefore,
# it is safer to be too big than too small. Therefore $SizeIfTrueSizeUnknown
# should be as big as possible, but do not make $SizeIfTrueSizeUnknown
# bigger than the amount of free space on your hard drive.
# If $SizeIfTrueSizeUnknown is greater than $MaxSizeUnReadArchiveData and
# the archive is not seekable, then kxarc resets $SizeIfTrueSizeUnknown to
# $MaxSizeUnReadArchiveData

$FileCopyingBlockSize = 400000;
#$FileCopyingBlockSize = 786432;
# 2097152 1048576 524288 262144 131072 65536
# $FileCopyingBlockSize controls how much memory kxarc uses when copying
# files. More memory allows kxarc to copy files more efficiently; but if
# $FileCopyingBlockSize is too big it will slow kxarc down by
# forcing the kernel to use virtual memory, and by starving the kernel file
# buffering functions of memory.
# When kxarc reads a file, it reads $FileCopyingBlockSize and then checks
# for end of file, because this is easier than getting the file size and
# reading the file size if that is less than $FileCopyingBlockSize.
# Apparently, this causes perl to allocate $FileCopyingBlockSize bytes of
# memory, even though it might read only a few bytes, or even 0 bytes.
# Therefore, a bigger number for $FileCopyingBlockSize will waste memory
# and slow kxarc down.
# The ideal size for $FileCopyingBlockSize would be slightly larger than the
# size of the files you are archiving or unarchiving;
# but in the real world files usually have different sizes.
# $FileCopyingBlockSize should be smaller than $MaxSizeUnReadArchiveData.
# $FileCopyingBlockSize must be a multiple of 4 because this makes it
#      easier to calculate checksums
# Try 1/16 of the amount of memory in your computer.

$CompareCommand = 'diff --brief -';

# For $CompareCommand, use any command which compares standard input with a
# file, and returns 0 if it matches and nonzero if it does not. Note that
# before $CompareCommand runs, kxarc adds a space, a single quote, the name
# of the file, another single quote, and ' > /dev/null'; so 'diff --brief -'
# becomes 'diff --brief - 'foo' > /dev/null' for file 'foo'.

$MaxHeaderSize = 65536;
# If kxarc finds that the header of a file in an archive is larger than
# $MaxHeaderSize, then it will assume that the file is corrupted. Headers
# can therorectically be very very large, but are usually less than 1K.
# kxarc tries to copy the header into memory before it checks the checksum.
# If an archive is corrupted, it might contain random bytes, which might
# appear to say that there is a very large header; and if the apparent size
# of the header is larger than the amount of memory in your computer, your
# computer will crash. So $MaxHeaderSize should be set to less than the
# amount of memory in your computer to prevent a possible but unlikely
# crash. Any number more than 1K and less than the amount of memory in your
# computer is fine.

# Also in the catagory of user configuration, you can configure kxarc
# to continue instead of aborting when bugs are detected. That is about
# 1300 lines down. Search for  'sub del'

# the next section is sort of user configurable, except if you change any
# of these, your version of kxarc will be incompatible with other versions!
# actions:
$NoAction = 0;
$Archive = 1;
$UnArchive = 2;
$Compare = 3;
$Fix = 4;
$List = 5;
$Delete = 6;
$Message = 7;
$Display = 8;
# file types:
# One might think that the file types should be integers rather than one byte
# strings, because usually it is faster to compare two integers than two
# strings. But the file type has to be put into the archive as a string;
# by defining the file types as strings we do not have to convert them to
# strings when we put the file type into the archive
$Nothing = chr(0);  # or we want this for a real file type, how about 'no'?
$DataAboutFormatOfArchive = chr(1);
$Index = chr(2);
$ArchiveMessage = chr(3);
$ArchivedCommands = chr(4);
$NormalFile = '-';  # chr(45)
$BlockDevice = 'b';  # chr(98)
$CharacterDevice = 'c';  # chr(99)
$Directory = 'd';  # chr(100)
$SoftLink = 'l';  # chr(108)
$Pipe = 'p';  # chr(112)
# $ExtendedFileType = chr(255);
$CommandSeperator = chr(10);   # "\n";
$OutsideMarker = 'KMH';   # used outside the compressed data
$InsideMarker = 'kh';    # used inside the compressed data

# global data initializations and defaults
$ArchiveIsNormalFile = $No;
$Action = $NoAction;
$AddBeginCommand = $No;
$AddEndCommand = $No;
$AddMessage = $No;
$BeginCommandString = '';
$EndCommandString = '';
$ExitOnNonFatalErrors = $No;
$ExitCodeZeroIfFilesDidNotMatch = $No;
$FileIsCorrupted = $No;
$FileListFile = '';
$FilesDidNotMatch = $No;
$TypeOfFile = $Nothing;
$IgnoreCommands = $No;
$IgnoreDataAboutFormatOfArchive = $No;
$IgnoreUidGid = $No;
$Interactive = $No;
$MakeIndex = $No;
$MustSearch = $No;
$NameOfArchive = '';
$NameOfFile = '';
$NameOfFileList = '';
$NonFatalErrorsOccurred = $No;
$Overwrite = $Yes;
$PerlLargeIntegerBugs = $No;
$PossibleUnReadCount = 0;
$SkipCorruptedFiles = $Yes;
$SizeOfDataReadSinceUnReadMark = 0;
$UncompressHeaderFirst = $No;
$UnReadArchiveData = '';
$UseFileList = $No;
$UsingIndex = $No;
$Verbose = $No;

$NA = -1;
while ( $NA < $#ARGV ) {
  $NA = $NA + 1;
  #print("current option is $ARGV[$NA]\n");
  if ($ARGV[$NA] eq '--archive') {
    $Action = $Archive;
    next;
    }
  if ($ARGV[$NA] eq '--unarchive') {
    $Action = $UnArchive;
    next;
    }
  if (substr($ARGV[$NA],0,20) eq '--add-begin-command=') {
    $AddBeginCommand = $Yes;
    $BeginCommandString = $BeginCommandString . substr($ARGV[$NA],20) .
                           $CommandSeperator;
    next;
    }
  if (substr($ARGV[$NA],0,18) eq '--add-end-command=') {
    $AddEndCommand = $Yes;
    $EndCommandString = $EndCommandString . substr($ARGV[$NA],18) .
                          $CommandSeperator;
    next;
    }
  if (substr($ARGV[$NA],0,14) eq '--add-message=') {
    if ( $AddMessage == $Yes )  {
      print(STDERR "\nkxarc: there is more than one message file\n");
      exit(10);
      }
    $NameOfMessageFile = substr($ARGV[$NA],14);
    if ( ! -r $NameOfMessageFile )  {
      print(STDERR "\nkxarc: unable to read from message file $NameOfMessageFile\n");
      &deleteTmpFilesAndExit(17);
      }
    $AddMessage = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--ask-about-corrupted-files') {
    $SkipCorruptedFiles = $Ask;
    next;
    }
  if ($ARGV[$NA] eq '--ask-before-overwrite') {
    $Overwrite = $Ask;
    next;
    }
  if ($ARGV[$NA] eq '--compare') {
    $Action = $Compare;
    next;
    }
  if (substr($ARGV[$NA],0,22) eq '--compression-command=') {
    $CompressionCommand = substr($ARGV[$NA],22);
    next;
    }
  if ($ARGV[$NA] eq '--create') {
    $Action = $Archive;
    next;
    }
  if ($ARGV[$NA] eq '--delete') {
    $Action = $Delete;
    next;
    }
  if ($ARGV[$NA] eq '--display') {
    $Action = $Display;
    next;
    }
  if ($ARGV[$NA] eq '--do-not-overwrite') {
    $Overwrite = $No;
    next;
    }
  if ($ARGV[$NA] eq '--do-not-skip-corrupted-files') {
    $SkipCorruptedFiles = $No;
    next;
    }
  if ($ARGV[$NA] eq '--exit-code-zero-if-files-did-not-match') {
    $ExitCodeZeroIfFilesDidNotMatch = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--exit-on-nonfatal-errors') {
    $ExitOnNonFatalErrors = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--extract') {
    $Action = $UnArchive;
    next;
    }
  if (substr($ARGV[$NA],0,18) eq '--file-names-from=') {
    $NameOfFileList = substr($ARGV[$NA],18);
    $UseFileList = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--help') {
    print("
kxarc [action] [options] [name of archive]
          
actions:  (one and only one action is required)
--archive      --unarchive    --compare         --list
--fix          --help         --version       --delete
--display      --message

options:
--ask-before-overwrite     --do-not-overwrite
--add-begin-command=<command>   --add-end-command=<command>
--file-names-from=<name of file list>     --ignore-uid-gid
--verbose           --interactive     --exit-on-nonfatal-errors
--exit-code-zero-if-files-did-not-match   --make-index
--do-not-skip-corrupted-files   --ask-about-corrupted-files
--add-message=<name of message file>        --ignore-archived-commands
--ignore-data-about-format-of-archive      --uncompress-header-first
--compression-command=<compression command>
--uncompression-command=<uncompression command>

unless you say otherwise, the following options will be assumed:
--compression-command=\'$CompressionCommand\'
--uncompression-command=\'$UncompressionCommand\'
");
    exit(0);
    }
  if ($ARGV[$NA] eq '--ignore-archived-commands') {
    $IgnoreCommands = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--ignore-data-about-format-of-archive') {
    $IgnoreDataAboutFormatOfArchive = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--ignore-uid-gid') {
    $IgnoreUidGid = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--interactive') {
    $Interactive = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--list') {
    $Action = $List;
    next;
    }
  if ($ARGV[$NA] eq '--make-index') {
    $MakeIndex = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--message') {
    $Action = $Message;
    next;
    }
  if ($ARGV[$NA] eq '--uncompress-header-first') {
    $UncompressHeaderFirst = $Yes;
    next;
    }
  if (substr($ARGV[$NA],0,24) eq '--uncompression-command=') {
    $UncompressionCommand = substr($ARGV[$NA],24);
    next;
    }
  if ($ARGV[$NA] eq '--verbose') {
    $Verbose = $Yes;
    next;
    }
  if ($ARGV[$NA] eq '--version') {
    print("
kxarc, Kenneth Howlett's unix archive program, version 0.5
");
    exit(0);
    }
  if ( $NA == $#ARGV    and    $NameOfArchive eq '' )  {
    $NameOfArchive = $ARGV[$NA];
    next;
    }
  print(STDERR "
kxarc: unknown command line option:  $ARGV[$NA]
try     kxarc --help
");
  exit(10);
  }

if ( $Action == $NoAction )  {
  print(STDERR
        "\nkxarc: you have not told kxarc what action you want it to do.\n",
        "try     kxarc --help\n");
  exit(10);
  }

if ( $NameOfArchive eq '' )  { $NameOfArchive = '-' }
if ( $UseFileList == $No     and     $Action == $Archive )  {
  #     or      $Action == $Delete
  $UseFileList = $Yes;
  $NameOfFileList = '-';
  }
if ( $NameOfArchive eq $NameOfFileList    and     $Action != $Archive )  {
  if ( $NameOfArchive eq '-' )  {
    print(STDERR "\nkxarc: you cannot read both the archive and the\n",
         "file list from standard input\n");
    }
  else  {
    print(STDERR "\nkxarc: you cannot read both the archive and the\n",
         "file list from $NameOfArchive\n");
    }
  exit(10);
  }
if ( $UseFileList == $Yes )  {
  if (!open(FH_FileList,"< $NameOfFileList")) {
     print(STDERR "\nkxarc: unable to read from file list $NameOfFileList\n");
     exit(11);
     }
  }

if ( ! open(FH_T, "> $CompressionInputFile") )  {
  print(STDERR
        "\nkxarc: unable to write to temporary file $CompressionInputFile\n");
  &deleteTmpFilesAndExit(12);
  }
close(FH_T);
if ( ! open(FH_T, "< $CompressionInputFile") )  {
  print(STDERR
      "\nkxarc: unable to read from temporary file $CompressionInputFile\n");
  &deleteTmpFilesAndExit(12);
  }
close(FH_T);
if ( ! open(FH_T, "> $CompressionOutputFile") )  {
  print(STDERR
      "\nkxarc: unable to write to temporary file $CompressionOutputFile\n");
  &deleteTmpFilesAndExit(12);
  }
close(FH_T);
if ( ! open(FH_T, "< $CompressionOutputFile") )  {
  print(STDERR
      "\nkxarc: unable to read from temporary file $CompressionOutputFile\n");
  &deleteTmpFilesAndExit(12);
  }
close(FH_T);
# perl version 5.004_04 built for i386-linux has bugs in working
# with integers which require more than 31 bits, which is integers
# larger than 0x7FFFFFFF, about 2 billion.
# I am assuming this will not be a big problem, because file sizes
# are rarely that big.
# If it is a problem, we will have to use Math::BigInt or something
# If you get tired of seeing the following warning, delete it.
if ( 2147483648 < 0 )  {
  print(STDERR '
kxarc: this perl cannot handle integers bigger than 0x7FFFFFFF.
Do not ask kxarc to archive files which are bigger than 2 gigabytes, and do
not do anything with archives which contain files bigger than 2 gigabytes.
');
  $PerlLargeIntegerBugs = $Yes;
  if ( $MaxSizeUnReadArchiveData < 0 )  {
    print(STDERR '
kxarc: $MaxSizeUnReadArchiveData is less than 0. You are probably using
a number which is too big for this perl.
Resetting $MaxSizeUnReadArchiveData to 2147483647
');
    $MaxSizeUnReadArchiveData = 2147483647;
    }
  if ( $FileCopyingBlockSize < 0 )  {
    print(STDERR '
kxarc: $FileCopyingBlockSize is less than 0. You are probably using
a number which is too big for this perl.
Resetting $FileCopyingBlockSize to 2147483644
');
    $FileCopyingBlockSize = 2147483644;
    }
  }

if ( ($FileCopyingBlockSize%4) != 0 )  {
  print(STDERR '
kxarc: $FileCopyingBlockSize is not a multiple of 4.
Resetting $FileCopyingBlockSize to ',
  ($FileCopyingBlockSize - ($FileCopyingBlockSize%4)),"\n");
  $FileCopyingBlockSize = $FileCopyingBlockSize - ($FileCopyingBlockSize%4);
  }
if ( $FileCopyingBlockSize > $MaxSizeUnReadArchiveData )  {
  print(STDERR '
$MaxSizeUnReadArchiveData must greater than or equal to $FileCopyingBlockSize
Resetting $MaxSizeUnReadArchiveData to ',$FileCopyingBlockSize,"\n");
  $MaxSizeUnReadArchiveData = $FileCopyingBlockSize;
  }
$OutsideMarkerLength = length($OutsideMarker);
$InsideMarkerLength = length($InsideMarker);
if ( $Interactive == $Yes )  {
  $Overwrite = $Ask;
  $SkipCorruptedFiles = $Ask;
  }



if ( $Action == $Archive )  {
if ( $NameOfArchive eq '-'    and    $Verbose == $Yes )  {
  print(STDERR "
kxarc: you can NOT use '--verbose' when writing the archive to standard output
");
  &deleteTmpFilesAndExit(10);
  }
if ( $AddMessage == $Yes     and     $UseFileList == $Yes )  {
  if ( $NameOfMessageFile eq $FileListFile )  {
  print(STDERR "
kxarc: you can NOT read both the message and the file list from
$FileListFile
");
  &deleteTmpFilesAndExit(10);
  }}
if ( $AddMessage == $Yes     and     $UseFileList == $No )  {
  if ( $NameOfMessageFile eq '-' )  {
  print(STDERR "
kxarc: you can NOT read both the message and the file list from
standard input
");
  &deleteTmpFilesAndExit(10);
  }}
if ( $CompressionCommand eq '' )  { $UncompressionCommand = '' }
if ( ! open(FH_Archive,">> $NameOfArchive") ) {
  print(STDERR "\nkxarc: unable to write to archive $NameOfArchive\n");
  &deleteTmpFilesAndExit(11);
  }
if ( -f $NameOfArchive      and      $NameOfArchive ne '-' )
    { $ArchiveIsNormalFile = $Yes }
$TypeOfFile = $DataAboutFormatOfArchive;
$SA = $InsideMarker . $TypeOfFile . &Vstring($UncompressionCommand);
$CalculatedChecksum = unpack("%32L*","$SA\0\0\0");
#print(STDERR "checksum is $CalculatedChecksum for\n$SA\n");
$SA = $SA . pack('L',$CalculatedChecksum);
&writeOutsideMarkerAndDataSizeToArchive(length($SA));
print(FH_Archive $SA);
if ( $AddMessage == $Yes )  {
  $TypeOfFile = $ArchiveMessage;
  &addFileToTempFileAndCompressAndWriteToArchive
    (($InsideMarker . $TypeOfFile), $NameOfMessageFile);
  }
if ( $AddBeginCommand == $Yes )  {
  $TypeOfFile = $ArchivedCommands;
  &addNonFileToArchive($InsideMarker . $TypeOfFile . $BeginCommandString);
  $BeginCommandString = '';
  }
while ( <FH_FileList> )  {
  chomp($_);
  $NameOfFile = $_;
  $TypeOfFile = $Nothing;
  if ( -l $NameOfFile )  {      # if file is a soft link
    $TypeOfFile = $SoftLink;
    (undef,undef,undef,undef,$UIDfromFile,$GIDfromFile,undef)
       = lstat($NameOfFile);
    $LinksToFromFile = readlink($NameOfFile);
    &addNonFileToArchive($InsideMarker . $TypeOfFile . &packUidGid .
          &Vstring($NameOfFile) . &Vstring($LinksToFromFile));
    next;
    }
  # we do the -e test after the -l test because if we do -e on a soft link,
  # -e tests the file which the soft link links to.
  if ( ! -e $NameOfFile )  {    # if file does not exist
    print(STDERR
      "\nkxarc: $NameOfFile cannot be archived because it does not exist\n");
    &errorMaybeAbort;
    next;
    }
  if ( -f $NameOfFile )  {      # if file is a normal file
    if ( ! -r $NameOfFile )  {
      print(STDERR
       "\nkxarc: $NameOfFile cannot be archived because it is not readable\n");
      &errorMaybeAbort;
      next;
      }
    $TypeOfFile = $NormalFile;
    (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,undef,
        undef,undef,$TimeOfLastChangeFromFile,undef) = stat($NameOfFile);
    $PermissionsFromFile = $TypePermissions & 0x0FFF;
    $SA = pack('S', $PermissionsFromFile) . &packUidGid .
          &Vstring($NameOfFile) .
          "\0" . pack('L',$TimeOfLastChangeFromFile);
    &addFileToTempFileAndCompressAndWriteToArchive(($InsideMarker .
        $TypeOfFile . &VunsignedInteger(length($SA)) . $SA),$NameOfFile);
    if ( $Verbose == $Yes )  { print("$NameOfFile\n"); }
    next;
    }
  #print(STDERR " type of file is $TypeOfFile\n");
  if ( -d $NameOfFile )  { $TypeOfFile = $Directory }
  if ( -p $NameOfFile )  { $TypeOfFile = $Pipe }
  #print(STDERR " type of file is $TypeOfFile\n");
  if ( $TypeOfFile eq $Directory     or    $TypeOfFile eq $Pipe )  {
    (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,undef)
                = stat($NameOfFile);
    $PermissionsFromFile = $TypePermissions & 0x0FFF;
    &addNonFileToArchive($InsideMarker . $TypeOfFile .
          pack('S', $PermissionsFromFile) . &packUidGid .
          &Vstring($NameOfFile));
    next;
    }
  if ( -b $NameOfFile )  { $TypeOfFile = $BlockDevice }
  if ( -c $NameOfFile )  { $TypeOfFile = $CharacterDevice }
  #print(STDERR " type of file is $TypeOfFile\n");
  if ( $TypeOfFile eq $BlockDevice    or   $TypeOfFile eq $CharacterDevice ) {
    (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,$DeviceID,
             undef) = stat($NameOfFile);
    $PermissionsFromFile = $TypePermissions & 0x0FFF;
    $MajorNumberFromFile = ($DeviceID >> 8) & 0x00FF;
    $MinorNumberFromFile = $DeviceID & 0x00FF;
    #print(STDERR "major number from file is $MajorNumberFromFile\n");
    #print(STDERR "minor number from file is $MinorNumberFromFile\n");
    &addNonFileToArchive($InsideMarker . $TypeOfFile .
          pack('S', $PermissionsFromFile) . &packUidGid .
          &Vstring($NameOfFile) .
          pack('CC', $MajorNumberFromFile, $MinorNumberFromFile));
    next;
    }
  if ( -S $NameOfFile )  {      # if file is a socket
    print(STDERR "\nkxarc: skipping '$NameOfFile'\n",
               "because it is a socket\n");
    next;
    }
  print(STDERR
    "\nkxarc: '$NameOfFile' cannot be archived\n",
    "because it is an unknown file type\n");
  &errorMaybeAbort;
  next;
  }  # end of    while ( <FH_FileList> )  {
if ( $AddEndCommand == $Yes )  {
  $TypeOfFile = $ArchivedCommands;
  &addNonFileToArchive($InsideMarker . $TypeOfFile . $EndCommandString);
  }
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Archive )




if ( $Action == $Fix )  {
print(STDERR "\nkxarc: action '--fix' has not been coded yet.\n");
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Fix )  {




if ( $Action == $Message )  {
if ( $NameOfArchive eq '-' )  {
  if ( $Interactive == $Yes )  {
    print(STDERR "\nkxarc: you cannot use --interactive when reading\n",
         "the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  }
&openArchiveForRead;
until ( &endOfArchive )  {
&getNextFileFromArchive;
if ( $TypeOfFile eq $ArchiveMessage )  {
  if ( $Interactive == $Yes )  {
    print(STDERR
      "The archive contains a message. Display the message? (yn)\n");
    $SA = <STDIN>;
    if ( $SA eq "y\n"    or    $SA eq "Y\n" )
       { print("$BigStringData\n") }
    print(STDERR "search the archive for additional messages? (yn)\n");
    $SA = <STDIN>;
    if ( $SA eq "y\n"    or    $SA eq "Y\n" )  { next }
    }
  else  { print("$BigStringData\n") }
  &deleteTmpFilesAndExit(0);
  }
}  # end of    until ( &endOfArchive )  {
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Message )  {




if ( $Action == $List )  {
$UncompressHeaderFirst = $Yes;
if ( $Verbose == $Yes )  {
  print(STDERR '
kxarc: --list --verbose has not been coded yet. Try --list
');
  &deleteTmpFilesAndExit(0);
  }
&openArchiveForRead;
until ( &endOfArchive )  {
  &getNextFileFromArchive;
  if ( $NameOfFile ne '' )  { print("$NameOfFile\n") }
  }
#print(STDERR "end of archive\n");
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $List )  {




if ( $UseFileList == $Yes     and     $Interactive == $Yes )  {
  print(STDERR "\nkxarc: Load file list $NameOfFileList? y/n/a\n");
  while ( <STDIN> )  {
    if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
    if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
    if ( $_ eq "n\n"    or    $_ eq "N\n" )  { $UseFileList = $No; last }
    }
  }
if ( $UseFileList == $Yes )  {
  @FileList = <FH_FileList>;
  close(FH_FileList);
  chomp @FileList;
  }



if ( $Action == $UnArchive )  {
if ( $NameOfArchive eq '-' )  {
  if ( $Interactive == $Yes )  {
    print(STDERR "\nkxarc: you cannot use --interactive when reading\n",
         "the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  if ( $Overwrite == $Ask )  {
    print(STDERR "\nkxarc: you cannot use --ask-before-overwrite when\n",
         "reading the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  if ( $SkipCorruptedFiles == $Ask )  {
    print(STDERR "\nkxarc: you cannot use --ask-about-corrupted-files when\n",
         "reading the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  }
&openArchiveForRead;
L_Unarchive: until ( &endOfArchive )  {
&getNextFileFromArchive;
#print(STDERR "file type is $TypeOfFile; file name is '$NameOfFile'\n");
if ( $TypeOfFile eq $Index )  {
  print(STDERR '
kxarc: bug detected: sub getNextFileFromArchive returned with $TypeOfFile
set to $Index
');
  &deleteTmpFilesAndExit(40);
  }
if ( $TypeOfFile eq $DataAboutFormatOfArchive )  {
  print(STDERR '
kxarc: bug detected: sub getNextFileFromArchive returned with $TypeOfFile
set to $DataAboutFormatOfArchive
');
  &deleteTmpFilesAndExit(40);
  }
if ( $TypeOfFile eq $Nothing )  {
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR  "    skipping unreadable data\n");
    &errorMaybeAbort;
    }
  next;
  }
if ( $TypeOfFile eq $ArchiveMessage )  {
  if ( $Interactive == $Yes )  {
    print(STDERR
      "The archive contains a message. Display the message? y/n/a\n");
    while ( <STDIN> )  {
      if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
      if ( $_ eq "y\n"    or    $_ eq "Y\n" )  {
        print(STDERR "$BigStringData\n");
        next;
        }
      if ( $_ eq "n\n"    or    $_ eq "N\n" )  { next }
      }
    }
  next;
  }
if ( $TypeOfFile eq $ArchivedCommands )  {
  &executeCommandsInArchive;
  next;
  }
if ( $NameOfFile eq '' )  {
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR  "    skipping corrupted ",&fileTypeString," with no name\n");
    &errorMaybeAbort;
    next;
    }
  print(STDERR "
kxarc: bug detected: unable to get name of ",&fileTypeString," from archive
  file name is $NameOfFile
");
  &deleteTmpFilesAndExit(40);
  }
if ( &includeOrExclude($NameOfFile) == $Exclude )   { next }
if ( $FileIsCorrupted == $Yes     and     $SkipCorruptedFiles == $Yes )  {
  print(STDERR  "    skipping corrupted file\n");
  &errorMaybeAbort;
  next;
  }
if ( -e $NameOfFile )  { $FileAlreadyExists = $Yes }
else  { $FileAlreadyExists = $No }
if ( $FileAlreadyExists == $Yes     and     $Overwrite == $No )  {
  print(STDERR  "\nkxarc: skipping $NameOfFile to avoid overwrite\n");
  &errorMaybeAbort;
  next;
  }
# three reasons to have a prompt
if ( $Interactive == $Yes     or
  ($FileIsCorrupted == $Yes    and    $SkipCorruptedFiles == $Ask)     or
  ($FileAlreadyExists == $Yes    and    $Overwrite == $Ask) )  {
  # one of four possible prompts
  if ( $FileIsCorrupted == $Yes    and    $FileAlreadyExists == $Yes )
      { print(STDERR "
$NameOfFile already exists,
and the archive version is corrupted.
unarchive the corrupted version and overwrite the existing version? y/n/a
") }
  elsif ( $FileIsCorrupted == $Yes )
      { print(STDERR "    unarchive corrupted file? y/n/a\n") }
  elsif ( $FileAlreadyExists == $Yes )
      { print(STDERR "
$NameOfFile already exists.
unarchive $NameOfFile and overwrite existing file? y/n/a
") }
  else  { print(STDERR "unarchive $NameOfFile? y/n/a\n") }
  while ( <STDIN> )  {
    if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
    if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
    if ( $_ eq "n\n"    or    $_ eq "N\n" )   {
       print(STDERR "skipping $NameOfFile\n");
       next L_Unarchive;
       }
    }
  }
if ( $FileIsCorrupted == $Yes )  {
  print(STDERR "    unarchiving corrupted ",&fileTypeString," $NameOfFile\n");
  if ( $Interactive == $No    and    $MustSearch == $No )  { &setUnReadMark }
  }
if ( $FileAlreadyExists == $Yes )  {
    print(STDERR "    overwriting $NameOfFile\n");
    if ( -d $NameOfFile )  { rmdir($NameOfFile) }
    else { unlink($NameOfFile) }
    # Some types of files, like normal files, are easily overwritten;
    # so it is not really needed to delete the old files first. Other
    # types, like devices, can not be overwritten and must be deleted
    # first. And if we try to do something weird, like overwrite a
    # device with a file, then we need to delete it first; we do not
    # want to open the device and write data to it!
    # So it seems easiest to always delete before overwriting.
    # But if the file is a directory and it contains more
    # files, then it will not be deleted.
    # If we are going to overwrite the directory with a new directory,
    # that is not a problem, we can just keep the old directory;
    # but kxarc is not able to take a directory which contains files
    # and convert the directory to a normal file, link, pipe, or device;
    # hopefully no one will ever want to do that.
    }
if ( $TypeOfFile eq $NormalFile )  {
  if ( -e $NameOfFile )  {
    print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  if ( ! open(FH_FileToUnArchive,"> $NameOfFile") )  {
    print(STDERR "\nkxarc: unable to create $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  while  ( read(FH_CompressionOutputFile,$BigStringData,
                $FileCopyingBlockSize) )
      { $NA = print(FH_FileToUnArchive $BigStringData) }
  close(FH_FileToUnArchive);
  if ( $NA == $Failure )  {
    print(STDERR "\nkxarc: unable to write to $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  &setFilePermissionsGidUid;
  if ( ! utime(0,$TimeOfLastChangeFromArchive,$NameOfFile) )  {
    print(STDERR "\nkxarc: unable to change last change time of $NameOfFile\n");
    &errorMaybeAbort;
    }
  if ( $Verbose == $Yes )  {
    print("$NameOfFile\n");
    }
  next;
  }
if ( $TypeOfFile eq $Directory )  {
  # if it already exists and it is NOT a directory, error
  # if it does not exist, create it
  # if it already exists and it is a directory, do nothing
  if ( -e $NameOfFile ) {
    if ( ! -d $NameOfFile )  {
      print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
      &errorMaybeAbort;
      next;
      }
    }
  else {
    if ( ! mkdir($NameOfFile,0777) )  {
      print(STDERR "\nkxarc: unable to create $NameOfFile\n");
      &errorMaybeAbort;
      next;
      }
    }
  &setFilePermissionsGidUid;
  if ( $Verbose == $Yes )  { print("$NameOfFile\n") }
  next;
  }
if ( $TypeOfFile eq $SoftLink )  {
  if ( -e $NameOfFile )  {
    print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  if ( ! symlink($LinksToFromArchive,$NameOfFile) )  {
    print(STDERR "\nkxarc: unable to create symlink $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  if ( $IgnoreUidGid == $No )  {
    if ( ! chown($UIDfromArchive,$GIDfromArchive,$NameOfFile) )  {
      print(STDERR
          "\nkxarc: unable to change owner and group of $NameOfFile\n");
      &errorMaybeAbort;
      }}
  if ( $Verbose == $Yes )  {
    print("$NameOfFile\n");
    }
  next;
  }
if ( $TypeOfFile eq $BlockDevice )  {
  if ( -e $NameOfFile )  {
    print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  $ExternalCommand =
#    "'mknod $NameOfFile b $MajorNumberFromArchive $MinorNumberFromArchive'";
    "mknod $NameOfFile b $MajorNumberFromArchive $MinorNumberFromArchive";
  $ExitCodeOfExternalCommand = system($ExternalCommand);
  if ( $ExitCodeOfExternalCommand != 0 )  {
    &zExternalCommandError(" to make a block device");
    &errorMaybeAbort;
    next;
    }
  &setFilePermissionsGidUid;
  if ( $Verbose == $Yes )  {
    print("$NameOfFile\n");
    }
  next;
  }
if ( $TypeOfFile eq $CharacterDevice )  {
  if ( -e $NameOfFile )  {
    print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
  $ExternalCommand =
#   "'mknod $NameOfFile c $MajorNumberFromArchive $MinorNumberFromArchive'";
   "mknod $NameOfFile c $MajorNumberFromArchive $MinorNumberFromArchive";
  $ExitCodeOfExternalCommand = system($ExternalCommand);
  if ( $ExitCodeOfExternalCommand != 0 )  {
    &zExternalCommandError(" to make a character device");
    &errorMaybeAbort;
    next;
    }
  &setFilePermissionsGidUid;
  if ( $Verbose == $Yes )  {
    print("$NameOfFile\n");
    }
  next;
  }
if ( $TypeOfFile eq $Pipe )  {
  if ( -e $NameOfFile )  {
    print(STDERR "\nkxarc: unable to overwrite $NameOfFile\n");
    &errorMaybeAbort;
    next;
    }
#  $SA = "'mknod $NameOfFile p'";
  $ExternalCommand = "mknod $NameOfFile p";
  $ExitCodeOfExternalCommand = system($ExternalCommand);
  if ( $ExitCodeOfExternalCommand != 0 )  {
    &zExternalCommandError(" to make a pipe");
    &errorMaybeAbort;
    next;
    }
  &setFilePermissionsGidUid;
  if ( $Verbose == $Yes )  {
    print("$NameOfFile\n");
    }
  next;
  }

print(STDERR
   "\nkxarc: skipping $NameOfFile because it is an unknown file type\n");
}  # end of    until ( &endOfArchive )  {
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $UnArchive )  {




if ( $Action == $Compare )  {
if ( $NameOfArchive eq '-' )  {
  if ( $Interactive == $Yes )  {
    print(STDERR "\nkxarc: you cannot use --interactive when reading\n",
         "the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  if ( $SkipCorruptedFiles == $Ask )  {
    print(STDERR "\nkxarc: you cannot use --ask-about-corrupted-files when\n",
         "reading the archive from standard input\n");
    &deleteTmpFilesAndExit(10);
    }
  }
&openArchiveForRead;
L_Compare: until ( &endOfArchive )  {
&getNextFileFromArchive;
if ( $TypeOfFile eq $Index )  {
  print(STDERR '
kxarc: bug detected: sub getNextFileFromArchive returned with $TypeOfFile
set to $Index
');
  &deleteTmpFilesAndExit(40);
  }
if ( $TypeOfFile eq $DataAboutFormatOfArchive )  {
  print(STDERR '
kxarc: bug detected: sub getNextFileFromArchive returned with $TypeOfFile
set to $DataAboutFormatOfArchive
');
  &deleteTmpFilesAndExit(40);
  }
if ( $TypeOfFile eq $ArchiveMessage    or
                 $TypeOfFile eq $ArchivedCommands )  { next }
if ( $TypeOfFile eq $Nothing )  {
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR  "    skipping corrupted or unreadable data in archive\n");
    &errorMaybeAbort;
    }
  next;
  }
#print(STDERR "file type is $TypeOfFile\nfile name is $NameOfFile\n");
if ( $NameOfFile eq ''     and
      ($TypeOfFile eq $BlockDevice    or    $TypeOfFile eq $CharacterDevice
      or    $TypeOfFile eq $Directory    or    $TypeOfFile eq $NormalFile
      or    $TypeOfFile eq $Pipe    or    $TypeOfFile eq $SoftLink) )  {
  print(STDERR "
kxarc: bug detected: unable to get name of ",&fileTypeString," from archive
  file name is $NameOfFile
");
  &deleteTmpFilesAndExit(40);
  }
if ( &includeOrExclude($NameOfFile) == $Exclude )   { next }
if ( $FileIsCorrupted == $Yes    and    $SkipCorruptedFiles == $Yes )  {
  if ( $ExitOnNonFatalErrors == $Yes )  { &deleteTmpFilesAndExit(2) }
    print(STDERR
      "    skipping $NameOfFile\n    because it is corrupted\n");
    next;
    }
if ( $Interactive == $Yes     or
  ($FileIsCorrupted == $Yes    and    $SkipCorruptedFiles == $Ask) )  {
  if ( $FileIsCorrupted == $Yes )
    { print(STDERR
          "compare corrupted ",&fileTypeString," $NameOfFile? y/n/a") }
  else  { print(STDERR "compare $NameOfFile? y/n/a") }
  while ( <STDIN> )  {
    if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
    if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
    if ( $_ eq "n\n"    or    $_ eq "N\n" )   {
       print(STDERR "skipping $NameOfFile\n");
       next L_Compare;
       }
    }
  }
if ( $FileIsCorrupted == $Yes )  {
  print(STDERR "    comparing corrupted ",&fileTypeString," $NameOfFile\n");
  if ( $Interactive == $No    and    $MustSearch == $No )  { &setUnReadMark }
  }
if ( ! -e $NameOfFile )  {
  # if it exists, and if it is a soft link, and if what it links to does not
  # exist; then -e will say it does not exist
  if ( -l $NameOfFile )  {
    if ( $TypeOfFile ne $SoftLink )  {
      print("$NameOfFile is a symbolic link which does not link to anything\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    }
  else  {
    print("$NameOfFile does not exist\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  }
if ( $TypeOfFile eq $SoftLink )  {
  if ( ! -l $NameOfFile )  {
    print("$NameOfFile is not a soft link\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  (undef,undef,undef,undef,$UIDfromFile,$GIDfromFile,undef)
       = lstat($NameOfFile);
  $LinksToFromFile = readlink($NameOfFile);
  if ( $IgnoreUidGid == $No )  {
    if ( $UIDfromFile != $UIDfromArchive )  {
      print("$NameOfFile has a different UID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    if ( $GIDfromFile != $GIDfromArchive )  {
      print("$NameOfFile has a different GID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    }
  if ( $LinksToFromFile ne $LinksToFromArchive )  {
    print("$NameOfFile links to a different file\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $Verbose == $Yes )  { print("$NameOfFile is unchanged\n") }
  next;
  }
if ( -l $NameOfFile )  {
  print("$NameOfFile is not a ",&fileTypeString,"\n");
  $FilesDidNotMatch = $Yes;
  next;
  }
if ( $TypeOfFile eq $NormalFile )  {
  if ( ! -f $NameOfFile )  {
    print("$NameOfFile is not a normal file\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,undef,
        undef,undef,$TimeOfLastChangeFromFile,undef) = stat($NameOfFile);
  if ( $TimeOfLastChangeFromFile != $TimeOfLastChangeFromArchive )  {
    print("$NameOfFile has a different date/time\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  $PermissionsFromFile = $TypePermissions & 0x0FFF;
  if ( $PermissionsFromFile != $PermissionsFromArchive )  {
    print("$NameOfFile has different permissions\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $IgnoreUidGid == $No )  {
    if ( $UIDfromFile != $UIDfromArchive )  {
      print("$NameOfFile has a different UID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    if ( $GIDfromFile != $GIDfromArchive )  {
      print("$NameOfFile has a different GID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    }
  if ( ! open(FH_Compare, "| $CompareCommand '$NameOfFile' > /dev/null") )  {
    print(STDERR "
kxarc: unable to execute
    $CompareCommand '$NameOfFile' \> /dev/null
");
    &deleteTmpFilesAndExit(14);
    }
  while  ( read(FH_CompressionOutputFile,$BigStringData,
                $FileCopyingBlockSize) )
      { print(FH_Compare $BigStringData) }
  $NA = close(FH_Compare);
  if ( $NA == 0    and    $! != 0 )  {
    print(STDERR "
kxarc: error executing
    $CompareCommand '$NameOfFile' \> /dev/null
");
    &deleteTmpFilesAndExit(14);
    }
  if ( $? != 0 )  {
    print("$NameOfFile has different contents\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $Verbose == $Yes )  { print("$NameOfFile is unchanged\n") }
  next;
  }
if ( $TypeOfFile eq $Directory     or     $TypeOfFile eq $Pipe )  {
   if ( $TypeOfFile eq $Directory )  { if ( ! -d $NameOfFile )  {
    print("$NameOfFile is not a directory\n");
    $FilesDidNotMatch = $Yes;
    next;
    }}
  if ( $TypeOfFile eq $Pipe )  { if ( ! -p $NameOfFile )  {
    print("$NameOfFile is not a pipe\n");
    $FilesDidNotMatch = $Yes;
    next;
    }}
  (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,undef)
            = stat($NameOfFile);
  $PermissionsFromFile = $TypePermissions & 0x0FFF;
  if ( $PermissionsFromFile != $PermissionsFromArchive )  {
    print("$NameOfFile has different permissions\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $IgnoreUidGid == $No )  {
    if ( $UIDfromFile != $UIDfromArchive )  {
      print("$NameOfFile has a different UID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    if ( $GIDfromFile != $GIDfromArchive )  {
      print("$NameOfFile has a different GID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    }
  if ( $Verbose == $Yes )  { print("$NameOfFile is unchanged\n") }
  next;
  }
if ( $TypeOfFile eq $BlockDevice    or    $TypeOfFile eq $CharacterDevice )  {
   if ( $TypeOfFile eq $BlockDevice )  { if ( ! -b $NameOfFile )  {
    print("$NameOfFile is not a block device\n");
    $FilesDidNotMatch = $Yes;
    next;
    }}
  if ( $TypeOfFile eq $CharacterDevice )  { if ( ! -c $NameOfFile )  {
    print("$NameOfFile is not a character device\n");
    $FilesDidNotMatch = $Yes;
    next;
    }}
  (undef,undef,$TypePermissions,undef,$UIDfromFile,$GIDfromFile,$DeviceID,
            undef) = stat($NameOfFile);
  $PermissionsFromFile = $TypePermissions & 0x0FFF;
  $MajorNumberFromFile = ($DeviceID >> 8) & 0x00FF;
  $MinorNumberFromFile = $DeviceID & 0x00FF;
  #print(STDERR "major number from file is $MajorNumberFromFile\n");
  #print(STDERR "minor number from file is $MinorNumberFromFile\n");
  if ( $PermissionsFromFile != $PermissionsFromArchive )  {
    print("$NameOfFile has different permissions\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  #if ( ! defined($GIDfromFile) )  { print(STDERR "no from file\n") }
  #if ( ! defined($GIDfromArchive) )  { print(STDERR "no from archive\n") }
  if ( $IgnoreUidGid == $No )  {
    if ( $UIDfromFile != $UIDfromArchive )  {
      print("$NameOfFile has a different UID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    if ( $GIDfromFile != $GIDfromArchive )  {
      print("$NameOfFile has a different GID\n");
      $FilesDidNotMatch = $Yes;
      next;
      }
    }
  if ( $MajorNumberFromFile != $MajorNumberFromArchive )  {
    print("$NameOfFile has a different major number\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $MinorNumberFromFile != $MinorNumberFromArchive )  {
    print("$NameOfFile has a different minor number\n");
    $FilesDidNotMatch = $Yes;
    next;
    }
  if ( $Verbose == $Yes )  { print("$NameOfFile is unchanged\n") }
  next;
  }
print(STDERR
   "\nkxarc: skipping $NameOfFile because it is an unknown file type\n");
}  # end of    until ( &endOfArchive )  {
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Compare )  {





if ( $Action == $Delete )  {
print(STDERR "\nkxarc: action '--delete' has not been coded yet.\n");
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Delete )  {





if ( $Action == $Display )  {
&openArchiveForRead;
until ( &endOfArchive )  {
&getNextFileFromArchive;
if ( $TypeOfFile eq $NormalFile      and
         &includeOrExclude($NameOfFile) == $Include )  {
  while  ( read(FH_CompressionOutputFile,$BigStringData,
                     $FileCopyingBlockSize) )
    { print($BigStringData) }
  }
}  # end of    until ( &endOfArchive )  {
&deleteTmpFilesAndExit(0);
}  # end of    if ( $Action == $Display )  {





print(STDERR '
kxarc: bug detected: action command line parameter is not missing,
    but no action taken
');
&deleteTmpFilesAndExit(40);




sub addFileToTempFileAndCompressAndWriteToArchive  {
  open(FH_CompressionInputFile,"> $CompressionInputFile");
  print(FH_CompressionInputFile $_[0]);
  open(FH_FileToArchive,"< $_[1]");
  while  ( read(FH_FileToArchive,$BigStringData,
                $FileCopyingBlockSize) )
      { print(FH_CompressionInputFile $BigStringData) }
  close(FH_FileToArchive);
  close(FH_CompressionInputFile);
  &compressTempFileAndWriteToArchive;
  }


sub addNonFileToArchive  {
  # would it be better to do this with pipes or backticks instead of
  # temporary files?
  open(FH_CompressionInputFile,"> $CompressionInputFile");
  print(FH_CompressionInputFile $_[0]);
  close(FH_CompressionInputFile);
  &compressTempFileAndWriteToArchive;
  # if verbose, display file name
  if ( $Verbose == $Yes    and    $NameOfFile ne '' )
    { print("$NameOfFile\n") }
  }


sub compressTempFileAndWriteToArchive  {
  my $NA;
  (undef,undef,undef,undef,undef,undef,undef,$SizeOfUncompressedData,undef)
                    = stat($CompressionInputFile);
  if ( $SizeOfUncompressedData <= 0 )   {
    print(STDERR "\nkxarc: bug detected: \$SizeOfUnCompressedData for
$NameOfFile
is less than or equal to 0
");
    if ( $PerlLargeIntegerBugs == $Yes )
           { print(STDERR "maybe the file is too big\n") }
    &deleteTmpFilesAndExit(40);
    }
  $SizeOfUncompressedData = $SizeOfUncompressedData + 4;
  if ( $UncompressionCommand eq '' )
        { return(&writeUncompressedDataToArchive) }
  $ExternalCommand =
    "$CompressionCommand < $CompressionInputFile > $CompressionOutputFile";
  $ExitCodeOfExternalCommand = system($ExternalCommand);
  if ( $ExitCodeOfExternalCommand != 0 )
    { &zExternalCommandError(" to compress data"); &deleteTmpFilesAndExit(13) }
  (undef,undef,undef,undef,undef,undef,undef,$SizeOfCompressedData,undef)
                    = stat($CompressionOutputFile);
  if ( $SizeOfCompressedData <= 0 )   {
    print(STDERR "\nkxarc: bug detected: \$SizeOfCompressedData for
$NameOfFile
is less than or equal to 0
");
    if ( $PerlLargeIntegerBugs == $Yes )
           { print(STDERR "maybe the file is too big\n") }
    &deleteTmpFilesAndExit(40);
    }
  if ( $SizeOfUncompressedData > $SizeOfCompressedData )
    # the compressed data is smaller, so put the compressed data into
    # the archive
    { return(&writeCompressedDataToArchive) }
  else   # the compressed data is NOT smaller, so put the UNCOMPRESSED
    # data into the archive, with a checksum
    { return(&writeUncompressedDataToArchive) }
  }



sub deleteTmpFilesAndExit  {
# for emergency use only, you can configure kxarc to continue instead
# of aborting when bugs are detected by uncommenting the following line:
#if ( $_[0] == 40 )  { return }
unlink($CompressionInputFile,$CompressionOutputFile);
if ( $ExitCodeZeroIfFilesDidNotMatch )  { $FilesDidNotMatch = $No }
if ( $_[0] == 0    and    $NonFatalErrorsOccurred == $Yes    and
              $FilesDidNotMatch == $Yes )  { exit 3 }
if ( $_[0] == 0    and    $NonFatalErrorsOccurred == $Yes )  { exit 2 }
if ( $_[0] == 0    and    $FilesDidNotMatch == $Yes )  { exit 1 }
exit $_[0];
}


sub endOfArchive  {
if ( $FileIsCorrupted == $Yes )  { return($Failure) }
if ( (length($UnReadArchiveData) - $SizeOfDataReadSinceUnReadMark) == 0
       and     eof(FH_Archive) )   { return($Success) }
else  { return($Failure) }
}



sub errorMaybeAbort  {
if ( $Interactive == $Yes )  {
  print(STDERR
    "press   a [enter]   to abort kxarc, press any other key to continue\n");
  $SA = <STDIN>;
  if ( $SA eq "a\n"    or    $SA eq "A\n" )  { &deleteTmpFilesAndExit(2) }
  }
elsif ( $ExitOnNonFatalErrors == $Yes ) { &deleteTmpFilesAndExit(2) }
$NonFatalErrorsOccurred = $Yes;
}



sub executeCommandsInArchive  {
my @Commands = (); my $SA; my $SB; my $NA;
#print(STDERR "starting sub executeCommandsInArchive\n");
if ( $Interactive == $No    and    $IgnoreCommands == $Yes )
    { return($Success) }
if ( $FileIsCorrupted == $Yes    and    $SkipCorruptedFiles == $Yes
     and     $Interactive == $No )  {
  print(STDERR '
kxarc: skipping corrupted commands in archive
');
  return($Failure);
  }
#print(STDERR "starting until\n");
until ( $BigStringData eq '' )  {
  $NA = index($BigStringData,$CommandSeperator);
  #print(STDERR "\$NA is $NA\n");
  if ( $NA == -1 )  { last }
  #print(STDERR "not -1\n");
  $SA = substr($BigStringData,0,$NA);
  if ( $SA ne '' )  { @Commands = (@Commands,$SA) }
  $BigStringData = substr($BigStringData,($NA+1));
  }
#print(STDERR "done until\n");
if ( $BigStringData ne '' )
      { @Commands = (@Commands,$BigStringData) }
for ( @Commands )  {
  if ( substr($_,0,2) eq 'e ' )  {
    $ExternalCommand = substr($_,2);
    if ( $Interactive == $Yes )  {
      print(STDERR "
kxarc: The archive contains the following external command:
    $ExternalCommand
    execute this command? y/n/a
");
      while ( <STDIN> )  {
        if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
        if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
        if ( $_ eq "n\n"    or    $_ eq "N\n" )  { next }
        }
      }
    $ExitCodeOfExternalCommand = system($ExternalCommand);
    if ( $ExitCodeOfExternalCommand != 0 )
      { &zExternalCommandError(""); &errorMaybeAbort }
    next;
    }
  else  {
    print(STDERR "
kxarc: ignoring unknown command
$SA
");
    next;
    }
  }
}



sub fileTypeString  {
if ( $TypeOfFile eq $NormalFile )  { return('normal file') }
if ( $TypeOfFile eq $Directory )  { return('directory') }
if ( $TypeOfFile eq $SoftLink )  { return('soft link') }
if ( $TypeOfFile eq $BlockDevice )  { return('block device') }
if ( $TypeOfFile eq $CharacterDevice )  { return('character device') }
if ( $TypeOfFile eq $Pipe )  { return('pipe') }
if ( $TypeOfFile eq $ArchiveMessage )  { return('archive message') }
if ( $TypeOfFile eq $DataAboutFormatOfArchive )
   { return('data about format of archive') }
if ( $TypeOfFile eq $Index )  { return('archive index') }
if ( $TypeOfFile eq $ArchivedCommands )  { return('archive commands') }
if ( $TypeOfFile eq $Nothing )  { return('no file type') }
return("unknown file type \'$TypeOfFile\', ASCII ", ord($TypeOfFile));
}



sub getNextFileFromArchive  {
#print(STDERR "getNext\n");
$PreviousFileWasCorrupted = $FileIsCorrupted;
$FileIsCorrupted = $No;
if ( $Interactive == $Yes    and    $PreviousFileWasCorrupted == $Yes
        and    $MustSearch == $No )  {
  print(STDERR "
kxarc: search corrupted ",&fileTypeString," for other files? y/n/a
");
  while ( <STDIN> )  {
    if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
    if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
    if ( $_ eq "n\n"    or    $_ eq "N\n" )  { &setUnReadMark; last }
    }
  }
if ( $PreviousFileWasCorrupted == $Yes )  { &unReadDataSinceUnReadMark }
$MustSearch = $No;
$TypeOfFile = $Nothing;
$NameOfFile = '';
close(FH_CompressionOutputFile);
if ( $UsingIndex == $No )  { return(&getNextNoIndex) }
return(&getNextIndex);
}



sub getNextIndex  {  # not ready
return($Failure);
}



sub getNextNoIndex  {
my $SA;
#print(STDERR "starting sub getNextNoIndex\n");
if ( &endOfArchive )  { return($Failure) }
&readOutsideMarker;
&setUnReadMark;
$SizeOfArchiveData = &readSizeOfDataFromArchive;
if ( &endOfArchive )  { return($Failure) }
if ( $SizeOfArchiveData == 0 )  { return($Failure) }
$ArchiveDataLeftToRead = $SizeOfArchiveData;
#print(STDERR "reading size of $ArchiveDataLeftToRead\n");
# if $ArchiveDataLeftToRead is not a reasonable number, then we have a
# corrupted archive, and we should unread $SA
# and return with nothing, and let it search the archive for the next
# $OutsideMarker.
# The minimum size is $InsideMarkerLength plus one byte for the file
# type plus four bytes for the checksum.
# The maximum size is the remaining size of the archive, but it is hard
# to figure out how big the archive is, especially if the archive is
# a pipe. And if the archive is truncated, then $ArchiveDataLeftToRead
# might be larger than the remaining size of the archive. So there is
# no maximum value for $ArchiveDataLeftToRead
if ( $ArchiveDataLeftToRead < 0 )   { print(STDERR '
kxarc: bug detected: $ArchiveDataLeftToRead from archive is less than 0
');
    if ( $PerlLargeIntegerBugs == $Yes )
           { print(STDERR "maybe the file is too big\n") }
    &deleteTmpFilesAndExit(40);
    }
if ( $ArchiveDataLeftToRead < ($InsideMarkerLength + 5) )  {
  &unread($SA);
  return($Failure);
  }
&readFromArchive($SA,$InsideMarkerLength);
if ( $SA eq $InsideMarker )  { return(&readUncompressedDataFromArchive) }
else  { return(&readCompressedDataFromArchive) }
}


sub includeOrExclude  {
if ( $UseFileList == $No )  { return($Include) }
if ( $_[0] eq '' )  { return($Exclude) }
for ( @FileList )  { return($Include)   if   ($_ eq $_[0]) }
return($Exclude);
}


sub makeParentDirectoriesIfNeeded {
my @A = (); my $NA = $Success; my $SA = $_[0];
#print(STDERR "sub makeParentDirectoriesIfNeeded called with $_[0]\n");
until ( $SA = substr($SA,0, rindex($SA,'/') - length($SA)),
          $SA eq ''    or    -d $SA )
    { @A = ($SA,@A) }
#print(STDERR @A);
for (@A) { $NA = mkdir($_,0755) }
                   # '0755' should give permissions of rwxr-xr-x
#print(STDERR "exit code of mkdir is $NA\n");
return($NA);
}



sub openArchiveForRead  {
if ( ! open(FH_Archive,"< $NameOfArchive") ) {
  if ( ! -e $NameOfArchive )  {
    print(STDERR "\nkxarc: unable to find archive $NameOfArchive\n");
    &deleteTmpFilesAndExit(11);
    }
  print(STDERR "\nkxarc: unable to read from archive $NameOfArchive\n");
  &deleteTmpFilesAndExit(11);
  }
if ( -f $NameOfArchive      and      $NameOfArchive ne '-' )
    { $ArchiveIsNormalFile = $Yes }
elsif ( $SizeIfTrueSizeUnknown > $MaxSizeUnReadArchiveData )
    { $SizeIfTrueSizeUnknown = $MaxSizeUnReadArchiveData }
}



sub packUidGid  {
if ( $IgnoreUidGid )  { return(&VunsignedInteger(0) . &VunsignedInteger(0)) }
return(&VunsignedInteger($UIDfromFile) . &VunsignedInteger($GIDfromFile));
}



sub parseIndex  {
# the index data is in $BigStringData; add code here to sort the data;
# set $UsingIndex to $Yes
return($Failure);
}



sub readCompressedDataFromArchive  {
my $SA; my $NA;
if ( $UncompressionCommand eq '' )  {
    print(STDERR "
kxarc: inside marker not found, so file should be compressed; but there is no
uncompression command. This probably means an unknown file is corrupted.
kxarc can not determine the file name or file type.
");
    $TypeOfFile = $Nothing;
    $FileIsCorrupted = $Yes;
    return($Failure);
    }
&unReadNumberOfBytes($InsideMarkerLength);
open(FH_CompressionInputFile,"> $CompressionInputFile");
  while ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
    &readFromArchive($BigStringData,$FileCopyingBlockSize);
    print(FH_CompressionInputFile $BigStringData);
    $ArchiveDataLeftToRead = $ArchiveDataLeftToRead -
                              $FileCopyingBlockSize;
    }
  &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
  print(FH_CompressionInputFile $BigStringData);
  close(FH_CompressionInputFile);
$ExternalCommand =
  "$UncompressionCommand < $CompressionInputFile > $CompressionOutputFile";
$ExitCodeOfExternalCommand = system($ExternalCommand);
if ( -z $CompressionOutputFile ) {
  print(STDERR "
kxarc: the uncompression command produced no output. This probably means an
unknown file is corrupted, and may be bigger than it appears to be.
");
  $FileIsCorrupted = $Yes;
  if ( $SizeOfArchiveData >= $SizeIfTrueSizeUnknown )   { return($Failure) }
  if ( $Interactive == $Yes )  {
    print(STDERR "Try a bigger size? y/n/a\n");
    while ( <STDIN> )  {
      if ( $_ eq "a\n"    or    $_ eq "A\n" )  { &deleteTmpFilesAndExit(0) }
      if ( $_ eq "y\n"    or    $_ eq "Y\n" )  { last }
      if ( $_ eq "n\n"    or    $_ eq "N\n" )  { return($Failure) }
      }
    }
  if ( $Interactive == $No )  { print(STDERR
"kxarc will try again with a bigger size.
") }
  print(STDERR 
"kxarc will assume that this file is corrupted; but if no additional
uncompression errors occur, then maybe only the size is corrupted;
the file might be okay.
");
  $MustSearch = $Yes;
  $ArchiveDataLeftToRead = $SizeIfTrueSizeUnknown - $SizeOfArchiveData;
  open(FH_CompressionInputFile,">> $CompressionInputFile");
    while ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
      &readFromArchive($BigStringData,$FileCopyingBlockSize);
      print(FH_CompressionInputFile $BigStringData);
      $ArchiveDataLeftToRead = $ArchiveDataLeftToRead -
                              $FileCopyingBlockSize;
      }
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    print(FH_CompressionInputFile $BigStringData);
    close(FH_CompressionInputFile);
  $ExitCodeOfExternalCommand = system($ExternalCommand);
  if ( -z $CompressionOutputFile ) {
    print(STDERR "
kxarc: the following external command:
$ExternalCommand
produced no output and returned an exit code of: ",
                     ($ExitCodeOfExternalCommand/256),"
(perl system() returned: $ExitCodeOfExternalCommand)
This probably means an unknown file is corrupted. kxarc can not determine
the file name or file type
");
    $FileIsCorrupted = $Yes;
    return($Failure);
    }
  }
if ( $ExitCodeOfExternalCommand != 0     and     $FileIsCorrupted == $No )  {
    &zExternalCommandError(" to compress data");
    print(STDERR "this file is probably corrupted\n");
    $FileIsCorrupted = $Yes;
    }
open(FH_CompressionOutputFile,"< $CompressionOutputFile");
read(FH_CompressionOutputFile,$SA,$InsideMarkerLength);
if ( $SA ne $InsideMarker )  {
    print(STDERR "
kxarc: the uncompressed data does not begin with '$InsideMarker'
This probably means an unknown file is corrupted. kxarc can not determine
the file name or file type.
");
    $FileIsCorrupted = $Yes;
    return($Failure);
    }
read(FH_CompressionOutputFile,$TypeOfFile,1);
if  ( $TypeOfFile eq $NormalFile )  {
  read(FH_CompressionOutputFile,$SA,256);
  ($HeaderSize,$NA) = &UnpackVunsignedInteger($SA,0,1);
  if ( ! defined($NA) )  { &zNoHeaderSize; return($Failure) }
  if ( $HeaderSize > $MaxHeaderSize )  { &zHeaderTooBig; return($Failure) }
  seek(FH_CompressionOutputFile,($NA - length($SA)),1);
  read(FH_CompressionOutputFile,$SA,$HeaderSize);
  $PermissionsFromArchive = unpack('S', $SA);
  $SA = substr($SA,2);
  &unPackUidGid($SA);
  ($NameOfFile,$NA) = &UnpackVstring($SA,0,1);
  if ( defined($NA) )  { $SA = substr($SA,$NA) }
  else  { &zNoNameInHeader($SA) }
  $TimeOfLastChangeFromArchive = unpack('L',substr($SA,1,4));
  if ( length($SA) > 5 )  { print(STDERR "
kxarc: archive header for normal file
$NameOfFile
includes additional unknown data. kxarc is assuming that this means the
file was archived by a more advanced version of kxarc. This additional
data is being ignored. If this file was not archived by a more advanced
version of kxarc, then this file is corrupted.
") }
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR
        "file type is ",&fileTypeString,"; file name is '$NameOfFile'\n");
    &errorMaybeAbort;
    }
  return($Success);
  }
elsif  ( $TypeOfFile eq $Directory     or     $TypeOfFile eq $SoftLink     or
     $TypeOfFile eq $BlockDevice      or      $TypeOfFile eq $CharacterDevice
     or      $TypeOfFile eq $Pipe )  {
  read(FH_CompressionOutputFile,$SA,$FileCopyingBlockSize);
  if ( ! eof(FH_CompressionOutputFile) )  {
    print(STDERR "
kxarc: ",&fileTypeString," being read from archive is as big as
\$FileCopyingBlockSize or bigger. Reading first $FileCopyingBlockSize
bytes only.
");
    $FileIsCorrupted = $Yes;
    }
  if ( $TypeOfFile ne $SoftLink )  {
    $PermissionsFromArchive = unpack('S', $SA);
    $SA = substr($SA,2);
    }
  &unPackUidGid($SA);
  ($NameOfFile,$NA) = &UnpackVstring($SA,0,1);
  if ( defined($NA) )  { $SA = substr($SA,$NA) }
  else  { &zNoNameInHeader($SA) }
  if ( $TypeOfFile eq $SoftLink )  {
    ($LinksToFromArchive,$NA) = &UnpackVstring($SA,0,1);
    if ( defined($NA) )  { $SA = substr($SA,$NA) }
    else  { $LinksToFromArchive = $SA; $SA = '' }
    }
  if ( $TypeOfFile eq $BlockDevice      or
      $TypeOfFile eq $CharacterDevice )  {
    ($MajorNumberFromArchive, $MinorNumberFromArchive) = unpack('CC', $SA);
    #print(STDERR "major number from archive is $MajorNumberFromArchive\n");
    #print(STDERR "minor number from archive is $MinorNumberFromArchive\n");
    $SA = substr($SA,2);
    }
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR
        "file type is ",&fileTypeString,"; file name is '$NameOfFile'\n");
    &errorMaybeAbort;
    }
  if ( length($SA) > 0 )  { print(STDERR "
kxarc: ",&fileTypeString,"
$NameOfFile
includes additional unknown data. kxarc is assuming that this means it
was archived by a more advanced version of kxarc. This additional
data is being ignored. If it was not archived by a more advanced
version of kxarc, then it is corrupted.
") }
  return($Success);
  }
elsif  ( $TypeOfFile eq $ArchiveMessage     or     $TypeOfFile eq $Index     or
         $TypeOfFile eq $ArchivedCommands  )  {
  read(FH_CompressionOutputFile,$BigStringData,$FileCopyingBlockSize);
  if ( ! eof(FH_CompressionOutputFile) )  {
    print(STDERR "
kxarc: ",&fileTypeString," being read from archive is as big as
\$FileCopyingBlockSize or bigger. Reading first $FileCopyingBlockSize
bytes only.
");
    $FileIsCorrupted = $Yes;
    &errorMaybeAbort;
    }
  if ( $TypeOfFile eq $Index )  {
    if ( $FileIsCorrupted == $Yes )  {
      print(STDERR "ignoring corrupted index\n");
      $TypeOfFile = $Nothing;
      $FileIsCorrupted = $No;
      return($Success);
      }
    &parseIndex;
    }
  return($Success);
  }
elsif ( $TypeOfFile eq $DataAboutFormatOfArchive )  {
  print(STDERR '
kxarc: bug detected: uncompression command found compressed in archive
');
  &deleteTmpFilesAndExit(40);
  }
else  {
  print(STDERR "\nkxarc: skipping unknown compressed file\n");
  $TypeOfFile = $Nothing;
  return($Success);
  }
print(STDERR '
kxarc: bug detected: at end of sub readCompressedDataFromArchive
without return
');
&deleteTmpFilesAndExit(40);
}



sub readFromArchive  {
# $_[0] is the string to write data to
# $_[1] is the number of bytes to read
my $NA;
if ( $_[1] == 0 )  { $_[0] = ''; return($Success) }
if ( $_[1] < 0 )  {
  print(STDERR '
kxarc: bug detected: sub readFromArchive called to read fewer than 0 bytes
');
  &deleteTmpFilesAndExit(40);
  }
if ( $_[1] > $MaxSizeUnReadArchiveData )  {
  print(STDERR '
kxarc: bug detected: sub readFromArchive called to read more
than $MaxSizeUnReadArchiveData
');
  &deleteTmpFilesAndExit(40);
  }
# if possible, we do this the easy way:
# if the archive is a normal file, we do not have to save data for possible
# future unreads, we can just save the seek count
if ( $ArchiveIsNormalFile == $Yes )  {
  if ( length($UnReadArchiveData) != 0 )  {
    print(STDERR '
kxarc: bug detected: $UnReadArchiveData is in use when archive is a
normal file
');
    &deleteTmpFilesAndExit(40);
    }
  $NA = read(FH_Archive,$_[0],$_[1]);
  $PossibleUnReadCount = $PossibleUnReadCount - $NA;
  return($Success);
  }
# otherwise we do this the hard way:
# if the archive is not a normal file, then we need to check for data
# which has been read but then unread before reading from the archive;
# and we need to save the data read since the last &setUnReadMark
# for possible future unreads.
# Since we may be transferring data from the read but unread archive data to
# the data read since &setUnReadMark, and then back again if we do a unread,
# these two data are combined into one string called
# $UnReadArchiveData. The data read since &setUnReadMark
# is the first part of the string. The read but unread
# data is the last part of the string. $SizeOfDataReadSinceUnReadMark gives
# the number of bytes of data data read since &setUnReadMark;
# all the rest of $UnReadArchiveData is data which has been read but unread.
# To move data from one to the other, we just change
# $SizeOfDataReadSinceUnReadMark. This should be faster and use memory
# more efficiently than having two seperate strings.
if ( $PossibleUnReadCount != 0 )  {
  print(STDERR '
kxarc: bug detected: $PossibleUnReadCount is in use when archive is not
a normal file
');
  &deleteTmpFilesAndExit(40);
  }
$SizeOfUnReadArchiveData = length($UnReadArchiveData);
$AvailableUnReadData = $SizeOfUnReadArchiveData -
                       $SizeOfDataReadSinceUnReadMark;
if ( $AvailableUnReadData < 0 )  {
    print(STDERR '
kxarc: bug detected: $AvailableUnReadData is less than 0
');
    &deleteTmpFilesAndExit(40);
    }
if ( $AvailableUnReadData == 0 )  {
  # no unread data available, so just read from the archive, and add the
  # data to $UnReadArchiveData
  # but first make sure $UnReadArchiveData will not become too big.
  $NA = ($SizeOfUnReadArchiveData + $_[1]) - $MaxSizeUnReadArchiveData;
  if ( $NA > 0 )  {
    $UnReadArchiveData = substr($UnReadArchiveData,$NA);
    $SizeOfDataReadSinceUnReadMark = $SizeOfDataReadSinceUnReadMark - $NA;
    }
  read(FH_Archive,$_[0],$_[1]);
  $UnReadArchiveData = $UnReadArchiveData . $_[0];
  $SizeOfDataReadSinceUnReadMark = length($UnReadArchiveData);
  return($Success);
  }
if ( $AvailableUnReadData >= $_[1] )  {
  # read from data which has been read and then unread
  $_[0] = substr($UnReadArchiveData,$SizeOfDataReadSinceUnReadMark,$_[1]);
  $SizeOfDataReadSinceUnReadMark = $SizeOfDataReadSinceUnReadMark + $_[1];
  return($Success);
  }
if ( $AvailableUnReadData < $_[1] )  {
  # read part from unread data, and part from the archive
  my $SA;
  $BytesToReadFromArchive = $_[1] - $AvailableUnReadData;
  $NA = ($SizeOfUnReadArchiveData + $BytesToReadFromArchive)
         - $MaxSizeUnReadArchiveData;
  if ( $NA > 0 )  {
    if ( $NA > $SizeOfDataReadSinceUnReadMark )
       { $NA = $SizeOfDataReadSinceUnReadMark }
    $UnReadArchiveData = substr($UnReadArchiveData,$NA);
    $SizeOfDataReadSinceUnReadMark = $SizeOfDataReadSinceUnReadMark - $NA;
    }
  read(FH_Archive,$SA,$BytesToReadFromArchive);
  $UnReadArchiveData = $UnReadArchiveData . $SA;
  $_[0] = substr($UnReadArchiveData,$SizeOfDataReadSinceUnReadMark);
  $SizeOfDataReadSinceUnReadMark = length($UnReadArchiveData);
  return($Success);
  }
}


sub readOutsideMarker  {
my $SA; my $NA = -1;
&readFromArchive($SA,$OutsideMarkerLength);
if ( $SA eq $OutsideMarker )  { return($Success) }
if ( $PreviousFileWasCorrupted = $No )  {
  print(STDERR '
kxarc: $OutsideMarker not found where expected: archive is probably corrupted
');
  &errorMaybeAbort;
  }
until ( $NA != -1    or    &endOfArchive )  {
  &unReadNumberOfBytes($OutsideMarkerLength - 1);
  &readFromArchive($SA,$FileCopyingBlockSize);
  $NA = index($SA,$OutsideMarker);
  }
if ( $NA == -1 )  { return($Failure) }
# &unReadNumberOfBytes(length($SA) + $OutsideMarkerLength - $NA);
&unReadNumberOfBytes(length($SA) - $NA);
&readFromArchive($SA,$OutsideMarkerLength);
if ( $SA ne $OutsideMarker )  {
      print(STDERR "
kxarc: bug detected: index() found '$OutsideMarker',
but it was not '$OutsideMarker'
");
      &deleteTmpFilesAndExit(40);
      }
}



sub readSizeOfDataFromArchive  {
my $SA; my $NA; my $NB;
&readFromArchive($SA,256);
($NA,$NB) = &UnpackVunsignedInteger($SA,0,1);
if ( ! defined($NB) )  {
  &unReadNumberOfBytes(length($SA));
  return(0);
  }
&unReadNumberOfBytes(length($SA) - $NB);
return($NA);
}




sub readUncompressedDataFromArchive  {
my $SA; my $SB; my $NA; my $NB;
my $DataForChecksum = $InsideMarker;
$ArchiveDataLeftToRead = $ArchiveDataLeftToRead - $InsideMarkerLength;
&readFromArchive($TypeOfFile,1);
#print(STDERR "type is ",&fileTypeString,"\n");
$DataForChecksum = $DataForChecksum . $TypeOfFile;
$ArchiveDataLeftToRead = $ArchiveDataLeftToRead - 5;
# subtract 1 because we just read 1 byte, and subtract 4 because
# we do not want to read the 4 byte checksum yet.
if  ( $TypeOfFile eq $NormalFile )  {
  #print(STDERR "normal file\n");
  &readFromArchive($SA,256);
  ($HeaderSize,$NA) = &UnpackVunsignedInteger($SA,0,1);
  if ( ! defined($NA) )  { &zNoHeaderSize; return($Failure) }
  if ( $HeaderSize > $MaxHeaderSize )  { &zHeaderTooBig; return($Failure) }
  &unReadNumberOfBytes(length($SA) - $NA);
  $ArchiveDataLeftToRead = $ArchiveDataLeftToRead - $NA;
  $DataForChecksum = $DataForChecksum . substr($SA,0,$NA);
  &readFromArchive($SA,$HeaderSize);
  $DataForChecksum = $DataForChecksum . $SA;
  $PermissionsFromArchive = unpack('S', $SA);
  $SA = substr($SA,2);
  &unPackUidGid($SA);
  ($NameOfFile,$NA) = &UnpackVstring($SA,0,1);
  if ( defined($NA) )  { $SA = substr($SA,$NA) }
  else  { &zNoNameInHeader($SA) }
  $TimeOfLastChangeFromArchive = unpack('L',substr($SA,1,4));
  if ( length($SA) > 5 )  { print(STDERR "
kxarc: archive header for normal file
$NameOfFile
includes additional unknown data. kxarc is assuming that this means the
file was archived by a more advanced version of kxarc. This additional
data is being ignored. If this file was not archived by a more advanced
version of kxarc, then this file is corrupted.
") }
  $ArchiveDataLeftToRead = $ArchiveDataLeftToRead - $HeaderSize;
  if ( $ArchiveDataLeftToRead < 0 )  {
    print(STDERR "
kxarc: size of header of normal file
$NameOfFile
is bigger than total size of header plus file contents; archive is probably
corrupted. kxarc can not unarchive this normal file because kxarc can not
tell where it ends.
");
    $FileIsCorrupted = $Yes;
    $TypeOfFile = $Nothing;
    return($Failure);
    }
  open(FH_CompressionOutputFile,"> $CompressionOutputFile");
  if ( ($FileCopyingBlockSize + 3) >= ($InsideMarkerLength + 3 +
       $HeaderSize + $ArchiveDataLeftToRead) )  {  # file is small
    # enough to fit the whole header + file in memory at once
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    print(FH_CompressionOutputFile $BigStringData);
    $CalculatedChecksum = unpack("%32L*",
            ($DataForChecksum . $BigStringData . "\0\0\0"));
    }
  else  {  # file is too big to fit in memory at once; we will have to
    # read it and calculate the checksum in pieces
    $CalculatedChecksum = 0;
    $NA = length($DataForChecksum) % 4;
    if ( $NA != 0 )  {  # size of $DataForChecksum is not a multiple of 4,
      # read 1,2, or 3 bytes from the archive to make it a multiple of 4
      $NA = 4 - $NA;
      &readFromArchive($BigStringData,$NA);
      print(FH_CompressionOutputFile $BigStringData);
      $DataForChecksum = $DataForChecksum . $BigStringData;
      $ArchiveDataLeftToRead = $ArchiveDataLeftToRead - $NA;
      }
    $CalculatedChecksum =
            $CalculatedChecksum + unpack("%32L*",$DataForChecksum);
    if ( $ArchiveDataLeftToRead <= 0 )  {
      print(STDERR '
kxarc: bug detected: $ArchiveDataLeftToRead is less than or equal to 0
');
      &deleteTmpFilesAndExit(40);
      }
    while ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
      &readFromArchive($BigStringData,$FileCopyingBlockSize);
      $ArchiveDataLeftToRead = $ArchiveDataLeftToRead
                                - $FileCopyingBlockSize;
      print(FH_CompressionOutputFile $BigStringData);
      $CalculatedChecksum =
           $CalculatedChecksum + unpack("%32L*",$BigStringData);
      }
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    print(FH_CompressionOutputFile $BigStringData);
    $CalculatedChecksum =
           $CalculatedChecksum + unpack("%32L*","$BigStringData\0\0\0");
    }
  close(FH_CompressionOutputFile);
  &readFromArchive($SA,4);
  $ChecksumFromArchive = unpack('L',$SA);
  if ( $CalculatedChecksum != $ChecksumFromArchive )  {
    print(STDERR "
kxarc: normal file in archive
$NameOfFile
has an invalid checksum, archive is probably corrupted
");
    $FileIsCorrupted = $Yes;
    &errorMaybeAbort;
    }
  open(FH_CompressionOutputFile,"< $CompressionOutputFile");
  return($Success);
  }
elsif  ( $TypeOfFile eq $Directory     or     $TypeOfFile eq $SoftLink     or
     $TypeOfFile eq $BlockDevice      or      $TypeOfFile eq $CharacterDevice
     or      $TypeOfFile eq $Pipe )  {
  #print(STDERR "nonfile\n");
  if ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
    print(STDERR "
kxarc: ",&fileTypeString," being read from archive is as big as
\$FileCopyingBlockSize or bigger.
reading first $FileCopyingBlockSize bytes only and ignoring checksum
");
    $FileIsCorrupted = $Yes;
    &readFromArchive($BigStringData,$FileCopyingBlockSize);
    $CalculatedChecksum = 0;
    $ChecksumFromArchive = 0;
    }
  else  {
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    $CalculatedChecksum = unpack("%32L*",
          ($DataForChecksum . $BigStringData . "\0\0\0"));
    &readFromArchive($SB,4);
    $ChecksumFromArchive = unpack('L',$SB);
    #print(STDERR "checksum from archive is $ChecksumFromArchive\n");
    #print(STDERR "checksum is $CalculatedChecksum for\n$DataForChecksum$SA\n");
    }
  if ( $TypeOfFile ne $SoftLink )  {
    $PermissionsFromArchive = unpack('S', $BigStringData);
    $BigStringData = substr($BigStringData,2);
    }
  &unPackUidGid($BigStringData);
  ($NameOfFile,$NA) = &UnpackVstring($BigStringData,0,1);
  if ( defined($NA) )  { $BigStringData = substr($BigStringData,$NA) }
  else  { &zNoNameInHeader($BigStringData) }
  if ( $TypeOfFile eq $SoftLink )  {
    ($LinksToFromArchive,$NA) = &UnpackVstring($BigStringData,0,1);
    if ( defined($NA) )  { $BigStringData = substr($BigStringData,$NA) }
    else  { $LinksToFromArchive = $BigStringData; $BigStringData = '' }
    }
  if ( $TypeOfFile eq $BlockDevice      or
      $TypeOfFile eq $CharacterDevice )  {
    ($MajorNumberFromArchive, $MinorNumberFromArchive)
               = unpack('CC', $BigStringData);
    #print(STDERR "major number from archive is $MajorNumberFromArchive\n");
    #print(STDERR "minor number from archive is $MinorNumberFromArchive\n");
    $BigStringData = substr($BigStringData,2);
    }
  if ( $CalculatedChecksum != $ChecksumFromArchive )  {
    print(STDERR "\nkxarc: invalid checksum: file is probably corrupted\n");
    $FileIsCorrupted = $Yes;
    }
  if ( $FileIsCorrupted == $Yes )  {
    print(STDERR
        "file type is ",&fileTypeString,"; file name is '$NameOfFile'\n");
    &errorMaybeAbort;
    }
  if ( length($BigStringData) > 0 )  { print(STDERR "
kxarc: ",&fileTypeString,"
$NameOfFile
includes additional unknown data. kxarc is assuming that this means it
was archived by a more advanced version of kxarc. This additional
data is being ignored. If it was not archived by a more advanced
version of kxarc, then it is corrupted.
") }
  return($Success);
  }
elsif ( $TypeOfFile eq $Index     or     $TypeOfFile eq $ArchivedCommands
          or      $TypeOfFile eq $ArchiveMessage )  {
  #print(STDERR "IMC\n");
  if ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
    print(STDERR "
kxarc: ",&fileTypeString," being read from archive is as big as
\$FileCopyingBlockSize or bigger.
reading first $FileCopyingBlockSize bytes only and ignoring checksum
");
    $FileIsCorrupted = $Yes;
    &readFromArchive($BigStringData,$FileCopyingBlockSize);
    $CalculatedChecksum = 0;
    $ChecksumFromArchive = 0;
    }
  else  {
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    $CalculatedChecksum = unpack("%32L*",
             ($DataForChecksum . $BigStringData . "\0\0\0"));
    &readFromArchive($SB,4);
    $ChecksumFromArchive = unpack('L',$SB);
    #print(STDERR "checksum from archive is $ChecksumFromArchive\n");
    #print(STDERR "checksum is $CalculatedChecksum for\n$DataForChecksum$SA\n");
    }
  if ( $CalculatedChecksum != $ChecksumFromArchive )  {
    print(STDERR
      "\nkxarc: ",&fileTypeString," in archive has invalid checksum\n");
    &errorMaybeAbort;
    $FileIsCorrupted = $Yes;
    }
  if ( $TypeOfFile eq $Index )  {
    if ( $FileIsCorrupted == $Yes )  {
      print(STDERR "ignoring corrupted index\n");
      $TypeOfFile = $Nothing;
      $FileIsCorrupted = $No;
      return($Success);
      }
    &parseIndex;
    }
  return($Success);
  }
elsif ( $TypeOfFile eq $DataAboutFormatOfArchive )  {
  #print(STDERR "format data\n");
  if ( $ArchiveDataLeftToRead > $FileCopyingBlockSize )  {
    print(STDERR "
kxarc: ",&fileTypeString," being read from archive is as big as
\$FileCopyingBlockSize or bigger.
reading first $FileCopyingBlockSize bytes only and ignoring checksum
");
    $FileIsCorrupted = $Yes;
    &readFromArchive($BigStringData,$FileCopyingBlockSize);
    $CalculatedChecksum = 0;
    $ChecksumFromArchive = 0;
    }
  else  {
    &readFromArchive($BigStringData,$ArchiveDataLeftToRead);
    $CalculatedChecksum = unpack("%32L*",
             ($DataForChecksum . $BigStringData . "\0\0\0"));
    &readFromArchive($SB,4);
    $ChecksumFromArchive = unpack('L',$SB);
    #print(STDERR "checksum from archive is $ChecksumFromArchive\n");
    #print(STDERR "checksum is $CalculatedChecksum for\n$DataForChecksum$SA\n");
    }
  ($SA,$NA) = &UnpackVstring($BigStringData,0,1);
# since the data about the format of the archive is needed to unarchive
# everything, an error here could be considered a fatal error.
# However, kxarc is supposed to be able to cope with errors; if there
# is an error in one part of the archive, kxarc is supposed to be able
# to unarchive files in other parts of the archive. Therefore, any one
# with half a brain will realize that constantly changing the archive
# format is a *BAD IDEA*, even though kxarc allows it; unusual archive
# formats should be used only in special situations. Always use the same
# $UncompressionCommand for all your backups.
# Therefore, if the data about the format of the archive is corrupted,
# kxarc uses the default data and continues, on the assumption that
# the data in the archive is the same as the default data.
  if ( ! defined($NA) )  { &zUnpackUncompressionCommandFailed }
  elsif ( $CalculatedChecksum != $ChecksumFromArchive )
    { &zDataAboutFormatOfArchiveFailedChecksum($SA) }
  else  {
    if ( length($BigStringData) > $NA )  { print(STDERR '
kxarc: data about format of archive
includes additional unknown data. kxarc is assuming that this means it
was archived by a more advanced version of kxarc. This additional
data is being ignored. If it was not archived by a more advanced
version of kxarc, then it is corrupted.
') }
    if ( $IgnoreDataAboutFormatOfArchive == $No )
        { $UncompressionCommand = $SA }
    }
  #print(STDERR "file type is format data\n");
  $TypeOfFile = $Nothing;
  #print(STDERR "file type is nothing\n");
  return($Success);
  }
else  {
  print(STDERR "\nkxarc: skipping unknown uncompressed file\n");
  $TypeOfFile = $Nothing;
  return($Success);
  }
print(STDERR '
kxarc: bug detected: at end of sub readUncompressedDataFromArchive
without return
');
&deleteTmpFilesAndExit(40);
}



sub setFilePermissionsGidUid  {
# my first thought was to change permissions first, then change owner;
# because maybe on some systems once you have changed the owner, you might
# no longer have permission to change permissions, etc. But on most systems
# you are not allowed to change the owner to something you do not have
# permission to change back, so it probably does not matter. However, if a
# file has setuid permission, then perl 5.004_04 chown() removes setuid
# permission. So to handle setuid permission properly, we must change owner
# first, then change permissions.
#my $A;(undef,undef,$A)=stat($NameOfFile);print(STDERR"TypePermissions are $A\n");
if ( $IgnoreUidGid == $No )  {
  #print(STDERR "changing UID of $NameOfFile to $UIDfromArchive\n");
  #print(STDERR "changing GID of $NameOfFile to $GIDfromArchive\n");
  if ( ! chown($UIDfromArchive,$GIDfromArchive,$NameOfFile) )  {
    print(STDERR "\nkxarc: unable to change owner and group of $NameOfFile\n");
    &errorMaybeAbort;
    }
  }
#(undef,undef,$A)=stat($NameOfFile);print(STDERR"TypePermissions are $A\n");
#print(STDERR "changing permissions of $NameOfFile to $PermissionsFromArchive\n");
if ( ! chmod($PermissionsFromArchive,$NameOfFile) )  {
    print(STDERR "\nkxarc: unable to change permissions of $NameOfFile\n");
    &errorMaybeAbort;
    }
#(undef,undef,$A)=stat($NameOfFile);print(STDERR"TypePermissions are $A\n");
}



sub setUnReadMark  {
#print(STDERR "set unread mark\n");
if ( $ArchiveIsNormalFile == $Yes )  {
  if ( length($UnReadArchiveData) != 0 )  {
    print(STDERR '
kxarc: bug detected: $UnReadArchiveData is in use when archive is a
normal file
');
    &deleteTmpFilesAndExit(40);
    }
  $PossibleUnReadCount = 0;
  return($Success);
  }
if ( $PossibleUnReadCount != 0 )  {
  print(STDERR '
kxarc: bug detected: $PossibleUnReadCount is in use when archive is not
a normal file
');
  &deleteTmpFilesAndExit(40);
  }
$UnReadArchiveData = substr($UnReadArchiveData,$SizeOfDataReadSinceUnReadMark);
$SizeOfDataReadSinceUnReadMark = 0;
return($Success);
}




sub unPackUidGid  {
my $NA; my $NB;
($UIDfromArchive,$NA,$GIDfromArchive,$NB) =
     &UnpackVunsignedInteger($_[0],0,2);
if ( ! defined($NA) )  {
  $UIDfromArchive = $_[0];
  $_[0] = '';
  return($Success);
  }
$_[0] = substr($_[0],$NA);
if ( ! defined($NB) )  {
  $GIDfromArchive = $_[0];
  $_[0] = '';
  return($Success);
  }
$_[0] = substr($_[0],$NB);
return($Success);
}



sub UnpackVunsignedInteger  {
# $_[0] is the input string, $_[1] is the start position, $_[2] is the
# number of VunsignedIntegers to unpack
# call this sub like this:
#   ($UnsignedInteger1,$SizeOfVunsignedInteger1,$UnsignedInteger2,
#         $SizeOfVunsignedInteger2,...,$LastUnsignedInteger,
#         $SizeOfLastVunsignedInteger) = $&UnpackVunsignedInteger
#         ($InputString,$StartPosition,$NumberOfVunsignedIntegersToUnpack);
# Remember that this sub converts the VunsignedIntegers to regular unsigned
# integers as it unpacks them; and the size is not the size of the unsigned
# integer, it is the size of the VunsignedInteger, it is the number of bytes
# of $InputString used by the VunsignedInteger. The size is returned in case
# you want to know how much of $InputString was read or if you want to know
# where in $InputString is the end of the last VunsignedInteger; if you do
# not care, then throw away the size.
my $Start; my $End; my $UnsignedInteger; my @Output = ();
my $NA = 1; my $NB; my $NC; my $SA;
use integer;
if ( $#_ < 2 ) { return(@Output) }
$Start = $_[1];
$End = length($_[0]);
until ( $NA > $_[2] )  {
  if ( ($Start + 1) > $End )  { last }
  $NB = unpack('C',substr($_[0],$Start,1));
  $Start = $Start + 1;
  if ( ($Start + $NB) > $End )  { last }
  $SA = substr($_[0],$Start,$NB);
  $Start = $Start + $NB;
  $UnsignedInteger = 0;
  $NC = length($SA) - 1;
  until ( $NC < 0 )  {
    { no integer;
    $UnsignedInteger = $UnsignedInteger << 8;
    $UnsignedInteger = $UnsignedInteger | unpack('C',substr($SA,$NC,1)); }
    $NC = $NC - 1;
    }
  push(@Output,$UnsignedInteger,($NB + 1));
  $NA = $NA + 1;
  }
return(@Output);
}




sub UnpackVstring  {
# $_[0] is the input string, $_[1] is the start position, $_[2] is the
# number of Vstrings to unpack
# call this sub like this:
#   ($String1,$SizeOfVstring1,$String2,$SizeOfVstring2,...,$LastString,
#           $SizeOfLastVstring) = $&UnpackVstring
#           ($InputString,$StartPosition,$NumberOfVstringsToUnpack);
# Remember that this sub converts the Vstrings to regular strings as it
# unpacks them; and the size is not the size of the string, it is the size
# of the Vstring, it is the number of bytes of $InputString used by the
# Vstring. The size is returned in case you want to know how much of
# $InputString was read or if you want to know where in $InputString is the
# end of the last Vstring; if you do not care, then throw away the size.
my $Start; my $End; my $Size; my @Output = ();
my $NA = 1; my $NB; my $NC; my $SA;
use integer;
if ( $#_ < 2 ) { return(@Output) }
$Start = $_[1];
$End = length($_[0]);
until ( $NA > $_[2] )  {
  if ( ($Start + 1) > $End )  { last }
  $NB = unpack('C',substr($_[0],$Start,1));
  $Start = $Start + 1;
  if ( ($Start + $NB) > $End )  { last }
  $SA = substr($_[0],$Start,$NB);
  $Start = $Start + $NB;
  $Size = 0;
  $NC = length($SA) - 1;
  until ( $NC < 0 )  {
    { no integer;
    $Size = $Size << 8;
    $Size = $Size | unpack('C',substr($SA,$NC,1)); }
    $NC = $NC - 1;
    }
  if ( ($Start + $Size) > $End )  { last }
  push(@Output,substr($_[0],$Start,$Size),($Size + $NB + 1));
  $Start = $Start + $Size;
  $NA = $NA + 1;
  }
return(@Output);
}




sub unReadDataSinceUnReadMark  {
# unread the data we have read from the archive since the last time
# we did &setUnReadMark
# The easy way to unread is to seek() backwards. But if the archive
# is a pipe or device, I am not sure if it is safe to use seek().
# Therefore if the archive is a normal file, we unread by seeking
# backwards; otherwise we unread by resetting $SizeOfDataReadSinceUnReadMark
# (see sub readFromArchive)
# Hopefully we will not have to unread anyway. We only have to unread
# when we get lost in the archive, and that should only happen if the
# archive gets corrupted.
if ( $ArchiveIsNormalFile == $Yes )  {
  if ( length($UnReadArchiveData) != 0 )  {
    print(STDERR '
kxarc: bug detected: $UnReadArchiveData is in use when archive is a
normal file
');
    &deleteTmpFilesAndExit(40);
    }
  #print(STDERR "seeking $PossibleUnReadCount\n");
  seek(FH_Archive,$PossibleUnReadCount,1);
  $PossibleUnReadCount = 0;
  return($Success);
  }
if ( $PossibleUnReadCount != 0 )  {
  print(STDERR '
kxarc: bug detected: $PossibleUnReadCount is in use when archive is not
a normal file
');
  &deleteTmpFilesAndExit(40);
  }
$SizeOfDataReadSinceUnReadMark = 0;
return($Success);
}




sub unReadNumberOfBytes  {
# $_[0] is the number of bytes to unread
if ( $_[0] == 0 )  { return($Success) }
if ( $_[0] < 0 )  {
    print(STDERR "
kxarc: bug detected: sub unReadNumberOfBytes called to unread $_[0] bytes.
It is not supposed to unread less than 0 bytes.
");
    &deleteTmpFilesAndExit(40);
    }
if ( $ArchiveIsNormalFile == $Yes )  {
  if ( length($UnReadArchiveData) != 0 )  {
    print(STDERR '
kxarc: bug detected: $UnReadArchiveData is in use when archive is a
normal file
');
    &deleteTmpFilesAndExit(40);
    }
  seek(FH_Archive,(0 - $_[0]),1);
  $PossibleUnReadCount = $PossibleUnReadCount + $_[0];
  return($Success);
  }
if ( $PossibleUnReadCount != 0 )  {
  print(STDERR '
kxarc: bug detected: $PossibleUnReadCount is in use when archive is not
a normal file
');
  &deleteTmpFilesAndExit(40);
  }
$SizeOfDataReadSinceUnReadMark = $SizeOfDataReadSinceUnReadMark - $_[0];
if ( $SizeOfDataReadSinceUnReadMark < 0 )
     { $SizeOfDataReadSinceUnReadMark = 0 }
return($Success);
}





sub Vstring  {
# This is a variation of the pack() function, but it does not need a
# template to unpack() because the template is packed with it.
# A Vstring is a variable sized string. A Vstring includes a header which
# gives the length of the Vstring. If several Vstrings are combined into one
# long string, then the can be seperated again by using the size of the
# first one to find the end of the first one, which is the beginning of the
# second one; read the size of the second one and find the end of the second
# one, etc. Vstrings use few bytes but are tricky to manipulate; they are
# useful for when you want to pack multiple strings into a single file, and
# want to be able to read it back later and seperate the strings again, and
# you have absolutely no idea how big the strings are. A Vstring can be any
# size from 0 bytes to (256 to the 255th power minus 1) bytes. A 0 byte string
# becomes a 1 byte Vstring; a 1 byte string becomes a 3 byte Vstring; a 255
# byte string becomes a 257 byte Vstring; a 256 byte string becomes a 259
# byte Vstring; etc.
# A Vstring has a two part header. The first part is one byte, and it gives
# the number of bytes in the second part. The second part gives the number
# of bytes in the string. If you know what size strings you are storing,
# then it is probably simpler to use a one part header, which gives the size
# of the string, or to use the normal pack() function. But if you do not
# know what size string will be stored, then you do not know how many bytes
# are needed to store the size, and you should be using Vstrings. Or if you
# are writing a program to store small strings, but you think that maybe in
# a few years when memory, hard drives, and files are bigger then someone
# may try to use your program to store long strings; then you should use
# Vstrings.
my $NA; my $NB = 0; my $SA = '';
use integer;
if ( ! defined($_[0]) )  { return(undef) }
$NA = length($_[0]);
if ( $NA < 0 )  { return(undef) }  # $NA should not be negative, but if it
              # is, then it is an error. perl 5.004_04 has a bug such that
              # integers larger than 2G are mistaken for negative numbers.
while ( $NA > 0 )  {
  $SA = $SA . pack('C',($NA & 0xFF));
  $NA = $NA >> 8;
  $NB = $NB + 1;
  }
return(pack('C',$NB) . $SA . $_[0])
}




sub VunsignedInteger  {
# converts an unsigned integer into a string, can theoretically handle
# numbers from 0 to (256 to the 255th power minus 1). Like pack(), except
# you do not need a template, and you do not need to say long, short, char,
# etc, and allows a very large range of unsigned integers. It is more
# flexible than pack(); it can pack very large numbers, yet it does not
# waste space when packing small numbers.
# like Vstring, but for a unsigned integer
# why not signed integers? because if you packed a 32 bit signed integer and
# unpacked it as a 64 bit signed integer, it would be wrong. But there is no
# problem packing a 32 bit unsigned integer and unpacking it as a 64
# bit unsigned integer. Remember, this sub is for when you do not know how
# many bits the integer is. If you are packing signed integers, you have to
# know how many bits it is, and if you know how many bits it is, you should
# use the normal pack() function.
my $NA; my $NB = 0; my $SA = '';
use integer;
if ( ! defined($_[0]) )  { return(undef) }
if ( $_[0] < 0 )  { return(undef) }
$NA = $_[0];
while ( $NA > 0 )  {
  $SA = $SA . pack('C',($NA & 0xFF));
  $NA = $NA >> 8;
  $NB = $NB + 1;
  }
return(pack('C',$NB) . $SA)
}




sub writeCompressedDataToArchive  {
&writeOutsideMarkerAndDataSizeToArchive($SizeOfCompressedData);
open(FH_CompressionOutputFile,"< $CompressionOutputFile");
while  ( read(FH_CompressionOutputFile,$BigStringData,
                                 $FileCopyingBlockSize) )
          { print(FH_Archive $BigStringData); }
close(FH_CompressionOutputFile);
}



sub writeOutsideMarkerAndDataSizeToArchive  {
#print(STDERR "writing size of $_[0]\n");
if ( $_[0] < 5 )  {
    print(STDERR "
kxarc: bug detected: sub writeOutsideMarkerAndDataSizeToArchive called
with data size of less than 5 bytes; data size for ",&fileTypeString,"
$NameOfFile is $_[0]
");
    &deleteTmpFilesAndExit(40);
    }
print(FH_Archive $OutsideMarker,&VunsignedInteger($_[0]));
}



sub writeUncompressedDataToArchive  {
&writeOutsideMarkerAndDataSizeToArchive($SizeOfUncompressedData);
$CalculatedChecksum = 0;
open(FH_CompressionInputFile,"< $CompressionInputFile");
while  ( read(FH_CompressionInputFile,$BigStringData,
                                 $FileCopyingBlockSize) )  {
  print(FH_Archive $BigStringData);
  if ( eof(FH_CompressionInputFile) )
         { $BigStringData = $BigStringData . "\0\0\0" }
  $CalculatedChecksum =
           $CalculatedChecksum + unpack("%32L*",$BigStringData);
  }
close(FH_CompressionInputFile);
print(FH_Archive pack('L',$CalculatedChecksum));
}




sub zDataAboutFormatOfArchiveFailedChecksum  { print(STDERR "
kxarc: data about format of archive has invalid checksum; archive is
probably corrupted. It appears to say that \$UncompressionCommand is
$_[0]
"); &errorMaybeAbort; $FileIsCorrupted = $Yes; print(STDERR
"ignoring data about format of archive
") }

sub zExternalCommandError  { print(STDERR "
kxarc: for the following external command$_[0]:
    '$ExternalCommand'
    perl system() returned: $ExitCodeOfExternalCommand");
if ( ($ExitCodeOfExternalCommand & 0xFFFF) == 0xFF00 )  { print(STDERR "
    that means the command could not be run because:
    $!") }   # $! should be a string explaining the error
else  { print(STDERR "
  the exit code was ", (($ExitCodeOfExternalCommand & 0xFF00) >> 8), "
  the signal was ", ($ExitCodeOfExternalCommand & 0x7F), "
");
  if ( ($ExitCodeOfExternalCommand & 0x80) == 1 )
        { print(STDERR "core dumped\n") }
  else  { print(STDERR "no core dump\n") }
}
return }

sub zHeaderTooBig  { print(STDERR "
kxarc: the uncompressed data appears to be a normal file with a header size
of $HeaderSize. Since this is larger than the maximum header size of
$MaxHeaderSize, kxarc is assuming this file is corrupted.
"); $FileIsCorrupted = $Yes }

sub zNoHeaderSize  { print(STDERR "
kxarc: the uncompressed data appears to be a normal file, but the header
size can not be determined. kxarc is assuming this file is corrupted.
"); $FileIsCorrupted = $Yes }

sub zNoNameInHeader  { print(STDERR "
kxarc: The header says that the size of the file name is larger than the size
of the header. This is not possible, so the file is probably corrupted or
truncated.
"); $NameOfFile = $_[0]; $_[0] = ''; $FileIsCorrupted = $Yes }

sub zUnpackUncompressionCommandFailed  { print(STDERR "
kxarc: unable to read uncompression command from data about format of
archive. The archive is probably corrupted.
"); &errorMaybeAbort; $FileIsCorrupted = $Yes; print(STDERR
"ignoring data about format of archive
") }




__END__

Some of the files which are stored in an archive might have control
characters or other strange characters in their names. For example, on my
dos partition there is a file named 'nav&.sys'. For a compare, kxarc tries
to run the program 'diff nav&.sys -'. But the shell tends to think this is
supposed to be 'diff nav' & '.sys -', and so diff says there is no file
'nav', and the shell says it can not run program '.sys'. To avoid such
problems, kxarc always adds quote marks when it assembles a command string
for the system() function.

Data and subroutines have fairly long, descriptive names because that
makes it easy to understand the source code without a lot of comments.
If I used short, cryptic names I would have to add a lot of comments to
explain what was going on, and the resulting source code would not be
any smaller. If some data is used only once or twice in the next few
lines after it is created, I call that temporary data.
Temporary strings have names like $SA, $SB, etc. Temporary numbers have
names like $NA, $NB, etc.

Most string data are less than 100 bytes, but a few are much longer. I am
not sure exactly how the perl compiler handles strings, but I am guessing
that if a string is big, then becomes small, then the compiler may not
release the memory right away, in case the string becomes big again.
Therefore it might be more efficient to assign a new big string to the
same data name as the last big string. Therefore, kxarc has one big
string data, named $BigStringData. As long as I remember to make sure that
$BigStringData is only used for one thing at a time, this should allow
kxarc to use memory more efficiently, to run a little faster and use a
little less memory. And then again, maybe not.

Any line of code which begins with '#' and no space after the '#' is debugging
code. I comment out debugging code when I am done with it, but I never delete
the debugging code because you never know when you will want it again.
For example:
    #print(STDERR "starting sub x\n");

A line of code which begins with '#' with a space after the '#' is probably
vestigal code; something I commented out because I was not sure whether to
delete it or not, and then forgot about it.

I wrote kxarc for backing up my hard drive to tape. The parts of kxarc
which I am likely to use for this are finished, and just about everything
else is unfinished.

If kxarc is making an archive and NOT compressing files, it copies data
to a temporary file and prepares to compress it, then copies the data to
the archive. It would be better to skip the temporary file and copy the
data directly to the archive, but it was easier to code this way; and it did
not seem important because I may never make an archive with compression off.
In other words, the code is optimized for using compression.

kxarc compresses every part of a file, including the file name. If you are
trying to unarchive one file from an archive, kxarc has to uncompress every
file in the archive to find the one you want. It would be faster if there
was an index. There is no code in kxarc for making or reading an index; I
have not even defined a format for an index yet. kxarc can not have all the
information for making an index until it finishes the rest of the archive;
thus it would have to make the index after it makes the rest of the archive;
thus it would be easiest to put the index at the end of the archive. If kxarc
was making an archive onto a pipe or device, it would have to put the index
at the end of the archive. But if kxarc was reading an archive, it would need
to read the index before it read the rest of the archive. If the archive was
a normal file, kxarc could read the end of the file first. But if the archive
was a pipe or device, the index would be useless unless the index was at the
beginning of the archive. Therefore, kxarc should put the index at the
beginning of the archive if possible; otherwise it should put the index at
the end of the archive. When reading an archive, kxarc should first look
for an index at the beginning, then if possible it should look for an index
at the end, then it should do without an index. An index would be useless
for me for backing up my hard drive to a tape device; thus I have decided
that kxarc will not make an index by default, and I have not written any
code for making or reading an index.

--uncompress-header-first is not coded.

--list, --message, and --display were not important to me, but I coded
these anyway because they were easy. But I did not code --list --verbose,
which should give an ls style listing.

--fix and --delete would be useless on a tape, so I did not code these.
They are similar, because there really is no way to fix a corrupted file
without error correction codes, so --fix would mostly delete corrupted
files. However, there some rare errors which maybe could be fixed. The
markers and sizes are sort of redundant, so if the markers were corrupted
but not the sizes, the markers could be fixed; and vice versa; but if both
were corrupted there would be little hope. If the data about the format of
the archive were corrupted, maybe the format could be determined by
careful examination of the archive. If several archives were cat'ed
together into a single archive, there might be duplicate data about the
format of the archive, duplicate files, partial indexes, multiple messages,
and different parts of the archive could have different formats; all of these
things could be fixed. Maybe there should be an option --test, sort of
like --fix except it only finds errors, not fix them.

If we really want the ability to fix corrupted archives, we could add
error correction codes, but that seems like too much work to me. Besides,
the tape is supposed to provide error correction codes.

Maybe kxarc needs an option --ignore-trailing-nuls, in case someone had
to add nuls to the end of an archive to match the block size required
by the tape driver. I think block sizes are an abomination; if a device
driver wants some specific block size, it should convert the data itself,
and other programs should not have to worry about it. As it is, kxarc
will think the trailing nuls are a corrupted file; the purpose of
the option --ignore-trailing-nuls would be to suppress the error message.

There should be more types of commands. External commands allow kxarc to
execute external scripts, which can do quite a lot, but there should also
be commands for the few things which can not be done in an external child
process: change the current directory, change the current UID, change the
current GID; and maybe make a directory, delete the script. It would be
interesting if commands could decide whether or not to unarchive a file,
so a complex installation program could be built out of commands and run
when the files are unarchived. Probably the best way to do that is to define
a command to run an external program, and read the output of that program as
a file list. Then you could set up an archive so the first file is the install
script, then a command to run the install script, the install script would
examine the computer and decide what files needed to be unarchived, and output
a file list of the files to be unarchived; kxarc would read the file list,
go on to the next file, check it against the file list, etc. But if there was
already a file list? Also, it would be good if kxarc could unarchive a script,
run the script, check the exit code of the script, and decide whether or not
to abort archiving depending on the exit code of the script. Or the the script
could output an empty file list to abort.

Maybe gzip -d exit code 2 should not be considered a corrupted file,
since gzip -d usually returns 1 for an error and 2 for a warning;
2 is less serious than 1. Maybe gzip's error messages should be captured
and examined, so kxarc could do different things depending on the
error message. But that would make trouble for anyone using a compression
program other than gzip, but no one is going to use a compression program
other than gzip, are they?

Maybe the data about the format of the archive should include the version.
Maybe it should include a name for the archive, or some information to
identify the backup set and sequence number; although that sort of
information could go in the message.

If kxarc finds a corrupted file, it requests additional memory for an
unread buffer. It does not request this memory until it needs it. Maybe there
should be an option to request this memory when kxarc starts, and to abort
if the memory is not available. As it is, if there is not enough memory for
the unread buffer, then kxarc aborts when it finds a corrupted file. Maybe if
there is not enough memory for the unread buffer, kxarc should try a smaller
unread buffer.

There should be an option to compare the data in a file, even if the file is
obviously different, like if the file has a different date or size.

Now that I have kxarc working enough to use it for backups, I intend
to go on to other projects. It may be 10 or 20 years before I get around
to finishing kxarc. So if you want to take it over and finish it yourself,
go right ahead.




The format of the archive

kxarc allows small variations in the format of the archive, for two reasons:
it allows the format to be optimized for local conditions; and because when
I started working on kxarc, I was not sure what format I wanted to use, so I
created variable formats, so I could easily change my mind later. You can
even change formats in the middle of an archive, but you should not, because
it interferes with kxarc's ability to handle corrupted archives.

For example, you can change the inside marker and outside marker, except
that would make archives created by your version of kxarc incompatible with
other versions, and your kxarc could not read archives created by other
kxarcs, and you can not change the markers in the middle of an archive. If
you really want to change the markers, the length does not matter. I did not
test this, but I think if the outside marker is '', then you will get an
endless loop if you read a corrupted file without --exit-on-nonfatal-errors;
and if the inside marker is '' kxarc will think every file in the archive is
uncompressed. The last characters of the outside marker should not be the
same as the first characters; for example 'aeaea' would be bad because the
last three characters are a repeat of the first three characters, and also
the last character is a repeat of the first character; otherwise kxarc will
be unable to cope with some rare errors. This is because after a corrupted
file, kxarc searches for the outside marker, and it starts searching after
the last outside marker found. If that is a problem, you can always change
the code so the search starts with the second character of the last outside
marker found.

You can change the uncompression command in the middle of an archive, but
you should not. For backups, you can use any uncompression command you want,
but you should use the same uncompression command for all backups. For
archives you are distributing over the internet, everyone should use the
same uncompression command, such as 'gzip -d'. If the uncompression commands
can not be read from the archive, kxarc will assume the default
uncompression command; but this only works if you always use the same
uncompression command.

An archive is divided into files. Some files are normal files, but others
are directories, devices, etc; and others are special kxarc things like
messages, commands, indexes, etc. The first file is always data about
the format of the archive.

Each file is divided into two parts, an outside and an inside. The inside is
sometimes compressed and sometimes not; the outside is never compressed. The
outside is the part which is never compressed; the inside is the part which
might be compressed

The outside is the first part of the file. The first part of the outside is
the outside marker, which is a signature or magic number, a string which
marks the beginning of a file. Then comes the size of the inside in
VunsignedInteger format.

The inside might or might not be compressed; if first part of the inside
is not the inside marker, the inside is compressed. The inside is divided
into three parts; the first part is the header, and the second part is the
contents; but only normal files have contents; for every other kind of file
there is no contents. The third part is the checksum; the checksum only
exists if the inside was not compressed.

The first part of the header is the inside marker, then comes one byte
which gives the file type. The format of the rest of the header varies
depending on the file type.


# data format for data about format of archive:
inside marker
file type (1 byte)
uncompression command  (Vstring)
(maybe additional data)
(4 byte checksum)

# data format for message, commands, and index:
inside marker
file type (1 byte)
message/commands/index
(maybe 4 byte checksum)

# data format for soft link is:
inside marker
file type (1 byte)
user id  (VunsignedInteger)
group id  (VunsignedInteger)
file name  (Vstring)
what it links to  (Vstring)
(maybe additional data)
(maybe 4 byte checksum)

# data format for normal file is:
inside marker
file type (1 byte)
size of the header  (VunsignedInteger)
permissions (2 bytes)
user id  (VunsignedInteger)
group id  (VunsignedInteger)
file name  (Vstring)
nul  (1 byte)  (reserved for fifth byte of date/time)
file date/time (4 bytes)
(maybe additional data)
contents of file
(maybe 4 byte checksum)

# data format for directory or pipe:
inside marker
file type (1 byte)
permissions (2 bytes)
user id  (VunsignedInteger)
group id  (VunsignedInteger)
file name  (Vstring)
(maybe additional data)
(maybe 4 byte checksum)

# data format for bock or character device:
inside marker
file type (1 byte)
permissions (2 bytes)
user id  (VunsignedInteger)
group id  (VunsignedInteger)
file name  (Vstring)
major number (1 byte)
minor number (1 byte)
(maybe additional data)
(maybe 4 byte checksum)

If we want to put more information into the archive, we can extend the
archive format in two ways. We can put more information at the end of
the header. kxarc ignore extra information at the end of the header, so
future versions of kxarc could extend the header, and old versions of
kxarc could still read the archives.
Also, we could define new file types with different formats.


exit codes
0    ok

sort of ok
1   ok, but compare found that one or more files do not match
2   ok, but nonfatal errors occurred; or aborted on nonfatal errors
3   ok, but files do not match AND nonfatal errors occurred

fatal errors
10  incorrect, incomplete, incompatible, or illogical command line parameters
11  unable to read from or write to archive or file list
12  unable to read from or write to temporary files
13  external compression program returned nonzero exit code
14  error executing external compare program

impossible errors
40  bug detected

Note that if perl errors occur, then perl chooses the exit code. If perl
runs out of memory, you get exit code 1. If perl fails to compile, you
get exit code 2. To determine if the exit code came from perl or from
the program, read the error messages.
