From thompson@tgsoft.com Wed Jun  9 21:10:41 1999
Return-Path: <thompson@tgsoft.com>
Received: from moose.tgsoft.com (cx20270-a.pwy1.sdca.home.com [24.0.169.3])
	by hub.freebsd.org (Postfix) with SMTP id 23EAD14E86
	for <FreeBSD-gnats-submit@freebsd.org>; Wed,  9 Jun 1999 21:10:20 -0700 (PDT)
	(envelope-from thompson@tgsoft.com)
Received: (qmail 19601 invoked by uid 128); 10 Jun 1999 04:10:18 -0000
Message-Id: <19990610041018.19600.qmail@moose.tgsoft.com>
Date: 10 Jun 1999 04:10:18 -0000
From: thompson@tgsoft.com
Reply-To: thompson@tgsoft.com
To: FreeBSD-gnats-submit@freebsd.org
Subject: new feature for dump, when -a does not work
X-Send-Pr-Version: 3.2

>Number:         12107
>Category:       bin
>Synopsis:       Add switch to dump to support multiple dumps on one tape
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jun  9 21:20:00 PDT 1999
>Closed-Date:    Thu May 24 02:35:47 PDT 2001
>Last-Modified:  Thu May 24 02:35:55 PDT 2001
>Originator:     mark thompson
>Release:        FreeBSD 3.2-STABLE i386
>Organization:
tgsoft
>Environment:

I have an older HP-by-Phillips DAT drive that is used for
backups. Unfortunately, dump -a does not do the right thing on this
drive, which makes it challenging to pack several filesystems on a
single tape (which you want to do, since several of the filesystems
are way smaller than a tape-full.

>Description:

When putting the n'th dump on a tape, you would like to tell dump how
much tape is left, so that it will ask for a new one at an appropriate
time. This patch adds the ability to say "the tape is -B blocks long,
but -F blocks are already used; so ask for a new tape when this is
full. The new tape will be -B blocks long".

>How-To-Repeat:

On my machine, with this drive and 3.2, write a tape using dump -a
that extends over a tape boundary. Try to restore the resulting 2
tapes.

>Fix:
	
Simple patch enclosed. Also an example perl program to use it.
----------- PATCH -------------
--- dump.8.ORIG	Wed Jun  9 08:20:52 1999
+++ dump.8	Wed Jun  9 08:22:49 1999
@@ -104,6 +104,10 @@
 The number of 1 KB blocks per volume.
 This option overrides the calculation of tape size
 based on length and density.
+.It Fl F Ar records
+The number of 1 kB blocks that have already been written on the first volume.
+Those blocks will be considered used when calculating the amount of tape
+left on this volume only.
 .It Fl a
 .Dq auto-size .
 Bypass all tape length considerations, and enforce writing
--- main.c.ORIG	Wed Jun  9 08:15:24 1999
+++ main.c	Wed Jun  9 13:47:55 1999
@@ -85,6 +85,7 @@
 int	dokerberos = 0;	/* Use Kerberos authentication */
 long	dev_bsize = 1;	/* recalculated below */
 long	blocksperfile;	/* output blocks per file */
+long    blocksthisvol;	/* our position on the tape */
 char	*host = NULL;	/* remote host (if any) */
 
 static long numarg __P((char *, long, long));
@@ -122,9 +123,9 @@
 
 	obsolete(&argc, &argv);
 #ifdef KERBEROS
-#define optstring "0123456789aB:b:cd:f:h:kns:T:uWw"
+#define optstring "0123456789aB:b:cd:F:f:h:kns:T:uWw"
 #else
-#define optstring "0123456789aB:b:cd:f:h:ns:T:uWw"
+#define optstring "0123456789aB:b:cd:F:f:h:ns:T:uWw"
 #endif
 	while ((ch = getopt(argc, argv, optstring)) != -1)
 #undef optstring
@@ -159,6 +160,10 @@
 				ntrec = HIGHDENSITYTREC;
 			break;
 
+		case 'F':		/* first block number */
+		        blocksthisvol = numarg("starting block number",
+					       0L, 0L);
+			break;
 		case 'f':		/* output file */
 			tape = optarg;
 			break;
@@ -596,6 +601,7 @@
 		case 'b':
 		case 'd':
 		case 'f':
+		case 'F':
 		case 'h':
 		case 's':
 		case 'T':
--- tape.c.ORIG	Wed Jun  9 08:16:37 1999
+++ tape.c	Wed Jun  9 08:19:58 1999
@@ -74,7 +74,7 @@
 long	lastspclrec = -1;	/* tape block number of last written header */
 int	trecno = 0;		/* next record to write in current block */
 extern	long blocksperfile;	/* number of blocks per output file */
-long	blocksthisvol;		/* number of blocks on current output file */
+extern  long blocksthisvol;	/* number of blocks on current output file */
 extern	int ntrec;		/* blocking factor on tape */
 extern	int cartridge;
 extern	char *host;
@@ -619,9 +619,12 @@
 		enslave();  /* Share open tape file descriptor with slaves */
 
 		asize = 0;
-		blocksthisvol = 0;
-		if (top)
+		if (top) {
 			newtape++;		/* new tape signal */
+		}
+		else {
+		        blocksthisvol = 0;     /* Not the first tape */
+		}
 		spcl.c_count = slp->count;
 		/*
 		 * measure firstrec in TP_BSIZE units since restore doesn't


-------- Example of usage --------------
#!/usr/bin/perl
$TAPENAME="/dev/nrsa0";
$TAPELENG=1200000;		# in 1K blocks

$today=`date +%Y%m%d`;
chomp $today;
$LOGFILE="/var/log/dump.$today";

open LOG, ">>$LOGFILE";
print LOG `date`;

REW1:
while (1) {
    print "Please mount first tape\n";
    $red = read STDIN, $buf, 1;
    $rc = system "mt -f $TAPENAME rewind";
#    print "rewind: rc $rc\n";
    last REW1 if (($rc & 0xffff) == 0);
}

$skip=1;
$pos=0;
$| = 1;				# Immediate flush

# each file system

while ($ARGV[0]) {
    close LOG;
    open LOG, ">>$LOGFILE";
# prvent silly tapes
    $left = $TAPELENG - $pos;
    if ($left < 1000) {
	print LOG "--- new tape because $left blocks left ---\n";
	print "*** Not much tape left\n";
	system "mt -f $TAPENAME offline";
      REW2:
	while (1) {
	    print "Please mount next tape\n";
	    $red = read STDIN, $buf, 1;
	    $rc = system "mt -f $TAPENAME rewind";
#	    print "rewind: rc $rc\n";
	    last REW2 if (($rc & 0xffff) == 0);
	}
	$skip = 1;
	$pos = 0;
    }
#  now dump it
    print LOG "=== $ARGV[0]: restore ifbs $TAPENAME 1 $skip ===\n";
    print LOG "=== dump -0 -f $TAPENAME -B $TAPELENG -F $pos $ARGV[0] ===\n";
    $pid = open DUMP,
      "dump -0 -f $TAPENAME -B $TAPELENG -F $pos $ARGV[0] 2>&1 > /dev/tty </dev/tty |";
    die "can't start dump" if ($pid == 0);
    print "Dump: pid $pid\n";
    while (<DUMP>) {
	print ;
	print LOG $_;
	if (m+DUMP: DUMP: ([0-9]*) tape blocks on ([0-9]*) volumes+) {
	    $BLOCKS=$1;
	    $VOLUMS=$2;
	}
    }
# dump exits
    close DUMP;
    $rc = $? & 0xffff;
    die "Dump failed" if ($rc);
    print "Dump: $BLOCKS blocks $VOLUMS volumes\n";
# how much did we use
    $old="";
    if ($VOLUMS > 1) {	# tape changed in middle
	$pos = $BLOCKS;
	$old = " new";
	$skip = 2;
    }
    else {
	$pos  = $BLOCKS;
	$skip++;
    }
    $left = $TAPELENG - $pos;
    $pct  = $left / ($TAPELENG/100);
    print LOG "--- Estimate $left blocks ($pct) on$old tape ---\n";
    shift;			# next
}
system "mt -f $TAPENAME offline";

print LOG `date`;
close LOG;
exit 0;
---------- Thank you for your consideration ---------

>Release-Note:
>Audit-Trail:

From: Matthew Jacob <mjacob@feral.com>
To: freebsd-gnats-submit@FreeBSD.org, thompson@tgsoft.com
Cc:  
Subject: Re: bin/12107: Add switch to dump to support multiple dumps on one tape
Date: Sun, 29 Oct 2000 10:38:06 -0800

 It's not quite clear to me why the -a argument doesn't/didn't (hope you
 didn't die of old age waiting for a responnse :-)) work for you.
 

From Mark:

> >>  >
> >>  dump does not report an error... it just asks for the next tape. The
> >>  problem comes when you try to restore, when it gets the second tape 
> >>  it reports a synchronization error. I cannot be absolutely positive,
> >>  but i am reasonably sure it is in variable block mode - that should 
> >>  be the default for DAT. If anyone wants to try to track this down   
> >>  (I don't care enough to), i can donate the HP drive. Its only problem
> >>  is that it sometimes eats tapes (which probably is an adjustment thing).
> >
> >Did the console report an error?
> >
> >Why I'm trying to get at this is to understand the lossage.
> >
> >-matt
> 
> I understand. I don't believe that any error is reported during
> the dump, but i will have to try it out to be sure; my aging   
> brain cells do not retain that information any more.

 
I just reconfirmed that dump and restore do correctly span tapes (at least
in 4.2Beta).

The following is my assessment:

The tape spanning feature works for tape drives if:

You're in variable block mode and REW (Report Early Warning) is set
for your tape drive. This allows a write to complete fully but for a
check condition to be generated that indicates EOT detected. The driver
then correctly can set ENOSPC for the next operation (which dump
understands) and that terminating filemark(s) can be written.

This ensures all data is flushed to tape and can be read back later so
that the restore stream is not confused.

If the tape drive is in fixed-block mode, because of the asynchronous nature
with which blocks are flushed out, there's no way to ensure this.

If the tape drive does not have REW set (most drives do, by default), you
may or man not lose data on the last write, but more importantly, a terminating
filemark can't be written and the tape driver doesn't know how to treat
this particular failure (physical end of tape) differently from other
closing filemark writes.

Because, I believe, tape spanning works for most cases, while this is
a somewhat neat feature, I'm not sure we should add your change.

State-Changed-From-To: open->closed 
State-Changed-By: phk 
State-Changed-When: Thu May 24 02:35:47 PDT 2001 
State-Changed-Why:  
timed out 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=12107 
>Unformatted:
