From fwc@mt.net  Thu Dec 29 11:20:12 2005
Return-Path: <fwc@mt.net>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8DF8016A41F
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 29 Dec 2005 11:20:12 +0000 (GMT)
	(envelope-from fwc@mt.net)
Received: from mt.net (mail.MT.net [206.127.64.140])
	by mx1.FreeBSD.org (Postfix) with SMTP id C141C43D5E
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 29 Dec 2005 11:20:11 +0000 (GMT)
	(envelope-from fwc@mt.net)
Received: (qmail 79783 invoked from network); 29 Dec 2005 11:20:09 -0000
Received: from unknown (HELO backup.mt.net) (206.127.64.132)
  by mail.mt.net with SMTP; 29 Dec 2005 11:20:09 -0000
Received: by backup.mt.net (sSMTP sendmail emulation); Thu, 29 Dec 2005 11:20:03 +0000
Message-Id: <20051229112011.C141C43D5E@mx1.FreeBSD.org>
Date: Thu, 29 Dec 2005 11:20:03 +0000
From: Forrest W Christian <fwc@mt.net>
Reply-To: Forrest W Christian <fwc@mt.net>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: Patch to make dumps more rsync-friendly
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         91049
>Category:       bin
>Synopsis:       [patch] dump(8): make dumps more rsync-friendly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 29 11:30:04 GMT 2005
>Closed-Date:    Fri May 23 23:15:36 UTC 2008
>Last-Modified:  Fri May 23 23:20:00 UTC 2008
>Originator:     Forrest Christian
>Release:        FreeBSD 6.0-RELEASE i386
>Organization:
Montana Internet Corporation
>Environment:
System: FreeBSD backup.mt.net 6.0-RELEASE FreeBSD 6.0-RELEASE #6: Tue Dec 13 00:11:49 UTC 2005 root@backup.mt.net:/usr/src/sys/i386/compile/BACKUP i386

>Description:
	When using dump to generate level 0 dumps which are then rsync'ed to a remote
	machine, the fact that the dump date is stored with each header (inode) record makes
	rsync significantly less efficient than necessary.  This also applies to inode access
	times when they are not important data to retain.  When implementing an offsite backup
	solution of this type, these dates in particular are not important, especially if it
	prevents effective offsite backups.
>How-To-Repeat:
	generate a level 0 dump.  rsync to remote location.  Generate another level 0 dump,
	rsync again and note the large amount of data being transferred just because of the
	date change. 
>Fix:

	Patch to add -r (don't store dump dates) and -R (don't store dump dates and access times)
	options is attached.  This is against the 6.0-release version of the dump source (it
	this is the current revision of the code besides a minor manual change).


--- orig/dump.8	Thu Dec 29 10:25:32 2005
+++ ./dump.8	Thu Dec 29 11:09:58 2005
@@ -38,7 +38,7 @@
 .Nd file system backup
 .Sh SYNOPSIS
 .Nm
-.Op Fl 0123456789acLnSu
+.Op Fl 0123456789acLnrRSu
 .Op Fl B Ar records
 .Op Fl b Ar blocksize
 .Op Fl C Ar cachesize
@@ -268,6 +268,19 @@
 prompts for a new tape.
 It is recommended to be a bit conservative on this option.
 The default tape length is 2300 feet.
+.It Fl r
+Be rsync-friendly.   Normally dump stores the date of the current
+and prior dump in numerous places throughout the dump.  This causes
+problems when rsync or another incremental file transfer program is
+used to update a remote copy of a level 0 dump, since the date changes
+for each dump.   This option sets both dates to the epoch, permitting
+rsync to be much more efficient when transferring a dump file.  
+.It Fl R
+Be even more rsync-friendly.  This option disables the storage of the 
+actual inode access time (storing it instead as the epoch).
+This option also sets -r.  This permits rsync to be even more efficient
+when transferring dumps generated from filesystems with numerous files
+which are not changing other than their access times. 
 .It Fl T Ar date
 Use the specified date as the starting time for the dump
 instead of the time determined from looking in
--- orig/dump.h	Thu Dec 29 10:25:32 2005
+++ ./dump.h	Thu Dec 29 10:47:01 2005
@@ -76,6 +76,8 @@
 int	unlimited;	/* if set, write to end of medium */
 int	cachesize;	/* size of block cache in bytes */
 
+int	rsync_friendly;	/* be friendly with rsync */
+int	rsync_morefriendly;	/* be even more friendly with rsync */
 int	notify;		/* notify operator flag */
 int	blockswritten;	/* number of blocks written on current tape */
 int	tapeno;		/* current tape number */
--- orig/main.c	Thu Dec 29 10:25:32 2005
+++ ./main.c	Thu Dec 29 11:07:05 2005
@@ -117,13 +117,15 @@
 	if (TP_BSIZE / DEV_BSIZE == 0 || TP_BSIZE % DEV_BSIZE != 0)
 		quit("TP_BSIZE must be a multiple of DEV_BSIZE\n");
 	level = '0';
+	rsync_friendly = 0;
+	rsync_morefriendly = 0;
 
 	if (argc < 2)
 		usage();
 
 	obsolete(&argc, &argv);
 	while ((ch = getopt(argc, argv,
-	    "0123456789aB:b:C:cD:d:f:h:LnP:Ss:T:uWw")) != -1)
+	    "0123456789aB:b:C:cD:d:f:h:LnP:RrSs:T:uWw")) != -1)
 		switch (ch) {
 		/* dump level */
 		case '0': case '1': case '2': case '3': case '4':
@@ -189,6 +191,15 @@
 			popenout = optarg;
 			break;
 
+		case 'r':		/*store slightly less data to be friendly with rsync*/
+			rsync_friendly = 1;
+			break;
+
+		case 'R':		/*store even less data to be even more friendly with rsync*/
+			rsync_friendly = 1;
+			rsync_morefriendly = 1;
+			break;
+
 		case 'S':               /* exit after estimating # of tapes */
 			just_estimate = 1;
 			break;
@@ -236,6 +247,10 @@
 		(void)fprintf(stderr, "\n");
 		exit(X_STARTUP);
 	}
+	if (rsync_friendly && (level>'0')) {
+		(void)fprintf(stderr, "rsync friendly options should only be used with level 0 dumps.\n");
+		exit(X_STARTUP);
+	}
 	if (Tflag && uflag) {
 	        (void)fprintf(stderr,
 		    "You cannot use the T and u flags together.\n");
@@ -383,6 +398,11 @@
 	spcl.c_level = level - '0';
 	spcl.c_type = TS_TAPE;
 
+	if (rsync_friendly) {	/*don't store real dump times*/
+		spcl.c_date = 0;
+		spcl.c_ddate = 0;
+	}
+
 	if (spcl.c_date == 0) {
 		tmsg = "the epoch\n";
 	} else {
@@ -391,7 +411,7 @@
 	}
 	msg("Date of this level %c dump: %s", level, tmsg);
 
-	if (!Tflag)
+	if ((!Tflag)&&(!rsync_friendly))
 	        getdumptime();		/* /etc/dumpdates snarfed */
 	if (spcl.c_ddate == 0) {
 		tmsg = "the epoch\n";
--- orig/traverse.c	Thu Dec 29 10:25:32 2005
+++ ./traverse.c	Thu Dec 29 11:05:01 2005
@@ -682,6 +682,10 @@
 {
 	int32_t sum, cnt, *lp;
 
+	if (rsync_morefriendly) {
+		spcl.c_atime=0;
+		spcl.c_atimensec=0;
+	}
 	spcl.c_inumber = ino;
 	spcl.c_magic = FS_UFS2_MAGIC;
 	spcl.c_checksum = 0;

>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: mckusick 
State-Changed-When: Fri May 23 23:14:24 UTC 2008 
State-Changed-Why:  
Requested feature has been added to dump. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=91049 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/91049: commit references a PR
Date: Fri, 23 May 2008 23:13:19 +0000 (UTC)

 mckusick    2008-05-23 23:13:14 UTC
 
   FreeBSD src repository
 
   Modified files:
     sbin/dump            dump.8 dump.h main.c traverse.c 
   Log:
   When using dump to generate level 0 dumps which are then rsync'ed
   to a remote machine, the fact that the dump date is stored with
   each header (inode) record makes rsync significantly less efficient
   than necessary. This also applies to inode access times when they
   are not important data to retain. When implementing an offsite
   backup solution of this type, these dates in particular are not
   important, especially if it prevents effective offsite backups.
   
   PR:           bin/91049
   Submitted by: Forrest W Christian <fwc@mt.net>
   
   Revision  Changes    Path
   1.76      +22 -1     src/sbin/dump/dump.8
   1.29      +1 -0      src/sbin/dump/dump.h
   1.66      +23 -3     src/sbin/dump/main.c
   1.41      +5 -0      src/sbin/dump/traverse.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
