From eugen@grosbein.pp.ru  Tue Mar 27 18:07:31 2012
Return-Path: <eugen@grosbein.pp.ru>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id C9666106566C
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 27 Mar 2012 18:07:31 +0000 (UTC)
	(envelope-from eugen@grosbein.pp.ru)
Received: from grosbein.pp.ru (grosbein.pp.ru [89.189.172.146])
	by mx1.freebsd.org (Postfix) with ESMTP id 767238FC14
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 27 Mar 2012 18:07:29 +0000 (UTC)
Received: from grosbein.pp.ru (localhost [127.0.0.1])
	by grosbein.pp.ru (8.14.5/8.14.5) with ESMTP id q2RI3MPV003026
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 28 Mar 2012 01:03:22 +0700 (NOVT)
	(envelope-from eugen@grosbein.pp.ru)
Received: (from eugen@localhost)
	by grosbein.pp.ru (8.14.5/8.14.5/Submit) id q2RI3LEH003025;
	Wed, 28 Mar 2012 01:03:21 +0700 (NOVT)
	(envelope-from eugen)
Message-Id: <201203271803.q2RI3LEH003025@grosbein.pp.ru>
Date: Wed, 28 Mar 2012 01:03:21 +0700 (NOVT)
From: Eugene Grosbein <eugen@grosbein.pp.ru>
Reply-To: Eugene Grosbein <eugen@grosbein.pp.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: [patch] newsyslog -t fails to find previous rotated log
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         166448
>Category:       bin
>Synopsis:       [patch] newsyslog(8): newsyslog -t fails to find previous rotated log
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    markj
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 27 18:10:11 UTC 2012
>Closed-Date:    Fri May 10 20:59:11 UTC 2013
>Last-Modified:  Sun May 12 02:10:00 UTC 2013
>Originator:     Eugene Grosbein
>Release:        FreeBSD 8.3-PRERELEASE amd64
>Organization:
RDTC JSC
>Environment:
System: FreeBSD grosbein.pp.ru 8.3-PRERELEASE FreeBSD 8.3-PRERELEASE #19: Tue Mar 20 03:24:04 NOVT 2012 root@grosbein.pp.ru:/usr/local/obj/usr/local/src/sys/DADV amd64

>Description:

	newsyslog.conf(5) has "when" field instructing newsyslog to find
	previous rotated copy of log and look if it is older than "when" hours.

	newsyslog always looks for "file.0" and this is wrong when -t flag is used.

>How-To-Repeat:

	Use newsyslog -t and non-zero value in "when" field:

/var/log/cron 600 3 * 8@T C

	This should rotate log three times a day starting from midnight.

>Fix:

--- usr.sbin/newsyslog/newsyslog.c.orig	2012-03-27 22:43:06.000000000 +0700
+++ usr.sbin/newsyslog/newsyslog.c	2012-03-28 00:53:21.000000000 +0700
@@ -2206,6 +2206,77 @@
 	return (kbytes(dbtob(sb.st_blocks)));
 }
 
+/* Return the age of previous old log file, when using time based filenames. */
+static time_t
+find_oldest_timelog(const char *dir, const char *logfname)
+{
+	struct stat sb;
+	int c, valid;
+	size_t logfname_len;
+	struct tm tm;
+	time_t oldt;
+	struct dirent *dp;
+	DIR *dirp;
+	char *s;
+
+	if ((dirp = opendir(dir)) == NULL)
+		err(1, "Cannot open log directory '%s'", dir);
+
+	oldt = -1;
+	logfname_len = strlen(logfname);
+	while ((dp = readdir(dirp)) != NULL) {
+		if (dp->d_type != DT_REG)
+			continue;
+		/* Ignore everything but files with our logfile prefix */
+		if (strncmp(dp->d_name, logfname, logfname_len) != 0)
+			continue;
+		/* Ignore the actual non-rotated logfile */
+		if (dp->d_namlen == logfname_len)
+			continue;
+		/*
+		 * Make sure we have found a logfile, so the
+		 * postfix is valid, IE format is: '.<time>(.[bg]z)?'.
+		 */
+		if (dp->d_name[logfname_len] != '.') {
+			if (verbose)
+				printf("Ignoring %s which has unexpected "
+				    "extension '%s'\n", dp->d_name,
+				    &dp->d_name[logfname_len]);
+			continue;
+		}
+		if ((s = strptime(&dp->d_name[logfname_len + 1],
+			    timefnamefmt, &tm)) == NULL) {
+			if (verbose)
+				printf("Ignoring %s which does not "
+				    "match time format\n", dp->d_name);
+			continue;
+		}
+
+		valid = 0;
+		c = 0;
+		while (!valid && c < COMPRESS_TYPES)
+			if (strcmp(s, compress_type[c++].suffix) == 0)
+				valid = 1;
+
+		if (valid != 1) {
+			if (verbose)
+				printf("Ignoring %s which has unexpected "
+				    "extension '%s'\n", dp->d_name, s);
+			continue;
+		}
+		if (stat(dp->d_name, &sb) < 0)
+			err(1, "Cannot stat '%s'", dp->d_name);
+
+		/* We have found more recent old logfile */
+		if (oldt < sb.st_mtime)
+			oldt = sb.st_mtime;
+	}
+	closedir(dirp);
+
+	/* Return -1 if nothing found */
+	return oldt;
+}
+
 /* Return the age of old log file (file.0) */
 static int
 age_old_log(char *file)
@@ -2241,6 +2312,17 @@
 		(void) strlcpy(tmp, file, sizeof(tmp));
 	}
 
+	if (timefnamefmt != NULL) {
+                char *bd;
+                time_t t;
+
+	        if ((bd = dirname(tmp)) == NULL)
+                        err(1, "'%s'", tmp);
+                if ((t = find_oldest_timelog(bd, basename(tmp))) == -1)
+                        return (-1);
+                return ((int)(ptimeget_secs(timenow) - t + 1800) / 3600);
+	}
+
 	strlcat(tmp, ".0", sizeof(tmp));
 	logfile_suffix = get_logfile_suffix(tmp);
 	if (logfile_suffix == NULL)


>Release-Note:
>Audit-Trail:

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: bug-followup@FreeBSD.ORG
Cc:  
Subject: Re: bin/166448: [patch] newsyslog -t fails to find previous rotated
 log
Date: Wed, 28 Mar 2012 02:10:50 +0700

 Sorry, I've sent buggy patch (does stat() on wrong filename). Here is corrected one.
 
 --- usr.sbin/newsyslog/newsyslog.c.orig	2012-02-13 17:02:03.000000000 +0400
 +++ usr.sbin/newsyslog/newsyslog.c	2012-03-27 22:53:46.000000000 +0400
 @@ -2206,6 +2206,79 @@
  	return (kbytes(dbtob(sb.st_blocks)));
  }
  
 +/* Return the age of previous old log file, when using time based filenames. */
 +static time_t
 +find_oldest_timelog(const char *dir, const char *logfname)
 +{
 +	struct stat sb;
 +	int c, valid;
 +	size_t logfname_len;
 +	struct tm tm;
 +	time_t oldt;
 +	struct dirent *dp;
 +	DIR *dirp;
 +	char *s;
 +	char tmp[MAXPATHLEN];
 +
 +	if ((dirp = opendir(dir)) == NULL)
 +		err(1, "Cannot open log directory '%s'", dir);
 +
 +	oldt = -1;
 +	logfname_len = strlen(logfname);
 +	while ((dp = readdir(dirp)) != NULL) {
 +		if (dp->d_type != DT_REG)
 +			continue;
 +		/* Ignore everything but files with our logfile prefix */
 +		if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 +			continue;
 +		/* Ignore the actual non-rotated logfile */
 +		if (dp->d_namlen == logfname_len)
 +			continue;
 +		/*
 +		 * Make sure we have found a logfile, so the
 +		 * postfix is valid, IE format is: '.<time>(.[bg]z)?'.
 +		 */
 +		if (dp->d_name[logfname_len] != '.') {
 +			if (verbose)
 +				printf("Ignoring %s which has unexpected "
 +				    "extension '%s'\n", dp->d_name,
 +				    &dp->d_name[logfname_len]);
 +			continue;
 +		}
 +		if ((s = strptime(&dp->d_name[logfname_len + 1],
 +			    timefnamefmt, &tm)) == NULL) {
 +			if (verbose)
 +				printf("Ignoring %s which does not "
 +				    "match time format\n", dp->d_name);
 +			continue;
 +		}
 +
 +		valid = 0;
 +		c = 0;
 +		while (!valid && c < COMPRESS_TYPES)
 +			if (strcmp(s, compress_type[c++].suffix) == 0)
 +				valid = 1;
 +
 +		if (valid != 1) {
 +			if (verbose)
 +				printf("Ignoring %s which has unexpected "
 +				    "extension '%s'\n", dp->d_name, s);
 +			continue;
 +		}
 +		snprintf(tmp, sizeof(tmp), "%s/%s", dir, dp->d_name);
 +		if (stat(tmp, &sb) < 0)
 +			err(1, "Cannot stat '%s'", tmp);
 +
 +		/* We have found more recent old logfile */
 +		if (oldt < sb.st_mtime)
 +			oldt = sb.st_mtime;
 +	}
 +	closedir(dirp);
 +
 +	/* Return -1 if nothing found */
 +	return oldt;
 +}
 +
  /* Return the age of old log file (file.0) */
  static int
  age_old_log(char *file)
 @@ -2241,6 +2314,17 @@
  		(void) strlcpy(tmp, file, sizeof(tmp));
  	}
  
 +	if (timefnamefmt != NULL) {
 +                char *bd;
 +                time_t t;
 +
 +	        if ((bd = dirname(tmp)) == NULL)
 +                        err(1, "'%s'", tmp);
 +                if ((t = find_oldest_timelog(bd, basename(tmp))) == -1)
 +                        return (-1);
 +                return ((int)(ptimeget_secs(timenow) - t + 1800) / 3600);
 +	}
 +
  	strlcat(tmp, ".0", sizeof(tmp));
  	logfile_suffix = get_logfile_suffix(tmp);
  	if (logfile_suffix == NULL)
 

From: Marco Steinbach <coco@executive-computing.de>
To: bug-followup@FreeBSD.org
Cc: eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Wed, 23 May 2012 21:56:51 +0200

 I'm running into this, also.  Logrotation is done upon every invocation 
 of newsyslog(8), if 'when' is given as purely numerical hour interval, 
 and an alternate timeformat is used.
 
 E.g.:
 
 p90a1# newsyslog -t %Y%m%d_%H%M%S
 p90a1# ls -1 /var/log/sendmail.st.20*
 /var/log/sendmail.st.20120523_201321
 
 p90a1# newsyslog -t %Y%m%d_%H%M%S
 p90a1# ls -1 /var/log/sendmail.st.20*
 /var/log/sendmail.st.20120523_201321
 /var/log/sendmail.st.20120523_201343
 
 I've been successfuly using Eugenes patch against newsyslog(8) for about 
 24 hours now on 8.3 and 9.0 (second hunk needs to be moved for patching 
 9) i386/amd64 systems, tossing quite some logs around with and without 
 using alternate timeformats.
 
 
 MfG CoCo
Responsible-Changed-From-To: freebsd-bugs->markj 
Responsible-Changed-By: markj 
Responsible-Changed-When: Sat Dec 22 00:02:25 UTC 2012 
Responsible-Changed-Why:  
I'll take it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166448 
State-Changed-From-To: open->feedback 
State-Changed-By: markj 
State-Changed-When: Thu Dec 27 16:47:35 UTC 2012 
State-Changed-Why:  
Awaiting feedback. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166448 

From: Mark Johnston <markjdb@gmail.com>
To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru,
	coco@executive-computing.de
Cc:  
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Thu, 27 Dec 2012 11:43:18 -0500

 --FkmkrVfFsRoUs1wW
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 Hi Eugene and Marco,
 
 Would you be able to test the attached patch? It follows the same
 approach as the original patch, but I've rewritten it to avoid
 duplicating code from delete_oldest_timelog(). I've been testing it for
 about a day now and everything looks ok.
 
 I've attached a patch against HEAD and one against STABLE-8 (untested)
 as well. I can provide patches against other branches if you need it.
 
 Thanks!
 -Mark
 
 --FkmkrVfFsRoUs1wW
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="newsyslog_dash_t_fix.patch"
 
 diff --git a/usr.sbin/newsyslog/newsyslog.c b/usr.sbin/newsyslog/newsyslog.c
 index 878999c..1d7f4d6 100644
 --- a/usr.sbin/newsyslog/newsyslog.c
 +++ b/usr.sbin/newsyslog/newsyslog.c
 @@ -276,7 +276,7 @@ static void parse_args(int argc, char **argv);
  static int parse_doption(const char *doption);
  static void usage(void);
  static int log_trim(const char *logname, const struct conf_entry *log_ent);
 -static int age_old_log(char *file);
 +static int age_old_log(const char *file);
  static void savelog(char *from, char *to);
  static void createdir(const struct conf_entry *ent, char *dirpart);
  static void createlog(const struct conf_entry *ent);
 @@ -1447,20 +1447,78 @@ oldlog_entry_compare(const void *a, const void *b)
  }
  
  /*
 + * Check whether the file corresponding to dp is an archive of the logfile
 + * logfname, based on the timefnamefmt format string. Return true and fill out
 + * tm if this is the case; otherwise return false.
 + */
 +static int
 +validate_old_timelog(const struct dirent *dp, const char *logfname, struct tm *tm)
 +{
 +	size_t logfname_len;
 +	char *s;
 +	int c;
 +
 +	logfname_len = strlen(logfname);
 +
 +	if (dp->d_type != DT_REG)
 +		return (0);
 +	/* Ignore everything but files with our logfile prefix. */
 +	if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 +		return (0);
 +	/* Ignore the actual non-rotated logfile. */
 +	if (dp->d_namlen == logfname_len)
 +		return (0);
 +
 +	/*
 +	 * Make sure we created have found a logfile, so the
 +	 * postfix is valid, IE format is: '.<time>(.[bgx]z)?'.
 +	 */
 +	if (dp->d_name[logfname_len] != '.') {
 +		if (verbose)
 +			printf("Ignoring %s which has unexpected "
 +			    "extension '%s'\n", dp->d_name,
 +			    &dp->d_name[logfname_len]);
 +		return (0);
 +	}
 +	if ((s = strptime(&dp->d_name[logfname_len + 1],
 +	    timefnamefmt, tm)) == NULL) {
 +		/*
 +		 * We could special case "old" sequentially named logfiles here,
 +		 * but we do not as that would require special handling to
 +		 * decide which one was the oldest compared to "new" time based
 +		 * logfiles.
 +		 */
 +		if (verbose)
 +			printf("Ignoring %s which does not "
 +			    "match time format\n", dp->d_name);
 +		return (0);
 +	}
 +
 +	for (c = 0; c < COMPRESS_TYPES; c++)
 +		if (strcmp(s, compress_type[c].suffix) == 0)
 +			/* We're done. */
 +			return (1);
 +
 +	if (verbose)
 +		printf("Ignoring %s which has unexpected extension '%s'\n",
 +		    dp->d_name, s);
 +
 +	return (0);
 +}
 +
 +/*
   * Delete the oldest logfiles, when using time based filenames.
   */
  static void
  delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  {
  	char *logfname, *s, *dir, errbuf[80];
 -	int dir_fd, i, logcnt, max_logcnt, valid;
 +	int dir_fd, i, logcnt, max_logcnt;
  	struct oldlog_entry *oldlogs;
 -	size_t logfname_len;
  	struct dirent *dp;
  	const char *cdir;
  	struct tm tm;
  	DIR *dirp;
 -	int c;
  
  	oldlogs = malloc(MAX_OLDLOGS * sizeof(struct oldlog_entry));
  	max_logcnt = MAX_OLDLOGS;
 @@ -1478,7 +1536,6 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  		err(1, "basename()");
  	if ((logfname = strdup(s)) == NULL)
  		err(1, "strdup()");
 -	logfname_len = strlen(logfname);
  	if (strcmp(logfname, "/") == 0)
  		errx(1, "Invalid log filename - became '/'");
  
 @@ -1490,51 +1547,9 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  		err(1, "Cannot open log directory '%s'", dir);
  	dir_fd = dirfd(dirp);
  	while ((dp = readdir(dirp)) != NULL) {
 -		if (dp->d_type != DT_REG)
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
  			continue;
  
 -		/* Ignore everything but files with our logfile prefix */
 -		if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 -			continue;
 -		/* Ignore the actual non-rotated logfile */
 -		if (dp->d_namlen == logfname_len)
 -			continue;
 -		/*
 -		 * Make sure we created have found a logfile, so the
 -		 * postfix is valid, IE format is: '.<time>(.[bg]z)?'.
 -		 */
 -		if (dp->d_name[logfname_len] != '.') {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name,
 -				    &dp->d_name[logfname_len]);
 -			continue;
 -		}
 -		if ((s = strptime(&dp->d_name[logfname_len + 1],
 -			    timefnamefmt, &tm)) == NULL) {
 -			/*
 -			 * We could special case "old" sequentially
 -			 * named logfiles here, but we do not as that
 -			 * would require special handling to decide
 -			 * which one was the oldest compared to "new"
 -			 * time based logfiles.
 -			 */
 -			if (verbose)
 -				printf("Ignoring %s which does not "
 -				    "match time format\n", dp->d_name);
 -			continue;
 -		}
 -
 -		for (c = 0; c < COMPRESS_TYPES; c++)
 -			if (strcmp(s, compress_type[c].suffix) == 0)
 -				valid = 1;
 -		if (valid != 1) {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name, s);
 -			continue;
 -		}
 -
  		/*
  		 * We should now have old an old rotated logfile, so
  		 * add it to the 'list'.
 @@ -2254,13 +2269,66 @@ sizefile(const char *file)
  	return (kbytes(dbtob(sb.st_blocks)));
  }
  
 -/* Return the age of old log file (file.0) */
 +/*
 + * Return the mtime of the most recent archive of the logfile, using timestamp
 + * based filenames.
 + */
 +static time_t
 +mtime_old_timelog(const char *file)
 +{
 +	struct stat sb;
 +	struct tm tm;
 +	int dir_fd;
 +	time_t t;
 +	struct dirent *dp;
 +	DIR *dirp;
 +	char *s, *logfname, *dir;
 +
 +	t = -1;
 +
 +	if ((dir = dirname(file)) == NULL) {
 +		warn("dirname() of '%s'", file);
 +		return (t);
 +	}
 +	if ((s = basename(file)) == NULL) {
 +		warn("basename() of '%s'", file);
 +		return (t);
 +	} else if (s[0] == '/') {
 +		warnx("Invalid log filename '%s'", s);
 +		return (t);
 +	} else if ((logfname = strdup(s)) == NULL)
 +		err(1, "strdup()");
 +
 +	if ((dirp = opendir(dir)) == NULL) {
 +		warn("Cannot open log directory '%s'", dir);
 +		return (t);
 +	}
 +	dir_fd = dirfd(dirp);
 +	/* Open the archive dir and find the most recent archive of logfname. */
 +	while ((dp = readdir(dirp)) != NULL) {
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
 +			continue;
 +
 +		if (fstatat(dir_fd, logfname, &sb, 0) == -1) {
 +			warn("Cannot stat '%s'", file);
 +			continue;
 +		}
 +		if (t < sb.st_mtime)
 +			t = sb.st_mtime;
 +	}
 +	closedir(dirp);
 +
 +	return (t);
 +}
 +
 +/* Return the age in hours of the most recent archive of the logfile. */
  static int
 -age_old_log(char *file)
 +age_old_log(const char *file)
  {
  	struct stat sb;
  	const char *logfile_suffix;
  	char tmp[MAXPATHLEN + sizeof(".0") + COMPRESS_SUFFIX_MAXLEN + 1];
 +	time_t mtime;
  
  	if (archtodir) {
  		char *p;
 @@ -2289,14 +2357,21 @@ age_old_log(char *file)
  		(void) strlcpy(tmp, file, sizeof(tmp));
  	}
  
 -	strlcat(tmp, ".0", sizeof(tmp));
 -	logfile_suffix = get_logfile_suffix(tmp);
 -	if (logfile_suffix == NULL)
 -		return (-1);
 -	(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 -	if (stat(tmp, &sb) < 0)
 -		return (-1);
 -	return ((int)(ptimeget_secs(timenow) - sb.st_mtime + 1800) / 3600);
 +	if (timefnamefmt != NULL) {
 +		mtime = mtime_old_timelog(tmp);
 +		if (mtime == -1)
 +			return (-1);
 +	} else {
 +		strlcat(tmp, ".0", sizeof(tmp));
 +		logfile_suffix = get_logfile_suffix(tmp);
 +		if (logfile_suffix == NULL)
 +			return (-1);
 +		(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 +		if (stat(tmp, &sb) < 0)
 +			return (-1);
 +		mtime = sb.st_mtime;
 +	}
 +	return ((int)(ptimeget_secs(timenow) - mtime + 1800) / 3600);
  }
  
  /* Skip Over Blanks */
 
 --FkmkrVfFsRoUs1wW
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="newsyslog_dash_t_fix_stable_8.patch"
 
 diff --git a/usr.sbin/newsyslog/newsyslog.c b/usr.sbin/newsyslog/newsyslog.c
 index eaa8190..e788d80 100644
 --- a/usr.sbin/newsyslog/newsyslog.c
 +++ b/usr.sbin/newsyslog/newsyslog.c
 @@ -274,7 +274,7 @@ static void parse_args(int argc, char **argv);
  static int parse_doption(const char *doption);
  static void usage(void);
  static int log_trim(const char *logname, const struct conf_entry *log_ent);
 -static int age_old_log(char *file);
 +static int age_old_log(const char *file);
  static void savelog(char *from, char *to);
  static void createdir(const struct conf_entry *ent, char *dirpart);
  static void createlog(const struct conf_entry *ent);
 @@ -1442,15 +1442,74 @@ oldlog_entry_compare(const void *a, const void *b)
  }
  
  /*
 + * Check whether the file corresponding to dp is an archive of the logfile
 + * logfname, based on the timefnamefmt format string. Return true and fill out
 + * tm if this is the case; otherwise return false.
 +*/
 +static int
 +validate_old_timelog(const struct dirent *dp, const char *logfname, struct tm *tm)
 +{
 +	size_t logfname_len;
 +	char *s;
 +	int c;
 +
 +	logfname_len = strlen(logfname);
 +
 +	if (dp->d_type != DT_REG)
 +		return (0);
 +	/* Ignore everything but files with our logfile prefix. */
 +	if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 +		return (0);
 +	/* Ignore the actual non-rotated logfile. */
 +	if (dp->d_namlen == logfname_len)
 +		return (0);
 +
 +	/*
 +	 * Make sure we created have found a logfile, so the
 +	 * postfix is valid, IE format is: '.<time>(.[bgx]z)?'.
 +	 */
 +	if (dp->d_name[logfname_len] != '.') {
 +		if (verbose)
 +			printf("Ignoring %s which has unexpected "
 +			    "extension '%s'\n", dp->d_name,
 +			    &dp->d_name[logfname_len]);
 +		return (0);
 +	}
 +	if ((s = strptime(&dp->d_name[logfname_len + 1],
 +	    timefnamefmt, tm)) == NULL) {
 +		/*
 +		 * We could special case "old" sequentially named logfiles here,
 +		 * but we do not as that would require special handling to
 +		 * decide which one was the oldest compared to "new" time based
 +		 * logfiles.
 +		 */
 +		if (verbose)
 +			printf("Ignoring %s which does not "
 +			    "match time format\n", dp->d_name);
 +		return (0);
 +	}
 +
 +	for (c = 0; c < COMPRESS_TYPES; c++)
 +		if (strcmp(s, compress_type[c].suffix) == 0)
 +			/* We're done. */
 +			return (1);
 +
 +	if (verbose)
 +		printf("Ignoring %s which has unexpected extension '%s'\n",
 +		    dp->d_name, s);
 +
 +	return (0);
 +}
 +
 +/*
   * Delete the oldest logfiles, when using time based filenames.
   */
  static void
  delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  {
  	char *logfname, *s, *dir, errbuf[80];
 -	int dirfd, i, logcnt, max_logcnt, valid;
 +	int dirfd, i, logcnt, max_logcnt;
  	struct oldlog_entry *oldlogs;
 -	size_t logfname_len;
  	struct dirent *dp;
  	const char *cdir;
  	struct tm tm;
 @@ -1472,7 +1531,6 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  		err(1, "basename()");
  	if ((logfname = strdup(s)) == NULL)
  		err(1, "strdup()");
 -	logfname_len = strlen(logfname);
  	if (strcmp(logfname, "/") == 0)
  		errx(1, "Invalid log filename - became '/'");
  
 @@ -1484,51 +1542,9 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  		err(1, "Cannot open log directory '%s'", dir);
  	dirfd = dirfd(dirp);
  	while ((dp = readdir(dirp)) != NULL) {
 -		if (dp->d_type != DT_REG)
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
  			continue;
  
 -		/* Ignore everything but files with our logfile prefix */
 -		if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 -			continue;
 -		/* Ignore the actual non-rotated logfile */
 -		if (dp->d_namlen == logfname_len)
 -			continue;
 -		/*
 -		 * Make sure we created have found a logfile, so the
 -		 * postfix is valid, IE format is: '.<time>(.[bg]z)?'.
 -		 */
 -		if (dp->d_name[logfname_len] != '.') {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name,
 -				    &dp->d_name[logfname_len]);
 -			continue;
 -		}
 -		if ((s = strptime(&dp->d_name[logfname_len + 1],
 -			    timefnamefmt, &tm)) == NULL) {
 -			/*
 -			 * We could special case "old" sequentially
 -			 * named logfiles here, but we do not as that
 -			 * would require special handling to decide
 -			 * which one was the oldest compared to "new"
 -			 * time based logfiles.
 -			 */
 -			if (verbose)
 -				printf("Ignoring %s which does not "
 -				    "match time format\n", dp->d_name);
 -			continue;
 -		}
 -
 -		for (int c = 0; c < COMPRESS_TYPES; c++)
 -			if (strcmp(s, compress_type[c].suffix) == 0)
 -				valid = 1;
 -		if (valid != 1) {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name, s);
 -			continue;
 -		}
 -
  		/*
  		 * We should now have old an old rotated logfile, so
  		 * add it to the 'list'.
 @@ -2207,13 +2223,66 @@ sizefile(const char *file)
  	return (kbytes(dbtob(sb.st_blocks)));
  }
  
 -/* Return the age of old log file (file.0) */
 +/*
 + * Return the mtime of the most recent archive of the logfile, using timestamp
 + * based filenames.
 + */
 +static time_t
 +mtime_old_timelog(const char *file)
 +{
 +	struct stat sb;
 +	struct tm tm;
 +	int dir_fd;
 +	time_t t;
 +	struct dirent *dp;
 +	DIR *dirp;
 +	char *s, *logfname, *dir;
 +
 +	t = -1;
 +
 +	if ((dir = dirname(file)) == NULL) {
 +		warn("dirname() of '%s'", file);
 +		return (t);
 +	}
 +	if ((s = basename(file)) == NULL) {
 +		warn("basename() of '%s'", file);
 +		return (t);
 +	} else if (s[0] == '/') {
 +		warnx("Invalid log filename '%s'", s);
 +		return (t);
 +	} else if ((logfname = strdup(s)) == NULL)
 +		err(1, "strdup()");
 +
 +	if ((dirp = opendir(dir)) == NULL) {
 +		warn("Cannot open log directory '%s'", dir);
 +		return (t);
 +	}
 +	dir_fd = dirfd(dirp);
 +	/* Open the archive dir and find the most recent archive of logfname. */
 +	while ((dp = readdir(dirp)) != NULL) {
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
 +			continue;
 +
 +		if (fstatat(dir_fd, logfname, &sb, 0) == -1) {
 +			warn("Cannot stat '%s'", file);
 +			continue;
 +		}
 +		if (t < sb.st_mtime)
 +			t = sb.st_mtime;
 +	}
 +	closedir(dirp);
 +
 +	return (t);
 +}
 +
 +/* Return the age in hours of the most recent archive of the logfile. */
  static int
 -age_old_log(char *file)
 +age_old_log(const char *file)
  {
  	struct stat sb;
  	const char *logfile_suffix;
  	char tmp[MAXPATHLEN + sizeof(".0") + COMPRESS_SUFFIX_MAXLEN + 1];
 +	time_t mtime;
  
  	if (archtodir) {
  		char *p;
 @@ -2242,14 +2311,21 @@ age_old_log(char *file)
  		(void) strlcpy(tmp, file, sizeof(tmp));
  	}
  
 -	strlcat(tmp, ".0", sizeof(tmp));
 -	logfile_suffix = get_logfile_suffix(tmp);
 -	if (logfile_suffix == NULL)
 -		return (-1);
 -	(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 -	if (stat(tmp, &sb) < 0)
 -		return (-1);
 -	return ((int)(ptimeget_secs(timenow) - sb.st_mtime + 1800) / 3600);
 +	if (timefnamefmt != NULL) {
 +		mtime = mtime_old_timelog(tmp);
 +		if (mtime == -1)
 +			return (-1);
 +	} else {
 +		strlcat(tmp, ".0", sizeof(tmp));
 +		logfile_suffix = get_logfile_suffix(tmp);
 +		if (logfile_suffix == NULL)
 +			return (-1);
 +		(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 +		if (stat(tmp, &sb) < 0)
 +			return (-1);
 +		mtime = sb.st_mtime;
 +	}
 +	return ((int)(ptimeget_secs(timenow) - mtime + 1800) / 3600);
  }
  
  /* Skip Over Blanks */
 
 --FkmkrVfFsRoUs1wW--

From: Marco Steinbach <coco@executive-computing.de>
To: Mark Johnston <markjdb@gmail.com>
Cc: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Fri, 28 Dec 2012 03:35:06 +0100 (CET)

 On Thu, 27 Dec 2012, Mark Johnston wrote:
 
 > Hi Eugene and Marco,
 >
 > Would you be able to test the attached patch? It follows the same
 > approach as the original patch, but I've rewritten it to avoid
 > duplicating code from delete_oldest_timelog(). I've been testing it for
 > about a day now and everything looks ok.
 >
 > I've attached a patch against HEAD and one against STABLE-8 (untested)
 > as well. I can provide patches against other branches if you need it.
 >
 > Thanks!
 > -Mark
 
 That's great news, thanks for looking into this -- I've patched against
 8.3-STABLE r243477 (i386) for starters, which seems to yield the
 expected behaviour.
 
 I'll let the machine shuffle some logs over the weekend, and then report
 back to you.
 
 MfG CoCo
 

From: Mark Johnston <markj@freebsd.org>
To: bug-followup@freebsd.org
Cc:  
Subject: [coco@executive-computing.de: Re: bin/166448: [patch] newsyslog(8):
 newsyslog -t fails to find previous rotated log]
Date: Tue, 1 Jan 2013 13:11:31 -0500

 ----- Forwarded message from Marco Steinbach <coco@executive-computing.de> -----
 
 Date: Tue, 1 Jan 2013 17:12:36 +0100 (CET)
 From: Marco Steinbach <coco@executive-computing.de>
 To: Mark Johnston <markjdb@gmail.com>
 Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find previous rotated log
 User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
 
 On Fri, 28 Dec 2012, Marco Steinbach wrote:
 
 > On Thu, 27 Dec 2012, Mark Johnston wrote:
 > 
 >> Hi Eugene and Marco,
 >> 
 >> Would you be able to test the attached patch? It follows the same
 >> approach as the original patch, but I've rewritten it to avoid
 >> duplicating code from delete_oldest_timelog(). I've been testing it for
 >> about a day now and everything looks ok.
 >> 
 >> I've attached a patch against HEAD and one against STABLE-8 (untested)
 >> as well. I can provide patches against other branches if you need it.
 >> 
 >> Thanks!
 >> -Mark
 > 
 > That's great news, thanks for looking into this -- I've patched against
 > 8.3-STABLE r243477 (i386) for starters, which seems to yield the
 > expected behaviour.
 > 
 > I'll let the machine shuffle some logs over the weekend, and then report
 > back to you.
 > 
 > MfG CoCo
 > 
 > 
 
 No anomalies with your patch applied.  Works as advertised, I'd say.
 
 MfG CoCo
 
 
 ----- End forwarded message -----
State-Changed-From-To: feedback->analyzed 
State-Changed-By: markj 
State-Changed-When: Mon Jan 7 22:09:00 UTC 2013 
State-Changed-Why:  
A fix is in review. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166448 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/166448: commit references a PR
Date: Tue, 26 Mar 2013 23:11:39 +0000 (UTC)

 Author: markj
 Date: Tue Mar 26 23:11:30 2013
 New Revision: 248776
 URL: http://svnweb.freebsd.org/changeset/base/248776
 
 Log:
   Fix interval-based rotations when the -t flag is used. In this case, find
   the most-recently archived logfile and use its mtime to determine whether
   or not to rotate, as in the non-timestamped case.
   
   Previously we would just try to use the mtime of <logfile>.0, which always
   results in a rotation since it generally doesn't exist in the -t case.
   
   PR:		bin/166448
   Approved by:	emaste (co-mentor)
   Tested by:	Marco Steinbach <coco executive-computing.de>
   MFC after:	2 weeks
 
 Modified:
   head/usr.sbin/newsyslog/newsyslog.c
 
 Modified: head/usr.sbin/newsyslog/newsyslog.c
 ==============================================================================
 --- head/usr.sbin/newsyslog/newsyslog.c	Tue Mar 26 23:04:06 2013	(r248775)
 +++ head/usr.sbin/newsyslog/newsyslog.c	Tue Mar 26 23:11:30 2013	(r248776)
 @@ -276,7 +276,7 @@ static void parse_args(int argc, char **
  static int parse_doption(const char *doption);
  static void usage(void);
  static int log_trim(const char *logname, const struct conf_entry *log_ent);
 -static int age_old_log(char *file);
 +static int age_old_log(const char *file);
  static void savelog(char *from, char *to);
  static void createdir(const struct conf_entry *ent, char *dirpart);
  static void createlog(const struct conf_entry *ent);
 @@ -1447,20 +1447,78 @@ oldlog_entry_compare(const void *a, cons
  }
  
  /*
 + * Check whether the file corresponding to dp is an archive of the logfile
 + * logfname, based on the timefnamefmt format string. Return true and fill out
 + * tm if this is the case; otherwise return false.
 + */
 +static int
 +validate_old_timelog(const struct dirent *dp, const char *logfname, struct tm *tm)
 +{
 +	size_t logfname_len;
 +	char *s;
 +	int c;
 +
 +	logfname_len = strlen(logfname);
 +
 +	if (dp->d_type != DT_REG)
 +		return (0);
 +	/* Ignore everything but files with our logfile prefix. */
 +	if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 +		return (0);
 +	/* Ignore the actual non-rotated logfile. */
 +	if (dp->d_namlen == logfname_len)
 +		return (0);
 +
 +	/*
 +	 * Make sure we created have found a logfile, so the
 +	 * postfix is valid, IE format is: '.<time>(.[bgx]z)?'.
 +	 */
 +	if (dp->d_name[logfname_len] != '.') {
 +		if (verbose)
 +			printf("Ignoring %s which has unexpected "
 +			    "extension '%s'\n", dp->d_name,
 +			    &dp->d_name[logfname_len]);
 +		return (0);
 +	}
 +	if ((s = strptime(&dp->d_name[logfname_len + 1],
 +	    timefnamefmt, tm)) == NULL) {
 +		/*
 +		 * We could special case "old" sequentially named logfiles here,
 +		 * but we do not as that would require special handling to
 +		 * decide which one was the oldest compared to "new" time based
 +		 * logfiles.
 +		 */
 +		if (verbose)
 +			printf("Ignoring %s which does not "
 +			    "match time format\n", dp->d_name);
 +		return (0);
 +	}
 +
 +	for (c = 0; c < COMPRESS_TYPES; c++)
 +		if (strcmp(s, compress_type[c].suffix) == 0)
 +			/* We're done. */
 +			return (1);
 +
 +	if (verbose)
 +		printf("Ignoring %s which has unexpected extension '%s'\n",
 +		    dp->d_name, s);
 +
 +	return (0);
 +}
 +
 +/*
   * Delete the oldest logfiles, when using time based filenames.
   */
  static void
  delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  {
  	char *logfname, *s, *dir, errbuf[80];
 -	int dir_fd, i, logcnt, max_logcnt, valid;
 +	int dir_fd, i, logcnt, max_logcnt;
  	struct oldlog_entry *oldlogs;
 -	size_t logfname_len;
  	struct dirent *dp;
  	const char *cdir;
  	struct tm tm;
  	DIR *dirp;
 -	int c;
  
  	oldlogs = malloc(MAX_OLDLOGS * sizeof(struct oldlog_entry));
  	max_logcnt = MAX_OLDLOGS;
 @@ -1478,7 +1536,6 @@ delete_oldest_timelog(const struct conf_
  		err(1, "basename()");
  	if ((logfname = strdup(s)) == NULL)
  		err(1, "strdup()");
 -	logfname_len = strlen(logfname);
  	if (strcmp(logfname, "/") == 0)
  		errx(1, "Invalid log filename - became '/'");
  
 @@ -1490,51 +1547,9 @@ delete_oldest_timelog(const struct conf_
  		err(1, "Cannot open log directory '%s'", dir);
  	dir_fd = dirfd(dirp);
  	while ((dp = readdir(dirp)) != NULL) {
 -		if (dp->d_type != DT_REG)
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
  			continue;
  
 -		/* Ignore everything but files with our logfile prefix */
 -		if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 -			continue;
 -		/* Ignore the actual non-rotated logfile */
 -		if (dp->d_namlen == logfname_len)
 -			continue;
 -		/*
 -		 * Make sure we created have found a logfile, so the
 -		 * postfix is valid, IE format is: '.<time>(.[bg]z)?'.
 -		 */
 -		if (dp->d_name[logfname_len] != '.') {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name,
 -				    &dp->d_name[logfname_len]);
 -			continue;
 -		}
 -		if ((s = strptime(&dp->d_name[logfname_len + 1],
 -			    timefnamefmt, &tm)) == NULL) {
 -			/*
 -			 * We could special case "old" sequentially
 -			 * named logfiles here, but we do not as that
 -			 * would require special handling to decide
 -			 * which one was the oldest compared to "new"
 -			 * time based logfiles.
 -			 */
 -			if (verbose)
 -				printf("Ignoring %s which does not "
 -				    "match time format\n", dp->d_name);
 -			continue;
 -		}
 -
 -		for (c = 0; c < COMPRESS_TYPES; c++)
 -			if (strcmp(s, compress_type[c].suffix) == 0)
 -				valid = 1;
 -		if (valid != 1) {
 -			if (verbose)
 -				printf("Ignoring %s which has unexpected "
 -				    "extension '%s'\n", dp->d_name, s);
 -			continue;
 -		}
 -
  		/*
  		 * We should now have old an old rotated logfile, so
  		 * add it to the 'list'.
 @@ -2260,13 +2275,66 @@ sizefile(const char *file)
  	return (kbytes(dbtob(sb.st_blocks)));
  }
  
 -/* Return the age of old log file (file.0) */
 +/*
 + * Return the mtime of the most recent archive of the logfile, using timestamp
 + * based filenames.
 + */
 +static time_t
 +mtime_old_timelog(const char *file)
 +{
 +	struct stat sb;
 +	struct tm tm;
 +	int dir_fd;
 +	time_t t;
 +	struct dirent *dp;
 +	DIR *dirp;
 +	char *s, *logfname, *dir;
 +
 +	t = -1;
 +
 +	if ((dir = dirname(file)) == NULL) {
 +		warn("dirname() of '%s'", file);
 +		return (t);
 +	}
 +	if ((s = basename(file)) == NULL) {
 +		warn("basename() of '%s'", file);
 +		return (t);
 +	} else if (s[0] == '/') {
 +		warnx("Invalid log filename '%s'", s);
 +		return (t);
 +	} else if ((logfname = strdup(s)) == NULL)
 +		err(1, "strdup()");
 +
 +	if ((dirp = opendir(dir)) == NULL) {
 +		warn("Cannot open log directory '%s'", dir);
 +		return (t);
 +	}
 +	dir_fd = dirfd(dirp);
 +	/* Open the archive dir and find the most recent archive of logfname. */
 +	while ((dp = readdir(dirp)) != NULL) {
 +		if (validate_old_timelog(dp, logfname, &tm) == 0)
 +			continue;
 +
 +		if (fstatat(dir_fd, logfname, &sb, 0) == -1) {
 +			warn("Cannot stat '%s'", file);
 +			continue;
 +		}
 +		if (t < sb.st_mtime)
 +			t = sb.st_mtime;
 +	}
 +	closedir(dirp);
 +
 +	return (t);
 +}
 +
 +/* Return the age in hours of the most recent archive of the logfile. */
  static int
 -age_old_log(char *file)
 +age_old_log(const char *file)
  {
  	struct stat sb;
  	const char *logfile_suffix;
  	char tmp[MAXPATHLEN + sizeof(".0") + COMPRESS_SUFFIX_MAXLEN + 1];
 +	time_t mtime;
  
  	if (archtodir) {
  		char *p;
 @@ -2295,14 +2363,22 @@ age_old_log(char *file)
  		(void) strlcpy(tmp, file, sizeof(tmp));
  	}
  
 -	strlcat(tmp, ".0", sizeof(tmp));
 -	logfile_suffix = get_logfile_suffix(tmp);
 -	if (logfile_suffix == NULL)
 -		return (-1);
 -	(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 -	if (stat(tmp, &sb) < 0)
 -		return (-1);
 -	return ((int)(ptimeget_secs(timenow) - sb.st_mtime + 1800) / 3600);
 +	if (timefnamefmt != NULL) {
 +		mtime = mtime_old_timelog(tmp);
 +		if (mtime == -1)
 +			return (-1);
 +	} else {
 +		strlcat(tmp, ".0", sizeof(tmp));
 +		logfile_suffix = get_logfile_suffix(tmp);
 +		if (logfile_suffix == NULL)
 +			return (-1);
 +		(void) strlcat(tmp, logfile_suffix, sizeof(tmp));
 +		if (stat(tmp, &sb) < 0)
 +			return (-1);
 +		mtime = sb.st_mtime;
 +	}
 +
 +	return ((int)(ptimeget_secs(timenow) - mtime + 1800) / 3600);
  }
  
  /* Skip Over Blanks */
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: analyzed->patched 
State-Changed-By: markj 
State-Changed-When: Tue Mar 26 23:21:49 UTC 2013 
State-Changed-Why:  
Fixed in r248776. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166448 
State-Changed-From-To: patched->closed 
State-Changed-By: markj 
State-Changed-When: Fri May 10 20:59:09 UTC 2013 
State-Changed-Why:  
Committed and MFCed, thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166448 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
Cc:  
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sat, 11 May 2013 14:44:36 +0200

 A similar issue may result because some file systems such as NFS may set
 the d_type field to DT_UNKNOWN. In that case,
 fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 
 "Normal" configurations using UFS or ZFS are unaffected, though.
 
 -- 
 Jilles Tjoelker

From: Mark Johnston <markj@freebsd.org>
To: Jilles Tjoelker <jilles@stack.nl>, bug-followup@freebsd.org,
	eugen@grosbein.pp.ru
Cc:  
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sat, 11 May 2013 18:30:13 -0400

 --ew6BAiZeqk4r7MaW
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 On Sat, May 11, 2013 at 12:50:01PM +0000, Jilles Tjoelker wrote:
 > The following reply was made to PR bin/166448; it has been noted by GNATS.
 > 
 > From: Jilles Tjoelker <jilles@stack.nl>
 > To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
 > Cc:  
 > Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 >  previous rotated log
 > Date: Sat, 11 May 2013 14:44:36 +0200
 > 
 >  A similar issue may result because some file systems such as NFS may set
 >  the d_type field to DT_UNKNOWN. In that case,
 >  fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 >  
 >  "Normal" configurations using UFS or ZFS are unaffected, though.
 
 Hi Jilles,
 
 It looks like it's even worse actually: delete_oldest_timelog() uses the
 same check to determine whether to delete an archived logfile. So if the
 log directory is on a filesystem with this property, old logs will never
 be deleted. Like the original problem, this isn't an issue if -t isn't
 used.
 
 I was able to reproduce this on NFSv4 and NFSv3 and the attached patch
 fixes the problem. Would you be able to review it when you have a
 chance?
 
 Thanks,
 -Mark
 
 --ew6BAiZeqk4r7MaW
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename=patch
 
 diff --git a/usr.sbin/newsyslog/newsyslog.c b/usr.sbin/newsyslog/newsyslog.c
 index 08598ac..299c989 100644
 --- a/usr.sbin/newsyslog/newsyslog.c
 +++ b/usr.sbin/newsyslog/newsyslog.c
 @@ -1452,16 +1452,27 @@ oldlog_entry_compare(const void *a, const void *b)
   * tm if this is the case; otherwise return false.
   */
  static int
 -validate_old_timelog(const struct dirent *dp, const char *logfname, struct tm *tm)
 +validate_old_timelog(int fd, const struct dirent *dp, const char *logfname,
 +    struct tm *tm)
  {
  	size_t logfname_len;
 +	struct stat sb;
  	char *s;
  	int c;
  
  	logfname_len = strlen(logfname);
  
 -	if (dp->d_type != DT_REG)
 -		return (0);
 +	if (dp->d_type != DT_REG) {
 +		/*
 +		 * Some filesystems (e.g. NFS) don't fill out the d_type field
 +		 * and leave it set to DT_UNKNOWN; in this case we must obtain
 +		 * the file type ourselves.
 +		 */
 +		if (dp->d_type != DT_UNKNOWN ||
 +		    fstatat(fd, dp->d_name, &sb, AT_SYMLINK_NOFOLLOW) != 0 ||
 +		    !S_ISREG(sb.st_mode))
 +			return (0);
 +	}
  	/* Ignore everything but files with our logfile prefix. */
  	if (strncmp(dp->d_name, logfname, logfname_len) != 0)
  		return (0);
 @@ -1547,7 +1558,7 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
  		err(1, "Cannot open log directory '%s'", dir);
  	dir_fd = dirfd(dirp);
  	while ((dp = readdir(dirp)) != NULL) {
 -		if (validate_old_timelog(dp, logfname, &tm) == 0)
 +		if (validate_old_timelog(dir_fd, dp, logfname, &tm) == 0)
  			continue;
  
  		/*
 @@ -2312,7 +2323,7 @@ mtime_old_timelog(const char *file)
  	dir_fd = dirfd(dirp);
  	/* Open the archive dir and find the most recent archive of logfname. */
  	while ((dp = readdir(dirp)) != NULL) {
 -		if (validate_old_timelog(dp, logfname, &tm) == 0)
 +		if (validate_old_timelog(dir_fd, dp, logfname, &tm) == 0)
  			continue;
  
  		if (fstatat(dir_fd, logfname, &sb, 0) == -1) {
 
 --ew6BAiZeqk4r7MaW--

From: Jilles Tjoelker <jilles@stack.nl>
To: Mark Johnston <markj@freebsd.org>
Cc: bug-followup@freebsd.org, eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sun, 12 May 2013 00:56:33 +0200

 On Sat, May 11, 2013 at 06:30:13PM -0400, Mark Johnston wrote:
 > On Sat, May 11, 2013 at 12:50:01PM +0000, Jilles Tjoelker wrote:
 > > The following reply was made to PR bin/166448; it has been noted by GNATS.
 > > 
 > > From: Jilles Tjoelker <jilles@stack.nl>
 > > To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
 > > Cc:  
 > > Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 > >  previous rotated log
 > > Date: Sat, 11 May 2013 14:44:36 +0200
 > > 
 > >  A similar issue may result because some file systems such as NFS may set
 > >  the d_type field to DT_UNKNOWN. In that case,
 > >  fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 > >  
 > >  "Normal" configurations using UFS or ZFS are unaffected, though.
 
 > Hi Jilles,
 
 > It looks like it's even worse actually: delete_oldest_timelog() uses the
 > same check to determine whether to delete an archived logfile. So if the
 > log directory is on a filesystem with this property, old logs will never
 > be deleted. Like the original problem, this isn't an issue if -t isn't
 > used.
 
 > I was able to reproduce this on NFSv4 and NFSv3 and the attached patch
 > fixes the problem. Would you be able to review it when you have a
 > chance?
 
 This looks like it can work but mtime_old_timelog() now does a
 potentially redundant fstatat() (it is different in that it follows
 symlinks but symlinks only happen if someone switched a regular file for
 a symlink concurrently).
 
 > diff --git a/usr.sbin/newsyslog/newsyslog.c b/usr.sbin/newsyslog/newsyslog.c
 > index 08598ac..299c989 100644
 > --- a/usr.sbin/newsyslog/newsyslog.c
 > +++ b/usr.sbin/newsyslog/newsyslog.c
 > @@ -1452,16 +1452,27 @@ oldlog_entry_compare(const void *a, const void *b)
 >   * tm if this is the case; otherwise return false.
 >   */
 >  static int
 > -validate_old_timelog(const struct dirent *dp, const char *logfname, struct tm *tm)
 > +validate_old_timelog(int fd, const struct dirent *dp, const char *logfname,
 > +    struct tm *tm)
 >  {
 >  	size_t logfname_len;
 > +	struct stat sb;
 >  	char *s;
 >  	int c;
 >  
 >  	logfname_len = strlen(logfname);
 >  
 > -	if (dp->d_type != DT_REG)
 > -		return (0);
 > +	if (dp->d_type != DT_REG) {
 > +		/*
 > +		 * Some filesystems (e.g. NFS) don't fill out the d_type field
 > +		 * and leave it set to DT_UNKNOWN; in this case we must obtain
 > +		 * the file type ourselves.
 > +		 */
 > +		if (dp->d_type != DT_UNKNOWN ||
 > +		    fstatat(fd, dp->d_name, &sb, AT_SYMLINK_NOFOLLOW) != 0 ||
 > +		    !S_ISREG(sb.st_mode))
 > +			return (0);
 > +	}
 >  	/* Ignore everything but files with our logfile prefix. */
 >  	if (strncmp(dp->d_name, logfname, logfname_len) != 0)
 >  		return (0);
 > @@ -1547,7 +1558,7 @@ delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir)
 >  		err(1, "Cannot open log directory '%s'", dir);
 >  	dir_fd = dirfd(dirp);
 >  	while ((dp = readdir(dirp)) != NULL) {
 > -		if (validate_old_timelog(dp, logfname, &tm) == 0)
 > +		if (validate_old_timelog(dir_fd, dp, logfname, &tm) == 0)
 >  			continue;
 >  
 >  		/*
 > @@ -2312,7 +2323,7 @@ mtime_old_timelog(const char *file)
 >  	dir_fd = dirfd(dirp);
 >  	/* Open the archive dir and find the most recent archive of logfname. */
 >  	while ((dp = readdir(dirp)) != NULL) {
 > -		if (validate_old_timelog(dp, logfname, &tm) == 0)
 > +		if (validate_old_timelog(dir_fd, dp, logfname, &tm) == 0)
 >  			continue;
 >  
 >  		if (fstatat(dir_fd, logfname, &sb, 0) == -1) {
 
 -- 
 Jilles Tjoelker

From: Mark Johnston <markj@freebsd.org>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@freebsd.org, eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sat, 11 May 2013 19:41:35 -0400

 On Sun, May 12, 2013 at 12:56:33AM +0200, Jilles Tjoelker wrote:
 > On Sat, May 11, 2013 at 06:30:13PM -0400, Mark Johnston wrote:
 > > On Sat, May 11, 2013 at 12:50:01PM +0000, Jilles Tjoelker wrote:
 > > > The following reply was made to PR bin/166448; it has been noted by GNATS.
 > > > 
 > > > From: Jilles Tjoelker <jilles@stack.nl>
 > > > To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
 > > > Cc:  
 > > > Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 > > >  previous rotated log
 > > > Date: Sat, 11 May 2013 14:44:36 +0200
 > > > 
 > > >  A similar issue may result because some file systems such as NFS may set
 > > >  the d_type field to DT_UNKNOWN. In that case,
 > > >  fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 > > >  
 > > >  "Normal" configurations using UFS or ZFS are unaffected, though.
 > 
 > > Hi Jilles,
 > 
 > > It looks like it's even worse actually: delete_oldest_timelog() uses the
 > > same check to determine whether to delete an archived logfile. So if the
 > > log directory is on a filesystem with this property, old logs will never
 > > be deleted. Like the original problem, this isn't an issue if -t isn't
 > > used.
 > 
 > > I was able to reproduce this on NFSv4 and NFSv3 and the attached patch
 > > fixes the problem. Would you be able to review it when you have a
 > > chance?
 > 
 > This looks like it can work but mtime_old_timelog() now does a
 > potentially redundant fstatat() (it is different in that it follows
 > symlinks but symlinks only happen if someone switched a regular file for
 > a symlink concurrently).
 
 Hm, I don't really see a nice way to avoid the reundant fstatat() without
 adding an unnecessary fstatat() call in more common cases. With my
 patch, the redundant call only happens on matching archived files for a
 log entry which uses interval-based rotation on NFS. If I do something
 like change validate_old_timelog() to fill out a struct stat for the
 caller, it'd either have to call fstatat(2) unconditionally (which is
 unnecessary most of the time), or it would only fill out the struct stat
 if d_type == DT_UNKNOWN, which forces mtime_old_timelog() to do the same
 check and is somewhat ugly IMHO.
 
 I think the fstatat() call in mtime_old_timelog() should be changed to
 use AT_SYMLINK_NOFOLLOW though, for consistency at least. At the moment
 it doesn't make much sense to use newsyslog on anything but regular
 files anyway. What do you think?

From: Jilles Tjoelker <jilles@stack.nl>
To: Mark Johnston <markj@freebsd.org>
Cc: bug-followup@freebsd.org, eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sun, 12 May 2013 02:02:25 +0200

 On Sat, May 11, 2013 at 07:41:35PM -0400, Mark Johnston wrote:
 > On Sun, May 12, 2013 at 12:56:33AM +0200, Jilles Tjoelker wrote:
 > > On Sat, May 11, 2013 at 06:30:13PM -0400, Mark Johnston wrote:
 > > > On Sat, May 11, 2013 at 12:50:01PM +0000, Jilles Tjoelker wrote:
 > > > > The following reply was made to PR bin/166448; it has been noted by GNATS.
 > > > > 
 > > > > From: Jilles Tjoelker <jilles@stack.nl>
 > > > > To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
 > > > > Cc:  
 > > > > Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 > > > >  previous rotated log
 > > > > Date: Sat, 11 May 2013 14:44:36 +0200
 > > > > 
 > > > >  A similar issue may result because some file systems such as NFS may set
 > > > >  the d_type field to DT_UNKNOWN. In that case,
 > > > >  fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 > > > >  
 > > > >  "Normal" configurations using UFS or ZFS are unaffected, though.
 
 > > > It looks like it's even worse actually: delete_oldest_timelog() uses the
 > > > same check to determine whether to delete an archived logfile. So if the
 > > > log directory is on a filesystem with this property, old logs will never
 > > > be deleted. Like the original problem, this isn't an issue if -t isn't
 > > > used.
 
 > > > I was able to reproduce this on NFSv4 and NFSv3 and the attached patch
 > > > fixes the problem. Would you be able to review it when you have a
 > > > chance?
 
 > > This looks like it can work but mtime_old_timelog() now does a
 > > potentially redundant fstatat() (it is different in that it follows
 > > symlinks but symlinks only happen if someone switched a regular file for
 > > a symlink concurrently).
 
 > Hm, I don't really see a nice way to avoid the reundant fstatat() without
 > adding an unnecessary fstatat() call in more common cases. With my
 > patch, the redundant call only happens on matching archived files for a
 > log entry which uses interval-based rotation on NFS. If I do something
 > like change validate_old_timelog() to fill out a struct stat for the
 > caller, it'd either have to call fstatat(2) unconditionally (which is
 > unnecessary most of the time), or it would only fill out the struct stat
 > if d_type == DT_UNKNOWN, which forces mtime_old_timelog() to do the same
 > check and is somewhat ugly IMHO.
 
 It can be left as it is. Someone who thinks it is too slow with
 DT_UNKNOWN-generating filesystems can change it.
 
 > I think the fstatat() call in mtime_old_timelog() should be changed to
 > use AT_SYMLINK_NOFOLLOW though, for consistency at least. At the moment
 > it doesn't make much sense to use newsyslog on anything but regular
 > files anyway. What do you think?
 
 For consistency it would be good to change it to AT_SYMLINK_NOFOLLOW,
 yes.
 
 -- 
 Jilles Tjoelker

From: Mark Johnston <markj@freebsd.org>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@freebsd.org, eugen@grosbein.pp.ru
Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 previous rotated log
Date: Sat, 11 May 2013 22:02:40 -0400

 On Sun, May 12, 2013 at 02:02:25AM +0200, Jilles Tjoelker wrote:
 > On Sat, May 11, 2013 at 07:41:35PM -0400, Mark Johnston wrote:
 > > On Sun, May 12, 2013 at 12:56:33AM +0200, Jilles Tjoelker wrote:
 > > > On Sat, May 11, 2013 at 06:30:13PM -0400, Mark Johnston wrote:
 > > > > On Sat, May 11, 2013 at 12:50:01PM +0000, Jilles Tjoelker wrote:
 > > > > > The following reply was made to PR bin/166448; it has been noted by GNATS.
 > > > > > 
 > > > > > From: Jilles Tjoelker <jilles@stack.nl>
 > > > > > To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
 > > > > > Cc:  
 > > > > > Subject: Re: bin/166448: [patch] newsyslog(8): newsyslog -t fails to find
 > > > > >  previous rotated log
 > > > > > Date: Sat, 11 May 2013 14:44:36 +0200
 > > > > > 
 > > > > >  A similar issue may result because some file systems such as NFS may set
 > > > > >  the d_type field to DT_UNKNOWN. In that case,
 > > > > >  fstatat(AT_SYMLINK_NOFOLLOW) has to be used to determine the file type.
 > > > > >  
 > > > > >  "Normal" configurations using UFS or ZFS are unaffected, though.
 > 
 > > > > It looks like it's even worse actually: delete_oldest_timelog() uses the
 > > > > same check to determine whether to delete an archived logfile. So if the
 > > > > log directory is on a filesystem with this property, old logs will never
 > > > > be deleted. Like the original problem, this isn't an issue if -t isn't
 > > > > used.
 > 
 > > > > I was able to reproduce this on NFSv4 and NFSv3 and the attached patch
 > > > > fixes the problem. Would you be able to review it when you have a
 > > > > chance?
 > 
 > > > This looks like it can work but mtime_old_timelog() now does a
 > > > potentially redundant fstatat() (it is different in that it follows
 > > > symlinks but symlinks only happen if someone switched a regular file for
 > > > a symlink concurrently).
 > 
 > > Hm, I don't really see a nice way to avoid the reundant fstatat() without
 > > adding an unnecessary fstatat() call in more common cases. With my
 > > patch, the redundant call only happens on matching archived files for a
 > > log entry which uses interval-based rotation on NFS. If I do something
 > > like change validate_old_timelog() to fill out a struct stat for the
 > > caller, it'd either have to call fstatat(2) unconditionally (which is
 > > unnecessary most of the time), or it would only fill out the struct stat
 > > if d_type == DT_UNKNOWN, which forces mtime_old_timelog() to do the same
 > > check and is somewhat ugly IMHO.
 > 
 > It can be left as it is. Someone who thinks it is too slow with
 > DT_UNKNOWN-generating filesystems can change it.
 > 
 > > I think the fstatat() call in mtime_old_timelog() should be changed to
 > > use AT_SYMLINK_NOFOLLOW though, for consistency at least. At the moment
 > > it doesn't make much sense to use newsyslog on anything but regular
 > > files anyway. What do you think?
 > 
 > For consistency it would be good to change it to AT_SYMLINK_NOFOLLOW,
 > yes.
 
 Ok, I've fixed this problem in r250545. Thanks a lot for pointing it
 out!
>Unformatted:
