From nobody@FreeBSD.org  Wed Oct 23 09:39:38 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTP id 303C863C
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 23 Oct 2013 09:39:38 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 1DA032065
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 23 Oct 2013 09:39:38 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r9N9dbKu015489
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 23 Oct 2013 09:39:37 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r9N9dbgC015488;
	Wed, 23 Oct 2013 09:39:37 GMT
	(envelope-from nobody)
Message-Id: <201310230939.r9N9dbgC015488@oldred.freebsd.org>
Date: Wed, 23 Oct 2013 09:39:37 GMT
From: Grzegorz <kostka.grzegorz@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ext2fs hash function incompatible with linux
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         183230
>Category:       kern
>Synopsis:       [ext2fs] ext2fs hash function incompatible with linux
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    pfg
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Oct 23 09:40:05 UTC 2013
>Closed-Date:    Thu Dec 26 01:16:10 UTC 2013
>Last-Modified:  Thu Dec 26 01:20:01 UTC 2013
>Originator:     Grzegorz
>Release:        9
>Organization:
.
>Environment:
.
>Description:
Hi,


probably I found a bug. Half md4 hash output value is assigned to hash and hash_minor. Linux fsck reports problems (HTREE max value is wrong) on ext partition with a lot directory entries.

FreeBSD implementation:
http://fxr.watson.org/fxr/source/fs/ext2fs/ext2_hash.c?v=FREEBSD9#L242

Linux implementation:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/ext3/hash.c

Look EXT2_HTREE_HALF_MD4:

FreeBSD:
major = hash[0];
minor = hash[1];

Linux:
minor_hash = buf[2];
hash = buf[1];



>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun Oct 27 22:11:01 UTC 2013 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=183230 
Responsible-Changed-From-To: freebsd-fs->pfg 
Responsible-Changed-By: pfg 
Responsible-Changed-When: Mon Oct 28 18:34:51 UTC 2013 
Responsible-Changed-Why:  
I will take this one. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=183230 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/183230: commit references a PR
Date: Mon, 28 Oct 2013 19:46:09 +0000 (UTC)

 Author: pfg
 Date: Mon Oct 28 19:46:01 2013
 New Revision: 257267
 URL: http://svnweb.freebsd.org/changeset/base/257267
 
 Log:
   MFC	r255338:
   
   ext2fs: temporarily disable htree directory index.
   
   In addition to our implementation not having workarounds for hash
   collisions, it appears we also have a compatibility problem.
   
   For now disable the htree code until we are able to re-examine
   both issues.
   
   PR:	kern/183230
 
 Modified:
   stable/9/sys/fs/ext2fs/ext2_htree.c
   stable/9/sys/fs/ext2fs/ext2_lookup.c
 Directory Properties:
   stable/9/sys/   (props changed)
   stable/9/sys/fs/   (props changed)
 
 Modified: stable/9/sys/fs/ext2fs/ext2_htree.c
 ==============================================================================
 --- stable/9/sys/fs/ext2fs/ext2_htree.c	Mon Oct 28 19:30:09 2013	(r257266)
 +++ stable/9/sys/fs/ext2fs/ext2_htree.c	Mon Oct 28 19:46:01 2013	(r257267)
 @@ -89,10 +89,12 @@ static int	ext2_htree_writebuf(struct ex
  int
  ext2_htree_has_idx(struct inode *ip)
  {
 +#ifdef EXT2FS_HTREE
  	if (EXT2_HAS_COMPAT_FEATURE(ip->i_e2fs, EXT2F_COMPAT_DIRHASHINDEX) &&
  	    ip->i_flags & EXT4_INDEX)
  		return (1);
  	else
 +#endif
  		return (0);
  }
  
 
 Modified: stable/9/sys/fs/ext2fs/ext2_lookup.c
 ==============================================================================
 --- stable/9/sys/fs/ext2fs/ext2_lookup.c	Mon Oct 28 19:30:09 2013	(r257266)
 +++ stable/9/sys/fs/ext2fs/ext2_lookup.c	Mon Oct 28 19:46:01 2013	(r257267)
 @@ -884,6 +884,7 @@ ext2_direnter(struct inode *ip, struct v
  	bcopy(cnp->cn_nameptr, newdir.e2d_name, (unsigned)cnp->cn_namelen + 1);
  	newentrysize = EXT2_DIR_REC_LEN(newdir.e2d_namlen);
  
 +#ifdef EXT2FS_HTREE
  	if (ext2_htree_has_idx(dp)) {
  		error = ext2_htree_add_entry(dvp, &newdir, cnp);
  		if (error) {
 @@ -904,6 +905,7 @@ ext2_direnter(struct inode *ip, struct v
  			return ext2_htree_create_index(dvp, cnp, &newdir);
  		}
  	}
 +#endif	/* EXT2FS_HTREE */
  
  	if (dp->i_count == 0) {
  		/*
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: Pedro Giffuni <pfg@FreeBSD.org>
To: bug-followup@FreeBSD.org, kostka.grzegorz@gmail.com
Cc:  
Subject: Re: kern/183230: [ext2fs] ext2fs hash function incompatible with
 linux
Date: Mon, 28 Oct 2013 15:27:35 -0500

 This is a multi-part message in MIME format.
 --------------090908010500000507030704
 Content-Type: text/plain; charset=UTF-8; format=flowed
 Content-Transfer-Encoding: 7bit
 
 Oops, I meant this.
 
 --------------090908010500000507030704
 Content-Type: text/plain; charset=UTF-8;
  name="patch-halfmd4.txt"
 Content-Transfer-Encoding: base64
 Content-Disposition: attachment;
  filename="patch-halfmd4.txt"
 
 SW5kZXg6IGZzL2V4dDJmcy9leHQyX2hhc2guYwo9PT09PT09PT09PT09PT09PT09PT09PT09
 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBmcy9leHQy
 ZnMvZXh0Ml9oYXNoLmMJKHJldmlzaW9uZSAyNTcyNjQpCisrKyBmcy9leHQyZnMvZXh0Ml9o
 YXNoLmMJKGNvcGlhIGxvY2FsZSkKQEAgLTI4OSw4ICsyODksOCBAQAogCQkJbGVuIC09IDMy
 OwogCQkJbmFtZSArPSAzMjsKIAkJfQotCQltYWpvciA9IGhhc2hbMF07Ci0JCW1pbm9yID0g
 aGFzaFsxXTsKKwkJbWFqb3IgPSBoYXNoWzFdOworCQltaW5vciA9IGhhc2hbMl07CiAJCWJy
 ZWFrOwogCWRlZmF1bHQ6CiAJCWdvdG8gZXJyb3I7Cg==
 --------------090908010500000507030704--

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/183230: commit references a PR
Date: Mon, 23 Dec 2013 19:47:42 +0000 (UTC)

 Author: pfg
 Date: Mon Dec 23 19:47:34 2013
 New Revision: 259780
 URL: http://svnweb.freebsd.org/changeset/base/259780
 
 Log:
   ext2fs: make the hashing algorithm match the linux code.
   
   There appears to be a hash function compatibility issue.
   The code is currently disabled but fix it nevertheless.
   
   PR:		kern/183230
   MFC after:	3 days
 
 Modified:
   head/sys/fs/ext2fs/ext2_hash.c
 
 Modified: head/sys/fs/ext2fs/ext2_hash.c
 ==============================================================================
 --- head/sys/fs/ext2fs/ext2_hash.c	Mon Dec 23 19:29:07 2013	(r259779)
 +++ head/sys/fs/ext2fs/ext2_hash.c	Mon Dec 23 19:47:34 2013	(r259780)
 @@ -289,8 +289,8 @@ ext2_htree_hash(const char *name, int le
  			len -= 32;
  			name += 32;
  		}
 -		major = hash[0];
 -		minor = hash[1];
 +		major = hash[1];
 +		minor = hash[2];
  		break;
  	default:
  		goto error;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: pfg 
State-Changed-When: Thu Dec 26 01:15:01 UTC 2013 
State-Changed-Why:  
Fix commited and MFC. 
Thank you for taking a look at the code! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=183230 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/183230: commit references a PR
Date: Thu, 26 Dec 2013 01:11:59 +0000 (UTC)

 Author: pfg
 Date: Thu Dec 26 01:11:51 2013
 New Revision: 259903
 URL: http://svnweb.freebsd.org/changeset/base/259903
 
 Log:
   MFC	r252397, r258904, r259780:
   Small ext2fs updates.
   
   Use the unsigned random() range in i_gen.
   Add two new reserved inodes.
   Make the hashing algorithm match the linux code.
   
   PR:		kern/183230
 
 Modified:
   stable/9/sys/fs/ext2fs/ext2_dinode.h
   stable/9/sys/fs/ext2fs/ext2_hash.c
   stable/9/sys/fs/ext2fs/ext2_vfsops.c
 Directory Properties:
   stable/9/sys/   (props changed)
   stable/9/sys/fs/   (props changed)
 
 Modified: stable/9/sys/fs/ext2fs/ext2_dinode.h
 ==============================================================================
 --- stable/9/sys/fs/ext2fs/ext2_dinode.h	Thu Dec 26 00:11:19 2013	(r259902)
 +++ stable/9/sys/fs/ext2fs/ext2_dinode.h	Thu Dec 26 01:11:51 2013	(r259903)
 @@ -44,6 +44,8 @@
  #define	EXT2_UNDELDIRINO	((ino_t)6)
  #define	EXT2_RESIZEINO		((ino_t)7)
  #define	EXT2_JOURNALINO		((ino_t)8)
 +#define	EXT2_EXCLUDEINO		((ino_t)9)
 +#define	EXT2_REPLICAINO		((ino_t)10)
  #define	EXT2_FIRSTINO		((ino_t)11)
  
  /*
 
 Modified: stable/9/sys/fs/ext2fs/ext2_hash.c
 ==============================================================================
 --- stable/9/sys/fs/ext2fs/ext2_hash.c	Thu Dec 26 00:11:19 2013	(r259902)
 +++ stable/9/sys/fs/ext2fs/ext2_hash.c	Thu Dec 26 01:11:51 2013	(r259903)
 @@ -289,8 +289,8 @@ ext2_htree_hash(const char *name, int le
  			len -= 32;
  			name += 32;
  		}
 -		major = hash[0];
 -		minor = hash[1];
 +		major = hash[1];
 +		minor = hash[2];
  		break;
  	default:
  		goto error;
 
 Modified: stable/9/sys/fs/ext2fs/ext2_vfsops.c
 ==============================================================================
 --- stable/9/sys/fs/ext2fs/ext2_vfsops.c	Thu Dec 26 00:11:19 2013	(r259902)
 +++ stable/9/sys/fs/ext2fs/ext2_vfsops.c	Thu Dec 26 01:11:51 2013	(r259903)
 @@ -1006,7 +1006,7 @@ ext2_vget(struct mount *mp, ino_t ino, i
  	 * already have one. This should only happen on old filesystems.
  	 */
  	if (ip->i_gen == 0) {
 -		ip->i_gen = random() / 2 + 1;
 +		ip->i_gen = random() + 1;
  		if ((vp->v_mount->mnt_flag & MNT_RDONLY) == 0)
  			ip->i_flag |= IN_MODIFIED;
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/183230: commit references a PR
Date: Thu, 26 Dec 2013 01:12:43 +0000 (UTC)

 Author: pfg
 Date: Thu Dec 26 01:12:35 2013
 New Revision: 259904
 URL: http://svnweb.freebsd.org/changeset/base/259904
 
 Log:
   MFC	r258904, r259780:
   Small ext2fs updates.
   
   Add two new reserved inodes.
   Make the hashing algorithm match the linux code.
   
   PR:		kern/183230
 
 Modified:
   stable/10/sys/fs/ext2fs/ext2_dinode.h
   stable/10/sys/fs/ext2fs/ext2_hash.c
 
 Modified: stable/10/sys/fs/ext2fs/ext2_dinode.h
 ==============================================================================
 --- stable/10/sys/fs/ext2fs/ext2_dinode.h	Thu Dec 26 01:11:51 2013	(r259903)
 +++ stable/10/sys/fs/ext2fs/ext2_dinode.h	Thu Dec 26 01:12:35 2013	(r259904)
 @@ -44,6 +44,8 @@
  #define	EXT2_UNDELDIRINO	((ino_t)6)
  #define	EXT2_RESIZEINO		((ino_t)7)
  #define	EXT2_JOURNALINO		((ino_t)8)
 +#define	EXT2_EXCLUDEINO		((ino_t)9)
 +#define	EXT2_REPLICAINO		((ino_t)10)
  #define	EXT2_FIRSTINO		((ino_t)11)
  
  /*
 
 Modified: stable/10/sys/fs/ext2fs/ext2_hash.c
 ==============================================================================
 --- stable/10/sys/fs/ext2fs/ext2_hash.c	Thu Dec 26 01:11:51 2013	(r259903)
 +++ stable/10/sys/fs/ext2fs/ext2_hash.c	Thu Dec 26 01:12:35 2013	(r259904)
 @@ -289,8 +289,8 @@ ext2_htree_hash(const char *name, int le
  			len -= 32;
  			name += 32;
  		}
 -		major = hash[0];
 -		minor = hash[1];
 +		major = hash[1];
 +		minor = hash[2];
  		break;
  	default:
  		goto error;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
