From nobody@www.freebsd.org  Sat Jun  8 13:18:53 2002
Return-Path: <nobody@www.freebsd.org>
Received: from nwww.freebsd.org (www.FreeBSD.org [216.136.204.117])
	by hub.freebsd.org (Postfix) with ESMTP id 8350537B404
	for <freebsd-gnats-submit@FreeBSD.org>; Sat,  8 Jun 2002 13:18:52 -0700 (PDT)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by nwww.freebsd.org (8.12.2/8.12.2) with ESMTP id g58KIphG011938
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 8 Jun 2002 13:18:51 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.2/8.12.2/Submit) id g58KIptL011937;
	Sat, 8 Jun 2002 13:18:51 -0700 (PDT)
Message-Id: <200206082018.g58KIptL011937@www.freebsd.org>
Date: Sat, 8 Jun 2002 13:18:51 -0700 (PDT)
From: Evgeny zislis <kesor@ulp.co.il>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Corrupted files on a FAT32 partition
X-Send-Pr-Version: www-1.0

>Number:         39043
>Category:       kern
>Synopsis:       [smbfs] Corrupted files on a FAT32 partition
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Jun 08 13:20:05 PDT 2002
>Closed-Date:    Mon May 11 06:33:53 UTC 2009
>Last-Modified:  Mon May 11 06:33:53 UTC 2009
>Originator:     Evgeny zislis
>Release:        FreeBSD 4.6-RC
>Organization:
>Environment:
FreeBSD finone.ulp.co.il 4.6-RC FreeBSD 4.6-RC #4: Fri May 24 18:24:45 IDT 2002 kesor@finone.km.dshapiro.org:/usr/obj/usr/src/sys/FINONE  i386
>Description:
      I have a WDC 80G hard drive, one primarly dos partition formatted as FAT32. It is mounted as /storage. I am using samba (2.2.4_1) to share this directory to windows machines on the network.
Copying files from and to this partition works good,
using smbclient 'get' command to get files from remote machines and put them in the shared mount point with the samba protocol doesnt do any problems.
On another UFS partition samba has no problems, it only occures with the case of FAT32 partition. (thus why samba people tell me its not a samba issue, but rather msdosfs kernel module bug)

The problem occures when using a windows machine, to copy files to the shared storage folder via the network.  When using \\machinename\storage in windows and writing to it - some parts of the file get overriden with 0roes.

I have used a 16mb file of random data, and from the hex dump its almost always more or less the same areas that are overriden with zeroes on the copied file :
080000-084000
100000-104000
180000-184000
200000-204000
280000-284000
d20000-end of file 1000000



I tested it in two completely different environments and networks, with completely different hardware and software that loosly matches my own configuration -- the problem repeats itself.
>How-To-Repeat:
      80G hard drive, formatted as FAT32, mounted as msdosfs in FreeBSD.
samba port (2.2.4_1) installed and configured to share that mount point over the network.
Take a file (the bigger the better, 10 bytes file didnt corrupt, while 3mb mp3s were getting corrupt 100% of the time)

Using a windows machine, open the shared samba mount and copy a file to it, then check it with md5 to see that it's changed.
>Fix:
      No known fix. there is a clue though,
in /sys/msdosfs_vnops.c : line 708
   * If the offset we are starting the write at is beyond the end of
   * the file, then they've done a seek.  Unix filesystems allow
   * files with holes in them, DOS doesn't so we must fill the hole
   * with zeroed blocks.

   if (uio->uio_offset > dep->de_FileSize) {
        error = deextend(dep, uio->uio_offset, cred);

   the bug is possible hiding in deextend or somewhere around it.

>Release-Note:
>Audit-Trail:

From: Stas Kysel <stas_k_freebsd@tiger.unisquad.com>
To: freebsd-gnats-submit@FreeBSD.org, kesor@ulp.co.il
Cc:  
Subject: Re: kern/39043: Corrupted files on a FAT32 partition
Date: Thu, 16 Jan 2003 20:47:30 +0200

 Hi.
 
 I have almost the same trouble.
 
 Differences are:
 1. 4.7-RELEASE
 2. samba-2.2.7a compiled from source
 3. 40Gb HDD
 4. I cannot reproduce it in 100%, but 2066 of 2870 big (>64k) files on fs
    are damaged
 5. Files are damaged starting form 0x8000.
 
 Common features:
 1. smbclient works fine
 2. FAT32 (Though I've experienced similar things long time ago on FFS).
 
 Additional details:
 1. Only first and last (incomplete) 32k blocks of small (<0x80000) files
 are intact. I did not watch into bigger files too much.
 2. Win2k/WinCommander were used to copy files
 
 More additional details?
 
 -- 
 Stas Kysel, UNIX Systems Administrator

From: Stas Kysel <stas_k_freebsd@tiger.unisquad.com>
To: freebsd-gnats-submit@FreeBSD.org, kesor@ulp.co.il
Cc:  
Subject: Re: kern/39043: Corrupted files on a FAT32 partition
Date: Wed, 22 Jan 2003 20:21:10 +0200

 Hi.
 
 Looks like samba guys are right - this is a bug in kernel, not in samba or
 windows :(
 ktrace/kdump shows that file (which appears corrupt on disk) was:
 1. opened:    CALL  open(0x81c1220,0xa02,0x1f4)
 2. truncated: CALL  ftruncate(0x13,0,0xbb0fb,0)
 then several times
   3. seeked to corresponding offset (0,0x10000,0x20000,...):
               CALL  lseek(0x13,0,0,0,0)
               CALL  lseek(0x13,0,0x10000,0,0)
               ...
   4. written by 64k chunks of data (not zeroes), except last write which was
      smaller of course:
               CALL  write(0x13,0x8229045,0x10000)
               CALL  write(0x13,0x8229045,0x10000)
               ...
 5. Then closed
 
 write() was always successful:
               RET   write 65536/0x10000
 
 Corruption is another type this time on all checked files - only first 32k
 are filled with data, the rest of file is zeroed.
 
 In addition it was stat()-ed several times and chmod() was tried on it,
 but unsuccessfully, because FAT32 does not support it.
 
 Not posting full kdump output here for brevity, but if someone wants
 to check me, I'll E-mail him corresponding parts of it.
 
 df output:
 ...
 /dev/ad2s1   39108736 7708608 31400128    20%    /share
 
 What can I do to help you to resolve this issue?
 
 Thank you for your attention.
 
 I've heard dmesg output is necessary. Not sure if it can help here, but
 just in case:
 ---------------------------------------------------------------------------
 Copyright (c) 1992-2002 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 	The Regents of the University of California. All rights reserved.
 FreeBSD 4.7-RELEASE #3: Thu Jan  9 12:23:15 EET 2003
     stas@localhost:/usr/src/sys/compile/kernel
 Timecounter "i8254"  frequency 1193182 Hz
 CPU: Pentium II/Pentium II Xeon/Celeron (434.32-MHz 686-class CPU)
   Origin = "GenuineIntel"  Id = 0x665  Stepping = 5
   Features=0x183f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR>
 real memory  = 133103616 (129984K bytes)
 config> di pcic0
 No such device: pcic0
 Invalid command or syntax.  Type `?' for help.
 config> di sn0
 No such device: sn0
 Invalid command or syntax.  Type `?' for help.
 config> di lnc0
 No such device: lnc0
 Invalid command or syntax.  Type `?' for help.
 config> di ie0
 No such device: ie0
 Invalid command or syntax.  Type `?' for help.
 config> di fe0
 No such device: fe0
 Invalid command or syntax.  Type `?' for help.
 config> di ed0
 No such device: ed0
 Invalid command or syntax.  Type `?' for help.
 config> di cs0
 No such device: cs0
 Invalid command or syntax.  Type `?' for help.
 config> di bt0
 No such device: bt0
 Invalid command or syntax.  Type `?' for help.
 config> di aic0
 No such device: aic0
 Invalid command or syntax.  Type `?' for help.
 config> di aha0
 No such device: aha0
 Invalid command or syntax.  Type `?' for help.
 config> di adv0
 No such device: adv0
 Invalid command or syntax.  Type `?' for help.
 config> q
 avail memory = 126320640 (123360K bytes)
 Preloaded elf kernel "kernel" at 0xc0326000.
 Preloaded userconfig_script "/boot/kernel.conf" at 0xc032609c.
 Pentium Pro MTRR support enabled
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 pcib0: <Intel 82810 (i810 GMCH) Host To Hub bridge> on motherboard
 pci0: <PCI bus> on pcib0
 pci0: <Intel 82810 (i810 GMCH) SVGA controller> at 1.0 irq 7
 pcib1: <Intel 82801AA (ICH) Hub to PCI bridge> at device 30.0 on pci0
 pci1: <PCI bus> on pcib1
 dc0: <Macronix 98715/98715A 10/100BaseTX> port 0xc000-0xc0ff mem 0xdd000000-0xdd0000ff irq 7 at device 0.0 on pci1
 dc0: Ethernet address: 00:80:ad:7e:2e:a4
 miibus0: <MII bus> on dc0
 dcphy0: <Intel 21143 NWAY media interface> on miibus0
 dcphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: <RealTek 8139 10/100BaseTX> port 0xc400-0xc4ff mem 0xdd001000-0xdd0010ff irq 5 at device 2.0 on pci1
 rl0: Ethernet address: 00:50:ba:83:6d:54
 miibus1: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus1
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 isab0: <Intel 82801AA (ICH) PCI to LPC bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH ATA66 controller> port 0xf000-0xf00f at device 31.1 on pci0
 ata0: at 0x1f0 irq 14 on atapci0
 ata1: at 0x170 irq 15 on atapci0
 pci0: <Intel 82801AA (ICH) USB controller> at 31.2 irq 12
 pcm0: <Intel 82801AA (ICH)> port 0xdc00-0xdc3f,0xd800-0xd8ff irq 11 at device 31.5 on pci0
 orm0: <Option ROM> at iomem 0xc0000-0xc7fff on isa0
 fdc0: ready for input in output
 fdc0: cmd 3 failed at out byte 1 of 3
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
 sio0: type 16550A
 sio1 at port 0x2f8-0x2ff irq 3 on isa0
 sio1: type 16550A
 DUMMYNET initialized (011031)
 IP packet filtering initialized, divert enabled, rule-based forwarding enabled, default to accept, logging limited to 100 packets/entry by default
 ad0: 2060MB <SAMSUNG WU32163A (2.16GB)> [4186/16/63] at ata0-master UDMA33
 ad2: 38204MB <SAMSUNG SV4012H> [77622/16/63] at ata1-master UDMA66
 Mounting root from ufs:/dev/ad0s1a
 WARNING: / was not properly dismounted
 ---------------------------------------------------------------------------
 
 -- 
 Stas Kysel, UNIX Systems Administrator

From: Tim Robbins <tjr@freebsd.org>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: kern/39043: Corrupted files on a FAT32 partition
Date: Mon, 6 Sep 2004 00:27:43 +1000

 Following up to note that kern/67555 has been closed as a duplicate of
 this PR.

From: Derek <derekm.nospam@rogers.com>
To: bug-followup@FreeBSD.org, kesor@ulp.co.il
Cc:  
Subject: Re: kern/39043: [samba] Corrupted files on a FAT32 partition
Date: Mon, 24 Oct 2005 20:35:58 -0400

 I have verified that this issue affects 6.0-RC1.  I am using from 
 packages samba-3.0.14a_1,1.  It may be worth noting that this PR 
 exhibits similar symptoms to kern/47628
State-Changed-From-To: open->analyzed 
State-Changed-By: linimon 
State-Changed-When: Wed Dec 14 19:07:43 UTC 2005 
State-Changed-Why:  
Note that this issue still seems to exist on 6.0. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=39043 

Adding to audit trail from email to list:

Date: Tue, 13 Dec 2005 17:10:00 +0100 (CET)
Message-Id: <200512131610.jBDGA0mK027336@lurza.secnetix.de>
From: Oliver Fromme <olli@lurza.secnetix.de>
To: freebsd-stable@FreeBSD.ORG

It's a long-standing bug that still exists.  It affects
FAT32 filesystems exported via Samba.  The workaround is
to re-format the drive with UFS instead of FAT.  I don't
think anyone is actively working on fixing the bug.  :-(

Best regards
   Oliver
State-Changed-From-To: analyzed->closed 
State-Changed-By: fjoe 
State-Changed-When: Mon May 11 06:30:08 UTC 2009 
State-Changed-Why:  
Dup of kern/47628. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=39043 
>Unformatted:
