https://goughlui.com/2017/05/30/experiment-8gb-usb-flash-drive-endurance-test/ [INS::INS] Gough's Tech Zone Reversing the mindless enslavement of humans by technology. [svg] Skip to content * Home * The VHS Corner + Intro - Inside the VHS Cassette & VCR + VHS Repair - Fix me baby, one more time! + Mini-Me: VHS-C Teardown & Adapter + I'm Dirty, Wash Me! - Head Cleaning Cassettes + VHS Cassette Library > o ACME Extra High Grade o ACME Professional Grade o ACME Super Grade o Akai o BASF Extra Quality o Casio AG o Foto Processing o Fuji AG o G-Tape Premium Performance o Hi-Tech o Hitachi High Resolution o JVC Excellent Resolution o Kodak o Maxell EX o Maxell G o Maxell GU o Maxell GX o Memorex HS o National NV o NEC o NEC High Definition o Nichei o Onix o Opus o Palsonic HI o Panasonic o Philips HG o Polaroid High Quality o Polaroid High Standard o QTape o Raks SQ o Sanyo High Grade o Scotch EG o Sheen o SKC HQ Maxplay Grade o Sony Clarity and Durability o Sony Dynamicron o Sony Dynamicron Durable & Excellent Quality o Sony Super Durable & Excellent Quality o TDK Sharp & Clear o TDK Super Avilyn o TDK TV o Teac ASX o Teac HX o Teac X-1 o Teac X-2 + VHS Clip Library > o Week 1: DSE, Tandy, Acer, IBM, Apple, Microsoft and Sony o Week 2: Apple, SEDA, Foxtel, Global Comp. Sega World, Sony, Motor Show, Vodafone * The CPU Corner + CPGA132 (Intel) + CPGA168 or Socket 0/1/486 (Intel/AMD/Cyrix) + Slot 1 (Intel) + Slot 2 (Intel) + Socket 370 (Intel) + Socket 423 (Intel) + Socket 462 or Socket A (AMD) + Socket 478 (Intel) + Socket 479 (Intel) + Socket 5 (Intel) + Socket 754 (AMD) + Socket 939 (AMD) + Socket AM2 (AMD) + Socket H2 or LGA 1155 (Intel) + Socket H3 or LGA 1150 (Intel) + Socket J or LGA 771 (Intel) + Socket M (Intel) + Socket P (Intel) + Socket T or LGA 775 (Intel) + Super Socket 7 (AMD) + IBM PowerPC (Apple) + Sun UltraSPARC IIi (Sun) * The Optical Disc Corner + Apple + Arita + BASF and EMTEC + BenQ + Fujifilm + Imation (CD-R & CD-RW) + Imation (DVD+-R/RW & BD-R/RE) + Kodak + LaCie + Laser + LG + Magmedia + Maxell + Mitsubishi + Mr. Data + Panasonic + Philips + Ricoh + RiDATA + RiTEK + Samsung + Sony (CD-R & CD-RW) + Sony (DVD+-R & DVD+-RW) + TDK (8cm DVD-R & DVD-RW) + TDK (BD-R & BD-RE) + TDK (CD-R & CD-RW) + TDK (DVD+-R) + TDK (DVD+-RW) + Verbatim (BD-R & BD-RE) + Verbatim (CD-R & CD-RW) + Verbatim (DVD+-R) + Verbatim (DVD+-RW) + Other Lesser Known Brands + Other Optical Disc Articles * The Hard Disk Corner + Hard Drive Performance Over the Years + Hard Drive Audio by Magnetic Pickup + Other Hard Drive Related Articles + Conner CFS210A (3.5'' 210MB 1994) + Fujitsu MHR2020AT (2.5'' 20GB 2002) + Fujitsu MHV2100BH PL (2.5'' 100GB 2006) + Fujitsu MHZ2120BH FFS G1 (2.5'' 120GB 2009) + Fujitsu MJA2160BH FFS G1 (2.5'' 160GB 2009) + Fujitsu MPA3026AT (3.5'' 2.62GB 1997) + Fujitsu MPC3064AT (3.5'' 6.48GB 1999) + Fujitsu MPD3084AT (3.5'' 8.45GB 1999) + Fujitsu MPE3064AT (3.5'' 6.48GB 2000) + Hitachi DK23CA-20 (2.5'' 20GB 2003) + Hitachi HTS541040G9SA00 Travelstar (2.5'' 40GB 2006) + Hitachi HTS541060G9SA00 Travelstar (2.5'' 60GB 2006) + Hitachi HTS543232A7A384 (2.5'' 320GB 2011) + Hitachi IC25N060ATMR04-0 Travelstar (2.5'' 60GB 2005) + IBM 19K1552 Fireball lct (3.5'' 10.1GB 2000) + IBM DPTA-372050 Deskstar (3.5'' 20.5GB 2000) + IBM DTLA-307030 Deskstar (3.5'' 30.7GB 2000) + Maxtor 4D040H2 (3.5'' 40GB 2002) + Maxtor 6L250R0 DiamondMax 10 (3.5'' 250GB 2008) + Maxtor 6Y120L0 DiamondMax Plus 9 (3.5'' 120GB 2005) + Maxtor 82161E2 (2.5'' 2014MB 1998) + Maxtor 92049U3 (3.5'' 20.4GB 2000) + Quantum 1080AT Fireball (3.5'' 1080MB 1995) + Quantum 6.4AT EX64A024 Fireball EX (3.5'' 6.4GB 1999) + Quantum Fireball Plus AS (3.5'' 20.5GB 2001) + Samsung SP2514N SpinPoint (3.5'' 250GB 2007) + Seagate ST3120022A Barracuda 7200.7 (3.5'' 120GB 2004) + Seagate ST3120026A Barracuda 7200.7 (3.5'' 120GB 2004) + Seagate ST320410A U6 (3.5'' 20GB 2002) + Seagate ST320413A U Series 5 (3.5'' 20GB 2002) + Seagate ST3250318AS Barracuda 7200.12 (3.5'' 250GB 2010) + Seagate ST32531A Medalist (3.5'' 2557MB 1998) + Seagate ST3320613AS Barracuda 7200.11 (3.5'' 320GB 2008) + Seagate ST3320620A Barracuda 7200.10 (3.5'' 320GB 2007) + Seagate ST340014A Barracuda 7200.7 (3.5'' 40GB 2004) + Seagate ST340810A U6 (3.5'' 40GB 2002) + Seagate ST34342A Medalist (3.5'' 4303MB 1998) + Seagate ST360021A Barracuda ATA IV (3.5'' 60GB 2002) + Seagate ST380021A Barracuda ATA IV (3.5'' 80GB 2003) + Seagate ST380023A Barracuda ATA V (3.5'' 80GB 2003) + Seagate ST950043AS Momentus (2.5'' 500GB 2011) + Seagate ST96812A Momentus 5400.2 (2.5'' 60GB 2006) + Toshiba MK1252GSX (2.5'' 120GB 2008) + Toshiba MK1652GSX (2.5'' 160GB 2008) + Toshiba MK2555GSXF (2.5'' 250GB 2010) + WD WD100BA-60AK Expert (3.5'' 10GB 1999) + WD WD1600AAJS-00PSA0 Caviar SE (3.5'' 160GB 2007) + WD WD1600BEVS-08VAT2 Scorpio Blue (2.5'' 160GB 2009) + WD WD1600BEVS-22RST0 Scorpio (2.5'' 160GB 2007) + WD WD2000JB-00GVC0 Caviar SE (3.5'' 200GB 2005) + WD WD2500AAJS-60Z0A0 Caviar Blue (3.5'' 250GB 2010) + WD WD2500JB-00GVA0 Caviar SE (3.5'' 250GB 2005) + WD WD2500YD-01NVB1 Caviar RE16 (3.5'' 250GB 2005) + WD WD2500YS-01SHB1 RE (3.5'' 250GB 2007) + WD WD3000JB-00KFA0 Caviar SE (3.5'' 300GB 2006) + WD WD3200JB-00KFA0 Caviar SE (3.5'' 320GB 2006) + WD WD400BB-22HEA1 Caviar (3.5'' 40GB 2004) + WD WD400BB-60DGA0 Caviar (3.5'' 40GB 2003) + WD WD5000AAKS-00TMA0 Caviar SE16 (3.5'' 500GB 2007) * Other Pages + Raspberry Pi: Element14 vs RS Components + SD/SDHC/SDXC CID CSD Register Data Database + SD/SDHC/SDXC Performance Test Database + Blu-ray Recordable Disc BCA Collection + LG GGW-H20L Supermultiblue Media Code Support List for Firmware YL07 + Terms of Use & Privacy Policy * Project FAX + Fax Technicalities & Audio Samples + Library of Received Faxes * About Me + Publications and Appearances + Contact Me * Gough's Review Challenge * Interesting Links - Project: THE (large) COLLECTION of V.34 Modem Sounds Review: Toshiba Exceria N301 64Gb SDXC & Samsung Evo+ 128Gb microSDXC - Experiment: 8Gb USB Flash Drive Endurance Test Posted on May 30, 2017 by lui_gough Flash memory is, and has been, a commodity item for a while now. Almost everyone has at least a few USB flash drives (sticks) and maybe even a few memory cards. When it comes to rapidly transferring large files between devices, or even storing working documents, USB flash drives are the universal choice.In recent years, we've also seen the popularization of solid-state drives (SSDs) which have put high demands on flash manufacturers in terms of volume, as they rapidly displace moderately high capacity hard drive storage from mainstream computers. However, while flash continues to grow in popularity, it is not without its downsides. The nature of flash memory itself operates on the principle of charge trapping and tunneling, which is not a perfect process. The charges that represent the data can be lost over time due to the insulator being leaky or damaged - the whole process of writing and erasing data relies on tunneling that induces damage into the insulator which is not readily repairable in-place. Worst still, owing to price pressure, the inclination to get more data onto less silicon has resulted in smaller feature sizes (e.g. 16nm) which result in smaller cells which hold less charge and have a shorter lifetime presumably due to the smaller insulator area as well. This is compounded by the move to triple-level cell (TLC) storage, which results in the need to accurately distinguish between eight levels of cell voltage rather than the four of MLC or two of SLC, thus reducing the margin for error. As a result, at least for planar NAND, the move to smaller lithography and TLC have resulted in reduced endurance. For the most part, most opinions do not focus on the issues of endurance as much as in the early days of SSDs. Larger drives, forced overprovisioning and better wear levelling technologies have largely resulted in consumer drives seeing only a limited number of write-erase cycles before being retired because they're too slow or too small. Simplistic calculations of say 10Gb written a day to a 1Tb SSD only results in 18.25 cycles (assuming no write amplification and perfect wear levelling) within a lifetime of five years. As a result, manufacturers seem to warrant SSDs based on terabytes written which can result in anywhere between 150-500 cycles for TLC drives in general, even though most of their drives will go further. However, the situation for inexpensive USB flash drives and memory cards are not so clear. Part of the issue is a lack of diagnostic data (e.g. SMART as on SSDs) which would allow users to understand the condition of their drives. Another is the lack of willingness on the manufacturer's behalf to state any information as to the endurance of their products. As a result, I decided to run a little experiment to try and find out just how robust inexpensive USB flash drives of today are. The Contenders and Methodology Three contenders were chosen for this experiment - namely, three different drives I have reviewed in the past on this site, of which I still had brand new samples which were never used. The drives are: * Comsol 8Gb UF4-8000 USB Flash Drive * Sandisk Cruzer Facet 8Gb * Verbatim Store'n'Go Pinstripe 8Gb They were attached to a computer with a USB 2.0 port and formatted to exFAT (to allow for large files >4Gb to be stored). Cygwin was used to continually write to the drive and log the progress using the following command: while :; do dd if=/dev/random of=/cygdrive/X/rawout.raw bs=8M &>> stress.log; sleep 5; done Note that the command continually recreates a file filled with pseudo-random contents, thus the drive cannot "cheat". A large block size was chosen to avoid write amplification due to small-block accesses being immediately purged to the drive. However, no effort to verify the validity of written data or verify that any stored data would be retained over a period of cold storage was undertaken. The experiment would be terminated and the log file examined to tally the number of write cycles endured as soon as the drive failed to continue to receive data. A sleep time was set for each loop to avoid excessive CPU utilization on failure (although I should have probably checked the return value of dd to terminate the loop instead). Post-failure examination of the drive status was undertaken. This experiment was conducted beginning November, prior to leaving for my holiday and for a variety of reasons (including a power failure and loss of system control due to loss of internet connectivity while I was overseas), was not completed until recently. Results Write Endurance Under the same conditions, the first drive to fail was the Sandisk Cruzer Facet, at 632 cycles written. This aligns with the rough expectation that planar TLC NAND may only achieve anywhere from 300-1000 cycles. [svg] The more devastating result was its failure mode. The drive exhibited a reluctance to conduct writes which resulted in the filesystem becoming corrupted. On a removal and reinsertion, the drive needed to be formatted as no valid filesystem was recognized. [svg] [svg] The damage was, however, more serious than that as the internal drive geometry appears to have been lost. The drive was unable to report its original size, and thus was unable to be formatted, nor read. Recovery of data is very unlikely even with some expertise, as the whole drive is a single system-on-package and thus the NAND and controller are encapsulated in the one package. [svg]In one case, I was able to get the geometry to recognize, however, the drive failed to return any data. The next drive to fail was the Comsol UF4-8000 which first hiccuped at 743 cycles by dropping out of the USB bus with partial corruption. A reformat was successful, and another 216 write cycles were completed before the drive again dropped out, for a total of 959 cycles. Partial corruption, especially of the filesystem, was experienced, however, the drive remained readable. [svg] [svg] Under this condition, it seems likely that data would be recoverable when the drive first faulters. However, wishing to explore the failure mode further, I reformatted the unit and attempted to run H2testW on it, for it to fail in less than a cycle of writes. [svg] Unfortunately, as it turns out, the act of trying to further write to the drive seems to have destroyed it - it now wants to be formatted, but fails to be formatted. It's likely that the drive has now failed in a read-only state, but in a corrupted one. As a result, it probably indicates that when a drive is exhibiting drop-out symptoms from high cycle use, that one should avoid any repairs to the filesystem and just image the card and recover from a copy of the data, as any writes may well exhaust the few spare-cells that remain. [svg] [svg] It's interesting to see that while both were based around Sandisk NAND, the Comsol did last a little longer (but not by that much) and fail differently as it used a third party controller. Another factor is likely due to the greater overprovisioning - 7.26GiB user accessible for the Comsol versus 7.44GiB for the Sandisk Cruzer Facet. The last to fail was the Verbatim Store'n'Go which achieved an impressive 9751 cycles. This drive was made of Toshiba eMMC memory, which is of a grade expected to be embedded in tablet devices, and thus its endurance was likely to be greater purely due to this fact. Another benefit may have been the greater overprovisioning on the Verbatim, which only exposed 7.21GiB of user accessible storage. [svg]At failure, the drive was recognized for size and format, with data still partially readable. Attempting to format and run H2testW resulted in all writes appearing to succeed very slowly but failure to verify written data. [svg]This may be a preferable end-of-life behaviour, as it would appear to "accept" any filesystem repair writes despite not being able to write it to memory, while allowing for whatever is still salvageable to be read. The cycle life result approaches that as claimed for MLC memory (of 10,000), and thus is quite respectable. Discussion and Other Key Points Most users of cheap commodity USB memory sticks will probably be wondering why any of this is important - after all, many of them may not even use 100 cycles before the stick is lost or damaged, so even 600 cycles is plenty. However, there's a few things to consider. For one, this cycle life test did not evaluate the data accuracy after writing - only when the drive failed to write or the filesystem got mangled, did we conclude the test and examine the drive. It's fairly probable that some data corruption or access failures may have occurred earlier if verification was undertaken. As a result, the numbers we are getting are an upper bound result. Secondly, the cycle life test did not evaluate how the persistence of the data stored on the drive. Over consistent cycling, the damage to the insulator is expected to increase the charge leakage rate, and thus it is quite probable that data stored on drives which have undergone cyclic writes will not be retained for as long, especially due to the more stringent voltage margins of TLC storage and the lack of sophistication in low-cost USB memory controllers. It seems quite probable that after even less than a year of storage, that data could be permanently lost to the point of the drive failing due to a loss of firmware or geometry metadata (note my experiences with microSD cards and even a Samsung 850 EVO SSD). Due to the cost pressures on this segment of the market, the majority of the USB memory sticks on the market are likely to be planar TLC NAND and suffer such issues. The cheaper, slower, all integrated miniature USB devices appear to be particularly vulnerable and problematic as any recovery from them will be complicated by the fact that the NAND can not be directly accessed. Regardless, the failure modes also vary, and in some cases result in sudden and complete loss of access to data, thus "self rescue" by software based recovery tools is not a possibility. With this in mind, these conclusions are likely to extend to many memory cards, particularly microSD cards where high densities and low costs prevail. In this application, it seems quite likely that cycle-life exhaustion could occur, especially when used in embedded computing with data logging, or in dashcams and security cameras which record in a loop. This voids the warranty of many microSD cards, but it's important to keep this in mind, as it would render the solution completely ineffective if the card were to fail or become unreadable when extracted from the camera. In the case of long-term storage, loss of data is indeed a possibility as such flash is not well suited for long term storage. Usage patterns will also make a difference, as will the sophistication of the wear levelling algorithm on the controller. Regardless, if a USB key is acting "iffy", it could well be a sign that it is running out of spare cells to reallocate, and could completely lock-up into read-only or become unreadable soon. It's unwise, with that in mind, to rely on cheap USB sticks as your sole storage of working documents despite the popularity of doing this. However, there's no guarantee that even more expensive units are better. Conclusion The endurance of cheap USB flash drives was examined in a write-only scenario, with the Sandisk Cruzer Facet failing at 632 cycles with complete failure to read any data, the Comsol UF4-8000 failing at 959 cycles with an earlier hiccup and partial corruption but in a readable state that rejected writes, and finally, the Verbatim Store'n'Go failing at 9751 cycles with partial corruption but a partially readable state that accepted writes but could not commit them to memory. The failure modes were varied and have an impact on the "recoverability" of the drive once end of life is reached. This test produces an upper bound figure on the endurance of cheap drives - it is likely if verification was completed that drives may have failed earlier, and that impacts on data retention over time would have also occurred. Even though the cycle figures may seem ample for most consumers, there are a number of applications (e.g. embedded data-logging, dash cam/surveillance) where it can be exhausted. Use of flash drives as primary storage for working documents or long term storage is probably unwise. Drives with drop-out symptoms or random write-failures/verification failures are likely experiencing a pre-failure symptom and should be imaged/ recovered without further writes to avoid complete drive failure. In light of this, the drives are still very suitable for occasional use and data interchange. Other technologies other than flash, for example, crosspoint memory as demonstrated in Intel Optane modules, may well be an alternative technology that overcomes some of these problems. Improvements in NAND geometry, such as 3D VNAND by Samsung and BiCS by Toshiba may also offset some of the loss in cycle life endurance, however, it is unlikely that you will find such improved technologies especially when shopping at the price sensitive bottom-end of the market, as many people do. However, even paying more is no guarantee as to quality, however, it seems quite likely that the NAND and controllers in SSDs are made to be significantly more reliable than that of such cheap drives. Like it? Share it! * Facebook * Twitter * Reddit * LinkedIn * More * * Tumblr * Pocket * * Pinterest * WhatsApp * * Telegram * Skype * * Related [svg] About lui_gough I'm a bit of a nut for electronics, computing, photography, radio, satellite and other technical hobbies. Click for more about me! View all posts by lui_gough - This entry was posted in Computing, Flash Memory and tagged computer hardware, flash, flash memory, storage, tested, usb. Bookmark the permalink. - Project: THE (large) COLLECTION of V.34 Modem Sounds Review: Toshiba Exceria N301 64Gb SDXC & Samsung Evo+ 128Gb microSDXC - 16 Responses to Experiment: 8Gb USB Flash Drive Endurance Test 1. [svg] Tesla1856 says: May 31, 2017 at 3:34 am Good post, thanks. Back in 2011 ... one of the first flash drives I ever bought was a SanDisk Cruzer 2gb from Staples. It was plain without any U3 or other security/encryption features. It always "long" formatted fine (as FAT32). I could even write small files to it ok. It worked fine at first, but then started to fail. When I wrote files to it, they would get secretly corrupted. Windows never was able to see that the file was getting corrupted during copy. I had to use a file-copy utility that verified every file as it was written to find the problem. Needless to say, I've never bought another SanDisk USB-Flash drive again (or ever tried their SSDs). I now stick to Verbatim and Lexar (both have lifetime warranties here is USA). Have never had one of those fail. Reply 2. [svg] Michael says: July 16, 2017 at 4:21 pm Hey Gough, Awesome post! I was always wondering regarding the endurance of contemporary flash drives, especially since all of the blog posts and S. Boboila's paper are from more than a couple years ago. There was just one thing I was curious about - it is for sure that SSDs can benefit from overpartitioning. However, is this also true for cheap flash drives? I identified a controller for a cheap Emtec flash drive a couple years ago, its technical advertising stated it could do static wear leveling - though some time ago dynamic wear leveling was the way USB flash usually went I wonder if that's still the case now. For example, would a cheap 8GB with a 6GB partition exhibit more endurance than the same model 8GB with a 7GB partition? Is that unpartitioned space going to be used for wear leveling? What do you think? Is this a question you'd be interested in investigating and writing about? I'd be thrilled if so! - Michael Reply + [svg] lui_gough says: July 17, 2017 at 9:27 am Dear Michael, I had a think about this and short of actually finding another fresh sample and destroying it, the answer is a qualified "it depends". The first thing is how sophisticated the flash controller is. As external drives *mostly* do not support TRIM over USB, the operating system has no way of signalling to the flash drive as to which blocks are no longer needed and can be discarded (e.g. a deleted file). Without this mechanism, the pool of "spare" blocks will be the overprovisioning, minus any bad blocks, assuming the drive had been filled completely at least once. The only way around this is if the controller is very intelligent (e.g. Sandforce SF-2281 controllers) and can utilize compression (where possible) to reduce block usage and give some extra room to move blocks around. The other way is if the controller has an understanding of the underlying filesystem and does things "on its own", although this can result in very unexpected clashes with OSes depending on how they manage their writes (e.g. write caches often mean data is written first, and metadata updated later). Of course, there *is* a way that it can work without any of this, and that is if the flash mapping table is completely blank after manufacturing and only populated as the sectors become used. I've seen some evidence - some units have very high read speeds when blank and slow down once completely written to the specification speed. In these cases, ensuring that the "extra" space is *never* used should leave it free for the controller to use at its own will. However, whether a particular controller does this is not entirely obvious, and if the spare area ever gets written, the space is ultimately allocated and taken away from the controller. The best way, although not without risk and not always possible, is to use the manufacturing process tools for your given controller to "re-certify" or "re-manufacture" the drive with a different capacity altogether. The downside is that doing this may not work in some cases where the compatible tool for a given controller cannot be found, sometimes it involves a firmware downgrade, it takes time, the verification process for the flash may be different since the "factory marked" bad block table may be destroyed, and the software could be laden with viruses/problematic filter drivers that cause USB port issues down the track. So hence the conclusion - a qualified "maybe". Sincerely, Gough Reply 3. [svg] dr0 says: July 14, 2019 at 7:51 am Brilliant article. I was curious about this subject for a while but have never come across anything like this experiment of yours. One question: how did you count the number of write cycles? Is there a diagnostic program for pen drives that can display this number or you just took this information from a log file created by Cygwin? Reply + [svg] lui_gough says: July 14, 2019 at 10:18 am As I wrote a simple infinite loop script, the results of the dd command are written appended to the log. By examining the logfile, you can easily count the instances where the dd process completed by first back-tracking from the end of the log and snipping away all the lines where the drive had an error and the write failed, then counting the lines where it shows the average write speed to determine the number of write cycles completed. - Gough Reply 4. [svg] dr0 says: July 15, 2019 at 8:35 am Thanks for the reply. One more question: can you help me convert your command to run on Linux? I tried to run it on one of my machines that runs Linux Mint 18.3, but it didn't work. I'm pretty bad with software side of things, especially terminal commands, so maybe I'm just doing something wrong. Reply + [svg] lui_gough says: July 15, 2019 at 9:29 am Please please please be extra careful. Don't go blindly running commands without understanding them, as dd can and will overwrite things including the drive you are booting from if you are unfortunate enough to make a critical typo and run it as the root user. Basically, the reason it won't work is because you need to change the paths. On Linux, your USB (I presume it's already mounted - if not, just open it up in the file browser) would normally be at /media/username/serialnumber or something similar. You want your test file created inside that - so where the dd command has of (short for output file), change that to reflect the drive you're using (e.g. /media/linuxuser /1234-5678). For drives larger than 4GiB, they MUST be formatted to a filesystem that can accept a file of the size of the drive (i.e. not FAT32). If you do not, then dd will only be able to overwrite the first 4GiB repeatedly and your cycle counts will not be accurate! On Linux, you could use ext2/3/4 and likely even NTFS. Or better yet, you can gain root privileges and instead write to the block device directly (the best case as this avoids a filesystem altogether allowing for more accuracy in testing the full capacity) but this is where you need root privileges and a typo in the device name will overwrite and destroy data that you want (and I can't do this on Windows on the box I was running unattended while on holiday). You should also change the logfile path/name to something sane for you or invoke it inside a directory where you can store the logfile (e.g. ~/ logfile.txt). Needless to say, trying to store the logfile on the USB stick itself would not end well. - Gough Reply o [svg] dr0 says: October 15, 2019 at 2:29 am Hello again. So, I changed the command to while :; do dd if=/dev/random of=/media/compaq/ 5a151eb8-c245-4896-8e91-bb713698c21d/rawout.raw bs=8M &>> ~/stress.log; sleep 5; done ... But it writes extremely slowly, literally a few bytes per second. The command have been executing for almost an hour, but the rawout.raw file is only 8.7KB in size and the log file is still empty. When I write to this very drive via regular file copy the speed is absolutely normal and hovers around 20MB/s. I have tried only two file system types for executing this command so far though (FAT32 and EXT4). Do you know what can be the cause of such slow write speeds while executing your script? Reply # [svg] lui_gough says: October 15, 2019 at 5:09 am This is likely due to entropy exhaustion and will depend on the sources of randomness available in the system. To overcome this, you can use /dev/urandom instead, which provides a faster stream of random data but perhaps is not quite as random as /dev/random is. This shouldn't really impact on the quality of the test and should improve speeds. - Gough Reply @ [svg] dr0 says: October 30, 2019 at 9:12 am Thanks for the tip. Using "urandom" instead of "random" did help to increase the write speed to its maximum level that my drive is capable of. But the resulting file gets overwritten every 4GB regardless of target filesystem. So I decided to fill my drive with random data and leave only 4GB of free space on it and constantly overwrite that free area. It's been running this way for a few days now. At first the write speed was around 18-21 MB/s, as was expected from this drive during normal operation. But after 1413 cycles of 4GB of free space being overwritten the write speed cut in half and now is hovering around 8-10 MB/s. At this point the drive already reached 3220 instances of its free area being overwritten. Frankly, I didn't expect this TLC-based drive to last this long. While I don't really care whether P/E cycles will be accurately counted or not. I just want to wear out the NAND cells on this drive as soon as I can. Any advice? @ [svg] lui_gough says: October 30, 2019 at 11:22 am The file should not be overwritten every 4GB - something must be wrong. This is most common on FAT32 formatted devices. I had no issues with NTFS on Windows via Cygwin. Your other alternative is to make a file of fixed size - e.g. bs=8M count=1000 would produce a file of 8000MB and check that it works properly. Perhaps you're running a 32-bit OS and there's something odd with how /dev/urandom is working for you. Perhaps you might need to try making a random file of the right size, store it with a hash on a drive, copy it to the device, hash it from the device and repeat. This would also check data integrity, as sometimes drives may silently corrupt data or pretend to write even though it has failed. It depends on the controller. Also remember, if half the drive has static data, assuming there is static wear levelling, writing half the drive will (on average) consume half a cycle from the standpoint of the whole device. Note that the drive may outlive your expectations - this is not unsurprising. Some devices have long outlived their specifications. Look at the Samsung SSD for example - https://techreport.com/ review/27909/ the-ssd-endurance-experiment-theyre-all-dead/ - Gough @ [svg] dr0 says: November 2, 2019 at 12:04 am >>> "Perhaps you might need to try making a random file of the right size, store it with a hash on a drive, copy it to the device, hash it from the device and repeat." - How do I do that? I figured out how to write a full drive capacity in one cycle but I have no clue how to automate the hash checking process... @ [svg] lui_gough says: November 2, 2019 at 3:20 pm Unfortunately, I've got many other things to do at this time than to solve this at this point in time - but if you're using Linux, you could go file-system-less by using dd with the block device itself (i.e. /dev/sdX). Likewise, in that case, as long as you know the size of the device (e.g. from fdisk -l), then you can produce a few sets of random data of that exact size that you store on a hard drive with their hashes (e.g. out of md5sum). Write each set in sequence using dd from the hard drive to the USB drive, then hash the drive itself and log results (e.g. md5sum / dev/sdX), comparing to the pre-recorded hashes. You might need to use sed/awk to extract just the hash portion from the returned text. Otherwise, maybe you'd prefer testing with patterns (e.g. as h2testw does) which makes it easier to verify the data, but then the issue is that static patterns may be liable to compression by the drive and may not detect all sorts of corruption. You might need to write your own program to do this most conveniently. - Gough 5. [svg] Zbig says: January 20, 2021 at 11:46 pm After dealing with a failure of SanDisk Cruzer storing ESXi image I found your blog and wondered if Verbatim still exists in 2021 and if so - does it still use eMMC memory in Store n' Go model you tested if that model is still available. I managed to find 8GB sticks that looks the same as yours however the memory chip looks different (smd package soldered on two shorter sides). Anyway I bought 20 of them and are in the process of sacrificing one. It's been holding up pretty well for over a month - 1257 write cycles with data validation so far without an error. Reply + [svg] Zbig says: April 28, 2021 at 1:05 am The stick failed on #4436 full write and read cycle in Check Flash 1.17.0 on April 23rd so it took serious beating quite well - about 6 months of non-stop writing at 8.48 MB/s followed by reading at 24.59 MB/s. Reply o [svg] lui_gough says: April 28, 2021 at 9:51 am That's not a bad result for a modern product! Thanks for sharing. - Gough Reply Error: Comment is Missing! Cancel reply * My Links + About Me + Publications and Appearances + Contact Me + Legacy Site + Facebook + Twitter + YouTube + LinkedIn + Google Scholar + ORCID ID + element14 Community Blog + Interesting Links + Terms of Use & Privacy Policy * Categories + Audio + Computing + DIY + Electronics + Event + Flash Memory + Lighting + Obituary + Opinion + Photography + Power Bank + Radio + Raspberry Pi + Salvage + Satellite + Tablet + Tech Flashback + Telecommunications + Travel + Uncategorized * Recent Posts + Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + SGJPHK2017 Tokyo Pt 6: Harajuku, Shibuya & Other Random Observations + Teardown, Repair: 500W "Pure Sine Wave" 12VDC to 220VAC+USB Inverter + SGJPHK2017 Tokyo Pt 5: Akihabara, Odaiba, Ikebukuro, Ginza & Tsukiji + SGJPHK2017 Tokyo Pt 4: Yoyogi Park, Sensoji Temple, Kanda, Meiji-Jingu & Yasukuni Shrines + Project: All Outputs Parallel Adapter for Rohde & Schwarz HMP4040 Power Supply + Lucy Helton's SSTV Ham Radio Project 2021 - The Beauty of Shortwave + RoadTest Review: Microchip PoE to USB-C Power & Data Adapter (PD-USB-DP60) + SGJPHK2017 Tokyo Pt 3: Ueno Park, Zojoji Temple, Kyu Shiba & Hamarikyu Gardens + SGJPHK2017 Tokyo Pt 2: Akasaka Palace, Imperial Palace & Gardens + SGJPHK2017 Tokyo Pt 1: Around the Hotel & Looking for Views + Quick Review, Teardown: 802.3at/bt Power-over-Ethernet (PoE) Power Injectors + Tips: Debugging SCPI & VISA Automation (Including NI I/O Trace) + Notes: Optimising the LTE Life (& My SIM Card Collection) + Random: Sydney in Lockdown, Public Transport & Random Observations (& More) + Tech Flashback: Kodak DC290 Zoom 2.1MP Digital Camera + Opinion: The COVID-19 Vaccination Experience (Pfizer @ NSW Health Vaccination Centre) + RoadTest Review: Maxim MAX31343SHLD# 5ppm MEMS Real-Time Clock Evaluation Kit + Tested: ZMI 10,000mAh High-Speed Rechargeable Li-Ion Portable Power Bank (QB810) + Tech Flashback: Philips PLC Electronic (9,11,15W/240V) Compact Fluorescent Lamp + Visited: Central Coast Amateur Radio Club (CCARC) Mayham Event (30 May 21) + Tech Flashback: CD-R/CD-RW Overburning & My Results Database + Quick Review: Google Nest Hub (7'' Display, 2nd Gen, GUIK2) + Review: Trouver POWER 11 Cordless Vacuum Cleaner (VPL4) + Review: The WORST 4-Port USB 2.0 Hub Ever? XHUB-3009A1 + Notes: Tektronix PA1000 Power Analyser with pyvisa & SCPI Quirks + Project: Implementing a (Proper) R&S NGM FastLog Client with pyvisa + Tutorial: Understanding the SCPI Status Model with pyvisa Examples + Quick Review: A Few Weekends with a Rohde & Schwarz RTE1104 Oscilloscope + Project: Modified Car Fridge "Pro" Mk II - Sabotaged by Cheapness? + Tech Flashback: Philips SL*13 (13W/240V) "Compact" Fluorescent Lamp + Tested: Vishay Siliconix PowerPAK SO-8 TrenchFET Power MOSFETs + Review, Teardown: Kingston G4 DataTraveler G4 64GB USB 3.1 Flash Drive + Random: Birthday Analysis, Public Transport, Shopping & Toothpaste + Experiment: 26650 Li-Ion Cell Usable Capacity vs. Temperature + Tech Flashback: Digital Video at 1.2Mbit/s - Developments in Video Compression + Project: Arduino-Based LAN-Controlled Input Scanner / Relay Multiplexer + Repair: A Relatively Generic Tianyin-branded Soymilk Maker + SGJPHK2017 Osaka Pt 6: More Parks, Day Trips to Ikeda, Mozu & Kobe + SGJPHK2017 Osaka Pt 5: Random Walking Observations + Note: Even Simple Banana Plugs Can Go Wrong - Especially Cheap Ones! + Tested: STMicroelectronics LDL1117S33R SOT-223 3.3V LDO Voltage Regulator + SGJPHK2017 Osaka Pt 4: Walking Around & Other Attractions + SGJPHK2017 Osaka Pt 3: Even More Temples, Shrines & Cathedrals + Review, Teardown: Orico QSE-4U 4-Port Smart Desktop QC2.0 USB Charger + Tested: MPS mEZD71202A-G 6.5-24V to 5V/2A DC-DC Power Supply Module + Tutorial: Introduction to SCPI Automation of Test Equipment with pyvisa + Note: Linear Regulator Woes - When is an AMS1117 not an AMS1117? + Review: Daiso Nevolution High-Power AA Alkaline Battery (5pk) + Radiofax: St. Petersburg, Russia (ROSKON, FGBU North-West UGMS?) * May 2017 M T W T F S S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 << Jan Jun >> * Archives + August 2021 (7) + July 2021 (7) + June 2021 (6) + May 2021 (13) + April 2021 (13) + March 2021 (10) + February 2021 (9) + January 2021 (12) + December 2020 (15) + November 2020 (1) + October 2020 (9) + September 2020 (14) + August 2020 (13) + July 2020 (7) + June 2020 (13) + May 2020 (14) + April 2020 (18) + March 2020 (2) + February 2020 (9) + January 2020 (21) + December 2019 (10) + November 2019 (8) + October 2019 (14) + September 2019 (8) + August 2019 (12) + July 2019 (9) + June 2019 (6) + May 2019 (4) + April 2019 (12) + March 2019 (23) + February 2019 (20) + January 2019 (23) + December 2018 (11) + November 2018 (7) + October 2018 (4) + September 2018 (3) + August 2018 (19) + July 2018 (6) + June 2018 (5) + May 2018 (21) + April 2018 (7) + March 2018 (8) + December 2017 (12) + November 2017 (19) + October 2017 (1) + July 2017 (4) + June 2017 (16) + May 2017 (6) + January 2017 (11) + December 2016 (30) + November 2016 (13) + October 2016 (15) + September 2016 (19) + August 2016 (16) + July 2016 (8) + June 2016 (31) + May 2016 (14) + April 2016 (21) + March 2016 (13) + February 2016 (17) + January 2016 (19) + December 2015 (20) + November 2015 (12) + October 2015 (15) + September 2015 (4) + August 2015 (12) + July 2015 (22) + June 2015 (14) + May 2015 (21) + April 2015 (25) + March 2015 (14) + February 2015 (16) + January 2015 (6) + December 2014 (11) + November 2014 (12) + October 2014 (19) + September 2014 (11) + August 2014 (21) + July 2014 (18) + June 2014 (20) + May 2014 (12) + April 2014 (7) + March 2014 (23) + February 2014 (18) + January 2014 (16) + December 2013 (15) + November 2013 (14) + October 2013 (21) + September 2013 (13) + August 2013 (17) + July 2013 (26) + June 2013 (26) + May 2013 (11) + April 2013 (17) + March 2013 (31) + February 2013 (27) + January 2013 (36) + December 2012 (31) + November 2012 (39) + October 2012 (3) + February 2012 (18) + January 2012 (1) * Tags + analysis + audio + battery + cheap + computer hardware + computer storage + ebay + electronics + element14 + event + experiment + flash + flash memory + improvise + led + lighting + made-in-china + mobile + networking + new stuff + opinion + photography + powerbank + power bank + project + public transport + radio + random + repair + review + rf + roadtest + salvage + storage + tablet + teardown + tested + test equipment + testing + travel + usb + video + vintage hardware + vintage stuff + visited * Recent Comments + Jon on Note: Choosing 5Ghz Wi-Fi Channels for 802.11ac in AU + lui_gough on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + postnut on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + postnut on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + Lefty on Teardown: Mains Double Adapters - New vs. Old + Roy Jacobs on SGJPHK2017 Tokyo Pt 6: Harajuku, Shibuya & Other Random Observations + David on Failed Repair: Samsung U28D590D 27'' UHD 4K LCD Monitor + lui_gough on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + TheDragonFire961 on Random: Site Updates, Spam/Scam Calls/ SMS, Flash Memory Woes & More + lui_gough on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + Krzys on Random: Site Updates, Spam/Scam Calls/SMS, Flash Memory Woes & More + brizdx on Radiofax Update: JFX (Japan), RBW41(?) & Vanino Radio (Russia) (Nov 20 - Jan 21) + Tom on Random: Sydney in Lockdown, Public Transport & Random Observations (& More) + naks110 on Weekend Probing Pt. 2: Keweisi KWS-V20 USB Tester IC Connections + naks110 on Weekend Probing: Keweisi KWS-V20 USB Tester IC Connections + Old Aussie on Teardown: Mains Double Adapters - New vs. Old + lui_gough on Review: Western Digital WD60EZRX 6Tb Green Hard Drive + TheDragonFire961 on Review: Western Digital WD60EZRX 6Tb Green Hard Drive + TheDragonFire961 on Analysis: Optus Fetch Mighty DVR Hard Drive + TheDragonFire961 on Analysis: Optus Fetch Mighty DVR Hard Drive * Meta + Log in + Entries feed + Comments feed + WordPress.org [INS::INS] Gough's Tech Zone All content (c) 2012 - 2021 Gough Lui unless otherwise stated. Proudly powered by WordPress.