One of the questions that comes up on occasion is "should I defrag my iPod, my SD card, or my USB drives?". To answer that, let's first take a step back and make note that these drives (also known by other names such as thumb drives, jump drives, solid state disks, etc) are flash-based storage devices (the largest I've seen is 32GB). They are used in Digital cameras under the names of SD cards, compact flash, memory stick, etc. The iPod and other MP3 players either have miniature hard disk drives (HDD), typically called microdrives, in the larger storage models, or flash-based drives in the smaller, 2Gb-4GB, such as with the iPod Nano. The exact nomenclature of a flash storage device depends on it's "interface". If it uses a USB interface it is typically called a jump drive, if it uses a SATA/SCSI interface and is intended to replace a hard disk drive, it is called a Solid State Drive (SSD). Other flash devices include the aforementioned digital media storage devices such as Memory Stick, Compact Flash, etc... In a nutshell, Flash based disks do not use a spinning disk and can access data randomly without any performance penalty. That may seem to obfuscate the benefit of defragmentation, and to a good degree it certainly mitigates the need. Flash and SSD devices are good at reading data, but are not as good at writing data. The reason for the poor write performance is that these (NAND based) devices must erase the space used for new file writes, immediately prior to writing the new data. This is known as erase-on-write or erase/write. Improvements in this area are coming (phase-change memory). However, flash devices running FAT or NTFS file system do still fragment the same way that a HDD would. Non-Windows products, like digital cameras/camcorders, use the FAT file system (FAT16 or FAT32, depending on the size of the drive). FAT file systems are more susceptible to fragmentation than NTFS. The greatest drawback of flash devices from the perspective of fragmentation is it is slow at random write I/O. Here's a quick test you can do yourself to show that severe free space fragmentation on Flash drives does affect performance. I did this myself, as a test run.
I took a brand new Kingston 1GB DataTraveler Hi-Speed USB drive with 24MB/sec read and 10MB/sec performance (per the manufacturer).
First I did a format of the disk - FAT16 (you'll need to use FAT32 for drives over 2GB). Then, using a development testing tool from Diskeeper Corp I fragmented the free space. I used Diskeeper 2007 to confirm the fragmentation as well as DiskView (a more granular tool available from Microsoft - formerly SysInternals). I created about 45MB of small files (16k to 48k in size) spread all across that Flash disk.
I then grabbed the VM Player install file (145MB), and made five more copies of it and zipped (Winzip) them into single 846MB zip file. This file was kept on a separate spindle (SATA disk) from the OS and paging files (to minimize variables from my time tests).
I used a simple stop-watch to time how long it took to copy this file from the SATA disk to the USB Flash drive with fragmented free space. It took 2:37 from start to finish.
I reformatted the USB drive, to the FAT16 file system again and rebooted the PC (just to make sure the cache was clear). I then copied that 846MB zip file from the same location over to that USB drive. This time the copy operation took 1:14, less than half the time required to copy than when the free space was fragmented.
Deleting a large, fragmented file also takes a long time.
From a "scientific" perspective the test can be run a few more times to come to an average, but given the difference was so significant, I personally did not feel the need to redo it. You can reverse the test order, and even use a program to zero-out the flash drive, just to eliminate any minor possible variables. Anyone else is certainly welcome to give this a go for themselves.
I did test one more case where I fragmented the free space into 24 even chunks and found no difference in copy time. While severe free space fragmentation is an issue, mild free space fragmentation is not - same concept as on physical disk. And yes the 846MB was fragmented in 19 pieces.
To create the free space fragmentation (without the development tool I'm privileged to have access to), you can copy a large number of small files to the Flash drive and deleted every other one, or other random deletion pattern (vary between deleting every third, fourth, fifth ...n file). If you have some programming skill this can be
scripted fairly simply. Just make sure there is enough room left on that USB drive after fragmenting the free space to copy the same test file. The more severe the free space fragmentation, the longer the copy operation will take.
That said, the degree to how this translates into actual usage depends. A real-world equivalent might be with a digital camera/camacorder where you mix various sized mpegs and jpegs, and use the device to delete some of these files from the drive. Unless you wipe the disk, the free space fragmentation will build up.
The test case I made up may be so extreme that it is unreal. I don't know what's "real-world" as I don't personally use Flash drives that often, and even then my actual usage isn't likely to equal yours. How often you want to consider free space consolidation depends; my best-guess is once every 6 months or so. The limited extent to which I use USB drives and the fact my 2GB mp3 player only ever gets minor and infrequent file changes, I doubt I'll personally ever need to worry about the free space fragmentation.
PS: We've been working with several of the technology leaders in the Flash/SSD industry for some time. They have been kind enough to send us pre-release devices for our R&D efforts. Expect future innovations from Diskeeper Corporation and those industry partners to improve performance and reliability on these storage devices.