How To Permanently
Data from a Hard Disk
Copyright©2003-2008 by Daniel B. Sedory
This page is mainly for the average consumers
who keep a great deal of personal and financial data on their computer hard
disks, and after obtaining a new computer or just accumulating
many small drives they finally decide to sell or give
away the old hard disks (or the whole computer; after
copying their data to new drives) and want to never worry
finding and using any of their data.
[ But, we highly recommend reading our other reasons, HERE, for wiping a hard disk before installing an operating system or using it to store data. ]
For executives concerned
about company secrets on a large number of disks in old computers that must
be sold for their remaining assets, your company's IT Dept. should have a security procedure
in place (most likely using a costly device or security program to wipe
the disks). But, we believe simply 'zeroing-out' a drive AND checking that
it has actually been done is more than adequate. For example, say a competitor
was able to buy all of your old machines (and assuming, of course, that your
IT Dept. employees follow the proper procedures for doing at least a single-pass
'wipe' of each drive*),
without knowing which drive contained any valuable data (or if any of them ever
did), we highly doubt anyone could justify the TIME and COST to employ an experienced
Electron Microscope operator in a Data Lab to hunt down bit by bit whatever
tiny amount of info they MIGHT find; with no assurances that ANYTHING (let alone
something of value) could ever be recovered! NOW IN THE YEAR 2008, after 15
years of people warning others and passing along their fears of possible
scenarios involving elecron microscopes, WE STILL have NEVER heard of a single
case from any lab actually using a microscope to discover any useful bit patterns!
Most hard disks manufactured after about 2000 employ error correcting
technologies which store all the raw data on platters in various ECC formats,
making it impossible to use even good data without the precise knowledge of
exactly how those bits were encoded!
[ *The problem with any wiping though is the larger the number of drives a company must wipe, with increasing time/costs and personnel involved in doing so, the easier it is for some drive to slip by the process completely intact. ]
If you need to delete data from disks used in Military, Government or Defense contractor's office, there are proscribed procedures for doing so! Personally, from the information we've read about such procedures, I consider them to be ridiculous in their extreme measures; hostile forces will always look for the 'weakest link' in obtaining desired information, and we believe it's still easier and quicker to obtain it from open communications and vulnerable human sources than any adequately overwritten hard disk. But, most likely due to many examples of human error when it comes to disposing of classified data, the US Government has orders that any media containing classified data must now be completely DEGAUSSED (making them entirely unusable) and/or PHYSICALLY DESTROYED, so there isn't even an issue involving erasure in those cases! [Quoting from the DSS (Defense Security Service) Clearing and Sanitization Matrix (C&SM), updated June 28, 2007: "There is currently no overwriting product or process that has been evaluated in accordance with the NIAP Common Criteria Evaluation and Validation Scheme (CCEVS). Therefore, .... Effective immediately, DSS will no longer approve overwriting procedures for the sanitization or downgrading (e.g. release to lower level classified information controls) of IS storage devices (e.g., hard drives) used for classified processing."] However, the NIST also came out with a recent study which appears to negate some of the DoD's stance on hard disks. You'll find all my comments on the DoD, NISP, NIST, etc. on this new page about: DoD 5220.22-M.
If you're a Billionaire (that keeps very sensitive data on disk), we doubt you'd ever NEED to resell your hard disks. You will, of course, have many backup disks full of your data [possibly on 'simultaneous' RAID arrays], and most likely you'd use them until they 'died,' then have TRUSTED and knowledgeable employees DESTROY those disks as you certainly would any floppy disk that failed.
there will always be widespread misuse of the terms: 'Wipe,' 'Erase,'
'Delete,' 'Destroy' and even the word 'Permanent.' However, when someone
describes a particular software program or technique using any of these terms,
you should pay close attention to what is meant (not just said
or written); especially if you're trying to make sure
NO ONE will ever be able to retrieve ANY DATA from your disks!
Here are a few statements we've found using this terminology, and WHY you need to be sure of their real meaning:
Because all data on your hard disk will be
destroyed by this procedure, you must back up your hard
disk before using this Debug script.
Microsoft Knowledge Base Article - 106419 (checked JULY 2008)
Want to WIPE a drive? .... "WARNING: Delpart is the easiest and most complete way to erase a drive. It warns you once and only once about what it will do, so be careful! Use it only on a drive you want to completely erase, as you will not be able to recover any data that was on the drive prior to running delpart." Supposedly from a reputible site; checked in July, 2008.
Another site states: "Use
Delpart to 'wipe clean' a hard drive." These
statements seem clear enough, right? WRONG!
The ONLY thing that DELPart
does is to remove a maximum of only 64 bytes of partitioning data from
each hard disk's Master Boot Record (MBR) and possibly 64 bytes
in each Extended Boot Record (EBR). Well...
If you gave me a drive that only Delpart had been used on, in about 10 minutes or less I could return the drive to its original state using a single software tool !
Power Quest (bought out by Symantec) in their old Partition Magic 4.0 program, made use of phrases such as: " WARNING: Deleting a partition will DESTROY any existing data" or even this one: "The DELETE operation deletes a partition and destroys all its data"; which certainly seems to imply that it does more than just remove partition information from the boot records of a drive! But, once again, these were nothing more than poorly worded statements trying to stress that the average home user would not be able to fix such a mistake, whereas, a person familiar with data recovery techniques could very easily do so!
|So, statements such as these are not true at all from the point of trying to make sure you have erased all traces of sensitive data!
First you need to realize that only by changing the contents of EVERY BYTE on a disk, can you be assured none of your data will ever be read again *. When you change all the bytes on a disk to a ZERO byte, it's commonly called 'zero-filling', 'zeroing-out' or 'initializing' a drive; it should not be called a low-level format (though we've increasingly seen this term used in the hard disk industry to incorrectly describe only an initialization process).
How Long will this take?
This procedure may take anywhere from only about 10 minutes for a small drive on a computer with a fast UDMA/UATA EIDE (or the new SATA) interface to 10 or more hours for large drives on old computers! The exact length of time depends upon the rate at which data can be written to the drive and the size (capacity) of the drive. So, a fully implemented UDMA or 'Ultra' ATA interface of 100 or 133 MHz (using 80-wire cables) will complete the process in much less time than some very old machine (of only 33 MHz) that did not have UDMA capability (40-wire drive cables can't handle UDMA speeds). Generally, you're looking at 20+ minutes to about 3.5 hours for most drives in use today (40 GB to 500 GB).
EXAMPLES: Some years ago, we noted a computer with a 1.5 GHz CPU correctly employing UATA 100 (a 100 MHz interface), took only 34 minutes to completely wipe a 60 GB drive. On the other hand, a 40 GB drive took all night (8+ hours?) to zero-out on a computer with an old 40-wire 33MHz interface! Of course, old boxes like that are very rarely used now.
Now we have S-ATA (Serial ATA) drives which have data transfer rates even higher than a P-ATA (Parallel ATA/IDE) drive with UDMA running at 133 MHz; though the increase in sizes to 300, 500 or even 750 GB, means overall 'wipe times' are still about the same (about an hour for the fastest drives). On a recent machine with a 320 GB SATA drive, correctly set up for SATA AHCI BIOS control, we achieved a sustained overwrite speed of 4.4 GB/min. This means it took only 10 minutes to clear 44 GB of data. About 1.2 hours for the whole disk.
So what programs can we use?! The best type of program (and often FREE) for what's called a SINGLE PASS ZERO-OUT is the same one some drive manufacturer may ask you to run before allowing a disk drive to be returned under warranty. For example, Western Digital®'s new DLG Diagnostic (or DLGDIAG; for DOS or Windows and included with their Data Lifeguard Tools 11 packages from Kroll Ontrack) will easily zero-fill every byte on a WD drive; just run dlgdiag.exe and choose Write Zeros to the drive. We've also used a similar program called SeaTools (No more Powermax: "Maxtor® has been acquired by Seagate Technology!"), to zero-out their drives. [Obtain these types of programs from your HD's maker, since they usually work with only one manufacturer's drives!]
But, many of the diagnostic functions of such programs will still give you some valuable info about other manufacturer's drives: For example, there's a nice DOS utility that will identify the Model, S/N and drive capacity of any HDD it finds on your system. It's from Hitachi (Global Storage Technologies) and it's called: "Drive Fitness Test (DFT)" http://www.hgst.com/hdd/support/download.htm . It's made for IBM® and Hitachi drives, so it will only zero-out those kind of drives from its Utilities menu! If you know how to make a floppy disk from just an image file (such as WinImage or by using the Linux 'dd' program), all you need is the 1440 KiB dft_v413.img (Right-Click and SAVE) image file. Otherwise, get the Windows floppy disk creator version (dft32_v413.exe) that makes its own boot/DFT floppy disk for you. To check for the latest software, go to their main page, then "Support" and lastly "Downloads" (last checked 12 JUL 2008). There's even a CD image (dft32_v413_b00.iso) available now!
There's a nice collection of links to many Drive Manufacturing sites with some type of drive utility program here: http://tacktech.com/display.cfm?ttid=287.
If your drive has a capacity under about 8.4 GB, you can use a program I compiled and tested on my own 6.4 GB drive: WIPE8.zip. It does not have the ability to easily test the bytes on your drive though; manufacturer's programs use special software functions their drives respond to directly, whereas WIPE8 (and other simple wiping tools) only use INT 13 calls to software routines in the BIOS code.
However, there's also a FREE program, with open source code, at SourceForge called Darik's Boot and Nuke (dban) which is a self-contained 1440 KiB boot disk that will wipe any drive! It has many options too; the simplest way to use it is to boot up with the disk, press the ENTER key at the "Boot: prompt" and then use its Interactive Menu. But you must exercise caution with this disk: It's easy to set it up to automatically wipe every drive on your computer! This is the same exact program that's now bundled with Eraser, since that program was never designed to do drive wiping. At the bottom of the dban web page, you'll find links to a list of "Similar Products" (all commercial). DBAN also creates and saves an interesting logfile of its operations at the end of a run.
Caution: Misuse of the term low-level format can even be found on the web pages of major HDD manufacturers! Due to the misconception by consumers (probably because of faulty media stories) that erasing a drive can only be accomplished with a "low-level format" utility, many HDD manufacturers have finally started to use that term to describe zero-fill utilities instead; but it's still incorrect usage! (Some techs/HDD manufacturers are also using the mixed term: low-level zero-fill which is more accurate).
To Low-Level Format (LLF) a drive, really means to set up the physical locations of its tracks and sectors on the platter itself! And embed that data in the control structures of the drive! Because of the complexity of a modern drive's internal structures (which includes zoned-bit recording and even servo data on the disk itself), a true LLF can only be done at the factory! Very old (less than 30 MiB in most cases) MFM drives did have LLF programs that consumers could occasionally use; thus the reason for the idea that they might still be useful.
The only site I've found which gives a comprehensive view and correct definitions for all formatting terms is: Low-Level Format, Zero-Fill and Diagnostic Utilities; or another page which specifically points out the misuse of the term: Low-level Formatting. One of the few HDD manufacturer pages we found that tried to guide the consumer in the right direction, is no longer avilable! If you search the Net for FAQs or forum discussions on the topic, you will probably find very confusing comments from both HDD manufacturers and end users alike! This WikiPedia article may be helpful: http://en.wikipedia.org/wiki/Disk_formatting .
NOTE: If you use Linux or have access to a Linux boot disk with the "dd" program on it, it's fairly easy to execute a command that will 'zero-out' a drive.
Here's an example
of how one would zero-out a 1440 KiB floppy diskette in
the first floppy drive:
where the "bs="
tells it to use 512-bytes per sector and the "count="
means to write zero bytes to 2,880 sequential sectors on the diskette in drive
Anyone who's skilled in the use of Linux scripts could easily create a program
for the unattended overwriting of a drive many times with different values;
and maybe even throw in a random number generator as the 'if' (input
file) device for good measure. [ If you're interested in learning more about
Linux, see my Intro. to Linux
Console Commands using 'tomsrtbt' Boot Disk here. ]
NOTE: This has already been accomplished!!! See my comments on the DBAN program above; it uses a Linux kernel inside a RAM drive to carry out its drive-wiping tasks!
There are at least two
other reasons (apart from security) for initializing (zero-filling) a drive:
1) When you first obtain a hard disk, you may wish to test every byte on the drive using a zero-out and test program from the drive's manufacturer. Most OEM drives (non-boxed; usually with only a very short term store warranty) probably do not undergo the same testing as fully-warranted products.
2) Whenever you decide to "start all over again" and reformat a drive, it's a very good idea to make sure there are no left over partition data structures (EBRs) just in case you need to run a data recovery program at some time in the future. If you've never had any Extended Partitions on the drive, you can forget about this. Problems for data recovery occur only when there are EBR sectors and Boot Records that were never deleted (zeroed-out) before a new Extended partition (with one or more logical drives) was created. Then if you try to get your data back after accidentally deleting a partition or the MBR, you (or a recovery program) might have a difficult time deciding which EBR sectors were used last! By 'zeroing-out' a disk that has had many Extended partitions created on it, you ensure data recovery programs will see ONLY the correct size of the partitions you are about to create.
* Data Recovery experts seem to have a different opinion than those working in, let's call it theoretical research, of what constitutes an 'adequate overwriting' of a hard disk to keep any previous data from ever being recovered. If you have the time and money to purchase and use a guaranteed secure wiping program, go ahead and do so; especially if you plan on reselling MANY lots of HDDs that have valuable data on them. However, for most individuals, I do not see any reason to use more than a single-pass overwrite as explained above.
If you want to read about the details regarding Data Erasure and Security, you could start with the very old (and "oft quoted" references to) paper by Peter Gutmann*. Though the paper is somewhat technical, it also appears to have been too theoretical even at the time it was written, and was already 'out-of-date' concerning any viable threat to your security many years ago! For example, the TPI (tracks per inch) value that he called "state of the art" for a disk's track density is at least 50 (or even more!) times LESS than what is actually being used today. For a reasonable review of Gutmann's paper from the perspective of a modern hard disk user, see: Can Intelligence Agencies Read Overwritten Data? by Daniel Feenberg. For those who wish to truly understand just how very complex recent hard disk drives are (including how magnetic data stored on their platters is thoroughly encoded by drive electronics), you should study (it may take you quite some time to work thorugh it) Charles Sobey's white paper: Recovering Unrecoverable Data (Note: this title concerns data that's at risk because a drive has physical damage; not data that was purposely overwritten!) Though mainly commissioned to describe the procedures and limitations of good Data Recovery labs, there's much you can learn from reading this paper.
*Secure Deletion of Data from Magnetic and Solid-State Memory - Read at least sections 1 thru 3 and conclusion.
NOTE: Prof. Gutmann added an Epilogue to his original paper (some time later) in which he commented that people treated his research more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques. (Please read his comments in full here. I think he probably shakes his head a bit every time he sees the phrase 'the Gutmann method,' since most people never applied his work correctly.) In a reply to an e-mail we sent Prof. Gutmann about advances in hard disk technology, he stated:
Not much has changed except that the older, easier-to-read formats are mostly dead and everyone is now using Extended PRML which is even harder to recover data from than PRML, as well as using smaller dimensions and assorted exotic new technologies, so the problem is slowly fixing itself.
[ February 01, 2002; quoted with Prof. Gutmann's permission. ]
We take this to mean even Prof. Gutmann reluctantly
stated the obvious about modern high density hard disks: You really don't need
to worry about anyone trying to use electron microscopy to find data from hard
disks you discard today! However, if you're the kind of person
who cherishes their paranoia(!), you can still find programs on the Net
such as: Sami Tolvanen's Eraser which
overwrites files up to 35 full passes! Maybe if you especially hate a particular
file this could give you some sort of satisfaction too? But, in order to
completely overwrite a whole drive (Sami's program will not 'clean out'
your Virtual Memory Swap Files!), you need to use the "dban" diskette
that now comes with it! Eraser is mainly for eliminating certain files;
not for 'wiping' the whole disk. Since traces of what's been in a file (if not
the whole file) can often be found in download caches, SWAP files and other
cache locations as well, you need to make sure you know what you're doing; especially
if you're trying to erase an encrypted message full of company secrets! For
example: Did the 'plain text' of the message get left anywhere else on the drive?
Note: Perhaps you should encrypt all of your sensitive data on a disk
drive first, then 'wipe' the drive afterwards! Sounds like a good method to
Personally, I'm not at all concerned about anything I 'zero-out' on a hard disk ever being seen by someone else again. If you think that any local law enforcement or government agency is going to attempt to find data on a drive with ALL zeros by taking a very long time and spending more money than they can really afford with no assurance whatsoever of finding anything(!), then you're living in a dream world that's already beyond all the fictional elements in TV shows like CSI and other such scientific forensic evidence dramas. It's much easier for criminals, terrorists and government agencies to obtain data about you using many other means!
I remember one TV drama where a tech told a D.A. that he couldn't find anything incriminating on a guy's HDD because he used a scrubber to clean out all of his cache files, etc. Then the tech added something like: If you gave me $50,000.00 to hire a lab with an electron microscope and skilled technicians, maybe I could come up with a few more words here or there in 6 months time. [We used to say here: "that's a fairly real attitude of most organizations that deal with evidence on HDDs," but by 2006 we thought even that statement had become fiction!] Do YOU as an average home user think you need to take even more precautions than a spy for some enemy government or terrorist group? Do YOU really need to worry about the CIA or a foreign government being interested enough in YOUR hard drive to expend all the necessary resources in an ATTEMPT to find only a tiny (if even that!) amount of its contents!? Get real.
though, are something that ALL the experts agree on:
The data tracks are so wide and the drives so 'forgiving' that an overwritten floppy could possibly be recovered using only a sophisticated software program to adjust the placement of the heads and hunt for alternate data on either side of the last overwrite. It takes so much TIME to adequately 'wipe' a floppy (with MANY passes), that the easiest way to dispose of one with a number of 'bad sectors' is to just crack it open and melt (or at least shred into tiny pieces ) the flexible disk media that's inside (it looks similar to the material used for audio tape). If a floppy has any classified data on it, our government will burn them, but they always take precautions when doing so since the material may become a toxic gas! (Many discarded computer circuit boards and other components are now a major environmental problem too!)
Last Update: July 12, 2008.
You can write to me using this:
Online Reply Form.
(It opens in a new window.)
MBR and Boot Records Index
The Starman's Realm Index Page