Low level formatting - when, why and how


Updated: January 2, 2012

You will sometimes hear geeks mention this three-word phrase and you might be impressed. The question is, now that your curiosity is piqued, what does it really mean and do you need it? Hopefully, today, I will be able to teach you when you might consider using low level formating, what it is good for and how to safely execute it.

Low level formatting is a hard disk operation that should make recovering data from your storage devices impossible once the operation is complete. It sounds like something you might want to do if giving away a hard disk or perhaps discarding an old computer that may have contained useful and important, private information. All right, let's take a closer look and try to figure out how we can go about geeking, safely and smartly.

Teaser

The jargon

Low level formatting, as opposed to high level formatting is an operation performed directly against disk sectors. You skip the file system layer and you go directly for the underlying storage. Let me elaborate.

Normally, operations against storage devices are performed using a logical abstraction layer called the filesystem. Humans do not think in terms of bits and sectors and such, they think in terms of file names and possibly file sizes. This is exactly what filesystems do, plus a few other things, like keep relations betweens files and directories, optimize read and writes, maintain integrity of operations, and more.

In Linux, for example, you have filesystem drivers, responsible for this kind of work, called ext2, ext3, ext4, reiserfs, and others. When you create a partition, say using GParted, you then format it with some filesystem. The choice will dictate what you can and cannot do with the underlying storage, which is now exposed to you.

This is the so-called high-level formatting, because you do not care what kind of hardware you have. You do not really need to know anything except that there's a hard disk that can now contain your stuff.

The high-level formatting can be considered a sort of a possibility map - it tells the kernel, which manages all those useful hard disk operations, where data can be stored and in what manner. This means that if you are reusing a hard disk, old data that has been previously stored on the device may still be available in its raw form, a series of zeros and ones physically written to the storage medium. Of course, this old data is meaningless, as the new filesystem residing on top it does not know about it and will freely overwrite segments during normal operations. But if you're only using 1-2% of your total storage, theoretically, there might be entire blocks of old data that you could read, bypassing the filesystem, so to speak.

Some consider this a privacy risk, as old disks given away or stolen or accidentally reused could be gleaned for old data. In most cases, determined and skilled users might be able to harvest bits and pieces of randomly stored data from the storage media. For some people, this is an unacceptable risk. Enter low level formatting.

Low-level formatting is a procedure where you write data directly to the storage medium, bypassing the filesystem layer. You do not care if the hard disk has one partition or more, NTFS or BTRFS or anything else. You are using the device driver, which can be IDE or SCSI or SATA or others, and you're writing data to physical sectors. More importantly, low level formatting will write to each and every bit on the storage device, making sure the old states are destroyed forever, and with them, any trace of former data previously stored.

Performing one low-level formatting operation is known as one-pass format. Some security consultants and experts and paranoids might recommend you perform three or more passes, to make absolutely sure no trace of old data can ever be retrieved. Statistically, this is a total overkill, but the choice is entirely yours.

Now that we know what this thing is, let's see it in practice.

Low level formatting, the combat trial

Before we begin, a super warning:

Low level formatting must be executed with extreme caution. You must be absolutely sure that you are going to perform this operation against the CORRECT storage device. A wrong choice might lead to total, absolute, irrecoverable destruction of your critical data. This is not a game. This is obliteration of disk information. Be warned!

What you need to do is: take the old disk and connect it to your machine. The disk may be still located inside the hard disk cage inside a computer case or connected externally. I recommend you try to find and buy an IDE/SATA to USB adapter, which allows you to connect external 3.5" disks to pretty much any machine. You may also use e-SATA connections if you have those.

Adapter

Once the disk is connected, let the system identify it. On Linux, which is the preferred platform for this kind of work, the new disk will show if you execute fdisk -l as root or sudo. Make sure you pay close attention to the disk notation, as we will soon need this.

Note: if you plan on zeroing the internal system disk, you won't be able to do this from the context of itself, so you will have to boot from another operating system, either a live CD or another instance in a multi-boot configuration. Once again, this emphasizes why Linux is the preferred choice. In this article, demoed using CentOS and Ubuntu.

It is possible that the disk already has a partition table with several partitions on it, filesystems and even data. Well, most likely. It is also possible that your system might automatically mount these partitions. While we're about to destroy the disk, it is still advisable that you gracefully unmount the partitions before you proceed. For more details on managing disks and partitions, my GParted and Linux commands tutorials should help you get around.

Next, we will execute the dd command against the device we want to zero.

dd if=/dev/zero of=/dev/<target device>

Effectively, this will write an endless stream of zeros to your device. Optionally, you can use /dev/urandom as the input source, if you want to mix zeros and ones.

Formatting

This will take a while. Since disks are much slower than your CPU, you may notice that your system is sort of churning CPU, but most of the activity will be spent on waiting for the disk to catch up, so you will probably see high %wa figures if you run top. dd itself will not use too many resources.

Top, dd

The procedure will take a while, depending on your connection and your disk type. If you're using USB, you will most likely be limited around 30-40MB/sec for version 2.0 of the protocol, which is the most popular choice out there. If you have an e-SATA connection, then the limiting factor will be your disk. For typical SATA, it's 7,200rpm or roughly 80MB/sec. This means that formatting a disk as large as 500GB can take 5-6 hours.

A slightly safer method

If you're uncomfortable doing the above, there's a safer method. Like before, plug the device in and let the system identify it. Next, unmount partitions. Next, open GParted and destroy all existing partitions. This can be done in one step by creating a new partition table. Next, create a new single partition and format it.

Destroy partition

New partition

After this step is complete, go back to command line and mount the new partition somewhere, like /mnt. The next step is to execute dd, only this time the target will be a file somewhere under /mnt. You can also specify block size (bs) value for the dd command to get things going a little faster, but the overall time won't be changed much.

Now, execute the dd command. Again, be careful what you do.

dd if=/dev/zero of=/mnt/file bs=1M

Sit back, relax and watch the target partition get filled with junk data. Once the target partition runs out of space, you're done. Your partition will contain lots of new and meaningless data. If you're really paranoid, you may want to destroy the partition table again and create a new one, performing yet another high-level format that should seal the coffin real tight.

Writing

Writing more

And here's an Ubuntu example:

No space left

Conclusion

There you go, another would-be geeky mystery resolved. Now you know what to do when you need to dispose of old hard disks. One thing you definitely do not need is to purchase expensive privacy tools that promise the same functionality.

Low-level formatting sounds wicked, but it's just a simple and fairly straightforward operation that will essentially zero your storage devices. Please be careful when doing this, as you might destroy important data forever. Lastly, if you're not comfortable working with raw devices, you might opt for the second method. All of this is just a brief teaser intro toward a bigger, more comprehensive article, which will teach you how to safely give away computers. Stay tuned.

Cheers.

RSS Feed icon

del.icio.us del.icio.us stumbleupon stumble digg digg reddit reddit slashdot slashdot



Advertise!

Would you like to advertise your product/site on Dedoimedo?

Read more

Donate to Dedoimedo!

Do you want to
help me take early retirement? How about donating
some dinero to
Dedoimedo?

Read more

Donate