Learn the DD command

Tinkster · 10-06-2005, 12:47 PM

Quote:

Originally posted by theYinYeti
This should work, although I've always thought that dd is far from the best method to clone a disk, because (unless there's something I misunderstand) dd only applies if both disk partitions are exactly the same number of raw bytes.

Yves.

Clone means "identical copy". If one does that to HDDs (or partitions,
or whatever one points dd at) this is what one gets. So yes, in case
the HDDs have different sizes you'll get odd results.

If you want to copy a partition from one drive to another there's
better tools out there, like partimage, but that won't work as seamlessly.

If it's only the files you could tar stuff through the same pipe, but
again, I said CLONE a HDD over the net, and that's exactly what
dd does.

Cheers,
Tink

Superion · 10-06-2005, 02:33 PM

Quote:

Originally posted by AwesomeMachine
Few linux newbies know much about the standard GNU untilities found in /bin. One of the most useful, and least understood of these commands is DD. Even people who have been using linux for years mostly don't know about the DD command. DD is found on every linux installation, and has been a standard since the 1970's in UNIX. There is a man page for DD, which gives a brief description, but no useful examples. Here are some things you need to know about DD.

Write random data over a file before deleting it:
first do an ls -l to find filesize. In this case it is 3769

ls -l afile
-rw------- ... 3769 Nov 2 13:41 <filename>
dd if=/dev/urandom of=<filename> \ bs=3769 count=1 conv=notrunc

This will write random characters over the entire file.

Could you do this for an entire drive? Would you happen to know how long it would take?

Tinkster · 10-06-2005, 03:00 PM

secure_delete is more suitable for 100% safe deletion, unless
you run the dd 32 times. To find out how long wiping the drive
might take try it

... it will depend on the drive, the CPU and how
busy the system is with other things at that time.

Cheers,
Tink

AwesomeMachine · 10-06-2005, 08:13 PM

Yes, you can write random data over an entire disk, almost. If you really, really want to erase a drive, the only way to do it is with the drives own internal erase mechanism. If there is anything on the drive, which would be of high value to someone else, smash it up with a hammer, and blast the remains with a blowtorch. This is known as the incineration method of disk cleansing. I kid you not, for classified data this is the only method the DoD approves of. They actually melt the drives with an acetylene torch. There is no known method to recover data from a drive so destroyed. It is difficult to imagine how one could do it, other than travel backwards in time to before the drive had been destroyed and somehow locate it.

Here is the command for dd to do a pretty good job

dd if=/dev/urandom of=/dev/sda bs=4096 conv=notrunc

This will cleanse a drive of all data, from any legitimate use. It would be difficult to recover data from the drive. This will take about 1.5 hours for a 120 GB drive. You would want to use a boot CD to do this. Otherwise there is an HPA on the drive. When linux boots from a hard drive it creates an HPA. This is an area which cannot be accessed by the operating system. It is for recovery purposes. If your drive goes bass-o-matic on you, it can still recover on the next boot. This security feature makes operating systems very robust, but compromises security. As a security measure, SECURITY_ERASE_UNIT was added to the security feature set of hard drives. To date it is almost impossible for a consumer to use this feature. There are few pieces of software which will trigger the command. Most are proprietary forensic programs, written by lab staff of a particular lab. There is also a SECURITY_LOCK_DRIVE feature. Only one company in the world has ever defeated it, Vogon. Nothing is secure besides physical destruction. That is only secure in the absence of time travel.

tumbelo · 10-09-2005, 07:56 AM

I have a question about using dd. I'm trying to dump a game cd to my hard drive. It's a mixed mode cd with both data and audio. It's an old game (Hexen 2) so I doubt it has any kind of copy protection. It doesn't even matter as long as I can just dump the CDDA part to an image so I can mount the image and play the in-game music from it instead of keeping the cd in the drive. It fails to copy because it's not data-only, I've experienced the same when ripping CDs in Windows with some programs...

dd: reading `/dev/hdd': Input/output error
332736+0 records in
332736+0 records out
170360832 bytes (170 MB) copied, 99.1102 seconds, 1.7 MB/s

I suppose I need a RAW dump? Is there a way to do it with dd?

AwesomeMachine · 10-10-2005, 02:50 AM

dd is a bitstream duplicator. I am not sure if it will copy music. Looks like not.

Emmanuel_uk · 10-15-2005, 02:11 AM

AwesomeMachine here is another application, crude, to check that a media is not corrupted
dd if=/dev/fd0 of=/dev/null

if dd stops with an error it means the media is faulty.
Can be applied to HD as well.

Very crude method, but why not?

jlinkels · 10-24-2005, 10:40 AM

AwesomeMachine,

It is great that you took the effort to write it all down.

Shorter:
DD: if you want to copy an arbitrary number of bytes from an arbitrary location to another arbitrary location. There are no restrictions.

Simple as that. Every administrator should be required to learn the man page by heart.

Occasionally I have to to some work on a Windoze platfrom. It is incredible how obfuscating this kind of copying/editing is, including (and most important) working with mbr and boot partitions. Dozens of (expensive) tools have been designed to cope with oen small task or the other.

While it is all so simple. What a charm.

jlinkels

AwesomeMachine · 10-24-2005, 07:50 PM

jlinkels,

See my post on Helix CD

jlinkels · 10-24-2005, 08:14 PM

I just checked your post. Not surprising that those tools are being used on a *nix platform. I guess many tool heavily lean on the dd command. I can't imagine that those tools do not exist on the M$ platform, but for a lot $$$, and classified.

jlinkels

mr_demilord · 10-25-2005, 03:57 AM

Amazing, thank you! :drool:

Emmanuel_uk · 11-08-2005, 02:07 AM

AwesomeMachine, I came accross these 2 (and dd_rhelp)

dd_rescue http://www.garloff.de/kurt/linux/ddrescue/
ddrescue http://savannah.gnu.org/projects/ddrescue/

If anybody read that far in the thread, then they must be keen
to learn, so well I decided to mention ddrescue

There was a review request for these, see
https://www.redhat.com/archives/fedo...msg01272.html.

I have not used them yet

Dragineez · 11-08-2005, 02:43 PM

Brilliant! This thread should be stickied (or is that stickyfied? stickyferred?)

arjaya · 11-19-2005, 07:37 AM

This was a great article! Sent it to my printer for reference!

I do have a question. is it possible to do something like this:

Code:

dd if=/dev/st0 of=/dev/hda

where /dev/hda is the ide drive that I have attached to the machine.

Thanks again for the great post!

AwesomeMachine · 11-20-2005, 12:00 AM

Quote:

Originally posted by arjaya
This was a great article! Sent it to my printer for reference!

I do have a question. is it possible to do something like this:

Code:

dd if=/dev/st0 of=/dev/hda

where /dev/hda is the ide drive that I have attached to the machine.

Thanks again for the great post!

/dev/st0 is a tape drive device. Ordinarily tape archives are made with some sort of alteration to the bit steam, such as "tar" or "gzip". If you have a tape drive connected, and recognized, you need to know the block size of the tape drive to do the copy.

dd if=/dev/st0 of=/dev/hda ibs=<tape drive block size, or some multiple of it> obs=<HDD block size> conv=noerror.

For tar archive backup

dd if=/dev/st0 ibs=<tape drive block size, or some multiple of it> | tar -xv | of=/dev/hda obs=<HDD block size> conv=noerror.

For gzipped tape archive

dd if=/dev/st0 ibs=<tape drive block size, or some multiple of it> | gunzip | of=/dev/hda obs=<HDD block size> conv=noerror.