cp of dd image failed, better to mount image and then copy?

bmcws · 06-22-2009, 05:00 AM

Hello,

I'm trying to copy a _large_ disk image file (made by dd - about 230GB) from an external USB disk to an internal disk. The cp (and nautilus copy and paste) fails with dmesg errors:

Code:

sd 5:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
sd 5:0:0:0: [sdb] Sense Key : Medium Error [current] 
sd 5:0:0:0: [sdb] Add. Sense: Unrecovered read error
end_request: I/O error, dev sdb, sector 646375991
Buffer I/O error on device sdb1, logical block 80796991
Buffer I/O error on device sdb1, logical block 80796992
Buffer I/O error on device sdb1, logical block 80796993
Buffer I/O error on device sdb1, logical block 80796994
Buffer I/O error on device sdb1, logical block 80796995
Buffer I/O error on device sdb1, logical block 80796996
Buffer I/O error on device sdb1, logical block 80796997
Buffer I/O error on device sdb1, logical block 80796998
Buffer I/O error on device sdb1, logical block 80796999
Buffer I/O error on device sdb1, logical block 80797000
__ratelimit: 20 callbacks suppressed
npviewer.bin[18191]: segfault at ff9bea2c ip 00000000ff9bea2c sp 00000000ffd66d0c error 14
npviewer.bin[2304]: segfault at ff9bea2c ip 00000000ff9bea2c sp 00000000ff88c83c error 14
sd 5:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
sd 5:0:0:0: [sdb] Sense Key : Medium Error [current] 
sd 5:0:0:0: [sdb] Add. Sense: Unrecovered read error
end_request: I/O error, dev sdb, sector 646376143
Buffer I/O error on device sdb1, logical block 80797010
sd 5:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
sd 5:0:0:0: [sdb] Sense Key : Medium Error [current] 
sd 5:0:0:0: [sdb] Add. Sense: Unrecovered read error
end_request: I/O error, dev sdb, sector 646376143
Buffer I/O error on device sdb1, logical block 80797010
sd 5:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
sd 5:0:0:0: [sdb] Sense Key : Medium Error [current] 
sd 5:0:0:0: [sdb] Add. Sense: Unrecovered read error
end_request: I/O error, dev sdb, sector 646376143
Buffer I/O error on device sdb1, logical block 80797010
sd 5:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
sd 5:0:0:0: [sdb] Sense Key : Medium Error [current] 
sd 5:0:0:0: [sdb] Add. Sense: Unrecovered read error
end_request: I/O error, dev sdb, sector 646376143
Buffer I/O error on device sdb1, logical block 80797010

What's the best way of recovering the data or as much data as possible (and find out which files are effected on the imaged disk)?

1. Would dd do any better at copying the file without errors?
2. Would mounting the image, then copying the files (using cp) on the mounted image copy all files in 'good' areas of the disk - and let me know which files did not copy properly; or would it just stop when it comes to the first error?
3. Any other ideas?

Help would be very much appreciated!

Thanks,

Bernard

jsteel · 06-22-2009, 06:38 AM

I don't think dd is what you want here. If you leave it copying (even if it is throwing up errors) does at least some data get copied across? If so I would call this the "good data", and everything else is corrupt in some way.

unSpawn · 06-22-2009, 06:45 AM

Quote:

Originally Posted by bmcws

The cp (and nautilus copy and paste) fails

Why would "nautilus copy and paste" behave different or more tolerant/resilient?

Quote:

Originally Posted by bmcws

Code:

        end_request: I/O error, dev sdb, sector 646375991

        end_request: I/O error, dev sdb, sector 646376143
        end_request: I/O error, dev sdb, sector 646376143
        end_request: I/O error, dev sdb, sector 646376143
        end_request: I/O error, dev sdb, sector 646376143

Buffer I/O error on device sdb1, logical block 80796991
Buffer I/O error on device sdb1, logical block 80796992
Buffer I/O error on device sdb1, logical block 80796993
Buffer I/O error on device sdb1, logical block 80796994
Buffer I/O error on device sdb1, logical block 80796995
Buffer I/O error on device sdb1, logical block 80796996
Buffer I/O error on device sdb1, logical block 80796997
Buffer I/O error on device sdb1, logical block 80796998
Buffer I/O error on device sdb1, logical block 80796999
Buffer I/O error on device sdb1, logical block 80797000

Buffer I/O error on device sdb1, logical block 80797010
Buffer I/O error on device sdb1, logical block 80797010
Buffer I/O error on device sdb1, logical block 80797010
Buffer I/O error on device sdb1, logical block 80797010

Quote:

Originally Posted by bmcws

What's the best way of recovering the data or as much data as possible (and find out which files are effected on the imaged disk)?

If your priority is to find out which files are effected then you could dry-run a fsck on the partition and see what gives. But modifying the disks contents in any way or wasting time while the drive is near end-of-life diminishes your chances of recovery. So if your priority is to get the 'dd' image off the removable then go save that data and leave the "find out" part for when time is a luxury.

Quote:

Originally Posted by bmcws

1. Would dd do any better at copying the file without errors?

Yes if the error is software-based. No if the error is hardware-based. Luckily there are versions of 'dd' that can retry on error, read from back to front, resume operations and such like dd_rescue, dcfldd or ddrescue. It's not a Holy Grail in that it will automagically save everything but it will likely save way more compared to 'cp'.

Quote:

Originally Posted by bmcws

2. Would mounting the image, then copying the files (using cp) on the mounted image copy all files in 'good' areas of the disk - and let me know which files did not copy properly; or would it just stop when it comes to the first error?

Prolly error out. If it's an IDE in a case and you loopmount the image then you stack looks like this: HD > usb_storage speaking SCSI > kernel speaking VFS for mounted filesystem > kernel providing loopmount for image > kernel speaking VFS for loopmounted filesystem. Anything that interpretes (VFS) a filesystem has software-based tolerance up to some point. Anything that "speaks" directly to the disk can have tolerances that go beyond the capabilities of VFS. If the problem is hardware-based or the image is way important you want to minimise risk, aim for HD > usb_storage speaking SCSI and dd_rescue the HD to an image somewhere safe.

bmcws · 06-23-2009, 04:44 AM

Quote:

Originally Posted by jsteel

I don't think dd is what you want here. If you leave it copying (even if it is throwing up errors) does at least some data get copied across? If so I would call this the "good data", and everything else is corrupt in some way.

The copy did actually copy across some data: about 90 percent of it... ddrescue seemed to do a better job of it... More details in my next post...

bmcws · 06-23-2009, 05:43 AM

Quote:

Why would "nautilus copy and paste" behave different or more tolerant/resilient?

No idea, just gave it a try: strangely, the copy and paste copied exactly same amount of data as a normal dd (about 90%), but the cp did only about half of the data.

Quote:

... hardware-based or the image is way important you want to minimise risk, aim for HD > usb_storage speaking SCSI and dd_rescue the HD to an image somewhere safe.

I ddrescue'd the file and there was only 2 errors/8192 B of errors. How do I find out where the errors are? Stupidly, I didn't specify a logfile: does ddrescue have a log somewhere else? I have block and sector numbers from dmesg, but how do I go about finding out which files, if any, are effected on the disk image?

unSpawn · 06-23-2009, 06:41 PM

Quote:

Originally Posted by bmcws

How do I find out where the errors are? (..) I have block and sector numbers from dmesg, but how do I go about finding out which files, if any, are effected on the disk image?

The location in bytes is logical sector * blocksize. For harddisks that's usually 512 bytes. Looking at the logical block numbers you have posted that would mean the errors start at `echo "80797010*512"|bc -l` bytes with a length of `echo "(80797010*512)-(80796991*512)"|bc -l`. Note dd_rescue supports "-s" so to verify (and obtain logging) you could start again using an offset a few logical blocks before 80796991 and a few past 80797010 on the original and the most recent disk image. Comparing output should show the imaged disk having zeroes where the original is b0rken. To see if the integrity of the image file that resides on this image file (of your external USB disk) is intact would be to loopmount this image file's partition, hash the image it contains, then hash the original disk. To pinpoint b0rkage, make it easier and more efficient you could use 'sha1sum' which provides "piecewise" mode. This means you could hash either the most recent disk image or the original harddisk (you know, the one you dd'ed the image file from onto your external USB disk of which you made an image file ;-p) and check it against the other.