Copy encrypted compact flash issues with dd in ubuntu
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I'll look into the md5 so i can try that. As for the other thought, i'm not sure what it's checking. We just so happen to have about 4 of this same type of unit, and i can remove the cf from any of them and put them in this unit with the now bad cf and it starts up. So something has to be in the cards that tells the machine it is the correct card, something that's not getting copied right with the dd command.
It's true that from my end it's hard to tell that i'm doing it correctly, that last attempt i tried with the oflag=sync definitely took a lot longer, so i figured it was probably the right way.
https://www.youtube.com/watch?v=Zj9H03rVef4 (not that i expect you guys to watch a ten minute video) This is where i got my method from, however i've already written in this thread what i was doing as well.
I did read on another thread that size might matter somehow in this type of situation, so i already ordered a 256 mb cf card that's on it's way. I also don't know if brand matters,i'm using a transcend brand. i was reading about industrial vs regular cf and that they are different. I'm using industrial cards because that's all we have here at work. but the smallest we have is 1 GB.
I'll keep reading about dd to make sure i'm not missing something and i'll read about md5 to use the suggestion from JeremyBoden.
Again thanks for all the suggestions and help I really appreciate you guys taking the time out.
There are a number of ways to look into the files too. You can take your dd file and actually mount that in a virtual machine as a disk. Use a live iso of linux or even BartsPE to view contents.
I suspect that some file some where is coded to use this special chip. Guess it could even be an onboard encryption or special chip. More likely it has a file that has been coded to read some unique metric on the chip.
While the md5 may help I'd suspect that would be more correct on a new never used cf. I'd be more tempted to do file by file diff.
I understand the idea of mounting the file as a disk in a vm. The 2nd part about live iso I don't follow. Also does that mean my dd file should be an iso to mount it to the vm?
also is file by file diff. manually comparing files, or is that a command? Just point me in the direction and i'll get started on that. I did the md5 btw, but I had already copied over my cf card with a copy from a different software someone suggested and the numbers weren't the same. So now i'm copying from my dd file back to my cf card and going to do the md5 again to see if they match. Might take a while thought so i'll start looking into what you said Jefro
Scratch part of that. I googled the diff command. I think if I understand it correctly I can write this command to do what you're saying:
diff -s /dev/sde home/folder/cfcard
where /dev/sde is where my cfcard reader will be with my good card installed and home/folder/cfcard is my copy that I used dd to make previously
and diff is my command with the added -s to report back that the files are the same when it finishes.
not sure if I structured that correctly, the example I was reading just said file1 and file2. let me know if you can, I still have a few hours before I can borrow a card from a working unit.
did the md5sum just for giggles. The numbers didn't match each other. Not sure what that means. the two that I did the md5sum on are the file on my hard drive and the cfcard that i just did the dd to.
also did the diff command on the hard drive copy and the cf card copy I just made using dd and it says they differ....how can a file differ from something I just used dd to create. am I missing something or shouldn't they be the exact same?
Last edited by solojayda3rd; 06-03-2015 at 12:17 AM.
would be nice to show us the commands (exactly) executed and results, not only speak about that.
md5sum will produce a string for you. In general if the two strings were identical that means the two sources are identical too. Different results mean different sources.
Nothing should be mounted on the CF card - nothing in /dev/sde[1-9]. You can't normally mount or unmount a device node (/dev/sde) - especially if partitions exist on it.
Seeing that my files differ made me start thinking back on this post....every time I plug in a cfcard it shows sde1. Everytime I do dd I specify sde (with no 1 because I thought I didn't need 1) however syg00 says nothing should be mounted. I took that to mean I shouldn't manually mount, but that's not what he said. He said it should be mounted.
so my question is, should I have been manually umount sde1 everytime I dd to my blank cfcard? Could that be why it's not copying right identical files?
would be nice to show us the commands (exactly) executed and results, not only speak about that.
md5sum will produce a string for you. In general if the two strings were identical that means the two sources are identical too. Different results mean different sources.
Ok here's the command for the card I'm writing to
Code:
sudo md5sum /dev/sde
The result was ea5b63fd4899e315798cc19ee45b6f3a
Here's the command for the file on my hard drive
Code:
sudo md5sum /home/folder/cfcard
the result was 704c0083ed50b0116f6720294ae3ec0
for the diff command my code was
Code:
sudo diff -s /home/folder/cfcard /dev/sde
the result was Binary files /home/folder/cfcard and /dev/sde differ
for me it means the command dd from cf to /home/folder/cfcard was not successful. Either the copy itself failed somehow or the device was already damaged, or ??? (probably a security issue?)
if that was the case, wouldn't I just be copying my bad identical information from my hard drive to the compact flash? I could understand it not working, but it should still be identical shouldn't it? Do you have any thoughts on my unmount question
You can try (several times):
sudo dd if=/dev/sde of=/home/folder/cfcard
sudo dd if=/dev/sde of=/home/folder/cfcard1
md5sum /home/folder/cfcard /home/folder/cfcard1
to check if that copy works properly
to umount: you should umount everything (/dev/sde*) before dd to the card, mounted filesystems (during dd) may cause problems. Obviously if there was any mounted filesystem you must umount that manually.
Yeah what you just said is the last thing I did which appears to be working better. I was finally able to get my hands on the good cf card and I went ahead and tried the diff command to compare it to the copy I made of it yesterday on my harddrive and it came back as differ. So I did the dd again, but this time I df -h, saw sde1 so I umount /dev/sde1 before I started the dd. I then did the dd
Code:
sudo dd if=/dev/sde of=/home/folder/cfcard
then proceeded to do the diff
Code:
sudo diff -s /dev/sde /home/folder/cfcard
and it came back identical this time. So finally, right now I'm doing the dd again (after I umount sde1 again since I plugged in my blank cf sde1 mounted automatically again) on to my blank card and hopefully it comes back identical when I do the diff after it finishes.
So I did as I mentioned in my last post, and for some reason the copy isn't identical to the file on my hard drive. The only good news is, after several checks I'm sure the copy on my hard drive is identical to the good card. I'm just not sure why my dd isn't copying from my hard drive to my blank cf correctly. I'm using
It's taking about an hour to copy, but when it finished I ran
Code:
diff -s /home/folder/cfcard /dev/sde
and it came back binary files differ. And as mentioned I did the umount as soon as I plugged the blank cfcard in
Code:
umount /dev/sde1
then I did the dd. I'm not sure if df -h, is causing some sort of issues, but I did perform that after every time I plugged in my cfcard reader, just to make sure it was automatically mounting my card. I'm going to try it again (not sure what another attempt is going to do different though) and I'll post if I get any good news.
instead of oflag/sync write oflag=sync
probably only a typo.
You need to check using the command mount if there was another mount (for example /dev/sde2).
As another step you can try to execute sync (as root) two times (after dd) to be really sure the content is copied to that device.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.