Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524
Original Poster
Rep:
Quote:
Originally Posted by brodo
How your skip and seek parameters relate to a 250GB disk to be cloned ?
Is it practical ?
If talking about directory archiving this is useful:
Code:
tar cvzf - /home/user/something/ | split --bytes=2000MB - something.tar.gz.
Can this idea be arranged with DD to clone HDD in this manner ?
Skip skips the number of blocks in the input. The drive you're reading from is the input. Seek you don't actually need in my example, so I edited it out. You can add compression to a dd command by piping and redirect or double piping. It's in the OP.
No, dd has no split switch. Usually when a user wants to make separate volumes with dd, it's to fit each volume to a DVD disk.
er, would it be possible to make 'smart dd', so that it reads partition table first, and, if it recognizes the (checked) filesystem, it reads only used blocks if user desires so?
so that later it writes less.
I think clonezilla does that?
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524
Original Poster
Rep:
The excellent feature of dd for back-ups is that it will create a bootable copy of a drive. If you don't need it to be bootable, there are other tools much more suited to the task, like clonezilla, rsync, bacula, etc.
dd will also clone partition table. if you are upgrading drive capacity, it will take some time to resize partitions if needed.
tar is also nice option, probably used by some of aforementioned tools.
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524
Original Poster
Rep:
Dd will copy everything on the drive. That's not technically true, but it will copy everything in the user-data storage areas of the disk.
Tar simply puts multiple files into one bigger file. Linux file-compression programs don't compress multiple files, so tar is necessary to compress a directory. I would estimate that tar is seldom used for it's original purpose, which was to make files ready for tape archives.
tar is better than most other similar utils, because, iirc, it can preserve linux file attributes, permissions..
one advantage over dd would be defragmentation on hdd.
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524
Original Poster
Rep:
dd tar rsync
Quote:
Originally Posted by qrange
tar is better than most other similar utils, because, iirc, it can preserve linux file attributes, permissions..
one advantage over dd would be defragmentation on hdd.
Dd preserves file attributes. Tar is ok, but you need room for the tar files. In this day and age I prefer rsync and then if it needs to be bootable, just boot with a live system and install grub to it.
Disks are getting too big to use dd for typical backups. But dd still has a lot of uses.
since `ps aux | awk '/bin\/dd/ && !/awk/ {print $2}'` is an empty string.
A solution would be to run:
`pidof dd | xargs kill -s USR1 $1` since I tested `pidof dd | xargs echo $1` is not empty.
In the end I use this twoliner found on this page :
(https://www.cyberciti.biz/faq/linux-...-while-coping/)
in pasting the result (the PID) of
`pidof dd` ### <--- say pid is 21145
in the next line :
`sudo kill -USR1 21145`
Further an interesting remark discussing the exact same extension to the dd-command you present later on your page 1
(reading: "Note that sending a SIGUSR1 signal to a running 'dd' process makes it ... ")
is that this will only work on (some) Linux, (see this discussion :
(https://unix.stackexchange.com/quest...d-mean-exactly)
-USR1 works like this only in the GNU version of 'dd' (AFAIK). A SIGUSR1 to 'dd' for other OSes (eg AIX, HP-UX & Solaris) will terminate the process, by default.
And instead of SIGUSR1 use SIGINFO (or INFO) on OSX (probably the same on BSD)
And like the last time I posted something here: Great page, thank you
im in the mid of a huge process of expanding my partition and this thread just helps to clear a lot of current questions,
thanks alot. I will bookmark this thread!
Oh? Never had a 'huge process' of repartitioning a drive before, it's always been pretty simple. And you've been asking about drive partitioning/re-partitioning now for EIGHT YEARS, under your other user ID, sayhello_to_the_world: https://www.linuxquestions.org/quest...to-4175463188/
What questions did this thread clear up for you? Which posts? Be great if you 'posted all of your findings', so that others with those questions could locate the posts in this thread easily.
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524
Original Poster
Rep:
DD in 2023
The needs for dd have changed radically since about 2015. SSDs parallelize writes, sdo the new way to wipe drives is with the SECURITY_ERASE_UNIT ATA command. But that only works up to SATA drives.
NVMe drives have a different command set. Basically the same command set--ATA--was used for 20 years. Now along comes NVM command set which is quite robust, quite flexible and powerful, fully accessible to the general public, but designed with SSD technology in mind.
A sequential bitstream copy program such as dd may not work predictably with the newer NVMe controllers. With wear concerns, a SSD won't write /dev/zero to an entire drive.
And, each manufacturer kind of rolls their own command set out of the immense NVM set of command sets. NVMe drives will be used in many consumer, government, military, institutional and industrial devices the likes of which cannot be fully imagined yet. Therefore the drive controllers do much more, and require a more robust command set that dd can't adequately exploit presently
Dd was made with a much simpler model in mind. Although I'm only aware of one area where it fails with SSDs, and that is writing sequential zeroes to an entire drive. That model is now obsolete.
Also, duplicating massive swathes of empty drive space is no longer rational, so at the very least, for mirroring, use clonezilla. Dd still has it's uses, but sequential mirroring is on the way out.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.