LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Ubuntu (https://www.linuxquestions.org/questions/ubuntu-63/)
-   -   What rsync command should I use for maximum preservation? (https://www.linuxquestions.org/questions/ubuntu-63/what-rsync-command-should-i-use-for-maximum-preservation-4175732619/)

maybeJosiah 01-09-2024 06:04 PM

What rsync command should I use for maximum preservation?
 
Quote:

sudo rsync -vrlDAXpHtUExog --progress --delete --open-noatime --exclude='(USB at)' (like '/' but not) (copy to directory)
That is what I think is most preservative. I would be on a bootable USB, navigate to where like my root directory would be, open terminal there, and run a command like this. Those things in parentheses, (), are to be replaced by what words inside them mean. That should both backup and restore but no need for --exclude upon restore. I just would like to be sure I do not have a mishap and not backup everything I can or make some mistake and lose something. I know there is -N but like my system does not support that. I am on a desktop 22.04.3 LTS Ubuntu with Ubuntu Pro Lenovo Thinkpad T460. I understand I do not need -v but I prefer to know everything. Also, I know there is -P but I do not want --partial. Feel free to give your preferred command whether it conflicts with like my preferences or not. I would be using a USB to back up. I would be using an ext4 USB. X E.

IsaacKuo 01-09-2024 06:25 PM

I use something like:

Code:

rsync -vaxAX --delete --progress --exclude home/kuo/.cache /mnt/sda1/. /mnt/sdc1/
This replicates all permissions that an Ubuntu OS (or Debian OS) cares about in order to function perfectly. They do not care about creation time or whatever.

You must be very careful about the specification of the source files/directories and destination directory. It is NOT intuitive. By default whatever files/directories are specified as source will be nested inside the destination directory. This is probably NOT what you are expecting.

There are some tricks, like placing "/." at the end of a directory in order to replicate the contents of the directory rather than creating a directory nested inside the destionation.

Another trick I like to use is something like this:

Code:

rsync -vaxAX --delete --progress --exclude home/kuo/.cache /mnt/sda1/* /mnt/sdc1/
This will expand out to a list of all files/directories within /mnt/sda1, copying them to just inside /mnt/sdc1/. The bad news is that this will fail to delete any contents of /mnt/sdc1/ that are not in /mnt/sda1/

Sounds bad?

The GOOD news is that if there's nothing at all in /mnt/sda1/, then rsync does nothing. It does NOT delete the contents of /mnt/sda1/. This is a good thing if /mnt/sda1/ isn't mounted, so there aren't any contents. I use this technique a LOT in automated tasks, so they don't go and delete the backup if things go wrong.

maybeJosiah 01-09-2024 07:10 PM

I know that works for other people but I would prefer to have all attributes and stuff. Like my code can handle all timestamps. Good tips maybe though. I guess I am like sentimental or something but I am trying for a complete clone, nothing left out, so thanks but like no thanks but still thanks for trying. X E.

IsaacKuo 01-09-2024 08:27 PM

A complete clone is possible with dd. The destination drive must be at least as big as the source drive, and both drives must lack problem sectors.

Note that there are numerous file system types which are very unhappy about complete clones, like anything LVM or btrfs. They really do not like UUIDs being replicated.

It's also generally extremely likely that various linux processes will get confused and operate on the wrong partition if there are duplicate UUIDs.

Generally you do NOT want to have a complete clone unless it's a very temporary situation and the clones will never again be attached to the same system.

maybeJosiah 01-09-2024 08:49 PM

I think mine is not LVM or btrfs. Also, if I am on a bootable USB when using this then those processes should not be in progress. I just try for an only filled space full, lacking nothing of original, backup. Also, would I not be making a separate inode and therefore separating processes and stuff? Are inodes like or are they UUIDs? I know I cannot set or restore inode number but at least I can record it. From what I heard from you "dd" sounds a lot like gnome disks, it stores empty space. With a 2TB storage and like 40GB of it used by me (not left over from rsync overload 228GB) I find "dd" and gnome disks impractical for me.

If you can point me toward what not to copy with a good explaination why then go ahead but otherwise, sticking with rsync clone USB drive. Thanks for trying to help. X E.

maybeJosiah 01-09-2024 08:57 PM

Oh, and could I just write UUIDs to a file and restore later for compatibility? What UUIDs are you referencing? Would you like either or both of like my email or phone number so we can have this talk as long as wanted? I am in Virginia though so unsure if it is long distance. X E.

maybeJosiah 01-09-2024 09:27 PM

I researched UUIDs some and found that there is a specific folder and file with it. Could I just copy that separately or something like that? Maybe would a separate folder work? Like I said I am trying to lose nothing of old install. X E.

wpeckham 01-09-2024 10:03 PM

I would never use rsync for maximum preservation.




Just sayin'

pan64 01-10-2024 06:50 AM

what does maximum preservation means? What problem do you want to solve?

Petri Kaukasoina 01-10-2024 09:24 AM

Quote:

Originally Posted by maybeJosiah (Post 6475730)
sudo rsync -vrlDAXpHtUExog --progress --delete --open-noatime --exclude='(USB at)' (like '/' but not) (copy to directory)

Have you thought about sparse files? Try this:

Code:

dd if=/dev/zero of=testfile1 bs=1G seek=1 count=0
rsync -vrlDAXpHtUExog testfile1 testfile2
ls -lsh testfile1 testfile2

The first number tells how much disk space the file uses.
Code:

ls -lsh /var/log/lastlog /var/log/faillog

maybeJosiah 01-10-2024 09:31 AM

By maximum preservation I mean that if it can be preserved it gets preserved. Problem I am trying to solve now is how to lose nothing if I lost like my computer or like purged python3 like I did last time. With that, in like my original post I had all rsync options I thought would preserve everything. I am now thinking since -axAXv with other options as posted by IsaacKuo seems to work on a root operating system I could try to write all not included properties of that to a file and restore from that. I see symbolic and hard links are not included. /bin is a symbolic link on like my system and to run Bash I need #! /bin/bash so I obviously need that. Do you think adding -H for hard links and another letter for symbolic links would work with a like root rsync command? Problem I am having is how to have a complete backup and like title says, if I want rsync to preserve all it can even if it is only for /home/norvel/ then what command options and stuff should I use? X E.

maybeJosiah 01-10-2024 09:36 AM

Petri Kaukasoina, no I did not try that, among other things like my goal is to back up only filled space and from what I can tell "dd" backs up all, including empty, space. For a 2TB volume that is completely impractical. X E.

Petri Kaukasoina 01-10-2024 09:43 AM

Quote:

Originally Posted by maybeJosiah (Post 6475827)
Petri Kaukasoina, no I did not try that, among other things like my goal is to back up only filled space and from what I can tell "dd" backs up all, including empty, space. For a 2TB volume that is completely impractical. X E.

dd in my post had nothing to do with backing up. It was used to create testfile1. rsync did the backing up of testfile1 to testfile2. And if you had tried it, ls would have shown you an interesting result.

maybeJosiah 01-10-2024 09:50 AM

Thanks for that thing about dd. Maybe I will try it or maybe not but what should be output of it? Is it to show something? If so what should I know? X E.

Petri Kaukasoina 01-10-2024 09:56 AM

Hint. man rsync:
Code:

      --sparse, -S
              Try  to  handle  sparse  files  efficiently so they take up less
              space on the destination.



All times are GMT -5. The time now is 01:37 AM.