What rsync command should I use for maximum preservation?
 

That is what I think is most preservative. I would be on a bootable USB, navigate to where like my root directory would be, open terminal there, and run a command like this. Those things in parentheses, (), are to be replaced by what words inside them mean. That should both backup and restore but no need for --exclude upon restore. I just would like to be sure I do not have a mishap and not backup everything I can or make some mistake and lose something. I know there is -N but like my system does not support that. I am on a desktop 22.04.3 LTS Ubuntu with Ubuntu Pro Lenovo Thinkpad T460. I understand I do not need -v but I prefer to know everything. Also, I know there is -P but I do not want --partial. Feel free to give your preferred command whether it conflicts with like my preferences or not. I would be using a USB to back up. I would be using an ext4 USB. X E.

I use something like:

This replicates all permissions that an Ubuntu OS (or Debian OS) cares about in order to function perfectly. They do not care about creation time or whatever.

You must be very careful about the specification of the source files/directories and destination directory. It is NOT intuitive. By default whatever files/directories are specified as source will be nested inside the destination directory. This is probably NOT what you are expecting.

There are some tricks, like placing "/." at the end of a directory in order to replicate the contents of the directory rather than creating a directory nested inside the destionation.

Another trick I like to use is something like this:

This will expand out to a list of all files/directories within /mnt/sda1, copying them to just inside /mnt/sdc1/. The bad news is that this will fail to delete any contents of /mnt/sdc1/ that are not in /mnt/sda1/

Sounds bad?

The GOOD news is that if there's nothing at all in /mnt/sda1/, then rsync does nothing. It does NOT delete the contents of /mnt/sda1/. This is a good thing if /mnt/sda1/ isn't mounted, so there aren't any contents. I use this technique a LOT in automated tasks, so they don't go and delete the backup if things go wrong.

I know that works for other people but I would prefer to have all attributes and stuff. Like my code can handle all timestamps. Good tips maybe though. I guess I am like sentimental or something but I am trying for a complete clone, nothing left out, so thanks but like no thanks but still thanks for trying. X E.

A complete clone is possible with dd. The destination drive must be at least as big as the source drive, and both drives must lack problem sectors.

Note that there are numerous file system types which are very unhappy about complete clones, like anything LVM or btrfs. They really do not like UUIDs being replicated.

It's also generally extremely likely that various linux processes will get confused and operate on the wrong partition if there are duplicate UUIDs.

Generally you do NOT want to have a complete clone unless it's a very temporary situation and the clones will never again be attached to the same system.

I think mine is not LVM or btrfs. Also, if I am on a bootable USB when using this then those processes should not be in progress. I just try for an only filled space full, lacking nothing of original, backup. Also, would I not be making a separate inode and therefore separating processes and stuff? Are inodes like or are they UUIDs? I know I cannot set or restore inode number but at least I can record it. From what I heard from you "dd" sounds a lot like gnome disks, it stores empty space. With a 2TB storage and like 40GB of it used by me (not left over from rsync overload 228GB) I find "dd" and gnome disks impractical for me.

If you can point me toward what not to copy with a good explaination why then go ahead but otherwise, sticking with rsync clone USB drive. Thanks for trying to help. X E.

Oh, and could I just write UUIDs to a file and restore later for compatibility? What UUIDs are you referencing? Would you like either or both of like my email or phone number so we can have this talk as long as wanted? I am in Virginia though so unsure if it is long distance. X E.

I researched UUIDs some and found that there is a specific folder and file with it. Could I just copy that separately or something like that? Maybe would a separate folder work? Like I said I am trying to lose nothing of old install. X E.

I would never use rsync for maximum preservation.




Just sayin'

what does maximum preservation means? What problem do you want to solve?

Have you thought about sparse files? Try this:

The first number tells how much disk space the file uses.

By maximum preservation I mean that if it can be preserved it gets preserved. Problem I am trying to solve now is how to lose nothing if I lost like my computer or like purged python3 like I did last time. With that, in like my original post I had all rsync options I thought would preserve everything. I am now thinking since -axAXv with other options as posted by IsaacKuo seems to work on a root operating system I could try to write all not included properties of that to a file and restore from that. I see symbolic and hard links are not included. /bin is a symbolic link on like my system and to run Bash I need #! /bin/bash so I obviously need that. Do you think adding -H for hard links and another letter for symbolic links would work with a like root rsync command? Problem I am having is how to have a complete backup and like title says, if I want rsync to preserve all it can even if it is only for /home/norvel/ then what command options and stuff should I use? X E.

Petri Kaukasoina, no I did not try that, among other things like my goal is to back up only filled space and from what I can tell "dd" backs up all, including empty, space. For a 2TB volume that is completely impractical. X E.

dd in my post had nothing to do with backing up. It was used to create testfile1. rsync did the backing up of testfile1 to testfile2. And if you had tried it, ls would have shown you an interesting result.

Thanks for that thing about dd. Maybe I will try it or maybe not but what should be output of it? Is it to show something? If so what should I know? X E.

Hint. man rsync:

Oh I should not that my examples do NOT include -H, which preserves hard links. I perhaps should start including it. Something like this:

Adding in that H preserves hard links. What my command currently does is create multiple copies of a file instead of hard linking them to a single file. This can be a problem for something that modifies a file in place (thus, modifying all hard links to it as well).

Obviously, it can also be a disk space issue.

But for my usage, this is not really a problem. The only place where I use hard links is in multiple snapshot type backups.

I am not trying for efficiency or size reduction, I just try for a full copy. I am starting to think "tar" might be a good idea but I like raw form for efficiency. Yes I have read about sparse files with rsync. I do not know what it would be used for with this though. X E.

IsaacKuo, how can I preserve symbolic links or create them again. I seem to need them. Storage space is no object. I have a 2.1TB USB stick. X E.

Looks like I can use "ln -s" for this so I can write this to a file and restore symbolic links from that. X E.

sudo rsync -vrlDAXpHtUExog --progress --delete --open-noatime --exclude='(USB at)' (like '/' but not) (copy to directory)

sudo rsync -vaxHAX --delete --progress --exclude home/kuo/.cache /mnt/sda1/* /mnt/sdc1/

Looks like missing "EUtp" so that is what I should store as a file. I just realized "l" is in "a" so symbolic links are preserved. That means executability, permissions, and I think rest are timestamps which I already record. Forgot what "t" was. Also, I like that --open-noatime so I preserve original access times. X E.

To explain that last post, that was for comparing and finding what I should record with "that means". Sorry I was confusing. X E.

If you want to save every and each file with all the attributes you need to create a full filesystem backup. No tool will manage everything.

maybe I would try the tool rsnapshot.

Just looked up again and yes, all I need to record extra is permissions and executability. I already have all timestamps. With that I think this is what to use.
Am I missing anything? X E.

This seems solved now but to be sure, is that last command I posted safe? I understand no one tool does all currently but with rsync and some code I think I can do a full backup. Am I correct in that? Also, I seem to be prefering raw paths while IsaacKuo is using /dev/... so why would I use that? I am trying for full as system was restorable. This includes what I delete stays deleted. X E.

Permissions seems complex to get and set. First 10 characters of "ls -l" are permissions but I do not know how to re-encode it for "chmod" Can I include "p" or should I try to re-encode? X E.

https://zzzcode.ai/answer-question?i...8-d98fb17da0c3

I just realized there is an easy way with "stat" and "chmod". Problem of permissions solved. Now just if like my previous to use command is safe. X E.

Symbolic links are replicated with the "l" option or anything that includes the "l" option. In particular, the "a" option includes the "l" option.

As to whether or not the replicated symlink works ... that depends on stuff later on, and may depend on whether the symlink is relative or absolute.

Either way, the "l" or "a" options will replicate all symlinks exactly as is - even broken ones.

Sorry I did not understand that "l" was in "a" before. X E.

Uh, no. I am not using /dev/, nor would it be advisable to ever try to use rsync with anything in /dev/. It only makes sense to use rsync with mounted normal file systems. (The currently running OS file system is indeed a normal file system, with mount point "/".)

Oops, it was /mnt/... not as I said, sorry. X E.

-A implies --perms. -E preserves the executability when --perms is not enabled. If --perms is enabled, this option is ignored.
Sorry, no use for your extra records of permissions/executability.

The only real difference between -vrlDAXHxog and -vaxHAX is that the latter preserves mtime. But for some reason you want to set it yourself.

As far as the OS is concerned /mnt/ is just some random normal directory. There's nothing special about it, and you can stuff it with whatever you want.

But by convention, it's a popular place to stick mount points. No special reason or benefit to putting them there.

Once upon a time, /mnt was used for mount points for removable media like floppies and CDs. But the proliferation of USB thumb drives made this increasingly confusing so they decided to move this stuff to /media.

That just left /mnt as a boring directory, so the only stuff that ever showed up there was manually put there.

I always create a /mnt/usbdrive directory there. Then when I attach a USB backup drive, I always manually mount it to /mnt/usbdrive. That way none of my scripts have to change when I run external backups. I personally dislike the /media/ concept, but at least I can get around it when I need to.

FWIW, -av --delete options to rsync are the only ones I've ever needed to backup my data to a linux formatted drive (say ext4).

I just like to be safe with backups. No losing stuff. Thank you all, especially IsaacKuo. Also, I try to store as little as possible so that thing about -A and --perms is good. If I ever have like a credits thing for a backup program that uses this, would anyone like to specify any more than user name to acknowledge? Like my email is maybejosiah@aol.com if you want a private message. X E.

Actually -a (--archive) equals -rlptgoD. Notice -p (--perms). Therefore if you use -a you already have --perms. Never used -A as don't use ACLs. In fact I had to just look them up to see why even present in Linux as owner/group/public seemed quite adequate for access control.

Thanks rclark, I was going to say that after reading rsync --help again but anyway did not get around to it. X E.

I found that IsaacKuo command does almost what like mine does. All I would add is "E". I read rsync --help again and could not find E included in -aAxXHv. Also, unsure if I should try to use rsync on that drive to back up or like my 22.04.2 desktop bootable USB flash drive version of rsync. That especially upon restore. I know how to get where rsync is. I am unsure if by default I would be using rsync on that drive or which rsync I would be using if I am in terminal at that drive. Obviously end all from directories with /. X E.

Let me shake your world: you can find all the info you need from the man page:

Okay, I read that "man rsync", so "E" is disabled if "--perms" is but I am unsure if executability is still preserved. I guess I will just include it even if it is ignored because it never said --perms preserves executability. Is there any way you know of I could preserve executability or do I need to record a file to save it? Also, is that "man rsync" online or on device and if it is on device, can I remove stuff like that? No need to answer about "man rsync", just executability. Many of like my desktop things are executable so that is one thing I would prefer to preserve. That and timestamps. Timestamps I may need to store all of because setting some with debugfs may affect others. X E.

-A, --perms, and -E all preserve executability.

-A includes --perms, which includes -E.

Yes, it is preserved: executability is only a subset of permissions. When permissions are copied, it includes read, write, exec for user, group, others.
Man pages are on your machine: file /usr/man/man1/rsync.1.gz or similar. 'locate rsync.1' should find it.
Only mtime is needed. And it is preserved with 'rsync -a' as everything else which matters.

Can you point me to some documentation that says executability is a subset of permissions? Also, call me sentimental but I prefer all timestamps, even if they do not normally matter. X E.

Okay, I take your word and like my research's word on that executability thing. Solved but I am still unsure whether I should specify where rsync is on like my main drive for backup or restore. X E.

Everything is in the man pages (usually):

It's up to you. Plain 'rsync' is enough but you can also specify the path. Maybe it's /usr/bin/rsync. 'whereis rsync' or 'which rsync' will tell.

Thanks, read. X E.

I think I prefer on main drive, newer stuff should have bug fixes and like and yes, I know how to find it. Thanks. X E.

Like my main concern about using rsync is if I overwrite rsync, what happens to that running program with restore. With that reply I assume it would keep running fine. X E.

If you type in the command "rsync", the bash shell will search for a command named "rsync" within the current $PATH. You can see the $PATH with this:

It searches these paths.

This is similar to how things work on, say, MS-DOS, but in Linux this $PATH will basically never include the current directory.

The bottom line is that it will only execute the copy of rsync that's from the currently running OS, and never any other copy of rsync unless you for some (stupid) reason specify otherwise with a full path or relative path to some other copy of rsync.

In general overwriting a file which is currently in use is not a good idea, that may have strange side effects.
In such cases, the operating system may continue to run the copy in RAM, as long as the system does not attempt to access the original file on disk. This can cause an instant crash (of that app).