LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)
-   -   Setting up an rsync server: advice needed (https://www.linuxquestions.org/questions/linux-general-1/setting-up-an-rsync-server-advice-needed-4175732064/)

hazel 02-24-2024 09:22 AM

Quote:

Originally Posted by Turbocapitalist (Post 6485590)
Is the target drive encrypted? I find that slows things way down. Depending on the brand/model of drive, encryption can slow it down by a factor of 10 or so.

No, I don't use encryption on either of my machines as they never leave the house. Besides, I would always be nervous of losing the encryption password!

It's probably slow because:
1) Littleboy is a very low spec machine.
2) The copy was done partly over wifi. I've tried to do it entirely via ethernet, but even when I used the address of the ethernet card on the laptop, it still used wifi for the transfer.

Most of today's update was the Mozilla cache, which imho is not worth copying. I must see if I can exclude that.
Quote:

This thread has encouraged some experimentation with these methods. On one project, I've moved to a rolling differential backup using Rsync and a --link-dest target based on the previous day.
Glad to have been of use.

Turbocapitalist 04-13-2024 04:00 AM

I've revisited this one today. I think there is an approach which gives a week's worth of backups, while automatically allowing for missed days such as weekends or holidays. The date of the previous successful backup is recorded in the file system by touching the one directory and that date is later retrieved using stat with date via the UNIX epoch.

Code:

...
d1=$(date +"%u")
d2=$(date -d "@$(stat --format='%X' ./backup/)" +"%u")

set -e
rsync -avH --link-dest=./backup/${d2} -e '...' \
        far@away:/source/ ./backup/${d1}/

touch ./backup/
...

If multiple runs happen on the same day, the --link-dest option is basically a non-operation. I'm not sure what rsync does with it under the hood, but if it $d2 and $d1 are the same it is either ignored or pointing to the same and thus unchanged files. Either way, those files are not copied while any differences from the remote server are still processed.

The set -e is just a quick hack, there are probably more appropriate ways using the $? variable to deal with tracking whether the backup was successful and the directory's date refreshed.

If a year's worth of backups are needed then %j can be used instead of %u in the date calculation. Or else use %d for a month's worth of backups.


All times are GMT -5. The time now is 02:56 PM.