Can not RSYNC files from native NAS volume to mayQNAPcloud Storage

mackowiakp · 12-25-2023, 12:23 AM

I would like to backup MariaDB in a container to myQNAPcloud Storage using RSYNC.
QNAP cloud resource mounted with QNAP HybridMount app.
I do this from the container using the command below. $path - a variable containing the path to the mounted resource from myQNAPcloud storage.

Code:

rsync -ratxvO --force --delete -e '/usr/bin/ssh -p668 -x -q -i /root/.ssh/id_rsa' /backup admin@media:$path
RSYNC cannot change the modification time settings of directories and files in the mounted cloud resource.

So I added the "O" option, which bypasses the time modification. However, something is still wrong. The directory structure is created but no files are transferred.
The message is that RSYNC cannot change the temporary name of the transmitted file (starting with a dot) to the target name. I would use RCLONE, which I use for backups to Google or M$ drives, and the problem would be solved.
But there is no QNAP drive in the RCLONE menu. So what should the RSYNC command look like to perform the above-mentioned backup?
At the moment, the disk mounted in this way certainly does not allow changing the modification time and has some problem with files whose names start with a dot.
Similarly, "normal" copying has problems with modification time:

Code:

cp -a nc-data /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/
cp: preserving times for `/share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/data/maciek/files/Dokumenty/FIRMA/2016/2-luty/Dokumenty': Is a directory
cp: preserving times for `/share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/data/maciek/files/Dokumenty/FIRMA/2016/2-luty/Faktury': Is a directory

However, cp is not a good solution for larger numbers of files, only some of which are different in reality of rather slow speed access to cloud resources.
Someone knows how to solve this problem?

lvm_ · 12-25-2023, 07:52 AM

Quote:

Originally Posted by mackowiakp

The message is that RSYNC cannot change the temporary name of the transmitted file (starting with a dot) to the target name.

Try running it with --inplace option.

mackowiakp · 12-26-2023, 01:53 AM

I try:

Code:

rsync -ratxvO --inplace --force --delete /share/Container/nc-data /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/

It synced several filesf/dirs but suddenly error and syncing stoped:

Code:

rsync: write failed on "/share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/.cache/ffmpeg-static-nodejs/d8e99a1ec077199ee27a98834289213261aa5bb0dfd73d566df7bb0c2f2df6eb02d4f5d7a2e9954111bb296ce199c489d2bd58d47933df0410485416b2d0f768.json": Timer expired (62)

So better but still not good...

Maybe problem is filesize but it is not huge:

Code:

[/share/homes/media/Pobrane/Moje] # ls -la /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/.cache/ffmpeg-static-nodejs/7282f931472805b88df890264a2e7b0f35223d7f6fcf43e54dcdc172e5cd21d1b61e4ffa8531bb3d3a3db02b75808d76c3f54fb97a9ed879edaa4efe5da75d09.body
-rw-r--r-- 1 911 911 28099886 2023-12-26 08:47 /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/.cache/ffmpeg-static-nodejs/7282f931472805b88df890264a2e7b0f35223d7f6fcf43e54dcdc172e5cd21d1b61e4ffa8531bb3d3a3db02b75808d76c3f54fb97a9ed879edaa4efe5da75d09.body

EDIT
Filesize is not a problem:

Code:

rsync: write failed on "/share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/keys/cert.crt": Timer expired (62)
rsync error: error in file IO (code 11) at receiver.c(376) [receiver=3.1.2]
[/share/homes/media/Pobrane/Moje] # ls -la /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/keys/cert.crt
-rw-r--r-- 1 911 911 1342 2023-12-26 08:36 /share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/keys/cert.crt

Adding "--timeout=200" does not resolve the problem

lvm_ · 12-26-2023, 02:42 AM

rsync doesn't retry on errors. If you have a shitty connection or a shitty cloud or whatever, put it in a loop until it returns 0.

mackowiakp · 12-26-2023, 02:55 AM

I dont think so because I sync to GDrive or OneCloud without problems. So it looks like some timers settings

And additional errors:

Code:

rsync: failed to set times on "/share/external/.cm/0/1344d097f-4b77-483b-8d2f-34bcf4228353/nc-data/config/www/nextcloud/apps/privacy/lib/Settings/UserDataManifestoSettings.php": No such file or directory (2)

pan64 · 12-26-2023, 03:51 AM

are these files in use on the remote site?

mackowiakp · 12-26-2023, 04:01 AM

mackowiakp · 12-27-2023, 05:05 AM

Using RCLONE instead RSYNC resolved the problem.