Moving directories between different machines

NotionCommotion · 11-24-2017, 08:31 PM

My end goal is to copy the "directoryB" directory from machine1:/home/user1/directoryA/directoryB/ to machine2:/home/user2/someDirectory/directoryA/directoryB/. Or if easier, I am okay with just copying the files within the directoryB directory of machine1 to the coorisponding directory in machine2. The directoryB directory has multiple subdirectories and I would like to speed up the transfer time, and by intention is to use tar -cvzf.

A couple of details:

user1@machine1's public ssh key is in user3@machine2's authorized keys.
user2@machine1's public ssh key is in user1@machine2's authorized keys.
user1@machine2 has sudo access and can access user3@machine2's home directory.

I can create the tar and zip it easy enough, but the file structure just keeps on getting messed up no matter how I change -C starting/directory.

I don't need it to be the most concise way ever, and wish to keep things relatively simple and relatively concise.

How would you recommend doing this? Thanks

frankbell · 11-24-2017, 08:46 PM

I would recommend rsync. There's a bit of a learning curve, but it's well worth the effort.

giis · 11-24-2017, 10:53 PM

Couple of ways, imo.

1 Use xz compression and rsync the .xz file
2 or simply rsync with archive option.

Turbocapitalist · 11-25-2017, 12:54 AM

As everyone has already mentioned, rsync is a good way to go. However, even though it uses SSH by default for the transport, you'll have to tell it which key to use via the -e option:

Code:

rsync -a -H -e 'ssh -i ~/.ssh/some_key' source  destination

chrism01 · 11-26-2017, 06:53 PM

If you are having trouble with the -C option, how about

Code:

# src m/c
cd /blah/src_dir/
scp -r * user@target:/dir1/dir2/tgt_dir/

AwesomeMachine · 11-26-2017, 07:52 PM

I don't think rsync uses ssh by default. I use it every day.

frankbell · 11-26-2017, 07:54 PM

I should have mentioned it last night, but there's also scp. I used it today to copy a directory from one machine to another.

There are some excellent examples here: https://haydenjames.io/linux-securel...les-using-scp/

The target machine must be running sshd (the ssh server daemon). The sending machine need only have the ssh client program installed.

JJJCR · 11-26-2017, 07:58 PM

scp -r /path/to/directory someuser@serverB:/path/to/files/.

rsync -avz /source/path user@desthost:/destination/path/

rsync -avz -e 'ssh -p 2222' /source/path user@desthost:/destination/path/

source:
https://serverfault.com/questions/17...erver-to-other
https://superuser.com/questions/6081...nother-via-ssh

or try exploring Midnight Commander:
http://linuxcommand.org/lc3_adv_mc.php

AwesomeMachine · 11-26-2017, 08:19 PM

Yes, but is scp just a variation on the primitive cp, without error checking or anything?

JJJCR · 11-26-2017, 08:51 PM

Quote:

Originally Posted by AwesomeMachine

Yes, but is scp just a variation on the primitive cp, without error checking or anything?

I'm not sure about it but I think there's a lot of difference.

Here's a thread from someone ascking scp source code:
https://www.linuxquestions.org/quest...e-code-539253/

frankbell · 11-26-2017, 09:06 PM

Per this link, scp does not do error checking by default, but I must say I've not encountered any errors in transferring files within my LAN via scp, and I've moved everything from text files to video files. Indeed, I'm watching one of those videos files as I type this and it has progressed flawlessly.

By the way, AwesomeMachine, excellent question.

AwesomeMachine · 11-26-2017, 11:08 PM

Thanks frankbell. Why do you need scp inside the LAN? And, you might not find errors transferring 100 files. But when you start transferring 10,000 files or more it can become more of a problem.

NotionCommotion · 12-08-2017, 03:20 PM

Quote:

Originally Posted by AwesomeMachine

Thanks frankbell. Why do you need scp inside the LAN?

Probably just my lack of understanding, but why is being on a LAN or WAN relevant?