Is there any faster way to copy a large amount (~ 2 TB) in different folders (in NTFS partitions); for example, from /dev/sdb1 to dev/sdc1 such as

/dev/sdb1/f1 /dev/sdb1/f2 /dev/sdb1/f3 to /dev/sdc1/f1 /dev/sdc1/f2 /dev/sdc1/f3

I like the integrated copy operation. I like it much better than Windows, but I have to copy around 2 TB of data and I want to be able to do it in the fastest way possible cause the situation is a bit urgent. I am not expecting a miracle, just trying to find out the fastest way.

Also, I am looking for a sync tool to check. I will be glad if you can give me suggestions about that too. I will be using the mirror method where right will be the same as left. I use freefilesync in Windows (which I do not like much that it uses 'delete and write' instead of 'rename' but it is the one I am using recently), but I do not know about what can I use in linux.

Thanks.

I guess the first question is - how often will you be doing this? If you are copying the large amounts of data frequently the answer may be faster hardware, raid, fiber channel drives, explore the file system you are using etc. If it is a one time thing, just copy it. I am not sure that the command line or a gui tool will be significantly different. I believe the hardware will be the limiting factor.

When going across a network (from my PC to server) I find that connecting to a nfs share point on the server is considerably quicker than using an ssh connection. Of course nfs is not nearly as secure.

As to a sync tool... I have been using mirrodir for quite a while. It might not be the best but it works for me.

Ken

1 members found this post helpful.

from what you posted they are local HDDs, or at least mounted USB type devices. if you are NOT needing to delete them (ie mv) then rsync might be faster then raw cp.

if you are going to use rsync you will need to include the --temp-dir=foo (i typically just use /tmp) as NTFS drives will NOT allow for true rsync on the local drive.

1 members found this post helpful.

I'd expect cp to be faster than rsync, because rsync does a differences check first; this is pointless if you are copying the whole file anyway.
OTOH I think rsync checksums as it works, but cp not so much.
Basically the total time is a combination of checking (either as you go or post-facto) and physically copying.
For a one off/full copy plan, I'd use cp for speed, then md5sum (or eg sha1sum) both sets (in parallel) and just compare the md5sums. I reckon that would be fairly quick.