LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 02-03-2013, 10:55 AM   #1
allidium
LQ Newbie
 
Registered: Feb 2013
Posts: 5

Rep: Reputation: Disabled
Angry Backup to external usb disk with ext4 or ext3 is dead slow


Hi,

I have an Oracle Linux Server and I have been backing up my server on an external usb 2 drive with a fat32 file system for ages without any problems. In order to backup about 20GB it took about 45 minutes. But I reached a point where I had some large files > 4GB which could not be backed up on a fat32 system.

So I decided to re-format the external drive to an ext4 partition. The results were devastating. It took about 15 hours to complete the backup.

I tried the same with an ext3 format, but the results were the same.

Am I doing something wrong?
 
Old 02-03-2013, 11:23 AM   #2
mcnalu
Member
 
Registered: Dec 2006
Location: Glasgow, UK
Distribution: Slackware current
Posts: 423

Rep: Reputation: 73
I'm not sure why you're seeing such a drop in performance but try disabling journaling on ext4. I've experienced some fairly substantial speed ups in usb drives from that.
 
1 members found this post helpful.
Old 02-04-2013, 01:08 AM   #3
allidium
LQ Newbie
 
Registered: Feb 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Yes, but the difference in performance is very big, i.e. 45 minutes vs 15 hours.

I do not think disabling journalling will do the trick. By the way, I just reformatted the external drive to NTFS and a backup of 20GB was done in just 15 minutes.

I am forced to use NTFS as I have not other choice. Am I the only one having such slow performance on ext3/ ext4?

By the way, the actual file system on the server is ext3.
 
Old 02-04-2013, 01:23 PM   #4
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,627

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
usb2 is only 35 MB/s . So 20GB will take a bit of time

it takes me about 5 to 8 min. to move a 14 gig image ( yes ONE 14 gig photo ) to my 3Tb storage drive , formatted to ext4 -- with journalling

the time difference is what is also running at the time

How is the drive mounted ?
I use a udev rule to force it to /dev/sdc and mount it in fstab

don't forget about the "bit VS. byte " sizes "Mb VS. MB"
 
1 members found this post helpful.
Old 02-05-2013, 12:19 AM   #5
allidium
LQ Newbie
 
Registered: Feb 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Hi John,

I do not create a specific entry in fstab, the drive simply gets mounted automatically when I plug it in. But you say that you copy 14GB in about 8 minutes. I need 10-15 hours to store 20GB. Do you think that the way it was mounted makes a difference?

If yes, can you please send me your example fstab mount command?

Thank you.
 
Old 02-05-2013, 01:16 AM   #6
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,627

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
Quote:
I have an Oracle Linux Server
seeing as that is based off ?? what RHEL 5 or RHEL6

As in what version is it

I take it there is a DE used ( guessing Gnome2 )

is the system using fuse ( 5 had an issue with fuse and the kernel, fixed in 5.3 ) and is gnome what is mounting it
or an init script

Right now i am only just guessing , so.....



my fstab entry is just a very basic one
Code:
/dev/sdc1      /mnt/USB.3Tb.Drive  ext4	rw,users 0 0
allowing the "users" group full read / write

and rhel 5 or 6 are not using "udev" so that is not an issue
 
1 members found this post helpful.
Old 02-05-2013, 01:38 AM   #7
allidium
LQ Newbie
 
Registered: Feb 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
I am using RHEL5 with KDE 3.5.4

Fuse was not even installed. I installed it a few days ago just to be able to use NTFS file systems. Currently I am backing up on an NTFS external drive. It takes only about 45 minutes to back up 33GB.

But I really would like to find out what am I doing wrong and ext3/4 takes so long.

By the way, the server is using ext3 for its main OS partition and I have not experienced any problems with that.

Thank you.
 
Old 02-05-2013, 09:01 AM   #8
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,784

Rep: Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214
Is this an "Advanced Format" drive with 4KB physical sectors? If so, a partition that is not aligned on a 4KB boundary (multiple of 8 512-byte logical sectors) can reduce write performance by a factor of 10. Example:
Code:
# hdparm -I /dev/sdd | grep 'Sector size'
	Logical  Sector size:                   512 bytes
	Physical Sector size:                  4096 bytes
 
1 members found this post helpful.
Old 02-05-2013, 11:47 AM   #9
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,627

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
rknichols has a very good point

Quote:
So I decided to re-format the external drive to an ext4 partition.
how was the drive formatted

the 3 Tb i am using is ( 4k as above )
 
1 members found this post helpful.
Old 02-06-2013, 12:43 AM   #10
allidium
LQ Newbie
 
Registered: Feb 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Unfortunately hdparm does not work with this drive, it gives the following error:

HDIO_DRIVE_CMD(identify) failed: Invalid argument
 
Old 02-06-2013, 02:04 AM   #11
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,627

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
reformat the drive

i use Gparted live cd
unless you must use special settings
leave it at the default settings and format to ext4
for a back up drive one partition is fine
if you need partitions, then 3 normal primary
one extended
then as many logical as needed
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] ntfs or ext4 on my external usb drive poly_s Linux - General 6 11-10-2011 01:35 PM
[SOLVED] Permanent filesystem corruption on reiserfs, ext3 and ext4 - disk failure? trumpet_tom Linux - Hardware 9 02-11-2010 05:29 AM
migrated from ext3-disk to a ext4 disk - now linux wont boot zilogZ80 Linux - Newbie 4 02-05-2010 03:19 AM
Mayday! How do I manually mount my encrypted ext3 external usb disk? simonsharry Linux - Newbie 11 02-15-2009 04:49 AM
ext3 RAID 1 on a USB External Two Disk Hard Drive - Possible? Runge_Kutta Linux - Hardware 3 08-17-2007 06:28 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 08:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration