Hi all,
I'm having some difficulty copying a large amount of data from an XFS filesystem (pre-existing, LVM) to an ext3 filesystem I've made. With both rsync and cp, I can copy a good deal of data (~250GB), but I eventually get this sort of thing in my /var/log/messages:
Code:
...
Feb 17 10:09:12 kernel: attempt to access beyond end of device
Feb 17 10:09:12 kernel: sdd1: rw=0, want=30372064920, limit=28671999967
Feb 17 10:09:12 kernel: EXT3-fs error (device sdd1): ext3_free_blocks: Freeing blocks not in datazone - block = 3796508114, count = 1
Feb 17 10:09:12 kernel: Aborting journal on device sdd1.
Feb 17 10:09:12 kernel: EXT3-fs error (device sdd1) in ext3_free_blocks_sb: Journal has aborted
...
Feb 17 10:09:22 kernel: ext3_abort called.
Feb 17 10:09:22 kernel: EXT3-fs error (device sdd1): ext3_journal_start_sb: Detected aborted journal
Feb 17 10:09:22 kernel: Remounting filesystem read-only
...
Feb 17 10:09:25 kernel: EXT3-fs error (device sdd1) in ext3_reserve_inode_write: Journal has aborted
Feb 17 10:09:25 kernel: EXT3-fs error (device sdd1) in ext3_orphan_del: Journal has aborted
Feb 17 10:09:25 kernel: EXT3-fs error (device sdd1) in ext3_truncate: Journal has aborted
Feb 17 10:09:25 kernel: attempt to access beyond end of device
Feb 17 10:09:25 kernel: sdd1: rw=1, want=33264745920, limit=28671999967
Feb 17 10:09:25 kernel: printk: 53 messages suppressed.
Feb 17 10:09:25 kernel: Buffer I/O error on device sdd1, logical block 4158093239
Feb 17 10:09:25 kernel: lost page write due to I/O error on sdd1
Feb 17 10:09:25 kernel: attempt to access beyond end of device
Feb 17 10:09:25 kernel: sdd1: rw=1, want=29308994896, limit=28671999967
Feb 17 10:09:25 kernel: Buffer I/O error on device sdd1, logical block 3663624361
Feb 17 10:09:25 kernel: lost page write due to I/O error on sdd1
...
Feb 17 10:09:25 kernel: __journal_remove_journal_head: freeing b_committed_data
Here's what I have that may be relevant:
I created a ~14 TB partition (gpt partition table) using parted:
Code:
Using /dev/sdd
(parted) print
Disk geometry for /dev/sdd: 0.000-22886400.000 megabytes
Disk label type: gpt
Minor Start End Filesystem Name Flags
1 0.017 14000000.000 ext3
I checked for kernel EFI support:
Code:
[mario ~]$ uname -r
2.6.9-89.33.1.ELsmp
[mario ~]$ grep EFI /boot/config-2.6.9-89.33.1.ELsmp
CONFIG_EFI_PARTITION=y
I made an ext3 filesystem using mkfs.ext3:
Code:
[mario ~]$ sudo mkfs.ext3 -c /dev/sdd1
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
896000000 inodes, 3583999995 blocks
179199999 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
109375 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544, 1934917632,
2560000000
Checking for bad blocks (read-only test): done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[mario ~]$ sudo tune2fs -l /dev/sdd1
tune2fs 1.41.14 (22-Dec-2010)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 2f4cecb5-a146-4284-b41f-d80ce1fe57d5
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags: signed_directory_hash
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 896000000
Block count: 3583999995
Reserved block count: 179199999
Free blocks: 3527723838
Free inodes: 895999989
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 169
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Filesystem created: Wed Feb 16 20:28:23 2011
Last mount time: Thu Feb 17 09:34:01 2011
Last write time: Thu Feb 17 09:34:01 2011
Mount count: 1
Maximum mount count: 27
Last checked: Wed Feb 16 20:28:23 2011
Check interval: 15552000 (6 months)
Next check after: Mon Aug 15 21:28:23 2011
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: 723a94c7-78b0-436d-adbe-694abd47b23e
Journal backup: inode blocks
The commands I used to copy were `rsync -aPv <dest> <target>`, `rsync -aPvSH <dest> <target>` and `cp -av <dest> <target>`.
The kernel version is (as seen above) 2.6.9-89.33.1.ELsmp and the box is running CentOS release 4.8 (Final). The originating disk is an LVM2 volume, created from a hardware RAID5 disk array (Dell PERC6 controller). The target disk is a RAID6 disk array attached to the same controller.
I'm not sure what I'm missing here but I figure it is likely something done incorrectly while making the filesystem or the partition. I suppose it is possible that the XFS/ext3 filesystem differences are trouble but that seems like something rsync and cp should be able to handle. I'm not extremely well-versed in all this so any help would be appreciated!
Thanks,
Mario
