It has been long since Linux has flabbergasted me as much as it did today:
I found a normal ascii-text (log)file of 3.8G on an ext3 filesystem, however, df shows me only 512M of the 4.5G is actually used on the filesystem and du shows me the file is only 61M:
output of ls :
Code:
# ls -liah /var/log/jobscheduler/scheduler.out
41291 -rw------- 1 root root 3.8G Apr 22 17:09 /var/log/jobscheduler/scheduler.out
output of du :
Code:
# du -h /var/log/jobscheduler/scheduler.out
61M /var/log/jobscheduler/scheduler.out
output of df :
Code:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgjobscheduler-lvjobscheduler_log 4.5G 521M 3.7G 13% /var/log/jobscheduler
I can copy the file using cp, multiple times on the same filesystem:
Code:
# cp /var/log/jobscheduler/scheduler.out /var/log/jobscheduler/scheduler.out.1
# cp /var/log/jobscheduler/scheduler.out /var/log/jobscheduler/scheduler.out.2
# cp /var/log/jobscheduler/scheduler.out /var/log/jobscheduler/scheduler.out.3
# ls -liah /var/log/jobscheduler/scheduler.out*
41291 -rw------- 1 root root 3.8G Apr 22 17:09 /var/log/jobscheduler/scheduler.out
41292 -rw------- 1 root root 3.8G Apr 22 17:28 /var/log/jobscheduler/scheduler.out.1
41293 -rw------- 1 root root 3.8G Apr 22 17:28 /var/log/jobscheduler/scheduler.out.2
41294 -rw------- 1 root root 3.8G Apr 22 17:28 /var/log/jobscheduler/scheduler.out.3
So now I have 15.2G on a 4.5G filesystem ..
OK, I though maybe the file has been allocated to 3.8G but actually uses only 61M which could explain this.. However:
When I cat the file to a new file, this new file actually does take 3.8G of diskspace:
Code:
# cat /var/log/jobscheduler/scheduler.out > /var/log/jobscheduler/scheduler.out.1
# ls -lia scheduler.out*
41291 -rw------- 1 root root 4045065364 Apr 22 17:09 scheduler.out
41292 -rw-r--r-- 1 root root 4045065364 Apr 22 17:34 scheduler.out.1
# du -h scheduler.out*
61M scheduler.out
3.8G scheduler.out.1
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgjobscheduler-lvjobscheduler_log 4.5G 4.3G 0 100% /var/log/jobscheduler
Also scp of the file to a remote machine gives me a full 3.8G file..
A line-count of the file shows me both files contain exactly the same amount of lines:
Code:
# cat scheduler.out | wc -l
294271
# cat scheduler.out.1 | wc -l
294271
and diff shows me there is absolutely no difference between those 2 files..
So the file actually contains 3.8G of data, but it only takes 61M on the filesystem.. as if it is compressed on FS-level.. but as far as I know, Ext3 doesn't support compression on filesystem level..
I checked for filesystem errors, but none where found..
With stat I found out that the original file actually takes a lot less blocks:
Code:
# stat scheduler.out
File: `scheduler.out'
Size: 4045065364 Blocks: 123472 IO Block: 4096 regular file
Device: fd06h/64774d Inode: 41291 Links: 1
Access: (0600/-rw-------) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2015-04-22 17:10:10.000000000 +0200
Modify: 2015-04-22 17:09:42.000000000 +0200
Change: 2015-04-22 17:09:42.000000000 +0200
Birth: -
than the 'catted' file:
Code:
# stat scheduler.out.1
File: `scheduler.out.1'
Size: 4045065364 Blocks: 7908248 IO Block: 4096 regular file
Device: fd06h/64774d Inode: 41292 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2015-04-22 17:34:19.000000000 +0200
Modify: 2015-04-22 17:34:32.000000000 +0200
Change: 2015-04-22 17:34:32.000000000 +0200
Birth: -
But where is then the 3.8G of data stored in this original file? ...
I manually checked the file contents of both files and verified that there are no blocks of zero's or so.. all looks like valid ascii log entries.. 294271 lines of ascii text..
Is anybody here that could explain what I am seeing here ?
