Alternatives to dd for skipping empty space

I am attempting to archive a 240G internal SSD.

In the past I've experience difficulties using dd with command line compression, speed & I/O errors.

Anyone using an alternative Linux utility for backup images of only used partition space (currently < 40GB)?

Have you looked into PARTCLONE or FSARCHIVER?

if you backup a full partition there is no used/free space, this information is not available.
If you backup files/dirs you can obviously skip "free" space. You can use for example rsync to do that.

Originally Posted by pan64 if you backup a full partition there is no used/free space, this information is not available.

Tools like partclone can produce a backup that does not include a supported filesystem's free space. Partclone's image format includes only the used space, and can optionally be compressed. If you play the game right with losetup, you can even produce a mountable sparse file that looks like the original filesystem but does not allocate any disk blocks for the free space.

Originally Posted by rknichols Tools like partclone can produce a backup that does not include a supported filesystem's free space. Partclone's image format includes only the used space, and can optionally be compressed. If you play the game right with losetup, you can even produce a mountable sparse file that looks like the original filesystem but does not allocate any disk blocks for the free space.

yes, partclone works on the filesystem, but dd has no any idea about that (including partclone.dd).

Originally Posted by pan64 If you backup files/dirs you can obviously skip "free" space. You can use for example rsync to do that.

Right, Linux, but this is a laptop Redmond partition with its hidden/system file nonsense. It needs to be a partition backup without the blank space.

I will check out PartClone & FSArchiver.

what do you mean by hidden/system file? What OS and filesystem is it?

Originally Posted by wpeckham Have you looked into PARTCLONE or FSARCHIVER?

Do you lean towards one or the other?

I see PartClone is part of Clonezilla & Wikipedia reads FSArchiver has a Windows version, but otherwise no obvious standout.

Originally Posted by pan64 what do you mean by hidden/system file? What OS and filesystem is it?

If you don't need Windows, consider yourself blessed.

TAR

Normally you'd move the files to be contiguous. Then you'd Zero out free space.

One can actually use windows backup feature on a hidden partition if they unhide it.

If you have a way to attach a WD drive they offer a free Acronis version that works great. Attach can be isci I think too.

Clonezilla is normally a file by file clone if it can read the filesystem.

I am just curious. What sort of error do you get from a command like this?

dd if=/dev/sda1 bs=1M skip=0 count=100 | gzip -c > test.bin.gz

Originally Posted by JASlinux I am attempting to archive a 240G internal SSD. In the past I've experience difficulties using dd with command line compression, speed & I/O errors. Anyone using an alternative Linux utility for backup images of only used partition space (currently < 40GB)?

Originally Posted by mw.decavia I am just curious. What sort of error do you get from a command like this? dd if=/dev/sda1 bs=1M skip=0 count=100 | gzip -c > test.bin.gz

If it were recent I could tell you. I remember a lot of frustration, vaguely recalling it could have been the file system (NTFS). It was also very slow (I use old computers).

What if we had room to clone the whole disk then split the .img before compressing each chunk after? I mull the possibilities.

I also need to learn why defragmenting is "trimming" the disk instead of the old graphical read/write routine. If it works the partition could be shrunk to used + required space.

Originally Posted by jefro TAR Normally you'd move the files to be contiguous. Then you'd Zero out free space.

I know what that means about not why it is a suggestion. If you .tar'd a partition that could be an archive if accurately replicated, not having to worry about what else is on a source disk.

One can actually use windows backup feature on a hidden partition if they unhide it.

Windows has a default system backup I use, but I need a Linux version.

If you have a way to attach a WD drive they offer a free Acronis version that works great. Attach can be isci I think too.

If you can download it as owner of a drive that sounds good.

Clonezilla is normally a file by file clone if it can read the filesystem.

Again, the goal is to skip the unused space, split into manageable chunks, & ideally compress, but maybe not all in one step if done manually.

And the question is again, what have you tried, what went wrong? What do you want to achieve?
If you have windows/ntfs better to use windows based tools to make backups.
Why do you need linux at all?
Zeroing the free space has a "side effect": you can easily backup the whole partition without storing all of those the zeroes (called sparse file).
But anyway, as far as I see you want to backup files, and you have a solution already. It will automatically skip free space. Tar can make an archive like that, including splitting and compressing too.
If you have I/O errors you need to fix them first, but it looks like it is not important. Copying 240GB may take some time, and there is no tool to overcome hardware limitations (if any).

The sample command I just offered you ...

dd if=/dev/sda1 bs=1M skip=0 count=100 | gzip -c > test.bin.gz

Would backup all of a windows "boot" partition with free/empty space compressed to almost nothing. (if it did not fail with an error)

For example, the 100 mb "boot" partition which was standard on windows7, results in a backup file "test.bin.gz" of less than 10 mb.

Of course, you would substitute the partition you want to backup in "if=/dev/", and put it's size (in mb) in "count="

This has worked for me for backing up fairly large windows partitions. But it can backup any partition that you can read from (even "hidden" partitions), no matter what filesystem/files it contains.

Originally Posted by JASlinux If it were recent I could tell you. I remember a lot of frustration, vaguely recalling it could have been the file system (NTFS). It was also very slow (I use old computers). What if we had room to clone the whole disk then split the .img before compressing each chunk after? I mull the possibilities. I also need to learn why defragmenting is "trimming" the disk instead of the old graphical read/write routine. If it works the partition could be shrunk to used + required space.