Building a distributed storage system from uneven-sized disks
 

I am in the fortunate position of being on the receiving side of a lot of hardware. From time to time some old servers (Compaq ML350) get into my possession. And from time to time I get some new disks.

At the moment I have 12 servers and approx 50-70 hard-drives in the sizes from 2GB to 250GB (most of them being 9GB, 20GB and 40GB).

At the moment I'm running 8 40GB disks per server, which results in a filesystem of approx. 250GB (RAID5) per. server. This is neat and good, but as the amount of servers increase, so does the maintenance.

What I would like is to have a sort of distributed filesystem that can grow/shrink as I add servers and disks.

Obviously, with this kind of setup, I don't have access to replacement drives/servers when something breaks, so basically when something breaks it needs to be taken out of commission. When this happens the replication should kick in and make sure I don't lose any data.

I have been thinking about a simple mirroring setup with 1 or two replicas, but some kind of system that makes sure that the replicas are always on different servers. Obviously 2 servers with one disk each is the minimum requirements.

One of my most crucial points is that I want it to be one BIG filesystem which is usable through the normal kernel filesystem layer. This filesystem will then be shared to clients using Samba and/or NFS.

High-availability is not crucial, as it is only for home use, but redundancy is important, as backups will be handled to a remote location where the network speed is very slow (read: ADSL).

Of course, performance when reading files sequentially is much more important than the speed of directory browsing.

Is this even possible with current open source tools?