|
Scalable Storage Clustering with redundancy? What to use?
I'm looking at implementing a large and scalable storage solution to replace lots of file servers which needs to meet the following requirements:
1. Redundancy
2. Scalability/Expandability
3. Performance
4. Single Directory tree for all storage
5. Preferably runs on Linux, my favourite.
6. Preferably on commodity hardware that is easy to replace/add, although not necessary
Ideally I'd like it to have something like a Global Filesystem like AFS or Microsoft's DFS where everything is organized into one directory tree, but also to have redundancy and data security like a Cluster Filesystem, so if one or more nodes breaks then it still works and we just put in new nodes. I also need to be able to extend it by putting in new nodes so that I can make the storage grow indefinitely.
At the moment my storage requirements are 15-30TB, but this will increase and so I need to able to add more space by adding more nodes.
What I think I am really after is a Cluster FileSystem, something like the Google File System, except that is not available cos Google are evil and eat up lots of talented open source people and keep cool stuff like that to themselves.
Ideally the solution should also be fast and reliable, so that I can crunch data on it from other servers.
Of course it doesn't __have__ to be open source or unixy or anything, but it would be nice...
Any ideas?
|
