Hi all,
I would like to build a small beowulf like cluster with few machines that I have here around from the pc-farm. The problem is that up to now I have only been able to find docs and links to old/unupdated/dead projects...

Do you have some hints? I slighly prefere debian-based systems, but any (reasonably recent....) distro would be ok.

Thank you!
Luigi


PS: I hope this is the correct forum for such question...

Well, "cluster" is a pretty general term. What exactly would you like to do with it? You need to have some kind of application in mind before you get started to figure out what sort of software you are looking at.

Beowulf is an advanced system for mathematical applications, but one could argue it is no more or less a cluster than two Debian boxes passing jobs between each other via SSH.

Hi MS3FGX,
I have a bunch of number-crunching applications that need to crunch few Tbytes of data in a "embarassing parallel" way (to say it in HPC jargon).

I did already the ssh trick between two machines but, as you know, there are lots of "little problems" like disk sharing, handling a machine/job failure, etc etc that must be handled and I would like not to reinvent the wheel...
So before extending it to more machines I would really be happy to try some beowulf clustering, i.e. the same game in pratice, but much simpler stuff for the end users.

What do you think? do you have some pointers to active projects?

Thanks!
Luigi

So I assume you are going to be writing the software for this cluster then?

In that case, you might want to look at OSCAR. I haven't used it personally, but I have heard good things. Basically is a complete toolkit to create and maintain Beowulf-style clusters on common Linux distributions. It handles all the mundane details that you mentioned like inter-node communication and leaves you free to just write up your code.

As I understand it, OSCAR can even create disk images with your code and configuration that are suitable for booting your nodes.

Hi MS3FGX,
thank you for the link
I have just tried OSCAR on a VirtualBox image with ubuntu 9.04, but I am not able to have it running: is it possible that OSCAR supports only ubuntu up to 8.04, debian up to 4, etc and not newer distros?
Any hint to have it running on a newer system?

Regards!
Luigi

Look at the video of Beowolf cluster

http://www.wbitt.com/hpcc-supercomputing-cbts.html