This is my first post, of any kind, anywhere, ever... so I was a bit hesitant to start a new thread but was unable to find an existing thread that seemed appropriate. Also I realize I am fishing a bit with the 'Subject' line but I wanted to cast a broad net. If I missed an existing thread I apologize.

I am contemplating a new project and would appreciate some feedback from anyone with experience clustering 32bit processors.

My goal is to build a series of appliances each consisting of 2 nodes joined as an HA cluster. For scalability I intend for the appliances to be capable of being joined to create a larger cluster. I will be running Xen on top of the aggregate cluster with minimal single-service hosts running as domU guests. I am experienced with both Xen and HA clustering and have worked out most of the technical details. However, for efficiency, I am considering using mini-ITX boards with fanless C7 processors for the appliances and the C7 is a 32 bit processor. I am wondering if there is going to be an issue with the memory addressing limit inherent in 32bit processors once the cluster as a whole exceeds 4GiB of RAM. With the mini-ITX/C7 setup each node would have a roughly 1Ghz clock speed and 1GiB RAM so when I add my 5th and 6th node the total RAM will exceed the 4,294,967,296 address limit of a 32bit processor, however, at that point the cluster will have 6 processors available to provide addresses...

If anyone has created a 32bit cluster with more than 4GiB of RAM or is familiar with how Xen addresses memory when multiple x86 cores are available to it I would be greatly appreciative of feedback, barring a qualified response I will scrap together some old x86 machines and post a definitive answer. However, I would like to avoid the experiment if anyone can verify that it is a waste of time.

Thanks for your time, Joe

My above post is probably more wordy than it needs to be. My question is not specific to Xen or even to clustering and can be summarized thus..,

Are SMP aware programs running in 32bit OSes on multiple x86 processors limited to 2^32 addresses? If not is the limit then (2^32)*[number of processors] or 2^(32*[number of processors])?

I would still be interested to hear from anyone who has created a 32bit cluster as there may be other considerations I have missed. I have found online articles where people have built x86 clusters but I haven't found any with performance figures that would indicate that they were utilizing more than 4GiB of RAM.

Thanks again, Joe

At this point I am just thinking this out out-loud for the benefit of anyone who stumbles upon this thread. I think I'm a little bit closer to an answer, let me know if my logic is flawed...

Although each processor has a limit to the number of addresses that it can assigned in parallel, the total number of addresses that can be assigned and tracked by a cluster is actually constrained by the OS kernel regardless of the number of processors and their combined capabilities. If I use a Xen kernel with PAE support I should be able to support 64 nodes at 1GiB per node.

If the above is correct then I should be able to cluster 32 appliances before I hit the wall and that is far more than my target. If a client needs more than a dozen appliances to support their processing load it's time to pony up for a blade server and build a real 64bit cluster.

Have I missed any considerations?
Has anyone here really done this?
-Joe

I just came across a news release that Via is soon to release a low power C7 compatible 64 bit processor with Hardware Virtualization support. If anyone cares the processor code is CN and it is known as Isaiah. I guess I will leave the 32 bit clustering to someone else. -joe

P.S. For posterity, I think my previous reasoning was correct and I don't anticipate any memory issues with building a PAE enabled 32 bit cluster except for the obvious performance hit inherent in using the PAE extension.