Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
By jeremy at 2008-05-29 12:55
Using OCFS2, A Clustering File System
Get a grip on the next-gen Oracle Cluster File System, an alternative to GFS.
Jeremy Garcia
Linux Magazine
Traditionally in Linux, if you wanted to access a file system from more than one machine you would mount it on one machine, and then export it to the others via NFS, CIFS or something similar. This is a stable, mature and well documented procedure, but it does have its downsides. Performance is not great, there can be locking issues, there are potential security implications and worst of all you have a single point of failure.
If your NFS server goes down, all clients immediately lose access. Luckily, there is another way to accomplish this task. A clustered file system is a file system which is simultaneously mounted on multiple servers. Implementing a clustered file system used to be a complicated and expensive proposition. It typically involved a high priced SAN, Fiber Channel HBA’s and switches and a proprietary file system.
Now you have iSCSI and your choice of Open Source clustered file systems available. The most popular is probably GFS, which Red Hat Open Sourced after acquiring Sistina. GFS now ships with both Fedora and CentOS, in addition to being a supported option on top of Red Hat Enterprise Linux. It is included in the mainline kernel as of 2.6.19.
Another option is OCFS2, which is the next generation of the Oracle Cluster File System. It is an extent based, POSIX compliant file system. Unlike the previous release (OCFS), OCFS2 is a general-purpose file system. As of this writing, you should be aware that OCFS2 does not support mmap. While support is planned for the near future, if you need a clustering file system that supports mmap now, you will want to look into GFS. (NOTE: since this article was published, mmap support has been added).
The OCFS2 user space tools are available for a variety of distributions, either directly from Oracle or from the distribution manufacturer. OCFS2 is licensed under the GPL. Check here for the main project page. To utilize OCFS2 you’ll need to install the ocfs2-tools package and the appropriate kernel modules (ocfs2-`uname -r`).
The ocfs2console package is not required but is recommended for ease-of-use. With these installed we’re ready to setup an OCFS2 partition. Setting up your shared storage is beyond the scope of this article and should be completed before you proceed. The examples used in this article are based on a RHEL 5.1 installation.
Some commands or file locations might differ slightly on other distributions. Note that if the current RHEL kernel does not have a corresponding OCFS2 kernel module RPM available directly from Oracle, you can easily rebuild one from the official ocfs2 tarball. As with GFS, OCFS2 is now in the mainline kernel, so if you are using a newer distribution you should check to see if you already have OCFS2 support before looking for the appropriate kernel module package.
First, you’ll need to edit /etc/sysconfig/o2cb and ensure it contains this line:
Code:
O2CB_ENABLED=true
Next, create /etc/ocfs2/cluster.conf on each machine. The following is for a three node cluster. For multi-homed machines, the node parameter should match the host name, regardless of which IP you use.
Code:
node:
ip_port = 7777
ip_address = 192.168.100.1
number = 0
name = host1.domain.com
cluster = ocfs2
node:
ip_port = 7777
ip_address = 192.168.100.2
number = 1
name = host2.domain.com
cluster = ocfs2
node:
ip_port = 7777
ip_address = 192.168.100.3
number = 2
name = host3.domain.com
cluster = ocfs2
cluster:
node_count = 3
name = ocfs2
You can now run /etc/init.d/o2cb start to start the cluster services. If everything worked, running /etc/init.d/o2cb status should look as follows:
With everything working you should add the partition to your fstab and ensure both o2cb and ocfs2 are set to start at boot.
Code:
/sbin/chkconfig o2cb on
/sbin/chkconfig ocfs2 on
This article has covered all the basics, and should be enough to get you a working OCFS2 setup. If you plan to run OCFS2 in production you should read the FAQ, which includes more advanced topics such as resizing, quorum and fencing, limitations and rolling upgrades.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.