edoceo

How To: GlusterFS Installation and Configuration

Installing and Configuring GlusterFS

Install GlusterFS, FUSE and other requirements.

emerge fuse glusterfs
[ebuild  N    ] sys-fs/fuse-2.8.5
[ebuild  N    ] sys-cluster/glusterfs-3.3.1  USE="-emacs -extras fuse (-infiniband) -static-libs -vim-syntax"
apt-get install glusterfs-server attr

On Gluster Server systems ensure that the glusterd daemon is started on boot.

rc-config add glusterd default

GlusterFS Single Server NFS Style §

In the most simplest configuration GlusterFS can be used in similar fashion to NFS. This configuration has a central GlusterFS server that allows multiple clients to connect. One nice thing to notice is that the clients have no configuration on themselves. The client configuration information is obtained from the server on demand.

This setup will simply use a single system as a GlusterFS host, similar to a single NFS type system. Replace, $name, $host, $path with proper value, $path can reference an existing directory.

gluster volume create $name transport tcp $host:$path
gluster volume start $name

Simple huh? Now when glusterd is stop/started this volume will be retained. If you are curious about this information look in /var/lib/glusterd.

GlusterFS Replica §

This configuration has two servers, which are like a mirror for the volume. Assume the servers are named gfs1 and gfs2, these commands are run on gfs1

gluster peer probe gfs2
gluster volume create BigVol replica 2 transport tcp gfs1:/brick/BigVol gfs2:/brick/BigVol
gluster volume start BigVol

GlusterFS Convert Distribute to Replica §

Suppose you start off with only one Gluster Brick and then get another server, hooray! Now, rather than Distribute you can convert this one Brick system to a Replica.

gluster peer probe $new_host
gluster volume add-brick $name replica 2 $new_host:/$path
gluster volume info $name

Yep, just adding a new Brick to an existing Distribute volume gets the job done.

GlusterFS Distribute Replica - Four(4) Hosts

This places files across two hosts, in a four host volume.

gluster peer probe gfs2
gluster peer probe gfs3
gluster peer probe gfs4
gluster volume create BigVol replica 2 transport tcp gfs1:/brick/BigVol gfs2:/brick/BigVol gfs3:/brick/BigVol gfs4:/brick/BigVol
gluster volume start BigVol

If you examine the volfile you will see that gfs1 and gfs2 make up a mirror, gfs3 and gfs4 also make up a mirror. Files, reads and writes will be distributed across these systems.

Connecting a Client §

Almost too easy, simply mount it. $host is the Gluster server, $name is the name of the exported volumne and $path is the local path to mount to.

root@desk # mount -t glusterfs $host:$name $path
# eg:
root@desk # mount -t glusterfs carbon:raid /mnt/raid

You can examine the Volume using the getspec sub-command.

gluster --remote-host=gfs1 system getspec BigVol

Configuring Quota §

Quota can be set on the directory, with '/' referencing the entire volume. Here we set a volume wide limit of 10GB and a limit of 2G on a specific directory. Then we set a quota timeout of 5 seconds. Then, using list to show the quota settings

gluster volume quota BigVol enable
gluster volume quota BigVol limit-usage / 10GB
gluster volume quota BigVol limit-usage /path/in/volume 2G
gluster volume set BigVol features.quota-timeout 30
gluster volume quota BigVol list
gluster volume quota BigVol list /path/in/volume

Clear these out with

gluster volume quota BigVol remove /
gluster volume quota BigVol remove /path/in/volume

Performance Options §

Over TCP you really must have fat ethernet cards, gigabit or better; dual nics are advisable.

gluster volume set BigVol diagnostics.brick-log-level WARNING
gluster volume set BigVol diagnostics.client-log-level WARNING
gluster volume set BigVol nfs.enable-ino32 on
gluster volume set BigVol nfs.addr-namelookup off
gluster volume set BigVol nfs.disable on
gluster volume set BigVol performance.cache-max-file-size 2MB
gluster volume set BigVol performance.cache-refresh-timeout 4
gluster volume set BigVol performance.cache-size 256MB
gluster volume set BigVol performance.write-behind-window-size 4M
gluster volume set BigVol performance.io-thread-count 32

See Also

Loading Comments from Disqus...