Configuration glusterfs en mode AFR (auto replication file)


Machine cliente : IP 192.168.1.236
Fichier de conf : /etc/glusterfs/afr

### Add client feature and attach to remote subvolume of server1
volume client1
type protocol/client
option transport-type tcp/client # for TCP/IP transport
option remote-host 192.168.1.237 # IP address of the remote brick
option remote-port 6996 # default server port is 6996
option remote-subvolume brick1 # name of the remote volume
end-volume

### Add client feature and attach to remote subvolume of server2
volume client2
type protocol/client
option transport-type tcp/client # for TCP/IP transport
option remote-host 192.168.1.238 # IP address of the remote brick
option remote-port 6997 # default server port is 6996
option remote-subvolume brick2 # name of the remote volume
end-volume

## Add AFR (Automatic File Replication) feature.
volume afr
type cluster/afr
subvolumes client1 client2
# option replicate *:2
end-volume

Serveur 1 : IP 192.168.1.237
Fichier de conf : /etc/glusterfs/afr
# **** server1 spec file ****

### Export volume "brick" with the contents of "/home/export" directory.
volume posix1
type storage/posix # POSIX FS translator
option directory /home/export # Export this directory
end-volume

### Add POSIX record locking support to the storage brick
volume brick1
type features/posix-locks
option mandatory on # enables mandatory locking on all files
subvolumes posix1
end-volume

### Add network serving capability to above brick.
volume server
type protocol/server
option transport-type tcp/server # For TCP/IP transport
option listen-port 6996 # Default is 6996
# option client-volume-filename /etc/glusterfs/glusterfs-client.vol
subvolumes brick1
option auth.ip.brick1.allow * # access to "brick" volume
end-volume

Serveur 2 : IP 192.168.1.238
Fichier de conf : /etc/glusterfs/afr
# **** server2 spec file ****
volume posix2
type storage/posix # POSIX FS translator
option directory /home/export # Export this directory
end-volume

### Add POSIX record locking support to the storage brick
volume brick2
type features/posix-locks
option mandatory on # enables mandatory locking on all files
subvolumes posix2
end-volume

### Add network serving capability to above brick.
volume server
type protocol/server
option transport-type tcp/server # For TCP/IP transport
option listen-port 6997 # Default is 6996
subvolumes brick2
option auth.ip.brick2.allow * # Allow access to "brick" volume
end-volume


On demarre ensuite le demon glusterfsd sur chacun des serveurs en lancant :
glusterfsd -f /etc/glusterfs/afr

Une petite verif par netstat -lnp |grep tcp :
tcp 0 0 0.0.0.0:6996 0.0.0.0:* LISTEN 2380/[glusterfsd]


Cote client il ne reste plus qu'a monter la ressource distante:
glusterfs -f /etc/glusterfs/afr /mnt (j'ai choisi /mnt comme mount-point)

On verifie que le montage s'est bien effectué:
cliglus:/etc/glusterfs# df -hT
Sys. de fich. Type Tail. Occ. Disp. %Occ. Monté sur
/dev/sda1 ext3 1,9G 596M 1,2G 34% /
tmpfs tmpfs 81M 0 81M 0% /lib/init/rw
udev tmpfs 10M 52K 10M 1% /dev
tmpfs tmpfs 81M 0 81M 0% /dev/shm
glusterfs fuse 3,7G 1,2G 2,3G 34% /mnt


Je verifie le contenu de /mnt :
cliglus:/# ll /mnt
total 12
drwxr-xr-x 2 1001 1001 4096 2008-02-21 10:59 .
drwxr-xr-x 23 root root 4096 2008-02-20 16:25 ..

Meme chose cote serveurs :
srv2glus:/# ll /home/export
total 12
drwxr-xr-x 2 export export 4096 2008-02-21 10:59 .
drwxr-xr-x 4 root root 4096 2008-02-20 17:15 ..


srvglus:/# ll /home/export
total 12
drwxr-xr-x 2 export export 4096 2008-02-21 10:59 .
drwxr-xr-x 4 root root 4096 2008-02-20 17:15 ..


Je teste maintenant la copie d'un fichier du client vers /mnt :

cliglus:/# cp /var/cache/apt/archives/libc6_2.3.6.ds1-13etch5_amd64.deb /mnt
et je verifie ...
1) chez le client
cliglus:/# ll /mnt
total 4108
drwxr-xr-x 2 1001 1001 4096 2008-02-21 11:03 .
drwxr-xr-x 23 root root 4096 2008-02-20 16:25 ..
-rw-r--r-- 1 root root 4183448 2008-02-21 11:03 libc6_2.3.6.ds1-13etch5_amd64.deb

2) sur le serveur 1
srvglus:/# ll /home/export
total 4108
drwxr-xr-x 2 export export 4096 2008-02-21 11:03 .
drwxr-xr-x 4 root root 4096 2008-02-20 17:15 ..
-rw-r--r-- 1 root root 4183448 2008-02-21 11:03 libc6_2.3.6.ds1-13etch5_amd64.deb

3) sur le serveur 2
srv2glus:/# ll /home/export
total 4108
drwxr-xr-x 2 export export 4096 2008-02-21 11:03 .
drwxr-xr-x 4 root root 4096 2008-02-20 17:15 ..
-rw-r--r-- 1 root root 4183448 2008-02-21 11:03 libc6_2.3.6.ds1-13etch5_amd64.deb

OK c'est bon.

Pour finir un coup d'oeil sur les logs (/var/log/glusterfs):

client:
2008-02-21 11:00:29 W [fuse-bridge.c:373:fuse_entry_cbk] glusterfs-fuse: 2: / => 1 Rehashing 0/0

server1:
2008-02-21 10:58:58 E [protocol.c:259:gf_block_unserialize_transport] server: EOF from peer (192.168.1.236:1023)
2008-02-21 10:58:58 C [tcp.c:87:tcp_disconnect] server: connection disconnected
2008-02-21 10:59:05 W [glusterfsd.c:229:glusterfsd_cleanup_and_exit] glusterfsd: shutting down server

server2:
2008-02-21 10:58:59 E [protocol.c:259:gf_block_unserialize_transport] server: EOF from peer (192.168.1.236:1022)
2008-02-21 10:58:59 C [tcp.c:87:tcp_disconnect] server: connection disconnected
2008-02-21 10:59:15 W [glusterfsd.c:229:glusterfsd_cleanup_and_exit] glusterfsd: shutting down server

Evidemment il s'agit là  d'une config de base mais elle a le mérite d'etre operationnelle en quelques minutes.
Pour les confis plus poussées je vous recommande le site glusterfs.org .