[Gluster-users] One node goes offline, the other node can't see the replicated volume anymore
Greg Scott
GregScott at infrasupport.com
Mon Jul 15 20:37:00 UTC 2013
Hang on a second, I didn't show enough of /var/log/messages from fw1. Here is a larger extract, showing the result of the mount commands. It shows errors mounting my filesystem, but then when I first login, I can see it anyway. Welcome to the Twilight Zone.
I have no idea why libvirtd is starting here. I'm going to get rid of that and reboot again.
.
.
.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started LSB: Bring up/down networking.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Network.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Reached target Network.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started Login and scanning of iSCSI devices.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Virtualization daemon...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started Virtualization daemon.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounting /firewall-scripts...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Vsftpd ftp daemon...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting RPC bind service...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting OpenSSH server daemon...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting /etc/rc.d/rc.local Compatibility...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started Vsftpd ftp daemon.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started RPC bind service.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting GlusterFS an clustered file-system server...
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Making sure the Gluster stuff is mounted
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Mounted before mount -av
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started OpenSSH server daemon.
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Filesystem Size Used Avail Use% Mounted on
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-root 14G 3.8G 8.7G 31% /
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: devtmpfs 990M 0 990M 0% /dev
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /dev/shm
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 2.1M 994M 1% /run
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /sys/fs/cgroup
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /tmp
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/sda2 477M 87M 365M 20% /boot
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/sda1 200M 9.4M 191M 5% /boot/efi
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: extra arguments at end (ignored)
Jul 15 15:18:07 chicago-fw1 dbus-daemon[457]: dbus[457]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (u
sing servicehelper)
Jul 15 15:18:07 chicago-fw1 dbus[457]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper
)
Jul 15 15:18:07 chicago-fw1 kernel: [ 24.022605] fuse init (API version 7.21)
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounted /firewall-scripts.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Remote File Systems.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Reached target Remote File Systems.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounting FUSE Control File System...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounted FUSE Control File System.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Permit User Sessions...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Permit User Sessions.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Command Scheduler...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Command Scheduler.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Job spooling tools...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Job spooling tools.
Jul 15 15:18:09 chicago-fw1 libvirtd[1001]: libvirt version: 1.0.5.2, package: 1.fc19 (Fedora Project, 2013-06-12-22:00:57, buil
dvm-12.phx2.fedoraproject.org)
Jul 15 15:18:09 chicago-fw1 libvirtd[1001]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_xen.so not accessible
Jul 15 15:18:09 chicago-fw1 libvirtd[1001]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_libxl.so not accessible
Jul 15 15:18:09 chicago-fw1 libvirtd[1001]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not accessible
Jul 15 15:18:09 chicago-fw1 libvirtd[1001]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_uml.so not accessible
Jul 15 15:18:09 chicago-fw1 avahi-daemon[445]: Registering new address record for fe80::230:18ff:fea2:a340 on enp5s7.*.
Jul 15 15:18:10 chicago-fw1 libvirtd[1001]: open("/var/run/libvirt/network/nwfilter.ltmp"): No such file or directory
Jul 15 15:18:10 chicago-fw1 dbus[457]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Jul 15 15:18:10 chicago-fw1 dbus-daemon[457]: dbus[457]: [system] Successfully activated service 'org.fedoraproject.Setroublesho
otd'
Jul 15 15:18:10 chicago-fw1 kernel: [ 27.494142] Ebtables v2.0 registered
Jul 15 15:18:11 chicago-fw1 kernel: [ 27.575907] ip6_tables: (C) 2000-2006 Netfilter Core Team
Jul 15 15:18:13 chicago-fw1 setroubleshoot: SELinux is preventing /usr/sbin/glusterfsd from name_bind access on the tcp_socket .
For complete SELinux messages. run sealert -l 7a4fcd5d-209a-4206-ad06-93e6de5e4327
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:14 chicago-fw1 audispd: queue is full - dropping event
.
. skippped a zillion of these
.
Jul 15 15:18:15 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:15 chicago-fw1 audispd: queue is full - dropping event
Jul 15 15:18:17 chicago-fw1 systemd[1]: Started GlusterFS an clustered file-system server.
Jul 15 15:18:17 chicago-fw1 systemd[1]: Starting GlusterFS an clustered file-system server...
Jul 15 15:18:17 chicago-fw1 glusterfsd[1264]: [2013-07-15 20:18:17.309107] C [glusterfsd.c:1374:parse_cmdline] 0-glusterfs: ERRO
R: parsing the volfile failed (No such file or directory)
Jul 15 15:18:17 chicago-fw1 GlusterFS[1264]: [2013-07-15 20:18:17.309107] C [glusterfsd.c:1374:parse_cmdline] 0-glusterfs: ERROR
: parsing the volfile failed (No such file or directory)
Jul 15 15:18:17 chicago-fw1 glusterfsd[1264]: USAGE: /usr/sbin/glusterfsd [options] [mountpoint]
Jul 15 15:18:17 chicago-fw1 systemd[1]: glusterfsd.service: control process exited, code=exited status=255
Jul 15 15:18:17 chicago-fw1 systemd[1]: Failed to start GlusterFS an clustered file-system server.
Jul 15 15:18:17 chicago-fw1 systemd[1]: Unit glusterfsd.service entered failed state.
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: Mount failed. Please check the log file for more details.
Jul 15 15:18:20 chicago-fw1 mount[1002]: Mount failed. Please check the log file for more details.
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: / : ignored
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: /boot : already mounted
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: /boot/efi : already mounted
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: /gluster-fw1 : already mounted
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: swap : ignored
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: /firewall-scripts : successfully mounted
Jul 15 15:18:20 chicago-fw1 rc.local[1006]: Mounted after mount -av
Jul 15 15:18:20 chicago-fw1 systemd[1]: firewall\x2dscripts.mount mount process exited, code=exited status=1
Jul 15 15:18:20 chicago-fw1 sedispatch: AVC Message for setroubleshoot, dropping message
Jul 15 15:18:20 chicago-fw1 sedispatch: AVC Message for setroubleshoot, dropping message
Jul 15 15:18:20 chicago-fw1 sedispatch: AVC Message for setroubleshoot, dropping message
Jul 15 15:18:20 chicago-fw1 sedispatch: AVC Message for setroubleshoot, dropping message
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: df: â/firewall-scriptsâ: Transport endpoint is not connected
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: Filesystem Size Used Avail Use% Mounted on
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-root 14G 3.8G 8.7G 31% /
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: devtmpfs 990M 0 990M 0% /dev
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /dev/shm
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: tmpfs 996M 872K 996M 1% /run
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /sys/fs/cgroup
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /tmp
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: /dev/sda2 477M 87M 365M 20% /boot
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: /dev/sda1 200M 9.4M 191M 5% /boot/efi
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1
Jul 15 15:18:21 chicago-fw1 rc.local[1006]: Starting up firewall common items
Jul 15 15:18:21 chicago-fw1 systemd[1]: Started /etc/rc.d/rc.local Compatibility.
Jul 15 15:18:21 chicago-fw1 systemd[1]: Starting Terminate Plymouth Boot Screen...
Jul 15 15:18:21 chicago-fw1 systemd[1]: Starting Wait for Plymouth Boot Screen to Quit...
Jul 15 15:18:21 chicago-fw1 systemd[1]: Started Terminate Plymouth Boot Screen.
Jul 15 15:18:21 chicago-fw1 systemd[1]: Started Wait for Plymouth Boot Screen to Quit.
Jul 15 15:18:21 chicago-fw1 systemd[1]: Starting Getty on tty1...
Jul 15 15:18:21 chicago-fw1 systemd[1]: Started Getty on tty1.
-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Greg Scott
Sent: Monday, July 15, 2013 3:29 PM
To: 'Joe Julian'
Cc: gluster-users at gluster.org
Subject: Re: [Gluster-users] One node goes offline, the other node can't see the replicated volume anymore
Re: Joe
> I see the glusterfsd.service, but not the glusterd.service. Try:
>
> systemctl disable glusterfsd.service
> systemctl enable glusterd.service
Tried this on both nodes and rebooted. Life in the Twilight Zone. First fw1 immediately after logging back in:
[root at chicago-fw1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora-root 14G 3.8G 8.7G 31% /
devtmpfs 990M 0 990M 0% /dev
tmpfs 996M 0 996M 0% /dev/shm
tmpfs 996M 892K 996M 1% /run
tmpfs 996M 0 996M 0% /sys/fs/cgroup
tmpfs 996M 0 996M 0% /tmp
/dev/sda2 477M 87M 365M 20% /boot
/dev/sda1 200M 9.4M 191M 5% /boot/efi
/dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1
192.168.253.1:/firewall-scripts 7.6G 19M 7.2G 1% /firewall-scripts
[root at chicago-fw1 ~]#
[root at chicago-fw1 ~]# ls /firewall-scripts
allow-all failover-monitor.sh lost+found route-monitor.sh
allow-all-with-nat fwdate.txt rc.firewall start-failover-monitor.sh
etc initial_rc.firewall rcfirewall.conf var
[root at chicago-fw1 ~]#
But it's not mounted on fw2.
[root at chicago-fw2 rc.d]# reboot
login as: root
root at 10.10.10.72's password:
Last login: Mon Jul 15 13:53:40 2013 from tinahp100b.infrasupport.local
[root at chicago-fw2 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora-root 14G 4.1G 8.4G 33% /
devtmpfs 990M 0 990M 0% /dev
tmpfs 996M 0 996M 0% /dev/shm
tmpfs 996M 892K 996M 1% /run
tmpfs 996M 0 996M 0% /sys/fs/cgroup
tmpfs 996M 0 996M 0% /tmp
/dev/sda2 477M 90M 362M 20% /boot
/dev/sda1 200M 9.4M 191M 5% /boot/efi
/dev/mapper/fedora-gluster--fw2 7.6G 19M 7.2G 1% /gluster-fw2
[root at chicago-fw2 ~]#
Here is an extract from /var/log/messages on fw2.
.
.
.
Jul 15 15:18:26 chicago-fw2 audispd: queue is full - dropping event Jul 15 15:18:26 chicago-fw2 audispd: queue is full - dropping event Jul 15 15:18:28 chicago-fw2 systemd[1]: Started GlusterFS an clustered file-system server.
Jul 15 15:18:28 chicago-fw2 systemd[1]: Starting GlusterFS an clustered file-system server...
Jul 15 15:18:28 chicago-fw2 glusterfsd[1220]: [2013-07-15 20:18:28.304028] C [glusterfsd.c:1374:parse_cmdline] 0-glu
sterfs: ERROR: parsing the volfile failed (No such file or directory) Jul 15 15:18:28 chicago-fw2 glusterfsd[1220]: USAGE: /usr/sbin/glusterfsd [options] [mountpoint] Jul 15 15:18:28 chicago-fw2 GlusterFS[1220]: [2013-07-15 20:18:28.304028] C [glusterfsd.c:1374:parse_cmdline] 0-glus
terfs: ERROR: parsing the volfile failed (No such file or directory) Jul 15 15:18:28 chicago-fw2 systemd[1]: glusterfsd.service: control process exited, code=exited status=255 Jul 15 15:18:28 chicago-fw2 systemd[1]: Failed to start GlusterFS an clustered file-system server.
Jul 15 15:18:28 chicago-fw2 systemd[1]: Unit glusterfsd.service entered failed state.
Jul 15 15:18:28 chicago-fw2 mount[997]: Mount failed. Please check the log file for more details.
Jul 15 15:18:28 chicago-fw2 rpc.statd[1258]: Version 1.2.7 starting Jul 15 15:18:28 chicago-fw2 rc.local[1001]: Mount failed. Please check the log file for more details.
Jul 15 15:18:28 chicago-fw2 systemd[1]: firewall\x2dscripts.mount mount process exited, code=exited status=1 Jul 15 15:18:28 chicago-fw2 systemd[1]: Unit firewall\x2dscripts.mount entered failed state.
Jul 15 15:18:28 chicago-fw2 sm-notify[1259]: Version 1.2.7 starting
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: / : ignored
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /boot : already mounted
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /boot/efi : already mounted
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /gluster-fw2 : already mounted
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: swap : ignored
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /firewall-scripts : successfully mounted
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: Mounted after mount -av
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: Filesystem Size Used Avail Use% Mounted on
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /dev/mapper/fedora-root 14G 4.1G 8.4G 33% /
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: devtmpfs 990M 0 990M 0% /dev
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: tmpfs 996M 0 996M 0% /dev/shm
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: tmpfs 996M 880K 996M 1% /run
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: tmpfs 996M 0 996M 0% /sys/fs/cgroup
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: tmpfs 996M 4.0K 996M 1% /tmp
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /dev/sda2 477M 90M 362M 20% /boot
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /dev/sda1 200M 9.4M 191M 5% /boot/efi
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: /dev/mapper/fedora-gluster--fw2 7.6G 19M 7.2G 1% /gluster-fw2
Jul 15 15:18:28 chicago-fw2 rc.local[1001]: Starting up firewall common items Jul 15 15:18:28 chicago-fw2 systemd[1]: Started /etc/rc.d/rc.local Compatibility.
Jul 15 15:18:28 chicago-fw2 systemd[1]: Starting Terminate Plymouth Boot Screen...
Jul 15 15:18:28 chicago-fw2 systemd[1]: Starting Wait for Plymouth Boot Screen to Quit...
Jul 15 15:18:28 chicago-fw2 systemd[1]: Started Terminate Plymouth Boot Screen.
Jul 15 15:18:28 chicago-fw2 systemd[1]: Started Wait for Plymouth Boot Screen to Quit.
.
.
.
And the extract from /var/log/messages from fw1
.
.
.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting OpenSSH server daemon...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting /etc/rc.d/rc.local Compatibility...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started Vsftpd ftp daemon.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Started RPC bind service.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting GlusterFS an clustered file-system server...
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Making sure the Gluster stuff is mounted Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Mounted before mount -av Jul 15 15:18:07 chicago-fw1 systemd[1]: Started OpenSSH server daemon.
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: Filesystem Size Used Avail Use% Mounted on
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-root 14G 3.8G 8.7G 31% /
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: devtmpfs 990M 0 990M 0% /dev
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /dev/shm
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 2.1M 994M 1% /run
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /sys/fs/cgroup
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: tmpfs 996M 0 996M 0% /tmp
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/sda2 477M 87M 365M 20% /boot
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/sda1 200M 9.4M 191M 5% /boot/efi
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: /dev/mapper/fedora-gluster--fw1 7.9G 33M 7.8G 1% /gluster-fw1
Jul 15 15:18:07 chicago-fw1 rc.local[1006]: extra arguments at end (ignored) Jul 15 15:18:07 chicago-fw1 dbus-daemon[457]: dbus[457]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (u sing servicehelper) Jul 15 15:18:07 chicago-fw1 dbus[457]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper
)
Jul 15 15:18:07 chicago-fw1 kernel: [ 24.022605] fuse init (API version 7.21)
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounted /firewall-scripts.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Remote File Systems.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Reached target Remote File Systems.
Jul 15 15:18:07 chicago-fw1 systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounting FUSE Control File System...
Jul 15 15:18:07 chicago-fw1 systemd[1]: Mounted FUSE Control File System.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Permit User Sessions...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Permit User Sessions.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Command Scheduler...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Command Scheduler.
Jul 15 15:18:09 chicago-fw1 systemd[1]: Starting Job spooling tools...
Jul 15 15:18:09 chicago-fw1 systemd[1]: Started Job spooling tools.
.
.
.
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list