[Gluster-users] Gluster volume not automounted when peer is down

Wed Nov 26 00:59:27 UTC 2014

Hi,

This is interesting, I think this is the same problem as mine.

Quoting myself from 1st message:
If I shutdown one of the servers (192.168.2.10), and I reboot the
remaining one (192.168.0.10), it won't come up as fast as it should. It
lags a few minutes waiting for gluster. After it eventually starts,
mount point is not mounted and volume is stopped:
[...]
This didn't happen before, so fine, I first have to stop the volume and
then start it again. It now shows as online:
Brick 192.168.0.10:/var/aloha                           49155 Y       3473
Self-heal Daemon on localhost                           N/A Y       3507

# time mount -a
real    2m7.307s

# time mount -t glusterfs 192.168.0.10:/aloha /var/www/hawaii
real    2m7.365s
------------------------------------------------------------------------
Quoting myself from 2nd message:
Steps to reproduce:
1. probe server2, create and start volume
2. do not mount volume
3. reboot/poweroff server2; or add server1 to its iptables (with -j
DROP, not -j REJECT)
4. on server1 (while server2 is rebooting or dropping traffic from
server1): time mount -t glusterfs server1:/volume /some/path

PS: with -j REJECT it mounts instantly. with -j DROP it always waits
2mins 7secs
------------------------------------------------------------------------

Links at
http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019415.html
http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019488.html

Can you also take a look and confirm it is the same? Also, while keeping one node down, please try that "time mount" command.

On Nov 24, 2014 8:07 PM, Eric Ewanco wrote:

Hi all,

We're trying to use gluster as a replicated volume.  It works OK when both peers are up but when one peer is down and the other reboots, the "surviving" peer does not automount glusterfs.  Furthermore, after the boot sequence is complete, it can be mounted without issue.  It automounts fine when the peer is up during startup.  I tried to google this and while I found some similar issues, I haven't found any solutions to my problem.  Any insight would be appreciated.  Thanks.

gluster volume info output (after startup):
Volume Name: rel-vol
Type: Replicate
Volume ID: 90cbe313-e9f9-42d9-a947-802315ab72b0
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.250.1.1:/export/brick1
Brick2: 10.250.1.2:/export/brick1

gluster peer status output (after startup):
Number of Peers: 1

Hostname: 10.250.1.2
Uuid: 8d49b929-4660-4b1e-821b-bfcd6291f516
State: Peer in Cluster (Disconnected)

Original volume create command:
gluster volume create rel-vol rep 2 transport tcp 10.250.1.1:/export/brick1 10.250.1.2:/export/brick1

I am running Gluster 3.4.5 on OpenSuSE 12.2.
gluster --version:
glusterfs 3.4.5 built on Jul 25 2014 08:31:19
Repository revision:git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.

The fstab line is:

localhost:/rel-vol              /home        glusterfs  defaults,_netdev      0 0

lsof -i :24007-24100:
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
glusterd  4073 root    6u  IPv4  82170      0t0  TCP s1:24007->s1:1023 (ESTABLISHED)
glusterd  4073 root    9u  IPv4  13816      0t0  TCP *:24007 (LISTEN)
glusterd  4073 root   10u  IPv4  88106      0t0  TCP s1:exp2->s2:24007 (SYN_SENT)
glusterfs 4097 root    8u  IPv4  16751      0t0  TCP s1:1023->s1:24007 (ESTABLISHED)

This is shorter than it is when it works, but maybe that's because the mount spawns some more processes.

Some ports are down:

root at q50-s1  <http://supercolony.gluster.org/mailman/listinfo/gluster-users>:/root> telnet localhost 24007

Trying ::1...

telnet: connect to address ::1: Connection refused

Trying 127.0.0.1...

Connected to localhost.

Escape character is '^]'.

telnet> close

Connection closed.

root at q50-s1  <http://supercolony.gluster.org/mailman/listinfo/gluster-users>:/root> telnet localhost 24009

Trying ::1...

telnet: connect to address ::1: Connection refused

Trying 127.0.0.1...

telnet: connect to address 127.0.0.1: Connection refused

ps axww | fgrep glu:

4073 ?        Ssl    0:10 /usr/sbin/glusterd -p /run/glusterd.pid

4097 ?        Ssl    0:00 /usr/sbin/glusterfsd -s 10.250.1.1 --volfile-id rel-vol.10.250.1.1.export-brick1 -p /var/lib/glusterd/vols/rel-vol/run/10.250.1.1-export-brick1.pid -S /var/run/89ba432ed09e07e107723b4b266e18f9.socket --brick-name /export/brick1 -l /var/log/glusterfs/bricks/export-brick1.log --xlator-option *-posix.glusterd-uuid=3b02a581-8fb9-4c6a-8323-9463262f23bc --brick-port 49152 --xlator-option rel-vol-server.listen-port=49152

5949 ttyS0    S+     0:00 fgrep glu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141126/cb1935f9/attachment.html>