[Gluster-users] The continuing story ...
mark at mark.mielke.cc
Sat Sep 19 16:22:39 UTC 2009
On 09/19/2009 05:39 AM, Anand Avati wrote:
>> [root at wcarh033]~# ps -ef | grep gluster
>> root 1548 1 0 21:00 ? 00:00:00
>> /opt/glusterfs/sbin/glusterfsd -f /etc/glusterfs/glusterfsd.vol
>> root 1861 1 0 21:00 ? 00:00:00
>> /opt/glusterfs/sbin/glusterfs --log-level=NORMAL
>> --volfile=/etc/glusterfs/tools.vol /gluster/tools
>> root 1874 1861 0 21:00 ? 00:00:00 /bin/mount -i -f -t
>> fuse.glusterfs -o rw,allow_other,default_permissions,max_read=131072
>> /etc/glusterfs/tools.vol /gluster/tools
>> root 2426 2395 0 21:02 pts/2 00:00:00 grep gluster
>> [root at wcarh033]~# ls /gluster/tools
>> Yep - all three nodes locked up. All it took was a simultaneous reboot
>> of all three machines.
>> After I kill -9 1874 (kill 1874 without -9 has no effect) from a
>> different ssh session, I get:
>> ls: cannot access /gluster/tools: Transport endpoint is not connected
>> After this, mount works (unmount not necessary it turns out).
>> I am unable to strace -p the mount -t fuse without it freezing up. I
>> pstack, but it returns 0 lines of output fairly quickly.
>> The symptoms are identical on all three machines. 3-way replication,
>> each server has both a server exposing one volume, and a client, with
>> cluster/replication and a preferred read of the local server.
> This is a strange hang. I have a few more questions -
> 1. is this off glusterfs.git master branch or release-2.0? If this is master, there have been heavy un-QA'ed modifications to get rid of libfuse dependency.
This is release-2.0 - 2.0.6 plus a few patches to pick up the other fix
you completed for my previous problem.
> 2. what happens if you try to start the three daemons together now when the system is not booting? Is this hang somehow related to the system booting?
> 3. can you provide dmesg output and glusterfs trace level logs of this scenario?
I will try these. Regular output is nothing for the glusterfs (client)
mounts. It shows the volumes and then no other lines. I'll see about
turning tracing on.
Mark Mielke<mark at mielke.cc>
More information about the Gluster-users