[Gluster-users] Error between v3.7.8 and v3.7.0

Sat Feb 20 17:48:41 UTC 2016

I do not see any mount related failures in the glusterd log you have
pasted. Ideally if mount request fails it could be either the GlusterD is
down or the brick processes are down. There'd be an error log entry from
mgmt_getspec().

The log entries do indicate that the n/w is unstable. If you are still
stuck could you provide the mount log and glusterd log please along with
gluster volume info output and mount command semantics?

-Atin
Sent from one plus one
On 20-Feb-2016 4:21 pm, "Ml Ml" <mliebherr99 at googlemail.com> wrote:

> Hello List,
>
> i am running ovirt (CentOS) on top of glusterfs. I have a 3 Node
> replica. Versions see below.
>
> Looks like i can not get my node1 (v 3.7.8) together with the othet
> two (v3.7.0). The error i get when i try to " mount -t glusterfs
> 10.10.3.7:/RaidVolC /mnt/":
>
> [2016-02-20 10:27:30.890701] W [socket.c:869:__socket_keepalive]
> 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid
> argument
> [2016-02-20 10:27:30.890728] E [socket.c:2965:socket_connect]
> 0-management: Failed to set keep-alive: Invalid argument
> [2016-02-20 10:27:30.891296] W [socket.c:588:__socket_rwv]
> 0-management: readv on 10.10.3.7:24007 failed (No data available)
> [2016-02-20 10:27:30.891671] E [rpc-clnt.c:362:saved_frames_unwind]
> (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7ff82c50bab2]
> (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7ff82c2d68de]
> (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff82c2d69ee]
> (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7a)[0x7ff82c2d837a]
> (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x88)[0x7ff82c2d8ba8] )))))
> 0-management: forced unwinding frame type(GLUSTERD-DUMP) op(DUMP(1))
> called at 2016-02-20 10:27:30.891063 (xid=0x35)
> The message "W [MSGID: 106118]
> [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management:
> Lock not released for RaidVolC" repeated 3 times between [2016-02-20
> 10:27:24.873207] and [2016-02-20 10:27:27.886916]
> [2016-02-20 10:27:30.891704] E [MSGID: 106167]
> [glusterd-handshake.c:2074:__glusterd_peer_dump_version_cbk]
> 0-management: Error through RPC layer, retry again later
> [2016-02-20 10:27:30.891871] W
> [glusterd-locks.c:681:glusterd_mgmt_v3_unlock]
>
> (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)
> [0x7ff821062b9c]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)
> [0x7ff82106ce72]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a)
> [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolB not held
> [2016-02-20 10:27:30.892001] W
> [glusterd-locks.c:681:glusterd_mgmt_v3_unlock]
>
> (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)
> [0x7ff821062b9c]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)
> [0x7ff82106ce72]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a)
> [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolC not held
> The message "W [MSGID: 106118]
> [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management:
> Lock not released for RaidVolB" repeated 3 times between [2016-02-20
> 10:27:24.877923] and [2016-02-20 10:27:30.891888]
> [2016-02-20 10:27:30.892023] W [MSGID: 106118]
> [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management:
> Lock not released for RaidVolC
> [2016-02-20 10:27:30.895617] W [socket.c:869:__socket_keepalive]
> 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid
> argument
> [2016-02-20 10:27:30.895641] E [socket.c:2965:socket_connect]
> 0-management: Failed to set keep-alive: Invalid argument
> [2016-02-20 10:27:30.896300] W [socket.c:588:__socket_rwv]
> 0-management: readv on 10.10.1.6:24007 failed (No data available)
> [2016-02-20 10:27:30.896541] E [rpc-clnt.c:362:saved_frames_unwind]
> (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7ff82c50bab2]
> (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7ff82c2d68de]
> (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff82c2d69ee]
> (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7a)[0x7ff82c2d837a]
> (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x88)[0x7ff82c2d8ba8] )))))
> 0-management: forced unwinding frame type(GLUSTERD-DUMP) op(DUMP(1))
> called at 2016-02-20 10:27:30.895995 (xid=0x35)
> [2016-02-20 10:27:30.896703] W
> [glusterd-locks.c:681:glusterd_mgmt_v3_unlock]
>
> (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)
> [0x7ff821062b9c]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)
> [0x7ff82106ce72]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a)
> [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolB not held
> [2016-02-20 10:27:30.896584] I [MSGID: 106004]
> [glusterd-handler.c:5127:__glusterd_peer_rpc_notify] 0-management:
> Peer <ovirt-node06-stgt.stuttgart.imos.net>
> (<08884518-2db7-4429-ab2f-019d03a02b76>), in state <Peer in Cluster>,
> has disconnected from glusterd.
> [2016-02-20 10:27:30.896720] W [MSGID: 106118]
> [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management:
> Lock not released for RaidVolB
> [2016-02-20 10:27:30.896854] W
> [glusterd-locks.c:681:glusterd_mgmt_v3_unlock]
>
> (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c)
> [0x7ff821062b9c]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162)
> [0x7ff82106ce72]
>
> -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a)
> [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolC not held
>
>
> Any idea what the problem is? I had a network problem which is sloved now.
> But now i am stuck with this.
>
>
> Node1:
> ============
> rpm -qa |grep gluster
> glusterfs-fuse-3.7.8-1.el7.x86_64
> glusterfs-3.7.8-1.el7.x86_64
> glusterfs-cli-3.7.8-1.el7.x86_64
> glusterfs-client-xlators-3.7.8-1.el7.x86_64
> glusterfs-rdma-3.7.8-1.el7.x86_64
> vdsm-gluster-4.16.30-0.el7.centos.noarch
> glusterfs-api-3.7.8-1.el7.x86_64
> glusterfs-libs-3.7.8-1.el7.x86_64
> glusterfs-server-3.7.8-1.el7.x86_64
>
>
> Node2:
> ==============
>  rpm -qa |grep gluster
> glusterfs-fuse-3.7.0-1.el7.x86_64
> glusterfs-libs-3.7.0-1.el7.x86_64
> glusterfs-api-3.7.0-1.el7.x86_64
> glusterfs-cli-3.7.0-1.el7.x86_64
> glusterfs-server-3.7.0-1.el7.x86_64
> glusterfs-3.7.0-1.el7.x86_64
> glusterfs-rdma-3.7.0-1.el7.x86_64
> vdsm-gluster-4.16.14-0.el7.noarch
> glusterfs-client-xlators-3.7.0-1.el7.x86_64
>
>
> Node3:
> =================
> rpm -qa|grep glus
> glusterfs-3.7.0-1.el7.x86_64
> glusterfs-rdma-3.7.0-1.el7.x86_64
> glusterfs-client-xlators-3.7.0-1.el7.x86_64
> glusterfs-libs-3.7.0-1.el7.x86_64
> glusterfs-api-3.7.0-1.el7.x86_64
> glusterfs-cli-3.7.0-1.el7.x86_64
> glusterfs-server-3.7.0-1.el7.x86_64
> vdsm-gluster-4.16.14-0.el7.noarch
> glusterfs-fuse-3.7.0-1.el7.x86_64
>
>
> Thanks,
> Mario
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160220/32cae6ba/attachment.html>