<div dir="ltr"><font face="arial, sans-serif">That's unfortunately not possible in my case as the clients are K8s worker nodes. The gluster client version is tied to the Kuberntes release/deployer - Rancher in my case.</font><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">Disabling ctime "<span style="color:rgb(216,214,208);font-size:11.9px;background-color:rgba(220,224,228,0.1)">gluster volume set $volname ctime off" </span><span style="color:rgb(216,214,208);font-size:11.9px;background-color:rgba(220,224,228,0.1)">fixed the mounting issue.</span></font><br><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 30, 2020 at 4:16 AM Mahdi Adnan <<a href="mailto:mahdi@sysmin.io">mahdi@sysmin.io</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div> We had a similar issue when we upgraded one of our clusters to 6.5 and clients were running 4.1.5 and 4.1.9, both crashed after few seconds of mounting, we did not dig into the issue instead, we upgraded the clients to 6.5 and it worked fine.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 28, 2020 at 1:35 AM Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com" target="_blank">laurentfdumont@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">After some more digging, it seems I'm hitting the following bug - <a href="https://github.com/gluster/glusterfs/issues/658" target="_blank">https://github.com/gluster/glusterfs/issues/658</a></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 27, 2020 at 4:23 PM Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com" target="_blank">laurentfdumont@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi everyone,<div><br></div><div>Small question. I'm trying to mount a Gluster volume (server is at 6.7) and the client is at 4.1.5. I'm seeing the mount start on the client but it looks like the client crashes and is left in a strange state. Is there any inherent compatibility issues between the two versions? Is 4.1.5 too old when talking to a 6.7 server?</div><div><br></div><div>Client : </div><div>sh-4.4# glusterfs --version<br>glusterfs 4.1.5<br>Repository revision: git://<a href="http://git.gluster.org/glusterfs.git" target="_blank">git.gluster.org/glusterfs.git</a><br>Copyright (c) 2006-2016 Red Hat, Inc. <<a href="https://www.gluster.org/" target="_blank">https://www.gluster.org/</a>><br>GlusterFS comes with ABSOLUTELY NO WARRANTY.<br>It is licensed to you under your choice of the GNU Lesser<br>General Public License, version 3 or any later version (LGPLv3<br>or later), or the GNU General Public License, version 2 (GPLv2),<br>in all cases as published by the Free Software Foundation.<br></div><div><br></div><div>Server :</div><div>root@gluster01:~# glusterd --version<br>glusterfs 6.7<br>Repository revision: git://<a href="http://git.gluster.org/glusterfs.git" target="_blank">git.gluster.org/glusterfs.git</a><br>Copyright (c) 2006-2016 Red Hat, Inc. <<a href="https://www.gluster.org/" target="_blank">https://www.gluster.org/</a>><br>GlusterFS comes with ABSOLUTELY NO WARRANTY.<br>It is licensed to you under your choice of the GNU Lesser<br>General Public License, version 3 or any later version (LGPLv3<br>or later), or the GNU General Public License, version 2 (GPLv2),<br>in all cases as published by the Free Software Foundation.<br></div><div><br></div><div>Steps :</div><div>sh-4.4# mount -t glusterfs 10.10.99.29:kube_vol /test<br>Mount failed. Please check the log file for more details.<br></div><div><br></div><div><br></div><div>sh-4.4# cat /var/log/glusterfs/test.log <br>[2020-01-27 21:03:26.770751] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 4.1.5 (args: /usr/sbin/glusterfs --process-name fuse --volfile-server=10.10.99.29 --volfile-id=kube_vol /test)<br>[2020-01-27 21:03:26.775717] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1<br>pending frames:<br>frame : type(0) op(0)<br>patchset: git://<a href="http://git.gluster.org/glusterfs.git" target="_blank">git.gluster.org/glusterfs.git</a><br>signal received: 11<br>time of crash: <br>2020-01-27 21:03:26<br>configuration details:<br>argp 1<br>backtrace 1<br>dlfcn 1<br>libpthread 1<br>llistxattr 1<br>setfsid 1<br>spinlock 1<br>epoll.h 1<br>xattr.h 1<br>st_atim.tv_nsec 1<br>package-string: glusterfs 4.1.5<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x237aa)[0x7f2e1061d7aa]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_print_trace+0x2f7)[0x7f2e10627517]<br>/lib/x86_64-linux-gnu/libc.so.6(+0x33060)[0x7f2e0ec6f060]<br>/lib/x86_64-linux-gnu/libc.so.6(+0xbf944)[0x7f2e0ecfb944]<br>/lib/x86_64-linux-gnu/libc.so.6(fnmatch+0x61)[0x7f2e0ecfce41]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_volume_option_get_list+0x35)[0x7f2e10670905]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x7698e)[0x7f2e1067098e]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_foreach_match+0x87)[0x7f2e10614b97]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_foreach+0x18)[0x7f2e10614d78]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_options_validate_list+0x3f)[0x7f2e10670b4f]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_options_validate+0x39)[0x7f2e10670bc9]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x5ab99)[0x7f2e10654b99]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_activate+0x24)[0x7f2e106554d4]<br>/usr/sbin/glusterfs(glusterfs_process_volfp+0x100)[0x5643b3bc1f40]<br>/usr/sbin/glusterfs(mgmt_getspec_cbk+0x6bd)[0x5643b3bc889d]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(+0xe110)[0x7f2e103ec110]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(+0xe44f)[0x7f2e103ec44f]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7f2e103e8863]<br>/usr/lib/x86_64-linux-gnu/glusterfs/4.1.5/rpc-transport/socket.so(+0x6cfb)[0x7f2e0b4e9cfb]<br>/usr/lib/x86_64-linux-gnu/glusterfs/4.1.5/rpc-transport/socket.so(+0x9605)[0x7f2e0b4ec605]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x8052e)[0x7f2e1067a52e]<br>/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7f2e0f47c4a4]<br>/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f2e0ed24d0f]<br>---------<br>[2020-01-27 21:22:35.074154] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 4.1.5 (args: /usr/sbin/glusterfs --process-name fuse --volfile-server=10.10.99.29 --volfile-id=kube_vol /test)<br>[2020-01-27 21:22:35.078542] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1<br>pending frames:<br>frame : type(0) op(0)<br>patchset: git://<a href="http://git.gluster.org/glusterfs.git" target="_blank">git.gluster.org/glusterfs.git</a><br>signal received: 11<br>time of crash: <br>2020-01-27 21:22:35<br>configuration details:<br>argp 1<br>backtrace 1<br>dlfcn 1<br>libpthread 1<br>llistxattr 1<br>setfsid 1<br>spinlock 1<br>epoll.h 1<br>xattr.h 1<br>st_atim.tv_nsec 1<br>package-string: glusterfs 4.1.5<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x237aa)[0x7f6ef309a7aa]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_print_trace+0x2f7)[0x7f6ef30a4517]<br>/lib/x86_64-linux-gnu/libc.so.6(+0x33060)[0x7f6ef16ec060]<br>/lib/x86_64-linux-gnu/libc.so.6(+0xbf944)[0x7f6ef1778944]<br>/lib/x86_64-linux-gnu/libc.so.6(fnmatch+0x61)[0x7f6ef1779e41]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_volume_option_get_list+0x35)[0x7f6ef30ed905]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x7698e)[0x7f6ef30ed98e]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_foreach_match+0x87)[0x7f6ef3091b97]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_foreach+0x18)[0x7f6ef3091d78]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_options_validate_list+0x3f)[0x7f6ef30edb4f]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(xlator_options_validate+0x39)[0x7f6ef30edbc9]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x5ab99)[0x7f6ef30d1b99]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_activate+0x24)[0x7f6ef30d24d4]<br>/usr/sbin/glusterfs(glusterfs_process_volfp+0x100)[0x55b5e83a7f40]<br>/usr/sbin/glusterfs(mgmt_getspec_cbk+0x6bd)[0x55b5e83ae89d]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(+0xe110)[0x7f6ef2e69110]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(+0xe44f)[0x7f6ef2e6944f]<br>/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7f6ef2e65863]<br>/usr/lib/x86_64-linux-gnu/glusterfs/4.1.5/rpc-transport/socket.so(+0x6cfb)[0x7f6eedf66cfb]<br>/usr/lib/x86_64-linux-gnu/glusterfs/4.1.5/rpc-transport/socket.so(+0x9605)[0x7f6eedf69605]<br>/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x8052e)[0x7f6ef30f752e]<br>/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7f6ef1ef94a4]<br>/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f6ef17a1d0f]<br>---------<br></div></div>
</blockquote></div>
________<br>
<br>
Community Meeting Calendar:<br>
<br>
APAC Schedule -<br>
Every 2nd and 4th Tuesday at 11:30 AM IST<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
NA/EMEA Schedule -<br>
Every 1st and 3rd Tuesday at 01:00 PM EDT<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr">Respectfully<div>Mahdi</div></div></div>
</blockquote></div>