<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div><blockquote type="cite" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><span style="font-size: small;" class="">Clarification for below logs:</span></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><font size="2" class="">- 'dev_static' is the gluster volume.</font></div><div class=""><font size="2" class="">- 'int-kube-01' is the gluster client.</font></div><div class=""><font size="2" class="">- '10.51.70.151' is the first node in a three node (2 replica, 1 arbiter) gluster cluster.</font></div><div class=""><font size="2" class="">- '/var/lib/kubelet/...../iss3dev-static' is a directory on the client that should be mounting '10.51.70.151:/dev_static/iss3dev-static', where 'iss3dev-static' is a directory inside the gluster mount 'dev_static'.</font></div><div class=""><font size="2" class="">- FYI only, we are essentially mounting directories inside gluster volumes as per Kubernetes example: <a href="https://github.com/kubernetes/examples/blob/master/staging/volumes/glusterfs/README.md" class="">https://github.com/kubernetes/examples/blob/master/staging/volumes/glusterfs/README.md</a></font></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><div class=""><font face="Monaco" size="2" class="">root@int-kube-01:/var/log/glusterfs # tail -20 mnt.log</font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font face="Monaco" size="2" class="">[2017-11-07 21:15:03.561470] I [MSGID: 100030] [glusterfsd.c:2524:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.12.2 (args: /usr/sbin/glusterfs --volfile-server=10.51.70.151 --volfile-id=/dev_static /mnt)</font></div><div class=""><font face="Monaco" size="2" class="">[2017-11-07 21:15:03.571205] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-glusterfs: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction</font></div><div class=""><font face="Monaco" size="2" class="">[2017-11-07 21:15:03.584098] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1</font></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><font size="2" class="">No new log entries appear after executing the following to demonstrate the problem:</font></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><font size="2" class=""><br class=""></font></div><div class=""><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class=""> # touch test</font></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class=""> # mkdir testdir2</font></font></div></div><div class=""><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class=""> # ls -ltar</font></font></div><div class=""><font face="Monaco" size="2" class="">total 0</font></div><div class=""><font face="Monaco" size="2" class="">-rw-r--r--. 1 root root 0 Nov 6 10:10 test</font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class=""> # cd testdir2</font></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static/testdir2</span><font face="Monaco" class=""> # ls -la</font></font></div><div class=""><font face="Monaco" size="2" class="">total 0</font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class="">/testdir2 # cd ..</font></font></div><div class=""><font size="2" class=""><font face="Monaco" class="">root@int-kube-01:</font><span style="font-family: Monaco;" class="">/var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static</span><font face="Monaco" class=""># ls -ltar</font></font></div><div class=""><font face="Monaco" size="2" class="">total 0</font></div><div class=""><font face="Monaco" size="2" class="">-rw-r--r--. 1 root root 0 Nov 6 10:10 test</font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font size="2" class="">The mount entry looks like:</font></div><div class=""><font face="Monaco" size="2" class=""><br class=""></font></div><div class=""><font face="Monaco" size="2" class=""><div class="">root@int-kube-01:~ # mount | grep iss3dev-static</div><div class="">10.51.70.151:/dev_static on /var/lib/kubelet/pods/434cba8e-bf87-11e7-8389-1aa903709357/volumes/<a href="http://kubernetes.io/" class="">kubernetes.io</a>~glusterfs/iss3dev-static type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)</div></font></div></div></div></div>_______________________________________________<br class=""></blockquote></div><br class=""><div class=""><br class=""></div><div class="">Does anyone have any ideas with this?</div><div class=""><br class=""></div><div class="">It's essentially stopped us from being able to roll out Gluster as even the most simple mount using the standard gluster native client with default options means that clients can't see directories.</div></body></html>