<div dir="ltr"><div>Hi Vijay</div><div><br></div><div>Now it is unmounting every 30 mins ! <br></div><div><br></div><div>The server log at /var/log/glusterfs/bricks/glusteratlas-brics001-gv0.log have this line only</div><div><br></div><div>2018-06-12 09:53:19.303102] I [MSGID: 115013] [server-helpers.c:289:do_fd_cleanup] 0-atlasglust-server: fd cleanup on /atlas/atlasdata/zgubic/hmumu/histograms/v14.3/Signal<br>[2018-06-12 09:53:19.306190] I [MSGID: 101055] [client_t.c:443:gf_client_unref] 0-atlasglust-server: Shutting down connection <server-name> -2224879-2018/06/12-09:51:01:460889-atlasglust-client-0-0-0</div><div><br></div><div>There is no other information. Is there any way to increase log verbosity?</div><div><br></div><div>on the client <br></div><div><br></div><div>2018-06-12 09:51:01.744980] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-atlasglust-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br>[2018-06-12 09:51:01.746508] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-atlasglust-client-5: Connected to atlasglust-client-5, attached to remote volume '/glusteratlas/brick006/gv0'.<br>[2018-06-12 09:51:01.746543] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-atlasglust-client-5: Server and Client lk-version numbers are not same, reopening the fds<br>[2018-06-12 09:51:01.746814] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-atlasglust-client-5: Server lk version = 1<br>[2018-06-12 09:51:01.748449] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-atlasglust-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br>[2018-06-12 09:51:01.750219] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-atlasglust-client-6: Connected to atlasglust-client-6, attached to remote volume '/glusteratlas/brick007/gv0'.<br>[2018-06-12 09:51:01.750261] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-atlasglust-client-6: Server and Client lk-version numbers are not same, reopening the fds<br>[2018-06-12 09:51:01.750503] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-atlasglust-client-6: Server lk version = 1<br>[2018-06-12 09:51:01.752207] I [fuse-bridge.c:4205:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.14<br>[2018-06-12 09:51:01.752261] I [fuse-bridge.c:4835:fuse_graph_sync] 0-fuse: switched to graph 0<br></div><div><br></div><div><br></div><div>is there a problem with server and client 1k version?</div><div><br></div><div>Thanks for your help.</div><div><br></div><div>Kashif<br></div><div><br></div><div><br></div><div><br></div><div> </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 11, 2018 at 11:52 PM, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Mon, Jun 11, 2018 at 8:50 AM, mohammad kashif <span dir="ltr"><<a href="mailto:kashif.alig@gmail.com" target="_blank">kashif.alig@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi</div><div><br></div><div>Since I have updated our gluster server and client to latest version 3.12.9-1, I am having this issue of gluster getting unmounted from client very regularly. It was not a problem before update.<br></div><div><br></div><div>Its a distributed file system with no replication. We have seven servers totaling around 480TB data. Its 97% full. <br></div><div><br></div><div>I am using following config on server</div><div><br></div><div><br></div>gluster volume set atlasglust features.cache-invalidation on<br>gluster volume set atlasglust features.cache-invalidation-ti<wbr>meout 600<br>gluster volume set atlasglust performance.stat-prefetch on<br>gluster volume set atlasglust performance.cache-invalidation on<br>gluster volume set atlasglust performance.md-cache-timeout 600<br>gluster volume set atlasglust performance.parallel-readdir on<br>gluster volume set atlasglust performance.cache-size 1GB<br>gluster volume set atlasglust performance.client-io-threads on<br>gluster volume set atlasglust cluster.lookup-optimize on<br>gluster volume set atlasglust performance.stat-prefetch on<br>gluster volume set atlasglust client.event-threads 4<br>gluster volume set atlasglust server.event-threads 4<br><div><br></div><div>clients are mounted with this option</div><div><br></div><div>defaults,direct-io-mode=disabl<wbr>e,attribute-timeout=600,entry-<wbr>timeout=600,negative-timeout=<wbr>600,fopen-keep-cache,rw,_<wbr>netdev <br></div><div><br></div><div>I can't see anything in the log file. Can someone suggest that how to troubleshoot this issue?</div><div><br></div><div><br></div></div></blockquote><div><br></div><div><br></div></span><div>Can you please share the log file? Checking for messages related to disconnections/crashes in the log file would be a good way to start troubleshooting the problem.</div><div><br></div><div>Thanks,</div><div>Vijay </div></div></div></div>
</blockquote></div><br></div>