<html><head></head><body>Keep in mind a local disk is 3,6,12 Gbps but a network connection is typically 1Gbps. A local disk quad in raid 10 will outperform a 10G ethernet (especially using SAS drives). <br><br><div class="gmail_quote">On December 5, 2017 6:11:38 AM EST, Riccardo Murri <riccardo.murri@uzh.ch> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">Hello,<br /><br />I'm trying to set up a SAMBA server serving a GlusterFS volume.<br />Everything works fine if I locally mount the GlusterFS volume (`mount<br />-t glusterfs ...`) and then serve the mounted FS through SAMBA, but<br />the performance is slower by a 2x/3x compared to a SAMBA server with a<br />local ext4 filesystem.<br /><br />I gather that SAMBA vfs_glusterfs module can give better<br />performance. However, as soon as I switch SMBD to using GFAPI (`vfs<br />objects = glusterfs`), the SMBD process crashes leaving a backtrace in<br />the logs. I can see no errors in the associated GlusterFS log file<br />and everything runs fine when locally-mounting. I am copying below<br />the share definition, SMBD crash dump from the logs, and the GlusterFS<br />client logs (produced by the `vfs_<a href="http://glusterfs.so">glusterfs.so</a>` module in SMBD).<br /><br />I'm using Ubuntu 16.04.3 with samba package<br />2:4.3.11+dfsg-0ubuntu0.16.04.12 (I locally recompiled<br />`samba-vfs-modules` to enable the GlusterFS VFS which is by default<br />disabled in Ubuntu builds).<br /><br />Any suggestions on what/where to look?<br /><br />Alternatively: how to improve the performance of SAMBA over<br />locally-mounted GlusterFS? right now, I'm getting a 2x to 4x slowdown<br />compared to SAMBA with local disk.<br /><br />Thanks for any help!<br /><br />Riccardo<br /><br /><br />### share definition from /etc/samba/smb.conf<br /><br />[archive]<br /> browsable =yes<br /> writable = yes<br /> create mask = 0775<br /> directory mask = 0775<br /><br /> guest ok = yes<br /><br /> vfs objects = glusterfs<br /> kernel share modes = No<br /><br /> glusterfs:loglevel = 7<br /> glusterfs:logfile = /var/log/samba/glusterfs-vol-archive.log<br /> glusterfs:volfile_server = glusterfs-01<br /> glusterfs:volume = archive<br /> path = /<br /><br /><br />### SMBD crash from SAMBA logs<br /><br />Dec 5 10:20:13 samba-04 smbd[31932]: [2017/12/05 10:20:13.996199, 0] ../source3/modules/vfs_glusterfs.c:258(vfs_gluster_connect)<br />Dec 5 10:20:13 samba-04 smbd[31932]: archive: Initialized volume from server glusterfs-01<br />Dec 5 10:20:13 samba-04 smbd[31932]: [2017/12/05 10:20:13.999371, 0] ../lib/util/fault.c:78(fault_report)<br />Dec 5 10:20:13 samba-04 smbd[31932]:<hr /><br />Dec 5 10:20:14 samba-04 smbd[31932]: [2017/12/05 10:20:14.000042, 0] ../lib/util/fault.c:79(fault_report)<br />Dec 5 10:20:14 samba-04 smbd[31932]: INTERNAL ERROR: Signal 6 in pid 31932 (4.3.11-Ubuntu)<br />Dec 5 10:20:14 samba-04 smbd[31932]: Please read the Trouble-Shooting section of the Samba HOWTO<br />Dec 5 10:20:14 samba-04 smbd[31932]: [2017/12/05 10:20:14.000594, 0] ../lib/util/fault.c:81(fault_report)<br />Dec 5 10:20:14 samba-04 smbd[31932]:<hr /><br />Dec 5 10:20:14 samba-04 smbd[31932]: [2017/12/05 10:20:14.000954, 0] ../source3/lib/util.c:789(smb_panic_s3)<br />Dec 5 10:20:14 samba-04 smbd[31932]: PANIC (pid 31932): internal error<br />Dec 5 10:20:14 samba-04 smbd[31932]: [2017/12/05 10:20:14.001712, 0] ../source3/lib/util.c:900(log_stack_trace)<br />Dec 5 10:20:14 samba-04 smbd[31932]: BACKTRACE: 29 stack frames:<br />Dec 5 10:20:14 samba-04 smbd[31932]: #0 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbregistry.so">libsmbregistry.so</a>.0(log_stack_trace+0x1a) [0x7f5e8b5427aa]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #1 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbregistry.so">libsmbregistry.so</a>.0(smb_panic_s3+0x20) [0x7f5e8b542880]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #2 /usr/lib/x86_64-linux-gnu/<a href="http://libsamba-util.so">libsamba-util.so</a>.0(smb_panic+0x2f) [0x7f5e8c2b5f1f]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #3 /usr/lib/x86_64-linux-gnu/<a href="http://libsamba-util.so">libsamba-util.so</a>.0(+0x1b136) [0x7f5e8c2b6136]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #4 /lib/x86_64-linux-gnu/<a href="http://libpthread.so">libpthread.so</a>.0(+0x11390) [0x7f5e8c514390]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #5 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(gsignal+0x38) [0x7f5e88a87428]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #6 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(abort+0x16a) [0x7f5e88a8902a]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #7 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(+0x777ea) [0x7f5e88ac97ea]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #8 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(+0x8037a) [0x7f5e88ad237a]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #9 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(cfree+0x4c) [0x7f5e88ad653c]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #10 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(+0x117955) [0x7f5e8be88955]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #11 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(+0x1188b7) [0x7f5e8be898b7]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #12 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(make_connection_smb2+0x62) [0x7f5e8be8a662]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #13 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(smbd_smb2_request_process_tcon+0x6ad) [0x7f5e8be9e48d]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #14 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(smbd_smb2_request_dispatch+0x974) [0x7f5e8be98bd4]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #15 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(+0x1287fb) [0x7f5e8be997fb]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #16 /usr/lib/x86_64-linux-gnu/<a href="http://libsmbconf.so">libsmbconf.so</a>.0(run_events_poll+0x167) [0x7f5e8a1d5917]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #17 /usr/lib/x86_64-linux-gnu/<a href="http://libsmbconf.so">libsmbconf.so</a>.0(+0x2cb77) [0x7f5e8a1d5b77]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #18 /usr/lib/x86_64-linux-gnu/<a href="http://libtevent.so">libtevent.so</a>.0(_tevent_loop_once+0x8d) [0x7f5e88e1fd3d]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #19 /usr/lib/x86_64-linux-gnu/<a href="http://libtevent.so">libtevent.so</a>.0(tevent_common_loop_wait+0x1b) [0x7f5e88e1fedb]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #20 /usr/lib/x86_64-linux-gnu/samba/<a href="http://libsmbd-base.so">libsmbd-base.so</a>.0(smbd_process+0x718) [0x7f5e8be88578]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #21 /usr/sbin/smbd(+0x8e12) [0x55ba26c51e12]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #22 /usr/lib/x86_64-linux-gnu/<a href="http://libsmbconf.so">libsmbconf.so</a>.0(run_events_poll+0x167) [0x7f5e8a1d5917]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #23 /usr/lib/x86_64-linux-gnu/<a href="http://libsmbconf.so">libsmbconf.so</a>.0(+0x2cb77) [0x7f5e8a1d5b77]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #24 /usr/lib/x86_64-linux-gnu/<a href="http://libtevent.so">libtevent.so</a>.0(_tevent_loop_once+0x8d) [0x7f5e88e1fd3d]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #25 /usr/lib/x86_64-linux-gnu/<a href="http://libtevent.so">libtevent.so</a>.0(tevent_common_loop_wait+0x1b) [0x7f5e88e1fedb]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #26 /usr/sbin/smbd(main+0x1899) [0x55ba26c50099]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #27 /lib/x86_64-linux-gnu/<a href="http://libc.so">libc.so</a>.6(__libc_start_main+0xf0) [0x7f5e88a72830]<br />Dec 5 10:20:14 samba-04 smbd[31932]: #28 /usr/sbin/smbd(_start+0x29) [0x55ba26c50199]<br />Dec 5 10:20:14 samba-04 smbd[31932]: [2017/12/05 10:20:14.006804, 0] ../source3/lib/dumpcore.c:303(dump_core)<br />Dec 5 10:20:14 samba-04 smbd[31932]: dumping core in /var/log/samba/cores/smbd<br />Dec 5 10:20:14 samba-04 smbd[31932]:<br />Dec 5 10:20:14 samba-04 smbd[31944]: [2017/12/05 10:20:14.213971, 0] ../source3/smbd/smb2_server.c:547(smbd_smb2_request_create)<br />Dec 5 10:20:14 samba-04 smbd[31944]: Invalid SMB packet: first request: 0x0003<br /><br />### GlusterFS client logs (issued from the `vfs_<a href="http://gluster.so">gluster.so</a>` module within `smbd`)<br /><br />[2017-12-05 10:20:13.949861] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-gfapi: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction<br />[2017-12-05 10:20:13.965822] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-archive-disperse-0: Using 'avx' CPU extensions<br />[2017-12-05 10:20:13.969808] W [MSGID: 101174] [graph.c:363:_log_if_unknown_option] 0-archive-readdir-ahead: option 'parallel-readdir' is not recognized<br />[2017-12-05 10:20:13.970295] I [MSGID: 104045] [glfs-master.c:91:notify] 0-gfapi: New graph 70656c6b-6d61-6e73-6c61-622d73616d62 (0) coming up<br />[2017-12-05 10:20:13.970352] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-0: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.970718] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-1: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971011] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-2: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971205] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-3: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971401] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-4: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971576] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-5: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971770] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-6: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.971985] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-7: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.972254] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-8: parent translators are ready, attempting connect on transport<br />[2017-12-05 10:20:13.972495] I [MSGID: 114020] [client.c:2360:notify] 0-archive-client-9: parent translators are ready, attempting connect on transport<br />Final graph:<br />+<hr />+<br /> 1: volume archive-client-0<br /> 2: type protocol/client<br /> 3: option ping-timeout 42<br /> 4: option remote-host glusterfs-10<br /> 5: option remote-subvolume /srv/glusterfs/brick/archive<br /> 6: option transport-type socket<br /> 7: option transport.address-family inet<br /> 8: option transport.tcp-user-timeout 0<br /> 9: option transport.socket.keepalive-time 20<br /> 10: option transport.socket.keepalive-interval 2<br /> 11: option transport.socket.keepalive-count 9<br /> 12: option send-gids true<br /> 13: end-volume<br /> 14:<br /> 15: volume archive-client-1<br /> 16: type protocol/client<br /> 17: option ping-timeout 42<br /> 18: option remote-host glusterfs-09<br /> 19: option remote-subvolume /srv/glusterfs/brick/archive<br /> 20: option transport-type socket<br /> 21: option transport.address-family inet<br /> 22: option transport.tcp-user-timeout 0<br /> 23: option transport.socket.keepalive-time 20<br /> 24: option transport.socket.keepalive-interval 2<br /> 25: option transport.socket.keepalive-count 9<br /> 26: option send-gids true<br /> 27: end-volume<br /> 28:<br /> 28:<br /> 29: volume archive-client-2<br /> 30: type protocol/client<br /> 31: option ping-timeout 42<br />[2017-12-05 10:20:13.972846] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-0: changing port to 49153 (from 0)<br /> 32: option remote-host glusterfs-08<br /> 33: option remote-subvolume /srv/glusterfs/brick/archive<br /> 34: option transport-type socket<br /> 35: option transport.address-family inet<br /> 36: option transport.tcp-user-timeout 0<br /> 37: option transport.socket.keepalive-time 20<br /> 38: option transport.socket.keepalive-interval 2<br /> 39: option transport.socket.keepalive-count 9<br /> 40: option send-gids true<br /> 41: end-volume<br /> 42:<br /> 43: volume archive-client-3<br /> 44: type protocol/client<br /> 45: option ping-timeout 42<br /> 46: option remote-host glusterfs-07<br /> 47: option remote-subvolume /srv/glusterfs/brick/archive<br /> 48: option transport-type socket<br /> 49: option transport.address-family inet<br /> 50: option transport.tcp-user-timeout 0<br />[2017-12-05 10:20:13.972929] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-3: changing port to 49153 (from 0)<br /> 51: option transport.socket.keepalive-time 20<br /> 52: option transport.socket.keepalive-interval 2<br /> 53: option transport.socket.keepalive-count 9<br /> 54: option send-gids true<br /> 55: end-volume<br /> 56:<br /> 57: volume archive-client-4<br /> 58: type protocol/client<br /> 59: option ping-timeout 42<br /> 60: option remote-host glusterfs-06<br /> 61: option remote-subvolume /srv/glusterfs/brick/archive<br /> 62: option transport-type socket<br /> 63: option transport.address-family inet<br /> 64: option transport.tcp-user-timeout 0<br /> 65: option transport.socket.keepalive-time 20<br /> 66: option transport.socket.keepalive-interval 2<br /> 67: option transport.socket.keepalive-count 9<br /> 68: option send-gids true<br /> 69: end-volume<br /> 70:<br /> 71: volume archive-client-5<br /> 72: type protocol/client<br /> 73: option ping-timeout 42<br /> 74: option remote-host glusterfs-05<br /> 75: option remote-subvolume /srv/glusterfs/brick/archive<br /> 76: option transport-type socket<br /> 77: option transport.address-family inet<br /> 78: option transport.tcp-user-timeout 0<br /> 79: option transport.socket.keepalive-time 20<br /> 80: option transport.socket.keepalive-interval 2<br /> 81: option transport.socket.keepalive-count 9<br /> 82: option send-gids true<br /> 83: end-volume<br /> 84:<br /> 85: volume archive-client-6<br /> 86: type protocol/client<br /> 87: option ping-timeout 42<br /> 88: option remote-host glusterfs-04<br /> 89: option remote-subvolume /srv/glusterfs/brick/archive<br /> 90: option transport-type socket<br /> 91: option transport.address-family inet<br /> 92: option transport.tcp-user-timeout 0<br /> 93: option transport.socket.keepalive-time 20<br /> 94: option transport.socket.keepalive-interval 2<br /> 95: option transport.socket.keepalive-count 9<br /> 96: option send-gids true<br /> 97: end-volume<br /> 98:<br /> 99: volume archive-client-7<br />100: type protocol/client<br />101: option ping-timeout 42<br />102: option remote-host glusterfs-03<br />103: option remote-subvolume /srv/glusterfs/brick/archive<br />104: option transport-type socket<br />105: option transport.address-family inet<br />106: option transport.tcp-user-timeout 0<br />107: option transport.socket.keepalive-time 20<br />108: option transport.socket.keepalive-interval 2<br />109: option transport.socket.keepalive-count 9<br />110: option send-gids true<br />111: end-volume<br />112:<br />113: volume archive-client-8<br />114: type protocol/client<br />115: option ping-timeout 42<br />116: option remote-host glusterfs-02<br />117: option remote-subvolume /srv/glusterfs/brick/archive<br />118: option transport-type socket<br />119: option transport.address-family inet<br />120: option transport.tcp-user-timeout 0<br />121: option transport.socket.keepalive-time 20<br />122: option transport.socket.keepalive-interval 2<br />123: option transport.socket.keepalive-count 9<br />124: option send-gids true<br />125: end-volume<br />126:<br />127: volume archive-client-9<br />128: type protocol/client<br />129: option ping-timeout 42<br />130: option remote-host glusterfs-01<br />131: option remote-subvolume /srv/glusterfs/brick/archive<br />132: option transport-type socket<br />133: option transport.address-family inet<br />134: option transport.tcp-user-timeout 0<br />135: option transport.socket.keepalive-time 20<br />136: option transport.socket.keepalive-interval 2<br />137: option transport.socket.keepalive-count 9<br />138: option send-gids true<br />139: end-volume<br />140:<br />141: volume archive-disperse-0<br />142: type cluster/disperse<br />143: option redundancy 2<br />144: subvolumes archive-client-0 archive-client-1 archive-client-2 archive-client-3 archive-client-4 archive-client-5 archive-client-6 archive-client-7 archive-client-8 archive-client-9<br />145: end-volume<br />146:<br />147: volume archive-dht<br />148: type cluster/distribute<br />149: option lock-migration off<br />150: subvolumes archive-disperse-0<br />151: end-volume<br />152:<br />153: volume archive-write-behind<br />154: type performance/write-behind<br />155: subvolumes archive-dht<br />156: end-volume<br />157:<br />158: volume archive-read-ahead<br />159: type performance/read-ahead<br />160: subvolumes archive-write-behind<br />161: end-volume<br />162:<br />163: volume archive-readdir-ahead<br />164: type performance/readdir-ahead<br />165: option parallel-readdir off<br />166: option rda-request-size 131072<br />167: option rda-cache-limit 10MB<br />168: subvolumes archive-read-ahead<br />169: end-volume<br />170:<br />171: volume archive-io-cache<br />172: type performance/io-cache<br />173: subvolumes archive-readdir-ahead<br />174: end-volume<br />175:<br />176: volume archive-quick-read<br />177: type performance/quick-read<br />178: subvolumes archive-io-cache<br />179: end-volume<br />180:<br />181: volume archive-open-behind<br />182: type performance/open-behind<br />183: subvolumes archive-quick-read<br />184: end-volume<br />185:<br />186: volume archive-md-cache<br />187: type performance/md-cache<br />188: option cache-posix-acl true<br />189: subvolumes archive-open-behind<br />190: end-volume<br />191:<br />192: volume archive-io-threads<br />193: type performance/io-threads<br />194: subvolumes archive-md-cache<br />195: end-volume<br />196:<br />197: volume archive<br />198: type debug/io-stats<br />199: option log-level INFO<br />200: option latency-measurement off<br />201: option count-fop-hits off<br />202: subvolumes archive-io-threads<br />203: end-volume<br />204:<br />205: volume meta-autoload<br />206: type meta<br />207: subvolumes archive<br />208: end-volume<br />209:<br />+<hr />+<br />[2017-12-05 10:20:13.973638] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-4: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.973683] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-1: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.974177] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-6: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.974236] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-2: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.974818] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-5: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.974862] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.974877] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-7: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.974925] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.974936] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-9: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.975363] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-archive-client-8: changing port to 49153 (from 0)<br />[2017-12-05 10:20:13.975458] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-4: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.975482] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.975980] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976005] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976211] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-0: Connected to archive-client-0, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976244] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-0: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.976299] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-3: Connected to archive-client-3, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976314] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-3: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.976492] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976501] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-7: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976599] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976628] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-archive-client-8: Using Program GlusterFS 3.3, Num (1298437), Version (330)<br />[2017-12-05 10:20:13.976792] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-0: Server lk version = 1<br />[2017-12-05 10:20:13.976806] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-3: Server lk version = 1<br />[2017-12-05 10:20:13.976941] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-6: Connected to archive-client-6, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976953] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-6: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.976972] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-1: Connected to archive-client-1, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976806] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-3: Server lk version = 1<br />[2017-12-05 10:20:13.976941] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-6: Connected to archive-client-6, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976953] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-6: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.976972] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-1: Connected to archive-client-1, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.976986] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-1: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.977015] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-2: Connected to archive-client-2, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.977025] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-2: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.977318] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-6: Server lk version = 1<br />[2017-12-05 10:20:13.977411] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-2: Server lk version = 1<br />[2017-12-05 10:20:13.977427] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-1: Server lk version = 1<br />[2017-12-05 10:20:13.977453] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-8: Connected to archive-client-8, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.977463] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-8: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.977637] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-7: Connected to archive-client-7, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.977647] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-7: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.977815] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-8: Server lk version = 1<br />[2017-12-05 10:20:13.977993] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-7: Server lk version = 1<br />[2017-12-05 10:20:13.978005] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-9: Connected to archive-client-9, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.978017] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-9: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.978364] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-9: Server lk version = 1<br />[2017-12-05 10:20:13.979511] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-5: Connected to archive-client-5, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.979545] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-5: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.980131] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-5: Server lk version = 1<br />[2017-12-05 10:20:13.990225] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-archive-client-4: Connected to archive-client-4, attached to remote volume '/srv/glusterfs/brick/archive'.<br />[2017-12-05 10:20:13.990267] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-archive-client-4: Server and Client lk-version numbers are not same, reopening the fds<br />[2017-12-05 10:20:13.990350] I [MSGID: 122061] [ec.c:344:ec_up] 0-archive-disperse-0: Going UP<br />[2017-12-05 10:20:13.990942] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-archive-client-4: Server lk version = 1<br />[2017-12-05 10:20:13.996048] I [MSGID: 104041] [glfs-resolve.c:971:__glfs_active_subvol] 0-archive: switched to graph 70656c6b-6d61-6e73-6c61-622d73616d62 (0)<br /><br /><hr /><br />Gluster-users mailing list<br />Gluster-users@gluster.org<br /><a href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a><br /></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. All tyopes are thumb related and reflect authenticity.</body></html>