<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
"http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<title></title>
</head>
<body style="font-family:Arial">
<div>Sorry, maybe I wasn't clear. When I say "gluster native acces" I am talking about libgfapi.<br></div>
<div>That was just the doc I followed to set it up. It's working... but too slow (and log verbose) when starting.<br></div>
<div><br></div>
<div>Thanks<br></div>
<div>RM<br></div>
<div><br></div>
<div style="display:none" class="impComposeSignature"><br></div>
<div>----- Mensagem de Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> ---------<br></div>
<div>Data: Fri, 19 Mar 2021 17:09:41 +0000 (UTC)<br></div>
<div>De: Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>><br></div>
<div>Assunto: Re: [Gluster-users] fuse access seems ok but native qemu access is too slow<br></div>
<div>Para: <a href="mailto:algol@sapo.pt">algol@sapo.pt</a>, <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br></div>
<div><br></div>
<blockquote style="border:none; margin:0px; padding-left:0;margin-inline-start: 0;margin-inline-end: 0;" type="cite">
<p>Many oVirt users are using qemu with libgfapi for Virtualization needs.<br></p>
<div id="yMail_cursorElementTracker_1616173715635"><br></div>
<div id="yMail_cursorElementTracker_1616173715673">
<div>Have you checked this one : <br></div>
<div><a name="linkextractor__1616173731397" href="https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/libgfapi%20with%20qemu%20libvirt.md" data-yahoo-extracted-link="true" id="linkextractor__1616173731397">https://github.com/gluster/glusterfs-specs/blob/master/done/GlusterFS%203.5/libgfapi%20with%20qemu%20libvirt.md</a><br></div>
</div>
<div id="yMail_cursorElementTracker_1616173731566"><br></div>
<div id="yMail_cursorElementTracker_1616173731652">I've used only oVirt's way (which obscures the details), but the article looks legit.<br></div>
<div id="yMail_cursorElementTracker_1616173766884"><br></div>
<div id="yMail_cursorElementTracker_1616173766933"><br></div>
<div id="yMail_cursorElementTracker_1616173767093">Best Regards,<br></div>
<div id="yMail_cursorElementTracker_1616173770936">
<div>Strahil Nikolov<br></div>
<div><br></div>
<blockquote style="margin: 0 0 20px 0;">
<div style="font-family:Roboto, sans-serif; color:#6D00F6;">
<div style="padding: 10px 0 0 20px; margin: 10px 0 0 0; border-left: 1px solid #6D00F6;">
<div id="yiv6099233829">
<div>
<div>
<div>Hi,<br></div>
<div>
<div><br></div>
</div>
<div>I have just configure a distributed-replicate 2x3 gluster volume.<br></div>
<div>It seems to be working fine when I mount the volume with fuse but native gluster access with qemu-img is much slower and outputs errors. Log files are filled with warning and errors.<br></div>
<div>Starting vm with the created images is possible but it takes much more time when using gluster native access. Nevertheless, after starting up, there seems to be no penalty in using the gluster native access image being some times faster.<br></div>
<div><br></div>
<div>Can anyone, please, help me in figuring out if there is and where is the problem?<br></div>
<div><br></div>
<div>Thanks<br></div>
<div>rm<br></div>
<div><br></div>
<div><br></div>
<div><br></div>
<div>It seems to be working fine when I mount it with fuse:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# mount -t glusterfs <a href="http://srv-31.lan.example.com/ssd-volume/libvirt/images/" rel="nofollow noopener noreferrer">srv-32.lan.example.com/ssd-volume/</a> /mnt/ssd-volume/</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# echo 'This is a test' > /mnt/ssd-volume/test/text.txt</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# ll /data/glusterfs/ssd/brick[12]/brick/test/*</span></span><br></div>
</div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">-rw-r--r-- 2 root root 15 mar 17 14:08 /data/glusterfs/ssd/brick1/brick/test/text.txt<br>
<br>
root@srv-31:~# time qemu-img create -f qcow2 /mnt/ssd-volume/libvirt/images/test_5.qcow2.img 64G</span></span></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Formatting '/mnt/ssd-volume/libvirt/images/test_5.qcow2.img', fmt=qcow2 size=68719476736 cluster_size=65536 lazy_refcounts=off refcount_bits=16</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">real 0m0,204s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">user 0m0,009s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">sys 0m0,009s</span></span><br></div>
<div><br></div>
<div><br></div>
<div>
<div>Creating a image file onteh mouted gluster volume is quick but I get some warnings on log files:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# tail -f /var/log/glusterfs/glusterd.log /var/log/glusterfs/glustershd.log /var/log/glusterfs/bricks/data-glusterfs-ssd-brick[12]-brick.log</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
</div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.301388 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.303694 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.305808 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.307868 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.309863 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.312139 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.317780 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.319007 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.321096 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.323111 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.325284 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.327319 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.329488 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.331792 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.333597 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.335864 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.339551 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.341110 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.343079 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.345174 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.349569 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.351920 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.353282 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.355272 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.357467 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.359584 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.361771 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.364013 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.366094 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.368094 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.370261 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.372389 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.375550 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">(...)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.477472 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.479897 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.482318 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:37:11.484489 +0000] W [dict.c:1532:dict_get_with_ref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/9.0/xlator/features/locks.so(+0x26c3e) [0x7fb1dbdccc3e] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_uint32+0x37) [0x7fb1e1a1f3f7] -->/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get_with_ref+0x7d) [0x7fb1e1a1e86d] ) 0-dict: dict OR key (glusterfs.lk.lkmode) is NULL [Invalid argument]</span></span><br></div>
<div><br></div>
<div><br></div>
<div>Native gluster access with qemu-img is much slower and outputs errors:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# time qemu-img create -f qcow2 gluster://srv-31.lan.example.com/ssd-volume/libvirt/images/test_6.qcow2.img</span></span> <span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">64G</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Formatting 'gluster://<a href="http://srv-31.lan.example.com/ssd-volume/libvirt/images/test_6.qcow2.im" rel="nofollow noopener noreferrer">srv-31.lan.example.com/ssd-volume/libvirt/images/test_6.qcow2.img</a>', fmt=qcow2 size=68719476736 cluster_size=65536 lazy_refcounts=off refcount_bits=16</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.490844 +0000] I [io-stats.c:3708:ios_sample_buf_size_configure] 0-ssd-volume: Configure ios_sample_buf size is 1024 because ios_sample_interval is 0</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615942 +0000] E [MSGID: 108006] [afr-common.c:6146:__afr_handle_child_down_event] 0-ssd-volume-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up.</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.617033 +0000] E [MSGID: 108006] [afr-common.c:6146:__afr_handle_child_down_event] 0-ssd-volume-replicate-1: All subvolumes are down. Going offline until at least one of them comes back up.</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:52.502174 +0000] I [io-stats.c:4038:fini] 0-ssd-volume: io-stats translator unloaded</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.514364 +0000] I [io-stats.c:3708:ios_sample_buf_size_configure] 0-ssd-volume: Configure ios_sample_buf size is 1024 because ios_sample_interval is 0</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.654336 +0000] E [MSGID: 108006] [afr-common.c:6146:__afr_handle_child_down_event] 0-ssd-volume-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up.</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.655251 +0000] E [MSGID: 108006] [afr-common.c:6146:__afr_handle_child_down_event] 0-ssd-volume-replicate-1: All subvolumes are down. Going offline until at least one of them comes back up.</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:48:03.525947 +0000] I [io-stats.c:4038:fini] 0-ssd-volume: io-stats translator unloaded</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">real 0m22,068s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">user 0m0,055s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">sys 0m0,043s</span></span><br></div>
<div><br></div>
<div><br></div>
<div>In both cases images are written correctly (and usable):<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# l /data/glusterfs/ssd/brick[12]/brick/libvirt/images/test_[56]*</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">-rw-r--r-- 2 root root 193K mar 19 11:37 /data/glusterfs/ssd/brick2/brick/libvirt/images/test_5.qcow2.img</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">-rw------- 2 root root 193K mar 19 11:47 /data/glusterfs/ssd/brick2/brick/libvirt/images/test_6.qcow2.img</span></span><br></div>
<div><br></div>
<div>But I get lots of warnings on log files:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# tail -f /var/log/glusterfs/glusterd.log /var/log/glusterfs/glustershd.log /var/log/glusterfs/bricks/data-glusterfs-ssd-brick[12]-brick.log</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/glusterd.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.483736 +0000] I [MSGID: 106496] [glusterd-handshake.c:969:__server_getspec] 0-management: Received mount request for volume ssd-volume</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.498908 +0000] I [addr.c:54:compare_addr_and_update] 0-/data/glusterfs/ssd/brick1/brick: allowed = "*", received addr = "127.0.0.1"</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.498967 +0000] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 80c9a2bf-6fad-4023-8458-a175cef7f681</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.498989 +0000] I [MSGID: 115029] [server-handshake.c:561:server_setvolume] 0-ssd-volume-server: accepted client from CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0 (version: 9.0) with subvol /data/glusterfs/ssd/brick1/brick</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.502029 +0000] I [addr.c:54:compare_addr_and_update] 0-/data/glusterfs/ssd/brick2/brick: allowed = "*", received addr = "127.0.0.1"</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.502087 +0000] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 80c9a2bf-6fad-4023-8458-a175cef7f681</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.502109 +0000] I [MSGID: 115029] [server-handshake.c:561:server_setvolume] 0-ssd-volume-server: accepted client from CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0 (version: 9.0) with subvol /data/glusterfs/ssd/brick2/brick</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615182 +0000] W [socket.c:767:__socket_rwv] 0-tcp.ssd-volume-server: readv on 127.0.0.1:49151 failed (No data available)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615258 +0000] I [MSGID: 115036] [server.c:500:server_rpc_notify] 0-ssd-volume-server: disconnecting connection [{client-uid=CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615215 +0000] W [socket.c:767:__socket_rwv] 0-tcp.ssd-volume-server: readv on 127.0.0.1:49150 failed (No data available)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615316 +0000] I [MSGID: 115036] [server.c:500:server_rpc_notify] 0-ssd-volume-server: disconnecting connection [{client-uid=CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615459 +0000] I [MSGID: 101055] [client_t.c:397:gf_client_unref] 0-ssd-volume-server: Shutting down connection CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:42.615524 +0000] I [MSGID: 101055] [client_t.c:397:gf_client_unref] 0-ssd-volume-server: Shutting down connection CTX_ID:d0cc883b-859c-4143-ab22-83e72673904a-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.522422 +0000] I [addr.c:54:compare_addr_and_update] 0-/data/glusterfs/ssd/brick1/brick: allowed = "*", received addr = "127.0.0.1"</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.522480 +0000] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 80c9a2bf-6fad-4023-8458-a175cef7f681</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.522502 +0000] I [MSGID: 115029] [server-handshake.c:561:server_setvolume] 0-ssd-volume-server: accepted client from CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0 (version: 9.0) with subvol /data/glusterfs/ssd/brick1/brick</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.525344 +0000] I [addr.c:54:compare_addr_and_update] 0-/data/glusterfs/ssd/brick2/brick: allowed = "*", received addr = "127.0.0.1"</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.525396 +0000] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 80c9a2bf-6fad-4023-8458-a175cef7f681</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.525419 +0000] I [MSGID: 115029] [server-handshake.c:561:server_setvolume] 0-ssd-volume-server: accepted client from CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0 (version: 9.0) with subvol /data/glusterfs/ssd/brick2/brick</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.653652 +0000] W [socket.c:767:__socket_rwv] 0-tcp.ssd-volume-server: readv on 127.0.0.1:49146 failed (No data available)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.653731 +0000] I [MSGID: 115036] [server.c:500:server_rpc_notify] 0-ssd-volume-server: disconnecting connection [{client-uid=CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.653752 +0000] W [socket.c:767:__socket_rwv] 0-tcp.ssd-volume-server: readv on 127.0.0.1:49144 failed (No data available)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.653840 +0000] I [MSGID: 115036] [server.c:500:server_rpc_notify] 0-ssd-volume-server: disconnecting connection [{client-uid=CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.653928 +0000] I [MSGID: 101055] [client_t.c:397:gf_client_unref] 0-ssd-volume-server: Shutting down connection CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.654040 +0000] I [MSGID: 101055] [client_t.c:397:gf_client_unref] 0-ssd-volume-server: Shutting down connection CTX_ID:53e36496-2f5a-4599-9d6b-aeda7822a7a0-GRAPH_ID:0-PID:53235-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/glusterd.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:47:53.507207 +0000] I [MSGID: 106496] [glusterd-handshake.c:969:__server_getspec] 0-management: Received mount request for volume ssd-volume</span></span><br></div>
<div><br></div>
<div><br></div>
<div>Starting a vm with a virtual disk on the mounted gluster volume <span class="yiv6099233829font" style=""><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">(-drive file=/mnt/ssd-volume/libvirt/images/test_6.qcow2.img)</span></span>is quick:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# time virsh start vm-11</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Domain vm-11 started</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">real 0m0,277s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">user 0m0,020s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">sys 0m0,004s</span></span><br></div>
<div><br></div>
<div>Statring a vm with a virtual disk on gluster volume with native access <span class="yiv6099233829font" style=""><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">(-drive file=gluster://srv-31.lan.example.com:24007/ssd-volume/libvirt/images/test_7.qcow2.img)</span></span>is much slower:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# time virsh start vm-11</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Domain vm-11 started</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">real 0m22,339s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">user 0m0,021s</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">sys 0m0,007s</span></span><br></div>
<div><br></div>
<div><br></div>
<div><br></div>
<div>From time to time, even without user operations, log files report errors:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# tail -f /var/log/glusterfs/glusterd.log /var/log/glusterfs/glustershd.log /var/log/glusterfs/bricks/data-glusterfs-ssd-brick[12]-brick.log</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:09.525079 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:09.525212 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=11817}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:4b47408f-323c-4c6a-9a20-2ae2a3a2cdb8-GRAPH_ID:3-PID:2291-HOST:srv-32-PC_NAME:ssd-volume-client-0-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:13.525581 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:13.525711 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=10055}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:4b47408f-323c-4c6a-9a20-2ae2a3a2cdb8-GRAPH_ID:3-PID:2291-HOST:srv-32-PC_NAME:ssd-volume-client-3-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:38.829803 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=11820}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:974f4637-64ef-42e6-afad-1dc9c67c4a43-GRAPH_ID:3-PID:2062-HOST:srv-33-PC_NAME:ssd-volume-client-0-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:38.829970 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=10058}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:974f4637-64ef-42e6-afad-1dc9c67c4a43-GRAPH_ID:3-PID:2062-HOST:srv-33-PC_NAME:ssd-volume-client-3-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:38.829721 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick2-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:38.829883 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.005995 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006115 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=10082}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:4945546f-f368-4fa7-8bfc-3dd7abda5d1b-GRAPH_ID:3-PID:2486-HOST:srv-31-PC_NAME:ssd-volume-client-3-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/bricks/data-glusterfs-ssd-brick1-brick.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006096 +0000] E [MSGID: 113002] [posix-entry-ops.c:682:posix_mkdir] 0-ssd-volume-posix: gfid is null for (null) [Invalid argument]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006212 +0000] E [MSGID: 115056] [server-rpc-fops_v2.c:497:server4_mkdir_cbk] 0-ssd-volume-server: MKDIR info [{frame=11844}, {MKDIR_path=}, {uuid_utoa=00000000-0000-0000-0000-000000000001}, {bname=}, {client=CTX_ID:4945546f-f368-4fa7-8bfc-3dd7abda5d1b-GRAPH_ID:3-PID:2486-HOST:srv-31-PC_NAME:ssd-volume-client-0-RECON_NO:-0}, {error-xlator=ssd-volume-posix}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">==> /var/log/glusterfs/glustershd.log <==</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006255 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-3: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006352 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-0: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006408 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-4: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006505 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-5: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006562 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-1: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">[2021-03-19 11:45:50.006645 +0000] E [MSGID: 114031] [client-rpc-fops_v2.c:214:client4_0_mkdir_cbk] 3-ssd-volume-client-2: remote operation failed. [{path=(null)}, {errno=22}, {error=Invalid argument}]</span></span><br></div>
<div><br></div>
<div><br></div>
<div><br></div>
<div>Everything seems to be ok<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# gluster volume status ssd-volume</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Status of volume: ssd-volume</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Gluster process TCP Port RDMA Port Online Pid</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">------------------------------------------------------------------------------</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-31.lan.example.com:/data/glusterfs/ssd/brick1/brick 49156 0 Y 9242</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-32.lan.example.com:/data/glusterfs/ssd/brick1/brick 49156 0 Y 7968</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-33.lan.example.com:/data/glusterfs/ssd/brick1/brick 49156 0 Y 8211</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-31.lan.example.com:/data/glusterfs/ssd/brick2/brick 49157 0 Y 9258</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-32.lan.example.com:/data/glusterfs/ssd/brick2/brick 49157 0 Y 7984</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick srv-33.lan.example.com:/data/glusterfs/ssd/brick2/brick 49157 0 Y 8227</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Self-heal Daemon on localhost N/A N/A Y 2486</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Self-heal Daemon on srv-32.lan.example.com N/A N/A Y 2291</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Self-heal Daemon on srv-33.lan.example.com N/A N/A Y 2062</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Task Status of Volume ssd-volume</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">------------------------------------------------------------------------------</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">There are no active volume tasks</span></span><br></div>
<div><br></div>
<div><br></div>
<div>
<div>
<div><br></div>
</div>
<div>
<div><br></div>
</div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# gluster volume info ssd-volume</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Volume Name: ssd-volume</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Type: Distributed-Replicate</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Volume ID: a6f3426f-5b33-404b-ab61-581e24e0c36d</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Status: Started</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Snapshot Count: 0</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Number of Bricks: 2 x 3 = 6</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Transport-type: tcp</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Bricks:</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick1: srv-31.lan.example.com:/data/glusterfs/ssd/brick1/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick2: srv-32.lan.example.com:/data/glusterfs/ssd/brick1/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick3: srv-33.lan.example.com:/data/glusterfs/ssd/brick1/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick4: srv-31.lan.example.com:/data/glusterfs/ssd/brick2/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick5: srv-32.lan.example.com:/data/glusterfs/ssd/brick2/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Brick6: srv-33.lan.example.com:/data/glusterfs/ssd/brick2/brick</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Options Reconfigured:</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">server.allow-insecure: on</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">storage.owner-gid: 64055</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">storage.owner-uid: 64055</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ssl.cipher-list: HIGH:!SSLv2</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">performance.client-io-threads: off</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">nfs.disable: on</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">transport.address-family: inet</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">storage.fips-mode-rchecksum: on</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">client.ssl: off</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">server.ssl: off</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">auth.ssl-allow: *</span></span><br></div>
<div><br></div>
</div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# gluster peer status</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Number of Peers: 2</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Hostname: srv-32.lan.example.com</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Uuid: 0a11bfb9-5821-42f0-8ef3-18b36bfcfa8a</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">State: Peer in Cluster (Connected)</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Hostname: srv-33.lan.example.com</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Uuid: 111d93d2-64aa-463f-8f33-7b53a3f05cab</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">State: Peer in Cluster (Connected)</span></span><br></div>
<div><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# gluster pool list</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">UUID Hostname State</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">0a11bfb9-5821-42f0-8ef3-18b36bfcfa8a srv-32.lan.example.com Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">111d93d2-64aa-463f-8f33-7b53a3f05cab srv-33.lan.example.com Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">67e9da67-90c5-486b-acb1-eb91abebcefb localhost Connected</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-32:~# gluster pool list</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">UUID Hostname State</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">111d93d2-64aa-463f-8f33-7b53a3f05cab srv-33.lan.example.com Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">67e9da67-90c5-486b-acb1-eb91abebcefb 10.0.0.31 Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">0a11bfb9-5821-42f0-8ef3-18b36bfcfa8a localhost Connected</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-33:~# gluster pool list</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">UUID Hostname State</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">0a11bfb9-5821-42f0-8ef3-18b36bfcfa8a srv-32.lan.example.com Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">67e9da67-90c5-486b-acb1-eb91abebcefb 10.0.0.31 Connected</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">111d93d2-64aa-463f-8f33-7b53a3f05cab localhost Connected</span></span><br></div>
<div><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# cat /etc/glusterfs/glusterd.vol</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">volume management</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> type mgmt/glusterd</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option working-directory /var/lib/glusterd</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option transport-type socket</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option transport.socket.keepalive-time 10</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option transport.socket.keepalive-interval 2</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option transport.socket.read-fail-log off</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option transport.socket.listen-port 24007</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option ping-timeout 0</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option event-threads 1</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"># option lock-timer 180</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"># option transport.address-family inet6</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"># option base-port 49152</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option max-port 60999</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif"> option rpc-auth-allow-insecure on</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">end-volume</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# ss -lntp</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">State Recv-Q Send-Q Local Address:Port Peer Address:Port </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:49153 0.0.0.0:* users:(("glusterfsd",pid=2455,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:49156 0.0.0.0:* users:(("glusterfsd",pid=9242,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:49157 0.0.0.0:* users:(("glusterfsd",pid=9258,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:24007 0.0.0.0:* users:(("glusterd",pid=1954,fd=10)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:111 0.0.0.0:* users:(("rpcbind",pid=1716,fd=4),("systemd",pid=1,fd=36)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:2224 0.0.0.0:* users:(("pcsd",pid=2126,fd=4)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1963,fd=3)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 20 127.0.0.1:25 0.0.0.0:* users:(("exim4",pid=2377,fd=3)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 0.0.0.0:49152 0.0.0.0:* users:(("glusterfsd",pid=2446,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 [::]:111 [::]:* users:(("rpcbind",pid=1716,fd=6),("systemd",pid=1,fd=38)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">LISTEN 0 128 [::]:2224 [::]:* users:(("pcsd",pid=2126,fd=5))</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# ss -tnp | grep gluster</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49149 127.0.1.1:24007 users:(("glusterfsd",pid=2446,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49144 10.0.0.33:49153 users:(("glusterfs",pid=2486,fd=17)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:24007 127.0.0.1:49145 users:(("glusterd",pid=1954,fd=26)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49157 127.0.0.1:49140 users:(("glusterfsd",pid=9258,fd=10)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49137 127.0.1.1:49156 users:(("glusterfs",pid=2917,fd=10)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49146 10.0.0.32:49152 users:(("glusterfs",pid=2486,fd=25)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49153 10.0.0.33:49147 users:(("glusterfsd",pid=2455,fd=270)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:24007 127.0.0.1:49148 users:(("glusterd",pid=1954,fd=12)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49157 10.0.0.33:49141 users:(("glusterfsd",pid=9258,fd=271)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:24007 127.0.0.1:49131 users:(("glusterd",pid=1954,fd=31)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49152 10.0.0.32:49146 users:(("glusterfsd",pid=2446,fd=271)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49134 10.0.0.33:24007 users:(("glusterd",pid=1954,fd=33)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49127 10.0.0.33:49157 users:(("glusterfs",pid=2917,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49142 127.0.1.1:49156 users:(("glusterfs",pid=2486,fd=15)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49135 127.0.1.1:49157 users:(("glusterfs",pid=2917,fd=14)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49145 127.0.0.1:24007 users:(("glusterfs",pid=2486,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49157 127.0.0.1:49135 users:(("glusterfsd",pid=9258,fd=272)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49152 127.0.0.1:49141 users:(("glusterfsd",pid=2446,fd=10)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49157 10.0.0.32:49144 users:(("glusterfsd",pid=9258,fd=270)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:24007 127.0.0.1:49132 users:(("glusterd",pid=1954,fd=25)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49148 127.0.1.1:24007 users:(("glusterfsd",pid=2455,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49118 10.0.0.32:49156 users:(("glusterfs",pid=2486,fd=13)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49156 127.0.0.1:49137 users:(("glusterfsd",pid=9242,fd=272)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49121 10.0.0.33:49156 users:(("glusterfs",pid=2917,fd=13)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49132 127.0.1.1:24007 users:(("glusterfsd",pid=9242,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49141 127.0.1.1:49152 users:(("glusterfs",pid=2486,fd=22)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49156 10.0.0.32:49148 users:(("glusterfsd",pid=9242,fd=270)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49152 10.0.0.33:49149 users:(("glusterfsd",pid=2446,fd=270)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49140 127.0.1.1:49157 users:(("glusterfs",pid=2486,fd=11)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49123 10.0.0.32:49157 users:(("glusterfs",pid=2917,fd=12)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49141 10.0.0.33:49152 users:(("glusterfs",pid=2486,fd=23)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49131 127.0.1.1:24007 users:(("glusterfsd",pid=9258,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49153 10.0.0.32:49145 users:(("glusterfsd",pid=2455,fd=271)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49117 10.0.0.33:49156 users:(("glusterfs",pid=2486,fd=5)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49114 10.0.0.32:49157 users:(("glusterfs",pid=2486,fd=16)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49153 127.0.0.1:49138 users:(("glusterfsd",pid=2455,fd=10)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49143 10.0.0.32:24007 users:(("glusterfs",pid=2917,fd=9)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49113 10.0.0.33:49157 users:(("glusterfs",pid=2486,fd=18)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49156 10.0.0.33:49143 users:(("glusterfsd",pid=9242,fd=271)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.0.1:49138 127.0.1.1:49153 users:(("glusterfs",pid=2486,fd=19)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:24007 10.0.0.33:49150 users:(("glusterd",pid=1954,fd=32)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49131 10.0.0.32:49156 users:(("glusterfs",pid=2917,fd=7)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49145 10.0.0.32:49153 users:(("glusterfs",pid=2486,fd=8)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:24007 127.0.0.1:49149 users:(("glusterd",pid=1954,fd=13)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:24007 10.0.0.32:49150 users:(("glusterd",pid=1954,fd=7)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 10.0.0.31:49151 10.0.0.32:24007 users:(("glusterd",pid=1954,fd=8)) </span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">ESTAB 0 0 127.0.1.1:49156 127.0.0.1:49142 users:(("glusterfsd",pid=9242,fd=10))</span></span><br></div>
<div><br></div>
<div>hostnames resolve correctly:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# for i in {31..33}; do host srv-$i.lan.example.com; done</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-31.lan.example.com has address 10.0.0.31</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-32.lan.example.com has address 10.0.0.32</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-33.lan.example.com has address 10.0.0.33</span></span><br></div>
<div><br></div>
<div><br></div>
<div>And network conectivity is ok:<br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# for i in {31..33}; do ping -c 1 -q srv-$i.lan.example.com | echo "srv-$i ok"; done</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-31 ok</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-32 ok</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">srv-33 ok</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# for i in {31..33}; do nc -v srv-$i.lan.example.com 24007; done</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Connection to srv-31.lan.example.com 24007 port [tcp/*] succeeded!</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Connection to srv-32.lan.example.com 24007 port [tcp/*] succeeded!</span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Connection to srv-33.lan.example.com 24007 port [tcp/*] succeeded!</span></span><br></div>
<div><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">root@srv-31:~# iptables -L</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Chain INPUT (policy ACCEPT)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">target prot opt source destination </span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Chain FORWARD (policy ACCEPT)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">target prot opt source destination </span></span><br></div>
<div><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">Chain OUTPUT (policy ACCEPT)</span></span><br></div>
<div><span style="" class="yiv6099233829font"><span class="font" style="font-family:Courier New, Courier, monospace, sans-serif">target prot opt source destination</span></span><br></div>
<div><br></div>
<div><br></div>
<div><br></div>
</div>
</div>
<div>________<br></div>
<div><br></div>
<div><br></div>
<div><br></div>
<div>Community Meeting Calendar:<br></div>
<div><br></div>
<div>Schedule -<br></div>
<div>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br></div>
<div>Bridge: <a href="https://meet.google.com/cpu-eiue-hvk">https://meet.google.com/cpu-eiue-hvk</a><br></div>
<div>Gluster-users mailing list<br></div>
<div><a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br></div>
<div><a href="https://lists.gluster.org/mailman/listinfo/gluster-users">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br></div>
</div>
</div>
</blockquote>
</div>
</blockquote>
<div><br></div>
<div>----- Fim da mensagem de Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> -----<br></div>
<div><br></div>
</body>
</html>