<div dir="ltr"><div>Thanks for that input. Adding Niels since the issue is reproducible only with libgfapi.<br><br></div>-Krutika<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 18, 2018 at 1:39 PM, Ing. Luca Lazzeroni - Trend Servizi Srl <span dir="ltr">&lt;<a href="mailto:luca@gvnet.it" target="_blank">luca@gvnet.it</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <p>Another update.</p>
    <p>I&#39;ve setup a replica 3 volume without sharding and tried to
      install a VM on a qcow2 volume on that device; however the result
      is the same and the vm image has been corrupted, exactly at the
      same point.</p>
    <p>Here&#39;s the volume info of the create volume:</p>
    <p>Volume Name: gvtest<br>
      Type: Replicate<br>
      Volume ID: e2ddf694-ba46-4bc7-bc9c-<wbr>e30803374e9d<span class=""><br>
      Status: Started<br>
      Snapshot Count: 0<br></span>
      Number of Bricks: 1 x 3 = 3<br>
      Transport-type: tcp<br>
      Bricks:<br>
      Brick1: gluster1:/bricks/brick1/gvtest<br>
      Brick2: gluster2:/bricks/brick1/gvtest<br>
      Brick3: gluster3:/bricks/brick1/gvtest<br>
      Options Reconfigured:<br>
      user.cifs: off<br>
      features.shard: off<span class=""><br>
      cluster.shd-wait-qlength: 10000<br>
      cluster.shd-max-threads: 8<br>
      cluster.locking-scheme: granular<br>
      cluster.data-self-heal-<wbr>algorithm: full<br>
      cluster.server-quorum-type: server<br>
      cluster.quorum-type: auto<br>
      cluster.eager-lock: enable<br>
      network.remote-dio: enable<br>
      performance.low-prio-threads: 32<br>
      performance.io-cache: off<br>
      performance.read-ahead: off<br>
      performance.quick-read: off<br>
      transport.address-family: inet<br></span>
      nfs.disable: on<br>
      performance.client-io-threads: off<br>
      <br>
    </p><div><div class="h5">
    <br>
    <div class="m_-592553901259022859moz-cite-prefix">Il 17/01/2018 14:51, Ing. Luca
      Lazzeroni - Trend Servizi Srl ha scritto:<br>
    </div>
    <blockquote type="cite">
      
      <p>Hi,</p>
      <p>after our IRC chat I&#39;ve rebuilt a virtual machine with FUSE
        based virtual disk. Everything worked flawlessly.</p>
      <p>Now I&#39;m sending you the output of the requested getfattr
        command on the disk image:</p>
      <p># file: TestFUSE-vda.qcow2<br>
        trusted.afr.dirty=<wbr>0x000000000000000000000000<br>
        trusted.gfid=<wbr>0x40ffafbbe987445692bb31295fa4<wbr>0105<br>
trusted.gfid2path.<wbr>dc9dde61f0b77eab=<wbr>0x31326533323631662d373839332d<wbr>346262302d383738632d3966623765<wbr>306232336263652f54657374465553<wbr>452d7664612e71636f7732<br>
        trusted.glusterfs.shard.block-<wbr>size=0x0000000004000000<br>
trusted.glusterfs.shard.file-<wbr>size=<wbr>0x00000000c1530000000000000000<wbr>0000000000000060be900000000000<wbr>000000<br>
      </p>
      <p>Hope this helps.</p>
      <p><br>
      </p>
      <br>
      <div class="m_-592553901259022859moz-cite-prefix">Il 17/01/2018 11:37, Ing. Luca
        Lazzeroni - Trend Servizi Srl ha scritto:<br>
      </div>
      <blockquote type="cite">
        
        <p>I actually use FUSE and it works. If i try to use &quot;libgfapi&quot;
          direct interface to gluster in qemu-kvm, the problem appears.</p>
        <p><br>
        </p>
        <br>
        <div class="m_-592553901259022859moz-cite-prefix">Il 17/01/2018 11:35, Krutika
          Dhananjay ha scritto:<br>
        </div>
        <blockquote type="cite">
          <div dir="ltr">
            <div>Really? Then which protocol exactly do you see this
              issue with? libgfapi? NFS? <br>
              <br>
            </div>
            -Krutika<br>
          </div>
          <div class="gmail_extra"><br>
            <div class="gmail_quote">On Wed, Jan 17, 2018 at 3:59 PM,
              Ing. Luca Lazzeroni - Trend Servizi Srl <span dir="ltr">&lt;<a href="mailto:luca@gvnet.it" target="_blank">luca@gvnet.it</a>&gt;</span>
              wrote:<br>
              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                <div text="#000000" bgcolor="#FFFFFF">
                  <p>Of course. Here&#39;s the full log. Please, note that
                    in FUSE mode everything works apparently without
                    problems. I&#39;ve installed 4 vm and updated them
                    without problems. <br>
                  </p>
                  <div>
                    <div class="m_-592553901259022859h5">
                      <p><br>
                      </p>
                      <br>
                      <div class="m_-592553901259022859m_-2254898158214265998moz-cite-prefix">Il
                        17/01/2018 11:00, Krutika Dhananjay ha scritto:<br>
                      </div>
                      <blockquote type="cite">
                        <div dir="ltr"><br>
                          <div class="gmail_extra"><br>
                            <div class="gmail_quote">On Tue, Jan 16,
                              2018 at 10:47 PM, Ing. Luca Lazzeroni -
                              Trend Servizi Srl <span dir="ltr">&lt;<a href="mailto:luca@gvnet.it" target="_blank">luca@gvnet.it</a>&gt;</span>
                              wrote:<br>
                              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                <div text="#000000" bgcolor="#FFFFFF">
                                  <p>I&#39;ve made the test with raw image
                                    format (preallocated too) and the
                                    corruption problem is still there
                                    (but without errors in bricks&#39; log
                                    file).</p>
                                  <p>What does the &quot;link&quot; error in
                                    bricks log files means ? <br>
                                  </p>
                                  <p>I&#39;ve seen the source code looking
                                    for the lines where it happens and
                                    it seems a warning (it doesn&#39;t imply
                                    a failure).</p>
                                </div>
                              </blockquote>
                              <div><br>
                              </div>
                              <div>Indeed, it only represents a
                                transient state when the shards are
                                created for the first time and does not
                                indicate a failure.</div>
                              <div>Could you also get the logs of the
                                gluster fuse mount process? It should be
                                under /var/log/glusterfs of your client
                                machine with the filename as a
                                hyphenated mount point path.</div>
                              <div><br>
                              </div>
                              <div>For example, if your volume was
                                mounted at /mnt/glusterfs, then your log
                                file would be named mnt-glusterfs.log.</div>
                              <div><br>
                              </div>
                              <div>-Krutika<br>
                              </div>
                              <div> <br>
                              </div>
                              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                <div text="#000000" bgcolor="#FFFFFF">
                                  <div>
                                    <div class="m_-592553901259022859m_-2254898158214265998h5">
                                      <p><br>
                                      </p>
                                      <br>
                                      <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-cite-prefix">Il
                                        16/01/2018 17:39, Ing. Luca
                                        Lazzeroni - Trend Servizi Srl ha
                                        scritto:<br>
                                      </div>
                                      <blockquote type="cite">
                                        <p>An update:</p>
                                        <p>I&#39;ve tried, for my tests, to
                                          create the vm volume as</p>
                                        <p>qemu-img create -f qcow2 -o
                                          preallocation=full
                                          gluster://gluster1/Test/Test-v<wbr>da.img
                                          20G</p>
                                        <p>et voila !</p>
                                        <p>No errors at all, neither in
                                          bricks&#39; log file (the &quot;link
                                          failed&quot; message disappeared),
                                          neither in VM (no corruption
                                          and installed succesfully).</p>
                                        <p>I&#39;ll do another test with a
                                          fully preallocated raw image.</p>
                                        <p><br>
                                        </p>
                                        <br>
                                        <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-cite-prefix">Il
                                          16/01/2018 16:31, Ing. Luca
                                          Lazzeroni - Trend Servizi Srl
                                          ha scritto:<br>
                                        </div>
                                        <blockquote type="cite">
                                          <p>I&#39;ve just done all the
                                            steps to reproduce the
                                            problem. <br>
                                          </p>
                                          <p>Tha VM volume has been
                                            created via &quot;qemu-img create
                                            -f qcow2 Test-vda2.qcow2
                                            20G&quot; on the gluster volume
                                            mounted via FUSE. I&#39;ve tried
                                            also to create the volume
                                            with preallocated metadata,
                                            which moves the problem a
                                            bit far away (in time). The
                                            volume is a replice 3
                                            arbiter 1 volume hosted on
                                            XFS bricks.<br>
                                          </p>
                                          <p>Here are the informations:</p>
                                          <p>[root@ovh-ov1 bricks]#
                                            gluster volume info gv2a2<br>
                                             <br>
                                            Volume Name: gv2a2<br>
                                            Type: Replicate<br>
                                            Volume ID:
                                            83c84774-2068-4bfc-b0b9-3e6b93<wbr>705b9f<br>
                                            Status: Started<br>
                                            Snapshot Count: 0<br>
                                            Number of Bricks: 1 x (2 +
                                            1) = 3<br>
                                            Transport-type: tcp<br>
                                            Bricks:<br>
                                            Brick1:
                                            gluster1:/bricks/brick2/gv2a2<br>
                                            Brick2:
                                            gluster3:/bricks/brick3/gv2a2<br>
                                            Brick3:
                                            gluster2:/bricks/arbiter_brick<wbr>_gv2a2/gv2a2
                                            (arbiter)<br>
                                            Options Reconfigured:<br>
                                            storage.owner-gid: 107<br>
                                            storage.owner-uid: 107<br>
                                            user.cifs: off<br>
                                            features.shard: on<br>
                                            cluster.shd-wait-qlength:
                                            10000<br>
                                            cluster.shd-max-threads: 8<br>
                                            cluster.locking-scheme:
                                            granular<br>
cluster.data-self-heal-algorit<wbr>hm: full<br>
                                            cluster.server-quorum-type:
                                            server<br>
                                            cluster.quorum-type: auto<br>
                                            cluster.eager-lock: enable<br>
                                            network.remote-dio: enable<br>
performance.low-prio-threads: 32<br>
                                            performance.io-cache: off<br>
                                            performance.read-ahead: off<br>
                                            performance.quick-read: off<br>
                                            transport.address-family:
                                            inet<br>
                                            nfs.disable: off<br>
performance.client-io-threads: off<br>
                                            <br>
                                          </p>
                                          <p>/var/log/glusterfs/glusterd.lo<wbr>g:</p>
                                          <p>[2018-01-15
                                            14:17:50.196228] I [MSGID:
                                            106488]
                                            [glusterd-handler.c:1548:__glu<wbr>sterd_handle_cli_get_volume]
                                            0-management: Received get
                                            vol req<br>
                                            [2018-01-15 14:25:09.555214]
                                            I [MSGID: 106488]
                                            [glusterd-handler.c:1548:__glu<wbr>sterd_handle_cli_get_volume]
                                            0-management: Received get
                                            vol req<br>
                                          </p>
                                          <p>(empty because today it&#39;s
                                            2018-01-16)</p>
                                          <p>/var/log/glusterfs/glustershd.<wbr>log:</p>
                                          <p>[2018-01-14
                                            02:23:02.731245] I
                                            [glusterfsd-mgmt.c:1821:mgmt_g<wbr>etspec_cbk]
                                            0-glusterfs: No change in
                                            volfile,continuing<br>
                                          </p>
                                          <p>(empty too)</p>
                                          <p>/var/log/glusterfs/bricks/bric<wbr>k-brick2-gv2a2.log
                                            (the interested volume):</p>
                                          <p>[2018-01-16
                                            15:14:37.809965] I [MSGID:
                                            115029]
                                            [server-handshake.c:793:server<wbr>_setvolume]
                                            0-gv2a2-server: accepted
                                            client from
                                            ovh-ov1-10302-2018/01/16-15:14<wbr>:37:790306-gv2a2-client-0-0-0
                                            (version: 3.12.4)<br>
                                            [2018-01-16 15:16:41.471751]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.4
                                            failed<br>
                                            [2018-01-16 15:16:41.471745]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.4
                                            -&gt; 
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/a0/14/a0144df3-8d89-4aed-872<wbr>e-5fef141e9e1efailed 
                                            [File exists]<br>
                                            [2018-01-16 15:16:42.593392]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.5
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/eb/04/eb044e6e-3a23-40a4-9ce<wbr>1-f13af148eb67failed 
                                            [File exists]<br>
                                            [2018-01-16 15:16:42.593426]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.5
                                            failed<br>
                                            [2018-01-16 15:17:04.129593]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.8
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/dc/92/dc92bd0a-0d46-4826-a4c<wbr>9-d073a924dd8dfailed 
                                            [File exists]<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.8
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/dc/92/dc92bd0a-0d46-4826-a4c<wbr>9-d073a924dd8dfailed 
                                            [File exists]&quot; repeated 5
                                            times between [2018-01-16
                                            15:17:04.129593] and
                                            [2018-01-16 15:17:04.129593]<br>
                                            [2018-01-16 15:17:04.129661]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.8
                                            failed<br>
                                            [2018-01-16 15:17:08.279162]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.9
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/c9/b7/c9b71b00-a09f-4df1-b87<wbr>4-041820ca8241failed 
                                            [File exists]<br>
                                            [2018-01-16 15:17:08.279162]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.9
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/c9/b7/c9b71b00-a09f-4df1-b87<wbr>4-041820ca8241failed 
                                            [File exists]<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.9
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/c9/b7/c9b71b00-a09f-4df1-b87<wbr>4-041820ca8241failed 
                                            [File exists]&quot; repeated 2
                                            times between [2018-01-16
                                            15:17:08.279162] and
                                            [2018-01-16 15:17:08.279162]</p>
                                          <p>[2018-01-16
                                            15:17:08.279177] E [MSGID:
                                            113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.9
                                            failed<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.4
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/a0/14/a0144df3-8d89-4aed-872<wbr>e-5fef141e9e1efailed 
                                            [File exists]&quot; repeated 6
                                            times between [2018-01-16
                                            15:16:41.471745] and
                                            [2018-01-16 15:16:41.471807]<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.5
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/eb/04/eb044e6e-3a23-40a4-9ce<wbr>1-f13af148eb67failed 
                                            [File exists]&quot; repeated 2
                                            times between [2018-01-16
                                            15:16:42.593392] and
                                            [2018-01-16 15:16:42.593430]<br>
                                            [2018-01-16 15:17:32.229689]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.14
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/53/04/530449fa-d698-4928-a26<wbr>2-9a0234232323failed 
                                            [File exists]<br>
                                            [2018-01-16 15:17:32.229720]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.14
                                            failed<br>
                                            [2018-01-16 15:18:07.154330]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.17
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/81/96/8196dd19-84bc-4c3d-909<wbr>f-8792e9b4929dfailed 
                                            [File exists]<br>
                                            [2018-01-16 15:18:07.154375]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.17
                                            failed<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.14
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/53/04/530449fa-d698-4928-a26<wbr>2-9a0234232323failed 
                                            [File exists]&quot; repeated 7
                                            times between [2018-01-16
                                            15:17:32.229689] and
                                            [2018-01-16 15:17:32.229806]<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.17
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/81/96/8196dd19-84bc-4c3d-909<wbr>f-8792e9b4929dfailed 
                                            [File exists]&quot; repeated 3
                                            times between [2018-01-16
                                            15:18:07.154330] and
                                            [2018-01-16 15:18:07.154357]<br>
                                            [2018-01-16 15:19:23.618794]
                                            W [MSGID: 113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.21
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/6d/02/6d02bd98-83de-43e8-a7a<wbr>f-b1d5f5160403failed 
                                            [File exists]<br>
                                            [2018-01-16 15:19:23.618827]
                                            E [MSGID: 113020]
                                            [posix.c:1485:posix_mknod]
                                            0-gv2a2-posix: setting gfid
                                            on
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.21
                                            failed<br>
                                            The message &quot;W [MSGID:
                                            113096]
                                            [posix-handle.c:770:posix_hand<wbr>le_hard]
                                            0-gv2a2-posix: link
                                            /bricks/brick2/gv2a2/.shard/62<wbr>335cb9-c7b5-4735-a879-59cff93f<wbr>e622.21
                                            -&gt;
                                            /bricks/brick2/gv2a2/.glusterf<wbr>s/6d/02/6d02bd98-83de-43e8-a7a<wbr>f-b1d5f5160403failed 
                                            [File exists]&quot; repeated 3
                                            times between [2018-01-16
                                            15:19:23.618794] and
                                            [2018-01-16 15:19:23.618794]<br>
                                            <br>
                                          </p>
                                          <p>Thank you,<br>
                                          </p>
                                          <br>
                                          <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-cite-prefix">Il
                                            16/01/2018 11:40, Krutika
                                            Dhananjay ha scritto:<br>
                                          </div>
                                          <blockquote type="cite">
                                            <div dir="ltr">
                                              <div>
                                                <div>
                                                  <div>Also to help
                                                    isolate the
                                                    component, could you
                                                    answer these:<br>
                                                    <br>
                                                  </div>
                                                  1. on a different
                                                  volume with shard not
                                                  enabled, do you see
                                                  this issue?<br>
                                                </div>
                                                2. on a plain 3-way
                                                replicated volume (no
                                                arbiter), do you see
                                                this issue?<br>
                                                <br>
                                              </div>
                                              <br>
                                              <div class="gmail_extra"><br>
                                                <div class="gmail_quote">On
                                                  Tue, Jan 16, 2018 at
                                                  4:03 PM, Krutika
                                                  Dhananjay <span dir="ltr">&lt;<a href="mailto:kdhananj@redhat.com" target="_blank">kdhananj@redhat.com</a>&gt;</span>
                                                  wrote:<br>
                                                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                    <div dir="ltr">
                                                      <div>Please share
                                                        the volume-info
                                                        output and the
                                                        logs under
                                                        /var/log/glusterfs/
                                                        from all your
                                                        nodes. for
                                                        investigating
                                                        the issue.</div>
                                                      <span class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927HOEnZb"><font color="#888888">
                                                          <div><br>
                                                          </div>
                                                          -Krutika<br>
                                                        </font></span></div>
                                                    <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927HOEnZb">
                                                      <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927h5">
                                                        <div class="gmail_extra"><br>
                                                          <div class="gmail_quote">On
                                                          Tue, Jan 16,
                                                          2018 at 1:30
                                                          PM, Ing. Luca
                                                          Lazzeroni -
                                                          Trend Servizi
                                                          Srl <span dir="ltr">&lt;<a href="mailto:luca@gvnet.it" target="_blank">luca@gvnet.it</a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
                                                          to everyone.<br>
                                                          <br>
                                                          I&#39;ve got a
                                                          strange
                                                          problem with a
                                                          gluster setup:
                                                          3 nodes with
                                                          Centos 7.4,
                                                          Gluster 3.12.4
                                                          from
                                                          Centos/Gluster
                                                          repositories,
                                                          QEMU-KVM
                                                          version 2.9.0
                                                          (compiled from
                                                          RHEL sources).<br>
                                                          <br>
                                                          I&#39;m running
                                                          volumes in
                                                          replica 3
                                                          arbiter 1 mode
                                                          (but I&#39;ve got
                                                          a volume in
                                                          &quot;pure&quot; replica
                                                          3 mode too).
                                                          I&#39;ve applied
                                                          the &quot;virt&quot;
                                                          group settings
                                                          to my volumes
                                                          since they
                                                          host VM
                                                          images.<br>
                                                          <br>
                                                          If I try to
                                                          install
                                                          something (eg:
                                                          Ubuntu Server
                                                          16.04.3) on a
                                                          VM (and so I
                                                          generate a bit
                                                          of I/O inside
                                                          it) and
                                                          configure KVM
                                                          to access
                                                          gluster volume
                                                          directly (via
                                                          libvirt),
                                                          install fails
                                                          after a while
                                                          because the
                                                          disk content
                                                          is corrupted.
                                                          If I inspect
                                                          the block
                                                          inside the
                                                          disk (by
                                                          accessing the
                                                          image directly
                                                          from outside)
                                                          I can found
                                                          many files
                                                          filled with
                                                          &quot;^@&quot;.<br>
                                                          </blockquote>
                                                          </div>
                                                        </div>
                                                      </div>
                                                    </div>
                                                  </blockquote>
                                                  <div><br>
                                                  </div>
                                                  <div>Also, what
                                                    exactly do you mean
                                                    by accessing the
                                                    image directly from
                                                    outside? Was it from
                                                    the brick
                                                    directories
                                                    directly? Was it
                                                    from the mount point
                                                    of the volume? Could
                                                    you elaborate? Which
                                                    files exactly did
                                                    you check?<br>
                                                  </div>
                                                  <div><br>
                                                  </div>
                                                  <div>-Krutika</div>
                                                  <div><br>
                                                  </div>
                                                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                    <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927HOEnZb">
                                                      <div class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927h5">
                                                        <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br>
                                                          If, instead, I
                                                          configure KVM
                                                          to access VM
                                                          images via a
                                                          FUSE mount,
                                                          everything
                                                          seems to work
                                                          correctly.<br>
                                                          <br>
                                                          Note that the
                                                          problem with
                                                          install is
                                                          verified 100%
                                                          time with
                                                          QCOW2 image,
                                                          while it
                                                          appears only
                                                          after with RAW
                                                          disk images.<br>
                                                          <br>
                                                          Is there
                                                          anyone who
                                                          experienced
                                                          the same
                                                          problem ?<br>
                                                          <br>
                                                          Thank you,<span class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927m_-4139169106555235646HOEnZb"><font color="#888888"><br>
                                                          <br>
                                                          <br>
                                                          -- <br>
                                                          Ing. Luca
                                                          Lazzeroni<br>
                                                          Responsabile
                                                          Ricerca e
                                                          Sviluppo<br>
                                                          Trend Servizi
                                                          Srl<br>
                                                          Tel:
                                                          0376/631761<br>
                                                          Web: <a href="https://www.trendservizi.it" rel="noreferrer" target="_blank">https://www.trendservizi.it</a><br>
                                                          <br>
______________________________<wbr>_________________<br>
                                                          Gluster-users
                                                          mailing list<br>
                                                          <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
                                                          <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br>
                                                          </font></span></blockquote>
                                                          </div>
                                                          <br>
                                                        </div>
                                                      </div>
                                                    </div>
                                                  </blockquote>
                                                </div>
                                                <br>
                                              </div>
                                            </div>
                                          </blockquote>
                                          <br>
                                          <pre class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
                                          <br>
                                          <fieldset class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927mimeAttachmentHeader"></fieldset>
                                          <br>
                                          <pre>______________________________<wbr>_________________
Gluster-users mailing list
<a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a></pre>
                                        </blockquote>
                                        <br>
                                        <pre class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
                                        <br>
                                        <fieldset class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927mimeAttachmentHeader"></fieldset>
                                        <br>
                                        <pre>______________________________<wbr>_________________
Gluster-users mailing list
<a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a></pre>
                                      </blockquote>
                                      <br>
                                      <pre class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859m_-2254898158214265998m_-2427999713053267927moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
                                    </div>
                                  </div>
                                </div>
                                <br>
                                ______________________________<wbr>_________________<br>
                                Gluster-users mailing list<br>
                                <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
                                <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br>
                              </blockquote>
                            </div>
                            <br>
                          </div>
                        </div>
                      </blockquote>
                      <br>
                      <pre class="m_-592553901259022859m_-2254898158214265998moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859m_-2254898158214265998moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
                    </div>
                  </div>
                </div>
              </blockquote>
            </div>
            <br>
          </div>
        </blockquote>
        <br>
        <pre class="m_-592553901259022859moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
        <br>
        <fieldset class="m_-592553901259022859mimeAttachmentHeader"></fieldset>
        <br>
        <pre>______________________________<wbr>_________________
Gluster-users mailing list
<a class="m_-592553901259022859moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="m_-592553901259022859moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a></pre>
      </blockquote>
      <br>
      <pre class="m_-592553901259022859moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
      <br>
      <fieldset class="m_-592553901259022859mimeAttachmentHeader"></fieldset>
      <br>
      <pre>______________________________<wbr>_________________
Gluster-users mailing list
<a class="m_-592553901259022859moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="m_-592553901259022859moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a></pre>
    </blockquote>
    <br>
    <pre class="m_-592553901259022859moz-signature" cols="72">-- 
Ing. Luca Lazzeroni
Responsabile Ricerca e Sviluppo
Trend Servizi Srl
Tel: 0376/631761
Web: <a class="m_-592553901259022859moz-txt-link-freetext" href="https://www.trendservizi.it" target="_blank">https://www.trendservizi.it</a></pre>
  </div></div></div>

<br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>