<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Yes, the shds did pick up the file for healing (I saw messages
      like " got entry: 1985e233-d5ee-4e3e-a51a-cf0b5f9f2aea") but no
      error afterwards.</p>
    <p>Anyway I reproduced it by manually setting the afr.dirty bit for
      a zero byte file on all 3 bricks. Since there are no afr pending
      xattrs indicating good/bad copies and all files are zero bytes,
      the data self-heal algorithm just picks the file with the latest
      ctime as source. In your case that was the arbiter brick. In the
      code, there is a check to prevent data heals if arbiter is the
      source. So heal was not happening and the entries were not removed
      from heal-info output.</p>
    <p>Perhaps we should add a check in the code to just remove the
      entries from heal-info if size is zero bytes in all bricks.<br>
    </p>
    <p>-Ravi<br>
    </p>
    <br>
    <div class="moz-cite-prefix">On 08/25/2017 06:33 PM, mabi wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:JCxdnUAL4IuxnsQyvvuflp3aAGsDlc6oA47KzMgshWxaJUF0o9O9rA1ftjclR4L-3RCLXjIj9Poajh21NhWQJ8rvJlXX1qbGI6RcHiaFzpc=@protonmail.ch">
      <div>Hi Ravi,<br>
      </div>
      <div><br>
      </div>
      <div>Did you get a chance to have a look at the log files I have
        attached in my last mail?<br>
      </div>
      <div><br>
      </div>
      <div>Best,<br>
      </div>
      <div>Mabi<br>
      </div>
      <div class="protonmail_signature_block
        protonmail_signature_block-empty">
        <div class="protonmail_signature_block-user
          protonmail_signature_block-empty">
          <div><br>
          </div>
        </div>
        <div class="protonmail_signature_block-proton
          protonmail_signature_block-empty"><br>
        </div>
      </div>
      <div><br>
      </div>
      <blockquote type="cite" class="protonmail_quote">
        <div>-------- Original Message --------<br>
        </div>
        <div>Subject: Re: [Gluster-users] self-heal not working<br>
        </div>
        <div>Local Time: August 24, 2017 12:08 PM<br>
        </div>
        <div>UTC Time: August 24, 2017 10:08 AM<br>
        </div>
        <div>From: <a class="moz-txt-link-abbreviated" href="mailto:mabi@protonmail.ch">mabi@protonmail.ch</a><br>
        </div>
        <div>To: Ravishankar N <a class="moz-txt-link-rfc2396E" href="mailto:ravishankar@redhat.com">&lt;ravishankar@redhat.com&gt;</a><br>
        </div>
        <div>Ben Turner <a class="moz-txt-link-rfc2396E" href="mailto:bturner@redhat.com">&lt;bturner@redhat.com&gt;</a>, Gluster Users
          <a class="moz-txt-link-rfc2396E" href="mailto:gluster-users@gluster.org">&lt;gluster-users@gluster.org&gt;</a><br>
        </div>
        <div><br>
        </div>
        <div>Thanks for confirming the command. I have now enabled DEBUG
          client-log-level, run a heal and then attached the glustershd
          log files of all 3 nodes in this mail.<br>
        </div>
        <div><br>
        </div>
        <div>The volume concerned is called myvol-pro, the other 3
          volumes have no problem so far.<br>
        </div>
        <div><br>
        </div>
        <div>Also note that in the mean time it looks like the file has
          been deleted by the user and as such the heal info command
          does not show the file name anymore but just is GFID which is:
          <br>
        </div>
        <div><br>
        </div>
        <div>gfid:1985e233-d5ee-4e3e-a51a-cf0b5f9f2aea<br>
        </div>
        <div><br>
        </div>
        <div class="protonmail_signature_block
          protonmail_signature_block-empty">
          <div class="protonmail_signature_block-user
            protonmail_signature_block-empty">
            <div><br>
            </div>
          </div>
          <div class="protonmail_signature_block-proton
            protonmail_signature_block-empty"><br>
          </div>
        </div>
        <div>Hope that helps for debugging this issue.<br>
        </div>
        <div><br>
        </div>
        <blockquote class="protonmail_quote" type="cite">
          <div>-------- Original Message --------<br>
          </div>
          <div>Subject: Re: [Gluster-users] self-heal not working<br>
          </div>
          <div>Local Time: August 24, 2017 5:58 AM<br>
          </div>
          <div>UTC Time: August 24, 2017 3:58 AM<br>
          </div>
          <div>From: <a class="moz-txt-link-abbreviated" href="mailto:ravishankar@redhat.com">ravishankar@redhat.com</a><br>
          </div>
          <div>To: mabi <a class="moz-txt-link-rfc2396E" href="mailto:mabi@protonmail.ch">&lt;mabi@protonmail.ch&gt;</a><br>
          </div>
          <div>Ben Turner <a class="moz-txt-link-rfc2396E" href="mailto:bturner@redhat.com">&lt;bturner@redhat.com&gt;</a>, Gluster Users
            <a class="moz-txt-link-rfc2396E" href="mailto:gluster-users@gluster.org">&lt;gluster-users@gluster.org&gt;</a><br>
          </div>
          <div><br>
          </div>
          <div><br>
          </div>
          <p>Unlikely. In your case only the afr.dirty is set, not the
            afr.volname-client-xx xattr.<br>
          </p>
          <div>`gluster volume set myvolume diagnostics.client-log-level
            DEBUG` is right.<br>
          </div>
          <div><br>
          </div>
          <div><br>
          </div>
          <div class="moz-cite-prefix">On 08/23/2017 10:31 PM, mabi
            wrote:<br>
          </div>
          <blockquote type="cite">
            <div>I just saw the following bug which was fixed in 3.8.15:<br>
            </div>
            <div><br>
            </div>
            <div><a
                href="https://bugzilla.redhat.com/show_bug.cgi?id=1471613"
                moz-do-not-send="true">https://bugzilla.redhat.com/show_bug.cgi?id=1471613</a><br>
            </div>
            <div><br>
            </div>
            <div>Is it possible that the problem I described in this
              post is related to that bug?<br>
            </div>
            <div class="protonmail_signature_block
              protonmail_signature_block-empty">
              <div class="protonmail_signature_block-user
                protonmail_signature_block-empty">
                <div><br>
                </div>
              </div>
              <div class="protonmail_signature_block-proton
                protonmail_signature_block-empty"><br>
              </div>
            </div>
            <div><br>
            </div>
            <blockquote type="cite" class="protonmail_quote">
              <div>-------- Original Message --------<br>
              </div>
              <div>Subject: Re: [Gluster-users] self-heal not working<br>
              </div>
              <div>Local Time: August 22, 2017 11:51 AM<br>
              </div>
              <div>UTC Time: August 22, 2017 9:51 AM<br>
              </div>
              <div>From: <a href="mailto:ravishankar@redhat.com"
                  class="moz-txt-link-abbreviated"
                  moz-do-not-send="true">ravishankar@redhat.com</a><br>
              </div>
              <div>To: mabi <a href="mailto:mabi@protonmail.ch"
                  class="moz-txt-link-rfc2396E" moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a><br>
              </div>
              <div>Ben Turner <a href="mailto:bturner@redhat.com"
                  class="moz-txt-link-rfc2396E" moz-do-not-send="true">&lt;bturner@redhat.com&gt;</a>,
                Gluster Users <a
                  href="mailto:gluster-users@gluster.org"
                  class="moz-txt-link-rfc2396E" moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <p><br>
              </p>
              <div><br>
              </div>
              <div class="moz-cite-prefix">On 08/22/2017 02:30 PM, mabi
                wrote:<br>
              </div>
              <blockquote type="cite">
                <div>Thanks for the additional hints, I have the
                  following 2 questions first:<br>
                </div>
                <div><br>
                </div>
                <div>- In order to launch the index heal is the
                  following command correct:<br>
                </div>
                <div>gluster volume heal myvolume<br>
                </div>
                <div><br>
                </div>
              </blockquote>
              <div>Yes<br>
              </div>
              <div><br>
              </div>
              <blockquote type="cite">
                <div>- If I run a "volume start force" will it have any
                  short disruptions on my clients which mount the volume
                  through FUSE? If yes, how long? This is a production
                  system that's why I am asking.<br>
                </div>
                <div><br>
                </div>
                <div class="protonmail_signature_block
                  protonmail_signature_block-empty">
                  <div class="protonmail_signature_block-user
                    protonmail_signature_block-empty">
                    <div><br>
                    </div>
                  </div>
                </div>
              </blockquote>
              <div>No. You can actually create a test volume on  your
                personal linux box to try these kinds of things without
                needing multiple machines. This is how we develop and
                test our patches :)<br>
              </div>
              <div>'gluster volume create testvol replica 3
                /home/mabi/bricks/brick{1..3} force` and so on. <br>
              </div>
              <div><br>
              </div>
              <div>HTH,<br>
              </div>
              <div>Ravi <br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <blockquote type="cite">
                <div><br>
                </div>
                <blockquote class="protonmail_quote" type="cite">
                  <div>-------- Original Message --------<br>
                  </div>
                  <div>Subject: Re: [Gluster-users] self-heal not
                    working<br>
                  </div>
                  <div>Local Time: August 22, 2017 6:26 AM<br>
                  </div>
                  <div>UTC Time: August 22, 2017 4:26 AM<br>
                  </div>
                  <div>From: <a class="moz-txt-link-abbreviated"
                      href="mailto:ravishankar@redhat.com"
                      moz-do-not-send="true">ravishankar@redhat.com</a><br>
                  </div>
                  <div>To: mabi <a class="moz-txt-link-rfc2396E"
                      href="mailto:mabi@protonmail.ch"
                      moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a>,
                    Ben Turner <a class="moz-txt-link-rfc2396E"
                      href="mailto:bturner@redhat.com"
                      moz-do-not-send="true">&lt;bturner@redhat.com&gt;</a><br>
                  </div>
                  <div>Gluster Users <a class="moz-txt-link-rfc2396E"
                      href="mailto:gluster-users@gluster.org"
                      moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <p>Explore the following:<br>
                  </p>
                  <p>- Launch index heal and look at the glustershd logs
                    of all bricks for possible errors<br>
                  </p>
                  <p>- See if the glustershd in each node is connected
                    to all bricks.<br>
                  </p>
                  <p>- If not try to restart shd by `volume start force`<br>
                  </p>
                  <p>- Launch index heal again and try.<br>
                  </p>
                  <div>- Try debugging the shd log by setting
                    client-log-level to DEBUG temporarily.<br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div class="moz-cite-prefix">On 08/22/2017 03:19 AM,
                    mabi wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div>Sure, it doesn't look like a split brain based
                      on the output:<br>
                    </div>
                    <div><br>
                    </div>
                    <div>Brick node1.domain.tld:/data/myvolume/brick<br>
                    </div>
                    <div>Status: Connected<br>
                    </div>
                    <div>Number of entries in split-brain: 0<br>
                    </div>
                    <div><br>
                    </div>
                    <div>Brick node2.domain.tld:/data/myvolume/brick<br>
                    </div>
                    <div>Status: Connected<br>
                    </div>
                    <div>Number of entries in split-brain: 0<br>
                    </div>
                    <div><br>
                    </div>
                    <div>Brick
                      node3.domain.tld:/srv/glusterfs/myvolume/brick<br>
                    </div>
                    <div>Status: Connected<br>
                    </div>
                    <div>Number of entries in split-brain: 0<br>
                    </div>
                    <div><br>
                    </div>
                    <div class="protonmail_signature_block
                      protonmail_signature_block-empty">
                      <div class="protonmail_signature_block-user
                        protonmail_signature_block-empty">
                        <div><br>
                        </div>
                      </div>
                      <div class="protonmail_signature_block-proton
                        protonmail_signature_block-empty"><br>
                      </div>
                    </div>
                    <div><br>
                    </div>
                    <blockquote type="cite" class="protonmail_quote">
                      <div>-------- Original Message --------<br>
                      </div>
                      <div>Subject: Re: [Gluster-users] self-heal not
                        working<br>
                      </div>
                      <div>Local Time: August 21, 2017 11:35 PM<br>
                      </div>
                      <div>UTC Time: August 21, 2017 9:35 PM<br>
                      </div>
                      <div>From: <a href="mailto:bturner@redhat.com"
                          class="moz-txt-link-abbreviated"
                          moz-do-not-send="true">bturner@redhat.com</a><br>
                      </div>
                      <div>To: mabi <a href="mailto:mabi@protonmail.ch"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a><br>
                      </div>
                      <div>Gluster Users <a
                          href="mailto:gluster-users@gluster.org"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
                      </div>
                      <div><br>
                      </div>
                      <div>Can you also provide:<br>
                      </div>
                      <div><br>
                      </div>
                      <div>gluster v heal &lt;my vol&gt; info
                        split-brain<br>
                      </div>
                      <div><br>
                      </div>
                      <div>If it is split brain just delete the
                        incorrect file from the brick and run heal
                        again. I haven"t tried this with arbiter but I
                        assume the process is the same.<br>
                      </div>
                      <div><br>
                      </div>
                      <div>-b<br>
                      </div>
                      <div><br>
                      </div>
                      <div>----- Original Message -----<br>
                      </div>
                      <div>&gt; From: "mabi" <a
                          href="mailto:mabi@protonmail.ch"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a><br>
                      </div>
                      <div>&gt; To: "Ben Turner" <a
                          href="mailto:bturner@redhat.com"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;bturner@redhat.com&gt;</a><br>
                      </div>
                      <div>&gt; Cc: "Gluster Users" <a
                          href="mailto:gluster-users@gluster.org"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
                      </div>
                      <div>&gt; Sent: Monday, August 21, 2017 4:55:59 PM<br>
                      </div>
                      <div>&gt; Subject: Re: [Gluster-users] self-heal
                        not working<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; Hi Ben,<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; So it is really a 0 kBytes file
                        everywhere (all nodes including the arbiter<br>
                      </div>
                      <div>&gt; and from the client).<br>
                      </div>
                      <div>&gt; Here below you will find the output you
                        requested. Hopefully that will help<br>
                      </div>
                      <div>&gt; to find out why this specific file is
                        not healing... Let me know if you need<br>
                      </div>
                      <div>&gt; any more information. Btw node3 is my
                        arbiter node.<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; NODE1:<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; STAT:<br>
                      </div>
                      <div>&gt; File:<br>
                      </div>
                      <div>&gt;
‘/data/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png’<br>
                      </div>
                      <div>&gt; Size: 0 Blocks: 38 IO Block: 131072
                        regular empty file<br>
                      </div>
                      <div>&gt; Device: 24h/36d Inode: 10033884 Links: 2<br>
                      </div>
                      <div>&gt; Access: (0644/-rw-r--r--) Uid: (
                        33/www-data) Gid: ( 33/www-data)<br>
                      </div>
                      <div>&gt; Access: 2017-08-14 17:04:55.530681000
                        +0200<br>
                      </div>
                      <div>&gt; Modify: 2017-08-14 17:11:46.407404779
                        +0200<br>
                      </div>
                      <div>&gt; Change: 2017-08-14 17:11:46.407404779
                        +0200<br>
                      </div>
                      <div>&gt; Birth: -<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; GETFATTR:<br>
                      </div>
                      <div>&gt; trusted.afr.dirty=0sAAAAAQAAAAAAAAAA<br>
                      </div>
                      <div>&gt;
                        trusted.bit-rot.version=0sAgAAAAAAAABZhuknAAlJAg==<br>
                      </div>
                      <div>&gt; trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==<br>
                      </div>
                      <div>&gt;
trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOyo=<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; NODE2:<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; STAT:<br>
                      </div>
                      <div>&gt; File:<br>
                      </div>
                      <div>&gt;
‘/data/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png’<br>
                      </div>
                      <div>&gt; Size: 0 Blocks: 38 IO Block: 131072
                        regular empty file<br>
                      </div>
                      <div>&gt; Device: 26h/38d Inode: 10031330 Links: 2<br>
                      </div>
                      <div>&gt; Access: (0644/-rw-r--r--) Uid: (
                        33/www-data) Gid: ( 33/www-data)<br>
                      </div>
                      <div>&gt; Access: 2017-08-14 17:04:55.530681000
                        +0200<br>
                      </div>
                      <div>&gt; Modify: 2017-08-14 17:11:46.403704181
                        +0200<br>
                      </div>
                      <div>&gt; Change: 2017-08-14 17:11:46.403704181
                        +0200<br>
                      </div>
                      <div>&gt; Birth: -<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; GETFATTR:<br>
                      </div>
                      <div>&gt; trusted.afr.dirty=0sAAAAAQAAAAAAAAAA<br>
                      </div>
                      <div>&gt;
                        trusted.bit-rot.version=0sAgAAAAAAAABZhu6wAA8Hpw==<br>
                      </div>
                      <div>&gt; trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==<br>
                      </div>
                      <div>&gt;
trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOVE=<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; NODE3:<br>
                      </div>
                      <div>&gt; STAT:<br>
                      </div>
                      <div>&gt; File:<br>
                      </div>
                      <div>&gt;
/srv/glusterfs/myvolume/brick/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png<br>
                      </div>
                      <div>&gt; Size: 0 Blocks: 0 IO Block: 4096 regular
                        empty file<br>
                      </div>
                      <div>&gt; Device: ca11h/51729d Inode: 405208959
                        Links: 2<br>
                      </div>
                      <div>&gt; Access: (0644/-rw-r--r--) Uid: (
                        33/www-data) Gid: ( 33/www-data)<br>
                      </div>
                      <div>&gt; Access: 2017-08-14 17:04:55.530681000
                        +0200<br>
                      </div>
                      <div>&gt; Modify: 2017-08-14 17:04:55.530681000
                        +0200<br>
                      </div>
                      <div>&gt; Change: 2017-08-14 17:11:46.604380051
                        +0200<br>
                      </div>
                      <div>&gt; Birth: -<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; GETFATTR:<br>
                      </div>
                      <div>&gt; trusted.afr.dirty=0sAAAAAQAAAAAAAAAA<br>
                      </div>
                      <div>&gt;
                        trusted.bit-rot.version=0sAgAAAAAAAABZe6ejAAKPAg==<br>
                      </div>
                      <div>&gt; trusted.gfid=0sGYXiM9XuTj6lGs8LX58q6g==<br>
                      </div>
                      <div>&gt;
trusted.glusterfs.d99af2fa-439b-4a21-bf3a-38f3849f87ec.xtime=0sWZG9sgAGOc4=<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; CLIENT GLUSTER MOUNT:<br>
                      </div>
                      <div>&gt; STAT:<br>
                      </div>
                      <div>&gt; File:<br>
                      </div>
                      <div>&gt;
                        "/mnt/myvolume/data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png"<br>
                      </div>
                      <div>&gt; Size: 0 Blocks: 0 IO Block: 131072
                        regular empty file<br>
                      </div>
                      <div>&gt; Device: 1eh/30d Inode:
                        11897049013408443114 Links: 1<br>
                      </div>
                      <div>&gt; Access: (0644/-rw-r--r--) Uid: (
                        33/www-data) Gid: ( 33/www-data)<br>
                      </div>
                      <div>&gt; Access: 2017-08-14 17:04:55.530681000
                        +0200<br>
                      </div>
                      <div>&gt; Modify: 2017-08-14 17:11:46.407404779
                        +0200<br>
                      </div>
                      <div>&gt; Change: 2017-08-14 17:11:46.407404779
                        +0200<br>
                      </div>
                      <div>&gt; Birth: -<br>
                      </div>
                      <div>&gt; <br>
                      </div>
                      <div>&gt; &gt; -------- Original Message --------<br>
                      </div>
                      <div>&gt; &gt; Subject: Re: [Gluster-users]
                        self-heal not working<br>
                      </div>
                      <div>&gt; &gt; Local Time: August 21, 2017 9:34 PM<br>
                      </div>
                      <div>&gt; &gt; UTC Time: August 21, 2017 7:34 PM<br>
                      </div>
                      <div>&gt; &gt; From: <a
                          href="mailto:bturner@redhat.com"
                          class="moz-txt-link-abbreviated"
                          moz-do-not-send="true">bturner@redhat.com</a><br>
                      </div>
                      <div>&gt; &gt; To: mabi <a
                          href="mailto:mabi@protonmail.ch"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a><br>
                      </div>
                      <div>&gt; &gt; Gluster Users <a
                          href="mailto:gluster-users@gluster.org"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt; ----- Original Message -----<br>
                      </div>
                      <div>&gt; &gt;&gt; From: "mabi" <a
                          href="mailto:mabi@protonmail.ch"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;mabi@protonmail.ch&gt;</a><br>
                      </div>
                      <div>&gt; &gt;&gt; To: "Gluster Users" <a
                          href="mailto:gluster-users@gluster.org"
                          class="moz-txt-link-rfc2396E"
                          moz-do-not-send="true">&lt;gluster-users@gluster.org&gt;</a><br>
                      </div>
                      <div>&gt; &gt;&gt; Sent: Monday, August 21, 2017
                        9:28:24 AM<br>
                      </div>
                      <div>&gt; &gt;&gt; Subject: [Gluster-users]
                        self-heal not working<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; Hi,<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; I have a replicat 2 with
                        arbiter GlusterFS 3.8.11 cluster and there is<br>
                      </div>
                      <div>&gt; &gt;&gt; currently one file listed to be
                        healed as you can see below but never gets<br>
                      </div>
                      <div>&gt; &gt;&gt; healed by the self-heal daemon:<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; Brick
                        node1.domain.tld:/data/myvolume/brick<br>
                      </div>
                      <div>&gt; &gt;&gt;
                        /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png<br>
                      </div>
                      <div>&gt; &gt;&gt; Status: Connected<br>
                      </div>
                      <div>&gt; &gt;&gt; Number of entries: 1<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; Brick
                        node2.domain.tld:/data/myvolume/brick<br>
                      </div>
                      <div>&gt; &gt;&gt;
                        /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png<br>
                      </div>
                      <div>&gt; &gt;&gt; Status: Connected<br>
                      </div>
                      <div>&gt; &gt;&gt; Number of entries: 1<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; Brick
                        node3.domain.tld:/srv/glusterfs/myvolume/brick<br>
                      </div>
                      <div>&gt; &gt;&gt;
                        /data/appdata_ocpom4nckwru/preview/1344699/64-64-crop.png<br>
                      </div>
                      <div>&gt; &gt;&gt; Status: Connected<br>
                      </div>
                      <div>&gt; &gt;&gt; Number of entries: 1<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; As once recommended on this
                        mailing list I have mounted that glusterfs<br>
                      </div>
                      <div>&gt; &gt;&gt; volume<br>
                      </div>
                      <div>&gt; &gt;&gt; temporarily through
                        fuse/glusterfs and ran a "stat" on that file
                        which is<br>
                      </div>
                      <div>&gt; &gt;&gt; listed above but nothing
                        happened.<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; The file itself is available on
                        all 3 nodes/bricks but on the last node it<br>
                      </div>
                      <div>&gt; &gt;&gt; has a different date. By the
                        way this file is 0 kBytes big. Is that maybe<br>
                      </div>
                      <div>&gt; &gt;&gt; the reason why the self-heal
                        does not work?<br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt; Is the file actually 0 bytes or is
                        it just 0 bytes on the arbiter(0 bytes<br>
                      </div>
                      <div>&gt; &gt; are expected on the arbiter, it
                        just stores metadata)? Can you send us the<br>
                      </div>
                      <div>&gt; &gt; output from stat on all 3 nodes:<br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt; $ stat &lt;file on back end
                        brick&gt;<br>
                      </div>
                      <div>&gt; &gt; $ getfattr -d -m - &lt;file on back
                        end brick&gt;<br>
                      </div>
                      <div>&gt; &gt; $ stat &lt;file from gluster
                        mount&gt;<br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt; Lets see what things look like on
                        the back end, it should tell us why<br>
                      </div>
                      <div>&gt; &gt; healing is failing.<br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt; -b<br>
                      </div>
                      <div>&gt; &gt;<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; And how can I now make this
                        file to heal?<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt; Thanks,<br>
                      </div>
                      <div>&gt; &gt;&gt; Mabi<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt;<br>
                      </div>
                      <div>&gt; &gt;&gt;
                        _______________________________________________<br>
                      </div>
                      <div>&gt; &gt;&gt; Gluster-users mailing list<br>
                      </div>
                      <div>&gt; &gt;&gt; <a
                          href="mailto:Gluster-users@gluster.org"
                          class="moz-txt-link-abbreviated"
                          moz-do-not-send="true">Gluster-users@gluster.org</a><br>
                      </div>
                      <div>&gt; &gt;&gt; <a
                          href="http://lists.gluster.org/mailman/listinfo/gluster-users"
                          class="moz-txt-link-freetext"
                          moz-do-not-send="true">http://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
                      </div>
                    </blockquote>
                    <div><br>
                    </div>
                    <div><br>
                    </div>
                    <div><br>
                    </div>
                    <pre wrap="">_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" class="moz-txt-link-abbreviated" moz-do-not-send="true">Gluster-users@gluster.org</a>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" class="moz-txt-link-freetext" moz-do-not-send="true">http://lists.gluster.org/mailman/listinfo/gluster-users</a>


</pre>
                  </blockquote>
                </blockquote>
                <div><br>
                </div>
              </blockquote>
            </blockquote>
            <div><br>
            </div>
          </blockquote>
        </blockquote>
        <div><br>
        </div>
      </blockquote>
      <div><br>
      </div>
    </blockquote>
    <br>
  </body>
</html>