<div dir="ltr">Please note, the file needs to be copied to all nodes.</div><div class="gmail_extra"><br><div class="gmail_quote">On 1 February 2018 at 09:31, Nithya Balachandran <span dir="ltr">&lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div>I think we have a workaround for until we have a fix in the code. The following worked on my system.<div><br></div><div>Copy the attached file to <b>/usr/lib*/glusterfs/</b><span style="font-family:Tahoma,sans-serif;font-size:12.8px"><b><font color="#000000">3.12.4</font></b></span><b>/<wbr>filter/</b>. (You might need to create the <b>filter</b> directory in this path.)</div><div>Make sure the file has execute permissions. On my system:</div><div><br></div><div><div><font face="garamond, serif">[root@rhgsserver1 fuse2]# cd /usr/lib/glusterfs/3.12.5/</font></div><div><font face="garamond, serif">[root@rhgsserver1 3.12.5]# l</font></div><div><font face="garamond, serif">total 4.0K</font></div><div><font face="garamond, serif">drwxr-xr-x.  2 root root   64 Feb  1 08:56 auth</font></div><div><font face="garamond, serif">drwxr-xr-x.  2 root root   34 Feb  1 09:12 filter</font></div><div><font face="garamond, serif">drwxr-xr-x.  2 root root   66 Feb  1 08:55 rpc-transport</font></div><div><font face="garamond, serif">drwxr-xr-x. 13 root root 4.0K Feb  1 08:57 xlator</font></div></div><div><font face="garamond, serif"><br></font></div><div><div><font face="garamond, serif">[root@rhgsserver1 fuse2]# cd filter</font></div><div><font face="garamond, serif">[root@rhgsserver1 filter]# pwd<br></font></div></div><div><div><font face="garamond, serif">/usr/lib/glusterfs/3.12.5/<wbr>filter</font></div><div><font face="garamond, serif">[root@rhgsserver1 filter]# ll</font></div></div><div><div><font face="garamond, serif">total 4</font></div><div><font face="garamond, serif">-rwxr-xr-x. 1 root root 95 Feb  1 09:12 shared-brick-count.sh</font></div></div><div><br></div><div>Rerun:</div><span class=""><div><span style="font-size:12.8px"><b>gluster v set dataeng cluster.min-free-inodes 6%</b></span></div><div><span style="font-size:12.8px"><b><br></b></span></div><div><span style="font-size:12.8px"><b><br></b></span></div></span><div>Check the .vol files to see if the value has changed. It should now be 1.You do not need to restart the volume.</div><div><br></div><div>See [1] for more details.</div><div><br></div><div>Regards,</div><div>Nithya</div><div><br></div><div><br></div><div><br></div><div>[1] <a href="http://docs.gluster.org/en/latest/Administrator%20Guide/GlusterFS%20Filter/" target="_blank">http://docs.gluster.org/<wbr>en/latest/Administrator%<wbr>20Guide/GlusterFS%20Filter/</a></div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 31 January 2018 at 23:39, Freer, Eva B. <span dir="ltr">&lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">







<div bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="m_3617041154886078633m_5832099881936252734WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Amar,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Thanks for your prompt reply. No, I do not plan to fix the code and re-compile. I was hoping it could be fixed with setting the shared-brick-count or some other
 option. Since this is a production system, we will wait until a fix is in a release.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks,<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva     (865) 574-6894</span><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"><u></u> <u></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Amar Tumballi &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;<br>
<b>Date: </b>Wednesday, January 31, 2018 at 12:15 PM<br>
<b>To: </b>Eva Freer &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;<br>
<b>Cc: </b>Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;, &quot;Greene, Tami McFarlin&quot; &lt;<a href="mailto:greenet@ornl.gov" target="_blank">greenet@ornl.gov</a>&gt;, &quot;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&quot; &lt;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&gt;</span></p><div><div class="m_3617041154886078633h5"><br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4<u></u><u></u></div></div><p></p>
</div><div><div class="m_3617041154886078633h5">
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Hi Freer, <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Our analysis is that this issue is caused by <a href="https://review.gluster.org/17618" target="_blank">https://review.gluster.org/<wbr>17618</a>. Specifically, in &#39;gd_set_shared_brick_count()&#39; from <a href="https://review.gluster.org/#/c/17618/9/xlators/mgmt/glusterd/src/glusterd-utils.c" target="_blank">https://review.gluster.or<wbr>g/#/c/17618/9/xlators/mgmt/glu<wbr>sterd/src/glusterd-utils.c</a>.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">But even if we fix it today, I don&#39;t think we have a release planned immediately for shipping this. Are you planning to fix the code and re-compile?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Amar<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">On Wed, Jan 31, 2018 at 10:00 PM, Freer, Eva B. &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Nithya,</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">I will be out of the office for ~10 days starting tomorrow. Is there any way we could possibly resolve it today?</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks,</span><u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva     (865) 574-6894</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:1.0in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;<br>
<b>Date: </b>Wednesday, January 31, 2018 at 11:26 AM</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<b>To: </b>Eva Freer &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;<br>
<b>Cc: </b>&quot;Greene, Tami McFarlin&quot; &lt;<a href="mailto:greenet@ornl.gov" target="_blank">greenet@ornl.gov</a>&gt;, &quot;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&quot; &lt;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&gt;,
 Amar Tumballi &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;<br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4<u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
On 31 January 2018 at 21:50, Freer, Eva B. &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">The values for shared-brick-count are still the same. I did not re-start the volume after setting the cluster.min-free-inodes to 6%. Do I need to restart it?</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
That is not necessary. Let me get back to you on this tomorrow.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
Nithya<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks,</span><u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:1.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva     (865) 574-6894</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:1.5in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;<br>
<b>Date: </b>Wednesday, January 31, 2018 at 11:14 AM<br>
<b>To: </b>Eva Freer &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;<br>
<b>Cc: </b>&quot;Greene, Tami McFarlin&quot; &lt;<a href="mailto:greenet@ornl.gov" target="_blank">greenet@ornl.gov</a>&gt;, &quot;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&quot; &lt;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&gt;,
 Amar Tumballi &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.0in">
<br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4<u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
On 31 January 2018 at 21:34, Freer, Eva B. &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Nithya,</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Responding to an earlier question: Before the upgrade, we were at 3.103 on these servers, but some of the clients were 3.7.6. From below, does this mean that “shared-brick-count” needs
 to be set to 1 for all bricks.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">All of the bricks are on separate xfs partitions composed hardware of RAID 6 volumes. LVM is not used. The current setting for cluster.min-free-inodes was 5%. I changed it to 6% per
 your instructions below. The df output is still the same, but I haven’t done the</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in;text-indent:.5in">
<i><span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">find /var/lib/glusterd/vols -type f|xargs sed -i -e &#39;s/option shared-brick-count [0-9]*/option shared-brick-count 1/g&#39;</span></i><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Should I go ahead and do this?</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
Can you check if the values have been changed in the .vol files before you try this? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
These files will be regenerated every time the volume is changed so changing them directly may not be permanent. I was hoping setting the cluster.min-free-inodes would have corrected this automatically and helped us figure out what was happening as we have
 not managed to reproduce this issue yet. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Output of stat –f for all the bricks:</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">[root@jacen ~]# stat -f /bricks/data_A*</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_A1&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 80100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 4530515093 Available: 4530515093</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250028064</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_A2&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 81100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 3653183901 Available: 3653183901</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250029262</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_A3&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 82100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 15134840607 Available: 15134840607</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250128031</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_A4&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 83100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 15626461604 Available: 15626461604</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250153857</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">[root@jaina dataeng]# stat -f /bricks/data_B*</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_B1&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 80100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 5689640723 Available: 5689640723</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250047934</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_B2&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 81100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 6623312785 Available: 6623312785</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250048131</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_B3&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 82100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 15106888485 Available: 15106888485</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250122139</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">  File: &quot;/bricks/data_B4&quot;</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">    ID: 83100000000 Namelen: 255     Type: xfs</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Block size: 4096       Fundamental block size: 4096</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Blocks: Total: 15626471424 Free: 15626461604 Available: 15626461604</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Consolas&quot;,sans-serif;color:black">Inodes: Total: 1250159424 Free: 1250153857</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks,</span><u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva     (865) 574-6894</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1.5in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:2.0in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;<br>
<b>Date: </b>Wednesday, January 31, 2018 at 10:46 AM<br>
<b>To: </b>Eva Freer &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;, &quot;Greene, Tami McFarlin&quot; &lt;<a href="mailto:greenet@ornl.gov" target="_blank">greenet@ornl.gov</a>&gt;<br>
<b>Cc: </b>Amar Tumballi &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:1.5in">
<br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4<u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
Thank you Eva. <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
>From the files you sent:<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jacen.bricks-data_A1-d<wbr>ataeng.vol:    option shared-brick-count 2<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jacen.bricks-data_A2-d<wbr>ataeng.vol:    option shared-brick-count 2<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jacen.bricks-data_A3-d<wbr>ataeng.vol:    option shared-brick-count 1<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jacen.bricks-data_A4-d<wbr>ataeng.vol:    option shared-brick-count 1<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jaina.bricks-data_B1-d<wbr>ataeng.vol:    option shared-brick-count 0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jaina.bricks-data_B2-d<wbr>ataeng.vol:    option shared-brick-count 0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jaina.bricks-data_B3-d<wbr>ataeng.vol:    option shared-brick-count 0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
dataeng.jaina.bricks-data_B4-d<wbr>ataeng.vol:    option shared-brick-count 0<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
Are all of these bricks on separate Filesystem partitions? If yes, can you please try running the following on one of the gluster nodes and see if the df output works post that?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
gluster v set dataeng cluster.min-free-inodes 6%<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
If it doesn;t work, please send us the stat -f output for each brick.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
Nithya<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
On 31 January 2018 at 20:41, Freer, Eva B. &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Nithya,</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">The file for one of the servers is attached.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks,</span><u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva     (865) 574-6894</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.0in">
<span style="font-size:12.0pt;font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:2.5in">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt;<br>
<b>Date: </b>Wednesday, January 31, 2018 at 1:17 AM<br>
<b>To: </b>Eva Freer &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt;<br>
<b>Cc: </b>&quot;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&quot; &lt;<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>&gt;, &quot;Greene, Tami McFarlin&quot; &lt;<a href="mailto:greenet@ornl.gov" target="_blank">greenet@ornl.gov</a>&gt;<br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
I found this on the mailing list: <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
<i><span style="font-size:9.5pt">I found the issue.<br>
<br>
The CentOS 7 RPMs, upon upgrade, modifies the .vol files. Among other things, it adds &quot;option shared-brick-count \d&quot;, using the number of bricks in the volume.<br>
<br>
This gives you an average free space per brick, instead of total free space in the volume.<br>
<br>
When I create a new volume, the value of &quot;shared-brick-count&quot; is &quot;1&quot;.<br>
<br>
find /var/lib/glusterd/vols -type f|xargs sed -i -e &#39;s/option shared-brick-count [0-9]*/option shared-brick-count 1/g&#39;</span></i><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-size:9.5pt">Eva, can you send me the contents of the /var/lib/glusterd/&lt;volname&gt; folder from any one node so I can confirm if this is the problem?</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-size:9.5pt">Regards,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-size:9.5pt">Nithya</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
On 31 January 2018 at 10:47, Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Hi Eva, <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
One more question. What version of gluster were you running before the upgrade?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Nithya<u></u><u></u></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
On 31 January 2018 at 09:52, Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Hi Eva, <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Can you send us the following:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
gluster volume info<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
gluster volume status<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
The log files and tcpdump for df on a fresh mount point for that volume.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
Nithya<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
On 31 January 2018 at 07:17, Freer, Eva B. &lt;<a href="mailto:freereb@ornl.gov" target="_blank">freereb@ornl.gov</a>&gt; wrote:<u></u><u></u></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">After OS update to CentOS 7.4 or RedHat 6.9 and update to Gluster 3.12.4, the ‘df’ command shows only part of the available space on the mount point for multi-brick volumes. All nodes are at 3.12.4.
 This occurs on both servers and clients. </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">We have 2 different server configurations.
</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Configuration 1: A distributed volume of 8 bricks with 4 on each server. The initial configuration had 4 bricks of 59TB each with 2 on each server. Prior to the update to CentOS 7.4 and gluster 3.12.4,
 ‘df’ correctly showed the size for the volume as 233TB. After the update, we added 2 bricks with 1 on each server, but the output of ‘df’ still only listed 233TB for the volume. We added 2 more bricks, again with 1 on each server. The output of ‘df’ now shows
 350TB, but the aggregate of 8 – 59TB bricks should be ~466TB.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Configuration 2: A distributed, replicated volume with 9 bricks on each server for a total of ~350TB of storage. After the server update to RHEL 6.9 and gluster 3.12.4, the volume now shows as having
 50TB with ‘df’. No changes were made to this volume after the update.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">In both cases, examining the bricks shows that the space and files are still there, just not reported correctly with ‘df’. All machines have been rebooted and the problem persists.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#2e74b5">Any help/advice you can give on this would be greatly appreciated.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4"> </span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Thanks in advance.</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:2.5in">
<span style="font-family:&quot;Tahoma&quot;,sans-serif;color:#4472c4">Eva Freer</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:2.5in">
______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:2.5in">
 <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:2.0in">
 <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:1.5in">
 <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:1.0in">
 <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">-- <u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Amar Tumballi (amarts)<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
</div>

</blockquote></div><br></div>
</div></div></blockquote></div><br></div>