<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Tahoma",sans-serif;
color:#2E74B5;
font-weight:normal;
font-style:normal;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Sam,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">For du –sh on my newer volume, the result is 161T. The sum of the Used space in the df –h output for all the bricks is ~163T. Close enough for me to believe everything
is there. The total for used space in the df –h of the mountpoint it 83T, roughly half what is used.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Relevant lines from df –h on server-A:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">Filesystem Size Used Avail Use% Mounted on<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sda1 59T 42T 17T 72% /bricks/data_A1<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdb1 59T 45T 14T 77% /bricks/data_A2<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdd1 59T 39M 59T 1% /bricks/data_A4<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdc1 59T 1.9T 57T 4% /bricks/data_A3<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">server-A:/dataeng 350T 83T 268T 24% /dataeng<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">And on server-B:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">Filesystem Size Used Avail Use% Mounted on<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdb1 59T 34T 25T 58% /bricks/data_B2<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdc1 59T 2.0T 57T 4% /bricks/data_B3<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sdd1 59T 39M 59T 1% /bricks/data_B4<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">/dev/sda1 59T 38T 22T 64% /bricks/data_B1<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;font-family:"Consolas",sans-serif;color:black">server-B:/dataeng 350T 83T 268T 24% /dataeng<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Eva Freer</span><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#4472C4"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Sam McLeod <mailinglists@smcleod.net><br>
<b>Date: </b>Tuesday, January 30, 2018 at 9:43 PM<br>
<b>To: </b>Eva Freer <freereb@ornl.gov><br>
<b>Cc: </b>"gluster-users@gluster.org" <gluster-users@gluster.org>, "Greene, Tami McFarlin" <greenet@ornl.gov><br>
<b>Subject: </b>Re: [Gluster-users] df does not show full volume capacity after update to 3.12.4<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">We noticed something similar. <o:p>
</o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Out of interest, does du -sh . show the same size?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:black">--<br>
Sam McLeod (protoporpoise on IRC)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:black"><a href="https://smcleod.net">https://smcleod.net</a><br>
https://twitter.com/s_mcleod<br>
<br>
Words are my own opinions and do not necessarily represent those of my employer or partners.<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">On 31 Jan 2018, at 12:47 pm, Freer, Eva B. <<a href="mailto:freereb@ornl.gov">freereb@ornl.gov</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">After OS update to CentOS 7.4 or RedHat 6.9 and update to Gluster 3.12.4, the ‘df’ command shows only part of the available
space on the mount point for multi-brick volumes. All nodes are at 3.12.4. This occurs on both servers and clients.<span class="apple-converted-space"> </span></span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">We have 2 different server configurations.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Configuration 1: A distributed volume of 8 bricks with 4 on each server. The initial configuration had 4 bricks of 59TB
each with 2 on each server. Prior to the update to CentOS 7.4 and gluster 3.12.4, ‘df’ correctly showed the size for the volume as 233TB. After the update, we added 2 bricks with 1 on each server, but the output of ‘df’ still only listed 233TB for the volume.
We added 2 more bricks, again with 1 on each server. The output of ‘df’ now shows 350TB, but the aggregate of 8 – 59TB bricks should be ~466TB.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Configuration 2: A distributed, replicated volume with 9 bricks on each server for a total of ~350TB of storage. After
the server update to RHEL 6.9 and gluster 3.12.4, the volume now shows as having 50TB with ‘df’. No changes were made to this volume after the update.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">In both cases, examining the bricks shows that the space and files are still there, just not reported correctly with
‘df’. All machines have been rebooted and the problem persists.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#2E74B5">Any help/advice you can give on this would be greatly appreciated.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#4472C4"> </span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#4472C4">Thanks in advance.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in;background:white"><span style="font-size:12.0pt;font-family:"Tahoma",sans-serif;color:#4472C4">Eva Freer<br>
<br>
<br>
</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;background:white">_______________________________________________</span><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif"><br>
<span style="background:white">Gluster-users mailing list</span><br>
</span><a href="mailto:Gluster-users@gluster.org"><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;color:#954F72;background:white">Gluster-users@gluster.org</span></a><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif"><br>
</span><a href="http://lists.gluster.org/mailman/listinfo/gluster-users"><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;color:#954F72;background:white">http://lists.gluster.org/mailman/listinfo/gluster-users</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
</div>
</body>
</html>