<div dir="ltr">Hi all,<div><br></div><div>We have a 4-node distributed cluster with 2 bricks per node running Gluster 7.7 + ZFS. We use directory quota to limit the space used by our members on each project. Two days ago we noticed inconsistent space used reported by Gluster in the quota list.</div><div><br></div><div>A small snippet of gluster volume quota vol list,</div><div><br></div><div> Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?<br></div><div>/projectA 5.0TB 80%(4.0TB) 3.1TB 1.9TB No No<br><b>/projectB 100.0TB 80%(80.0TB) 16383.4PB 740.9TB No No</b><br>/projectC 70.0TB 80%(56.0TB) 50.0TB 20.0TB No No<br></div><div><br></div><div>The total space available in the cluster is 360TB, the quota for projectB is 100TB and, as you can see, its reporting
16383.4PB used and 740TB available (already decreased from 750TB).</div><div><br></div><div>There was an issue in Gluster 3.x related to the wrong directory quota (<a href="https://lists.gluster.org/pipermail/gluster-users/2016-February/025305.html">https://lists.gluster.org/pipermail/gluster-users/2016-February/025305.html</a> and <a href="https://lists.gluster.org/pipermail/gluster-users/2018-November/035374.html">https://lists.gluster.org/pipermail/gluster-users/2018-November/035374.html</a>) but it's marked as solved (not sure if the solution still applies).</div><div><br></div><div><b>On projectB</b></div><div># getfattr -d -m . -e hex projectB<br># file:
projectB
<br>trusted.gfid=0x3ca2bce0455945efa6662813ce20fc0c<br>trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f35e69800098ed9<br>trusted.glusterfs.dht=0xe1a4060c000000003ffffffe5ffffffc<br>trusted.glusterfs.mdata=0x010000000000000000000000005f355c59000000000939079f000000005ce2aff90000000007fdacb0000000005ce2aff90000000007fdacb0<br>trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000ab0f227a860000000000478e33acffffffffffffc112<br>trusted.glusterfs.quota.dirty=0x3000<br>trusted.glusterfs.quota.limit-set.1=0x0000640000000000ffffffffffffffff<br>trusted.glusterfs.quota.size.1=0x0000ab0f227a860000000000478e33acffffffffffffc112<br></div><div><br></div><div><b>On projectA</b></div><div># getfattr -d -m . -e hex
projectA
<br># file: projectA<br>trusted.gfid=0x05b09ded19354c0eb544d22d4659582e<br>trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f1aeb9f00044c64<br>trusted.glusterfs.dht=0xe1a4060c000000001fffffff3ffffffd<br>trusted.glusterfs.mdata=0x010000000000000000000000005f1ac6a10000000018f30a4e000000005c338fab0000000017a3135a000000005b0694fb000000001584a21b<br>trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000067de3bbe20000000000000128610000000000033498<br>trusted.glusterfs.quota.dirty=0x3000<br>trusted.glusterfs.quota.limit-set.1=0x0000460000000000ffffffffffffffff<br>trusted.glusterfs.quota.size.1=0x0000067de3bbe20000000000000128610000000000033498<br></div><div><br></div><div>Any idea on what's happening and how to fix it?</div><div><br></div><div>Thanks!</div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="font-size:small"><b>João Baúto</b></div><div><font size="1">---------------</font></div><div><div><div dir="ltr"><b><font size="1">Scientific <span style="border-collapse:collapse">Computing and Software Platform<br></span></font></b><div><span style="border-collapse:collapse"><font size="1"><font color="#666666">Champalimaud Research<br>Champalimaud Center for the Unknown<br>Av. Brasília, Doca de Pedrouços<br>1400-038 Lisbon, Portugal</font><br><a href="https://www.fchampalimaud.org/" style="color:rgb(17,85,204)" target="_blank">fchampalimaud.org</a></font></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>