<div dir="ltr">Hi <span class="" id=":1gm.1" tabindex="-1" style="">Strahil</span>,<div><br></div><div>I have tried removing the quota for that specific directory and setting it again but it didn't work (maybe it has to be a quota disable and enable in the volume options). Currently testing a solution </div><div>by <span class="" id=":1gm.2" tabindex="-1" style="">Hari</span> with the quota_<span class="" id=":1gm.3" tabindex="-1" style="">fsck</span>.<span class="" id=":1gm.4" tabindex="-1" style="">py</span> script (<a href="https://medium.com/@harigowtham/glusterfs-quota-fix-accounting-840df33fcd3a">https://medium.com/@<span class="" id=":1gm.5" tabindex="-1" style="">harigowtham</span>/<span class="" id=":1gm.6" tabindex="-1" style="">glusterfs</span>-quota-fix-accounting-840df33fcd3a</a>) and its detecting a lot of size mismatch in files.</div><div><br></div><div>Thank you,</div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="font-size:small"><b><span class="" id=":1gm.7" tabindex="-1" style="">João</span> <span class="" id=":1gm.8" tabindex="-1" style="">Baúto</span></b></div><div><font size="1">---------------</font></div><div><div><div dir="ltr"><b><font size="1">Scientific <span style="border-collapse:collapse">Computing and Software Platform<br></span></font></b><div><span style="border-collapse:collapse"><font size="1"><font color="#666666"><span class="" id=":1gm.9" tabindex="-1" style="">Champalimaud</span> Research<br><span class="" id=":1gm.10" tabindex="-1" style="">Champalimaud</span> Center for the Unknown<br>Av. <span class="" id=":1gm.11" tabindex="-1" style="">Brasília</span>, <span class="" id=":1gm.12" tabindex="-1" style="">Doca</span> <span class="" id=":1gm.13" tabindex="-1" style="">de</span> <span class="" id=":1gm.14" tabindex="-1" style="">Pedrouços</span><br>1400-038 Lisbon, Portugal</font><br><a href="https://www.fchampalimaud.org/" style="color:rgb(17,85,204)" target="_blank"><span class="" id=":1gm.15" tabindex="-1" style="">fchampalimaud</span>.org</a></font></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> escreveu no dia sexta, 14/08/2020 à(s) 10:16:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi João,<br>
<br>
Based on your output it seems that the quota size is different on the 2 bricks.<br>
<br>
Have you tried to remove the quota and then recreate it ? Maybe it will be the easiest way to fix it.<br>
<br>
Best Regards,<br>
Strahil Nikolov<br>
<br>
<br>
На 14 август 2020 г. 4:35:14 GMT+03:00, "João Baúto" <<a href="mailto:joao.bauto@neuro.fchampalimaud.org" target="_blank">joao.bauto@neuro.fchampalimaud.org</a>> написа:<br>
>Hi all,<br>
><br>
>We have a 4-node distributed cluster with 2 bricks per node running<br>
>Gluster<br>
>7.7 + ZFS. We use directory quota to limit the space used by our<br>
>members on<br>
>each project. Two days ago we noticed inconsistent space used reported<br>
>by<br>
>Gluster in the quota list.<br>
><br>
>A small snippet of gluster volume quota vol list,<br>
><br>
> Path Hard-limit Soft-limit Used<br>
>Available Soft-limit exceeded? Hard-limit exceeded?<br>
>/projectA 5.0TB 80%(4.0TB) 3.1TB 1.9TB<br>
> No No<br>
>*/projectB 100.0TB 80%(80.0TB) 16383.4PB 740.9TB<br>
> No No*<br>
>/projectC 70.0TB 80%(56.0TB) 50.0TB 20.0TB<br>
> No No<br>
><br>
>The total space available in the cluster is 360TB, the quota for<br>
>projectB<br>
>is 100TB and, as you can see, its reporting 16383.4PB used and 740TB<br>
>available (already decreased from 750TB).<br>
><br>
>There was an issue in Gluster 3.x related to the wrong directory quota<br>
>(<br>
><a href="https://lists.gluster.org/pipermail/gluster-users/2016-February/025305.html" rel="noreferrer" target="_blank">https://lists.gluster.org/pipermail/gluster-users/2016-February/025305.html</a><br>
> and<br>
><a href="https://lists.gluster.org/pipermail/gluster-users/2018-November/035374.html" rel="noreferrer" target="_blank">https://lists.gluster.org/pipermail/gluster-users/2018-November/035374.html</a>)<br>
>but it's marked as solved (not sure if the solution still applies).<br>
><br>
>*On projectB*<br>
># getfattr -d -m . -e hex projectB<br>
># file: projectB<br>
>trusted.gfid=0x3ca2bce0455945efa6662813ce20fc0c<br>
>trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f35e69800098ed9<br>
>trusted.glusterfs.dht=0xe1a4060c000000003ffffffe5ffffffc<br>
>trusted.glusterfs.mdata=0x010000000000000000000000005f355c59000000000939079f000000005ce2aff90000000007fdacb0000000005ce2aff90000000007fdacb0<br>
>trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000ab0f227a860000000000478e33acffffffffffffc112<br>
>trusted.glusterfs.quota.dirty=0x3000<br>
>trusted.glusterfs.quota.limit-set.1=0x0000640000000000ffffffffffffffff<br>
>trusted.glusterfs.quota.size.1=0x0000ab0f227a860000000000478e33acffffffffffffc112<br>
><br>
>*On projectA*<br>
># getfattr -d -m . -e hex projectA<br>
># file: projectA<br>
>trusted.gfid=0x05b09ded19354c0eb544d22d4659582e<br>
>trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f1aeb9f00044c64<br>
>trusted.glusterfs.dht=0xe1a4060c000000001fffffff3ffffffd<br>
>trusted.glusterfs.mdata=0x010000000000000000000000005f1ac6a10000000018f30a4e000000005c338fab0000000017a3135a000000005b0694fb000000001584a21b<br>
>trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000067de3bbe20000000000000128610000000000033498<br>
>trusted.glusterfs.quota.dirty=0x3000<br>
>trusted.glusterfs.quota.limit-set.1=0x0000460000000000ffffffffffffffff<br>
>trusted.glusterfs.quota.size.1=0x0000067de3bbe20000000000000128610000000000033498<br>
><br>
>Any idea on what's happening and how to fix it?<br>
><br>
>Thanks!<br>
>*João Baúto*<br>
>---------------<br>
><br>
>*Scientific Computing and Software Platform*<br>
>Champalimaud Research<br>
>Champalimaud Center for the Unknown<br>
>Av. Brasília, Doca de Pedrouços<br>
>1400-038 Lisbon, Portugal<br>
><a href="http://fchampalimaud.org" rel="noreferrer" target="_blank">fchampalimaud.org</a> <<a href="https://www.fchampalimaud.org/" rel="noreferrer" target="_blank">https://www.fchampalimaud.org/</a>><br>
</blockquote></div>