[Gluster-users] Gluster 3.10.5: used disk size reported by quota and du mismatch

[Mauro Tridici]

Dear Hari,

I followed you suggestions, but, unfortunately, nothing is changed.
I tried to execute both the quota-fsck script with —fix-issues options both the "setfattr -n trusted.glusterfs.quota.dirty -v 0x3100” command against the files and directory mentioned by you (on each available brick).
Disk quota assigned to /tier2/ASC directory seems to be partially used (about 2,6 TB used), but the “real and current” situation is the following one (I deleted all files in primavera directory):

[root at s03 qc]# du -hsc /tier2/ASC/*
22G	/tier2/ASC/orientgate
26K	/tier2/ASC/primavera
22G	totale

So, I think that the problem should be only in "orientgate” or in “primavera” directory, right!?
For this reason, in order to collect some fresh logs, I executed again the check script starting from the top level directory “ASC”  using the following bash script (named hari-20180910) based on the new version of quota_fsck (rel. 9):

hari-20180910 script:

#!/bin/bash

#set -xv

host=$(hostname)

for i in {1..12}
do
 ./quota_fsck_r9.py --full-logs --sub-dir ASC /gluster/mnt$i/brick >> $host.log
done
~

In attachment, you can find the log files generated by the script.

SOME IMPORTANT NOTES:

- in the new log files, “primavera” directory is no more present

Is there something more that I can do?

Thank you very much for your patience.
Regards,
Mauro


> Il giorno 10 set 2018, alle ore 10:51, Hari Gowtham <hgowtham at redhat.com> ha scritto:
> 
> Hi,
> 
> Looking at the logs, I can see that the file:
> 
> /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB_Balkan_8km_1971-2005
> /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_ERA40_NMMB_Balkan_8km_1971-2000
> /orientgate/ftp/climate/3_urban_adaptation_health/6_budapest_veszprem_hungary/RHMSS_CMCC-CM_NMMB-RCP8.5_Balkan_8km_2010-2100
> /primavera/cam
> 
> has mismatch.
> 
> You can try setting dirty for this and then do a du on it.
> 
> A few corrections for my above comments.
> The contri size in the xattr and the aggregated size have to be checked.
> 
> On Mon, Sep 10, 2018 at 1:16 PM Mauro Tridici <mauro.tridici at cmcc.it> wrote:
>> 
>> 
>> Hi Hari,
>> 
>> thank you very much for your help.
>> I will try to use the latest available version of quota_fsck script and I will provide you a feedback as soon as possible.
>> 
>> Thank you again for the detailed explanation.
>> Regards,
>> Mauro
>> 
>> Il giorno 10 set 2018, alle ore 09:17, Hari Gowtham <hgowtham at redhat.com> ha scritto:
>> 
>> Hi Mauro,
>> 
>> The problem might be at some other place, So setting the xattr and
>> doing the lookup might not have fixed the issue.
>> 
>> To resolve this we need to read the log file reported by the fsck
>> script. In this log file we need to look for the size reported by the
>> xattr (the value "SIZE:" in the log file) and the size reported by the
>> stat on the file (the value after "st_size=" ).
> 
> The contri size in the xattr and the aggregated size have to be checked
> 
>> These two should be the same. If they mismatch, then we have to find
>> the top most dir which has the mismatch.
> 
> Bottom most dir/file has to be found. Replace top with bottom in the
> following places as well.
> 
>> On this top most directory you have to do a set dirty xattr and then
>> do a lookup.
>> 
>> If there are two different directories without a common top directory,
>> then both these have to undergo the above process.
>> 
>> The fsck script should work fine. can you try the "--fix-issue" with
>> the latest script instead of the 6th patch used above?
>> 
>> 
>> 
>> 
> 
> 
> -- 
> Regards,
> Hari Gowtham.


-------------------------
Mauro Tridici

Fondazione CMCC
CMCC Supercomputing Center
presso Complesso Ecotekne - Università del Salento -
Strada Prov.le Lecce - Monteroni sn
73100 Lecce  IT
http://www.cmcc.it

mobile: (+39) 327 5630841
email: mauro.tridici at cmcc.it

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log.20180910.tar.gz
Type: application/x-gzip
Size: 560447 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0001.gz>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180910/aa4a7c73/attachment-0003.html>