[Gluster-users] quota calculation gluster 3.2.3
Mohammed Junaid
junaid at gluster.com
Mon Sep 5 11:10:20 UTC 2011
Hi Anthony,
This is quiet unusual, to debug this further can you get the output of "du
-h" of mount-point and of directory soft (which is under mount point). This
is will help a lot in debugging this issue. Also, I noticed that the volume
info output provided is of a distributed-replicate setup which is different
from the one provided previously (it was plain replicate). Also please
provide the output of getfattr for the bricks and soft directory.
Junaid
On Mon, Sep 5, 2011 at 2:58 PM, anthony garnier <sokar6012 at hotmail.com>wrote:
>
> Volume configuration:
>
> yval1000:/ # gluster volume info
>
> Volume Name: venus
> Type: Distributed-Replicate
> Status: Started
> Number of Bricks: 2 x 2 = 4
>
> Transport-type: tcp
> Bricks:
> Brick1: ylal3020:/soft/venus
> Brick2: ylal3030:/soft/venus
> Brick3: yval1000:/soft/venus
> Brick4: yval1010:/soft/venus
> Options Reconfigured:
> features.quota-timeout: 120
> features.limit-usage: /test:100MB,/psa:200MB,/:7GB,/soft:5GB
> features.quota: on
> geo-replication.indexing: on
> network.ping-timeout: 10
> performance.cache-size: 1GB
> performance.cache-refresh-timeout: 60
> nfs.port: 2049
>
>
> Indeed values didn't converged automaticaly for big folder, for small
> folder like /test and /psa it was fine (I waited 1 day before checking the
> value). I was forced to run ls -lRa several times, but even with this
> workaround, values for big folder like /soft are not rightfully calculated :
>
>
> gluster volume quota venus list
> path limit_set size
>
> ----------------------------------------------------------------------------------
> /test 100MB 4.5MB <= OK
> /psa 200MB 167.0MB <=OK
> / 7GB 5.2GB <=Strange
> /soft 5GB 4.7GB <= Strange
>
> => the size of the entire filesystem is 1.4GB
>
>
>
> Anthony
>
>
> ------------------------------
> From: junaid at gluster.com
> Date: Mon, 5 Sep 2011 14:38:06 +0530
>
> Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
> To: sokar6012 at hotmail.com
> CC: gluster-users at gluster.org
>
> Hi Anthony,
>
> By volume configuration, I meant 'is the volume setup dht, afr, or
> dht-afr', I would like to know the output of
>
> volume info volume-name
>
> When data is already present before the activation of quota, it takes some
> time for quota values to converge because it has to crawl through the entire
> file-system to calculate the sizes. This is automatically done when you run
>
> volume quota vol-name enable
>
> The output of
> volume quota vol-name list
>
> will reflect the aggregated value at that point in time. If the values are
> not converging then its a issue. A simple work-around is to run ls -lR on
> the mount point. Please confirm if the values were not converging
> automatically, we will have a look into it.
>
> Junaid
>
>
> On Mon, Sep 5, 2011 at 12:51 PM, anthony garnier <sokar6012 at hotmail.com>wrote:
>
> Junaid,
>
> What do you mean by " the volume configuration on other two servers" ?
> Configuration is the same on all the server..
> I'm creating data with "dd" cmd, and yes, data was already present before
> activation of quota.
>
>
>
> ------------------------------
> From: junaid at gluster.com
> Date: Fri, 2 Sep 2011 22:55:02 +0530
>
> Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
> To: sokar6012 at hotmail.com
> CC: gluster-users at gluster.org
>
> Hi Anthony,
>
> Thanks for providing the input. What is the volume configuration on other
> two servers (yval1000 and yval1010)? My question is same, what data where
> you creating on these two machines? Was the data already present before
> quota was enabled?
>
> On Fri, Sep 2, 2011 at 1:49 PM, anthony garnier <sokar6012 at hotmail.com>wrote:
>
>
> I've done a ls -lRa throught the mount point and now i get :
> /soft/venus # gluster volume quota venus list
> path limit_set size
>
> ----------------------------------------------------------------------------------
> / 2GB 1.3GB
> /test 100MB 8.0KB
> /psa 200MB 151.0MB
> /soft 1500MB 826.3MB
>
> =>Seems to be consistent
>
> and now command getfattr on yval1000 and yval1010 show :
>
> yval1000 :
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
> trusted.glusterfs.quota.size=0x0000000028f9ca00
>
>
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
> trusted.glusterfs.quota.size=0x000000001421d400
>
>
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
> trusted.glusterfs.quota.size=0x00000000053d6000
>
>
> yval1010 :
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
> trusted.glusterfs.quota.size=0x000000002904ea00
>
>
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
> trusted.glusterfs.quota.size=0x000000002292ba00
>
>
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
> trusted.glusterfs.quota.size=0x00000000053d6000
>
> Regards,
>
> Anthony
>
>
> ------------------------------
> From: sokar6012 at hotmail.com
> To: junaid at gluster.com
> CC: gluster-users at gluster.org
> Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
> Date: Fri, 2 Sep 2011 07:57:19 +0000
>
>
> I forgot to check on the 2 other server.
> So ylal3020 and ylal3030 show similar output send previously.
>
> But yval1000 and yval1010 show with all command :
> trusted.glusterfs.quota.size=0x0000000000000000
>
> Is it normal ?
>
> Regards,
>
> Anthony
> ------------------------------
> From: sokar6012 at hotmail.com
> To: junaid at gluster.com
> CC: gluster-users at gluster.org
> Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
> Date: Fri, 2 Sep 2011 07:51:52 +0000
>
> Hi Junaid,
>
> Here is the output of the diffrent command executed on both server.
> Command show identical output on both server
>
> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus
> getfattr: Removing leading '/' from absolute path names
> # file: soft/venus
> trusted.glusterfs.quota.size=0x0000000022ccac00
>
> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
> getfattr: Removing leading '/' from absolute path names
> # file: soft/venus/test
> trusted.glusterfs.quota.size=0x0000000000000000
>
> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
> getfattr: Removing leading '/' from absolute path names
> # file: soft/venus/psa
> trusted.glusterfs.quota.size=0x000000000013d000
>
> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
> getfattr: Removing leading '/' from absolute path names
> # file: soft/venus/soft
> trusted.glusterfs.quota.size=0x000000000d15ce00
>
>
> # gluster volume quota venus list
> path limit_set size
>
> ----------------------------------------------------------------------------------
> / 2GB 556.8MB
> /test 100MB 0Bytes
> /psa 200MB 1.2MB
> /soft 1500MB 209.4MB
>
>
> Indeed there is 90% of small file (~60000 files).
>
> Thx
>
> Anthony
>
>
> ------------------------------
> From: junaid at gluster.com
> Date: Fri, 2 Sep 2011 06:18:30 +0530
> Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
> To: sokar6012 at hotmail.com
> CC: gluster-users at gluster.org
>
> Hi Anthony,
>
> To debug this further, can you send the output of
>
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
>
> from both the machine. Also check the output of
>
> gluster volume quota Replicate list
>
> Sometimes it takes a small amount of time to bring up the sizes. Also, what
> kind of data where you creating (I mean large amount of small files or large
> files because in case of small files, the directory sizes are not accounted
> by quota to calculate the size unlike the du -h command which uses the
> directory size as well).
>
> Junaid
> On Thu, Sep 1, 2011 at 1:37 PM, anthony garnier <sokar6012 at hotmail.com>wrote:
>
> Hi all,
>
> I've enable quota but I'm a bit confused by values displayed by GlusterFS
>
> Here is my volume :
>
> Volume Name: venus
> Type: Replicate
> Status: Started
> Number of Bricks: 2
> Transport-type: tcp
> Bricks:
> Brick1: ylal3020:/soft/venus
> Brick2: ylal3030:/soft/venus
> Options Reconfigured:
> features.limit-usage: /test:100MB,/psa:200MB,/soft:1500MB,/:2GB
> features.quota: on
> diagnostics.client-log-level: ERROR
> diagnostics.brick-log-level: ERROR
> network.ping-timeout: 10
> performance.cache-size: 2GB
> nfs.port: 2049
>
> I've got 3 folders in the backend (/soft/venus) :
> psa 160MB (with du -sh)
> soft 1.2GB
> test 12KB
> Total 1.4GB
>
> But when I list the quota with gluster I got :
> # gluster volume quota venus list
> path limit_set size
>
> ----------------------------------------------------------------------------------
> /test 100MB 12.0KB <=
> This one is OK
> /psa 200MB 64.4MB <= not OK
> /soft 1500MB 281.8MB <= not OK
> / 2GB 346.2MB <= not OK
>
> Any idea ?
>
> Regards,
>
> Anthony Garnier
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110905/75333c92/attachment.html>
More information about the Gluster-users
mailing list