[Gluster-users] quota calculation gluster 3.2.3
anthony garnier
sokar6012 at hotmail.com
Mon Sep 5 09:28:21 UTC 2011
Volume configuration:
yval1000:/ # gluster volume info
Volume Name: venus
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Brick3: yval1000:/soft/venus
Brick4: yval1010:/soft/venus
Options Reconfigured:
features.quota-timeout: 120
features.limit-usage: /test:100MB,/psa:200MB,/:7GB,/soft:5GB
features.quota: on
geo-replication.indexing: on
network.ping-timeout: 10
performance.cache-size: 1GB
performance.cache-refresh-timeout: 60
nfs.port: 2049
Indeed values didn't converged automaticaly for big folder, for small folder like /test and /psa it was fine (I waited 1 day before checking the value). I was forced to run ls -lRa several times, but even with this workaround, values for big folder like /soft are not rightfully calculated :
gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/test 100MB 4.5MB <= OK
/psa 200MB 167.0MB <=OK
/ 7GB 5.2GB <=Strange
/soft 5GB 4.7GB <= Strange
=> the size of the entire filesystem is 1.4GB
Anthony
From: junaid at gluster.com
Date: Mon, 5 Sep 2011 14:38:06 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org
Hi Anthony,
By volume configuration, I meant 'is the volume setup dht, afr, or dht-afr', I would like to know the output of
volume info volume-name
When data is already present before the activation of quota, it takes some time for quota values to converge because it has to crawl through the entire file-system to calculate the sizes. This is automatically done when you run
volume quota vol-name enable
The output of volume quota vol-name list
will reflect the aggregated value at that point in time. If the values are not converging then its a issue. A simple work-around is to run ls -lR on the mount point. Please confirm if the values were not converging automatically, we will have a look into it.
Junaid
On Mon, Sep 5, 2011 at 12:51 PM, anthony garnier <sokar6012 at hotmail.com> wrote:
Junaid,
What do you mean by " the volume configuration on other two servers" ? Configuration is the same on all the server..
I'm creating data with "dd" cmd, and yes, data was already present before activation of quota.
From: junaid at gluster.com
Date: Fri, 2 Sep 2011 22:55:02 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org
Hi Anthony,
Thanks for providing the input. What is the volume configuration on other two servers (yval1000 and yval1010)? My question is same, what data where you creating on these two machines? Was the data already present before quota was enabled?
On Fri, Sep 2, 2011 at 1:49 PM, anthony garnier <sokar6012 at hotmail.com> wrote:
I've done a ls -lRa throught the mount point and now i get :
/soft/venus # gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/ 2GB 1.3GB
/test 100MB 8.0KB
/psa 200MB 151.0MB
/soft 1500MB 826.3MB
=>Seems to be consistent
and now command getfattr on yval1000 and yval1010 show :
yval1000 :
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
trusted.glusterfs.quota.size=0x0000000028f9ca00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
trusted.glusterfs.quota.size=0x000000001421d400
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
trusted.glusterfs.quota.size=0x00000000053d6000
yval1010 :
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
trusted.glusterfs.quota.size=0x000000002904ea00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
trusted.glusterfs.quota.size=0x000000002292ba00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
trusted.glusterfs.quota.size=0x00000000053d6000
Regards,
Anthony
From: sokar6012 at hotmail.com
To: junaid at gluster.com
CC: gluster-users at gluster.org
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:57:19 +0000
I forgot to check on the 2 other server.
So ylal3020 and ylal3030 show similar output send previously.
But yval1000 and yval1010 show with all command :
trusted.glusterfs.quota.size=0x0000000000000000
Is it normal ?
Regards,
Anthony
From: sokar6012 at hotmail.com
To: junaid at gluster.com
CC: gluster-users at gluster.org
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:51:52 +0000
Hi Junaid,
Here is the output of the diffrent command executed on both server.
Command show identical output on both server
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus
getfattr: Removing leading '/' from absolute path names
# file: soft/venus
trusted.glusterfs.quota.size=0x0000000022ccac00
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/test
trusted.glusterfs.quota.size=0x0000000000000000
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/psa
trusted.glusterfs.quota.size=0x000000000013d000
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/soft
trusted.glusterfs.quota.size=0x000000000d15ce00
# gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/ 2GB 556.8MB
/test 100MB 0Bytes
/psa 200MB 1.2MB
/soft 1500MB 209.4MB
Indeed there is 90% of small file (~60000 files).
Thx
Anthony
From: junaid at gluster.com
Date: Fri, 2 Sep 2011 06:18:30 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org
Hi Anthony,
To debug this further, can you send the output of
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venusgetfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psagetfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
from both the machine. Also check the output of
gluster volume quota Replicate list
Sometimes it takes a small amount of time to bring up the sizes. Also, what kind of data where you creating (I mean large amount of small files or large files because in case of small files, the directory sizes are not accounted by quota to calculate the size unlike the du -h command which uses the directory size as well).
Junaid
On Thu, Sep 1, 2011 at 1:37 PM, anthony garnier <sokar6012 at hotmail.com> wrote:
Hi all,
I've enable quota but I'm a bit confused by values displayed by GlusterFS
Here is my volume :
Volume Name: venus
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Options Reconfigured:
features.limit-usage: /test:100MB,/psa:200MB,/soft:1500MB,/:2GB
features.quota: on
diagnostics.client-log-level: ERROR
diagnostics.brick-log-level: ERROR
network.ping-timeout: 10
performance.cache-size: 2GB
nfs.port: 2049
I've got 3 folders in the backend (/soft/venus) :
psa 160MB (with du -sh)
soft 1.2GB
test 12KB
Total 1.4GB
But when I list the quota with gluster I got :
# gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/test 100MB 12.0KB <= This one is OK
/psa 200MB 64.4MB <= not OK
/soft 1500MB 281.8MB <= not OK
/ 2GB 346.2MB <= not OK
Any idea ?
Regards,
Anthony Garnier
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110905/91db8f2d/attachment.html>
More information about the Gluster-users
mailing list