[Gluster-users] quota calculation gluster 3.2.3

anthony garnier sokar6012 at hotmail.com
Mon Sep 5 09:28:21 UTC 2011



Volume configuration: 

yval1000:/ # gluster volume info

Volume Name: venus
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Brick3: yval1000:/soft/venus
Brick4: yval1010:/soft/venus
Options Reconfigured:
features.quota-timeout: 120
features.limit-usage: /test:100MB,/psa:200MB,/:7GB,/soft:5GB
features.quota: on
geo-replication.indexing: on
network.ping-timeout: 10
performance.cache-size: 1GB
performance.cache-refresh-timeout: 60
nfs.port: 2049


Indeed values didn't converged automaticaly for big folder, for small folder like /test and /psa it was fine (I waited 1 day before checking the value). I was forced to run ls -lRa several times, but even with this workaround, values for big folder like /soft are not rightfully calculated : 

gluster volume quota venus list
    path          limit_set         size
----------------------------------------------------------------------------------
/test                     100MB                4.5MB <= OK
/psa                      200MB              167.0MB <=OK
/                           7GB                5.2GB <=Strange
/soft                       5GB                4.7GB <= Strange

=> the size of the entire filesystem is 1.4GB



Anthony


From: junaid at gluster.com
Date: Mon, 5 Sep 2011 14:38:06 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org

Hi Anthony,
By volume configuration, I meant 'is the volume setup dht, afr, or dht-afr', I would like to know the output of 
   volume info volume-name


When data is already present before the activation of quota, it takes some time for quota values to converge because it has to crawl through the entire file-system to calculate the sizes. This is automatically done when you run


   volume quota vol-name enable

The output of    volume quota vol-name list
will reflect the aggregated value at that point in time. If the values are not converging then its a issue. A simple work-around is to run ls -lR on the mount point. Please confirm if the values were not converging automatically, we will have a look into it.


Junaid

On Mon, Sep 5, 2011 at 12:51 PM, anthony garnier <sokar6012 at hotmail.com> wrote:







Junaid,

What do you mean by " the volume configuration on other two servers" ? Configuration is the same on all the server..
I'm creating data with "dd" cmd, and yes, data was already present before activation of quota.





From: junaid at gluster.com
Date: Fri, 2 Sep 2011 22:55:02 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3


To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org

Hi Anthony,
Thanks for providing the input. What is the volume configuration on other two servers (yval1000 and yval1010)? My question is same, what data where you creating on these two machines? Was the data already present before quota was enabled?





On Fri, Sep 2, 2011 at 1:49 PM, anthony garnier <sokar6012 at hotmail.com> wrote:

















I've done a ls -lRa throught the mount point and now i get : 
/soft/venus # gluster volume quota venus list
    path          limit_set         size
----------------------------------------------------------------------------------




/                           2GB                1.3GB
/test                     100MB                8.0KB
/psa                      200MB              151.0MB
/soft                    1500MB              826.3MB





=>Seems to be consistent

and now command  getfattr on yval1000 and yval1010 show : 

yval1000 : 
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/   
trusted.glusterfs.quota.size=0x0000000028f9ca00





getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
trusted.glusterfs.quota.size=0x000000001421d400

getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa


trusted.glusterfs.quota.size=0x00000000053d6000

 
yval1010 : 
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/   
trusted.glusterfs.quota.size=0x000000002904ea00

getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft




trusted.glusterfs.quota.size=0x000000002292ba00

getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
trusted.glusterfs.quota.size=0x00000000053d6000

Regards,



Anthony

From: sokar6012 at hotmail.com
To: junaid at gluster.com


CC: gluster-users at gluster.org
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:57:19 +0000












I forgot to check on the 2 other server.
So ylal3020 and ylal3030 show similar output send previously.

But yval1000 and yval1010 show with all command :
 trusted.glusterfs.quota.size=0x0000000000000000





Is it normal ?

Regards,

Anthony
From: sokar6012 at hotmail.com
To: junaid at gluster.com




CC: gluster-users at gluster.org
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:51:52 +0000








Hi Junaid,

Here is the output of the diffrent command executed on both server.
Command show identical output on both server

# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus
getfattr: Removing leading '/' from absolute path names




# file: soft/venus
trusted.glusterfs.quota.size=0x0000000022ccac00

 # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/test




trusted.glusterfs.quota.size=0x0000000000000000

 # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa     
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/psa




trusted.glusterfs.quota.size=0x000000000013d000

 # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/soft



trusted.glusterfs.quota.size=0x000000000d15ce00



# gluster volume quota venus list
    path          limit_set         size
----------------------------------------------------------------------------------
/                           2GB              556.8MB




/test                     100MB               0Bytes
/psa                      200MB                1.2MB
/soft                    1500MB              209.4MB


Indeed there is 90% of small file (~60000 files).





Thx

Anthony


From: junaid at gluster.com
Date: Fri, 2 Sep 2011 06:18:30 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3




To: sokar6012 at hotmail.com
CC: gluster-users at gluster.org

Hi Anthony,
To debug this further, can you send the output of 




getfattr -n trusted.glusterfs.quota.size -e hex /soft/venusgetfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test

getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psagetfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft


from both the machine. Also check the output of
gluster volume quota Replicate list






Sometimes it takes a small amount of time to bring up the sizes. Also, what kind of data where you creating (I mean large amount of small files or large files because in case of small files, the directory sizes are not accounted by quota to calculate the size unlike the du -h command which uses the directory size as well).







Junaid


On Thu, Sep 1, 2011 at 1:37 PM, anthony garnier <sokar6012 at hotmail.com> wrote:











Hi all,

I've enable quota but I'm a bit confused by values displayed by GlusterFS

Here is my volume : 

Volume Name: venus
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp






Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Options Reconfigured:
features.limit-usage: /test:100MB,/psa:200MB,/soft:1500MB,/:2GB
features.quota: on
diagnostics.client-log-level: ERROR






diagnostics.brick-log-level: ERROR
network.ping-timeout: 10
performance.cache-size: 2GB
nfs.port: 2049

I've got 3 folders in the backend (/soft/venus) : 
psa        160MB  (with du -sh)
soft         1.2GB






test         12KB
Total       1.4GB

But when I list the quota with gluster I got : 
# gluster volume quota venus list
    path          limit_set         size
----------------------------------------------------------------------------------






/test                     100MB               12.0KB                  <= This one is OK
/psa                      200MB               64.4MB           <= not OK
/soft                    1500MB              281.8MB           <= not OK






/                           2GB              346.2MB               <= not OK

Any idea ?

Regards,

Anthony Garnier



 		 	   		  

_______________________________________________

Gluster-users mailing list

Gluster-users at gluster.org

http://gluster.org/cgi-bin/mailman/listinfo/gluster-users



 		 	   		   		 	   		   		 	   		  

 		 	   		  

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110905/91db8f2d/attachment.html>


More information about the Gluster-users mailing list