[Gluster-users] Question about glusterfs quotas on debian wheezy?
Hristo Hristov
id at yaht.net
Thu Apr 19 16:58:35 UTC 2012
Dear list,
I forgot to add info about extended fs attributes for directories with
quota in my glusterfs setup. Here it is:
=== snip ===
(o) muzzy, aka g0
muzzy:~# getfattr -m . -d -e hex /g0/dav/128mb
getfattr: Removing leading '/' from absolute path names
# file: g0/dav/128mb
trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf
trusted.glusterfs.dht=0x0000000100000000aaaaaaaaffffffff
trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x0000000000003000
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.size=0x0000000000003000
root at g1:~# getfattr -m . -d -e hex /g1/dav/128mb
getfattr: Removing leading '/' from absolute path names
# file: g1/dav/128mb
trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf
trusted.glusterfs.dht=0x00000001000000000000000055555554
trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x000000000539f000
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.size=0x000000000539f000
root at mdx:~# getfattr -m . -d -e hex /g2/dav/128mb
getfattr: Removing leading '/' from absolute path names
# file: g2/dav/128mb
trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf
trusted.glusterfs.dht=0x000000010000000055555555aaaaaaa9
trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x0000000000004000
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.size=0x0000000000004000
=== snip ===
Also, I'm listing all the quotas added in my setup:
=== snip ===
muzzy:~# gluster volume quota m3d list
path limit_set size
----------------------------------------------------------------------------------
/1Gg 1GB
/10Gg 10GB
/5Gg 5GB
/512Mb 512MB 0Bytes
/dav/256MB 256MB 208.8MB
/dav/1GB 1GB 0Bytes
/dav/5GB 5GB 0Bytes
/dav/10GB 10GB 0Bytes
/cifs/256Mb 256MB 167.5MB
/dav/128mb 128MB 83.6MB
/dav/192mb 192MB 365.4MB
/cifs/192mb 192MB 210.9MB
/cifs/64mb 64MB 365.4MB
=== snip ===
Any ideas are welcome, and 10x for your attention.
Regards,
Hristo Hristov
On 4/19/12 5:21 PM, Hristo Hristov wrote:
> Hello list,
>
> I'm experimenting with a little GlusterFS cluster on debian wheezy:
>
> === snip ===
> muzzy:~# cat /etc/debian_version
> wheezy/sid
>
> muzzy:~# dpkg -l | grep gluster
> ii glusterfs-client 3.2.6-1 clustered file-system (client package)
> ii glusterfs-common 3.2.6-1 GlusterFS common libraries and translator
> modules
> ii glusterfs-server 3.2.6-1 clustered file-system (server package)
> === snip ===
>
> My volume has 3 bricks and is a distributed one:
>
> === snip ===
> muzzy:~# gluster volume info
>
> Volume Name: m3d
> Type: Distribute
> Status: Started
> Number of Bricks: 3
> Transport-type: tcp
> Bricks:
> Brick1: g0:/g0
> Brick2: g1:/g1
> Brick3: g2:/g2
> Options Reconfigured:
> features.quota-timeout: 0
> features.limit-usage:
> /1Gg:1GB,/10Gg:10GB,/5Gg:5GB,/512Mb:512MB,/dav/256MB:256MB,/dav/1GB:1GB,/dav/5GB:5GB,/dav/10GB:10GB,/cifs/256Mb:256MB,/dav/128mb:128MB,/dav/192mb:192MB,/cifs/192mb:192MB,/cifs/64mb:64MB
> features.quota: on
> auth.allow: 194.12.*,130.204.198.50
> === snip ===
>
> One of nodes, (g0), also mounts m3d volume (mount -t glusterfs) and
> "exports" it via samba and webdav. I'm using two different directories
> under volume root for each of them, i.e. dav and cifs. /This is not
> the best way to do export the volume, but I do not have more nodes at
> the moment :(/
>
> I'm mounting the volume with the following parameters:
> === snip ===
> mount -t glusterfs g0:/m3d /mnt/m3d -o
> acl,log-level=WARNING,log-file=/var/log/gluster.log
> === snip ===
>
> According the glusterfs documentation
> (Gluster_File_System-3.2.5-Administration_Guide-en-US.pdf, p. 67) we
> can set directory limit even on non-existing directory:
> === quote ===
> Note
> You can set the disk limit on the directory even if it is not created.
> The disk limit is enforced
> immediately after creating that directory. For more information on
> setting disk limit, see
> Section 10.3, “Setting or Replacing Disk Limit ”.
> === quote ===
>
> Here comes my problem. When I set directory quota on non-existing (or
> even existing directory) the quota limit is not enforced. I have to
> unmount the volume (i.e. stop apache and samba daemons before that)
> and mount it again (respectively starting back apache and samba
> daemons). Then quota limit is enforced correctly on existing
> directories. If I create a new directory and set quota limit, i have
> to umount and mount again in order to have working quota for that
> directory.
>
> Do you have any ideas, what may cause this problem? Will it be
> appropriate to blame Debian Wheezy :) /because I'm using It's testing
> branch/?
>
> Here is more info about my testing glusterfs installation:
> === snip ===
> muzzy:~# cat /etc/debian_version
> wheezy/sid
>
> muzzy:~# uname -a
> Linux muzzy 3.2.0-1-amd64 #1 SMP Fri Feb 17 05:17:36 UTC 2012 x86_64
> GNU/Linux
>
> muzzy:~# cat /var/log/gluster.log
> 2012-04-19 16:36:17.663509] W [write-behind.c:3023:init]
> 0-m3d-write-behind: disabling write-behind for first 0 bytes
> Given volfile:
> +------------------------------------------------------------------------------+
>
> 1: volume m3d-client-0
> 2: type protocol/client
> 3: option remote-host g0
> 4: option remote-subvolume /g0
> 5: option transport-type tcp
> 6: end-volume
> 7:
> 8: volume m3d-client-1
> 9: type protocol/client
> 10: option remote-host g1
> 11: option remote-subvolume /g1
> 12: option transport-type tcp
> 13: end-volume
> 14:
> 15: volume m3d-client-2
> 16: type protocol/client
> 17: option remote-host g2
> 18: option remote-subvolume /g2
> 19: option transport-type tcp
> 20: end-volume
> 21:
> 22: volume m3d-dht
> 23: type cluster/distribute
> 24: subvolumes m3d-client-0 m3d-client-1 m3d-client-2
> 25: end-volume
> 26:
> 27: volume m3d-quota
> 28: type features/quota
> 29: option limit-set
> /1Gg:1GB,/10Gg:10GB,/5Gg:5GB,/512Mb:512MB,/dav/256MB:256MB,/dav/1GB:1GB,/dav/5GB:5GB,/dav/10GB:10GB,/cifs/256Mb:256MB,/dav/128mb:128MB,/dav/192mb:192MB,/cifs/192mb:192MB
> 30: option timeout 0
> 31: subvolumes m3d-dht
> 32: end-volume
> 33:
> 34: volume m3d-write-behind
> 35: type performance/write-behind
> 36: subvolumes m3d-quota
> 37: end-volume
> 38:
> 39: volume m3d-read-ahead
> 40: type performance/read-ahead
> 41: subvolumes m3d-write-behind
> 42: end-volume
> 43:
> 44: volume m3d-io-cache
> 45: type performance/io-cache
> 46: subvolumes m3d-read-ahead
> 47: end-volume
> 48:
> 49: volume m3d-quick-read
> 50: type performance/quick-read
> 51: subvolumes m3d-io-cache
> 52: end-volume
> 53:
> 54: volume m3d-stat-prefetch
> 55: type performance/stat-prefetch
> 56: subvolumes m3d-quick-read
> 57: end-volume
> 58:
> 59: volume m3d
> 60: type debug/io-stats
> 61: option latency-measurement off
> 62: option count-fop-hits off
> 63: subvolumes m3d-stat-prefetch
> 64: end-volume
>
> +------------------------------------------------------------------------------+
>
> [2012-04-19 16:37:12.729422] W [dict.c:1153:data_to_str]
> (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc]
> (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a]
> (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x443)
> [0x7ff9bf0f7913]))) 0-dict: data is NULL
> [2012-04-19 16:37:12.729481] W [dict.c:1153:data_to_str]
> (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc]
> (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a]
> (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x59d)
> [0x7ff9bf0f7a6d]))) 0-dict: data is NULL
> [2012-04-19 16:37:12.729531] W [dict.c:1153:data_to_str]
> (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc]
> (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a]
> (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x72f)
> [0x7ff9bf0f7bff]))) 0-dict: data is NULL
> [2012-04-19 16:37:12.729580] W [dict.c:1153:data_to_str]
> (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc]
> (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a]
> (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x74b)
> [0x7ff9bf0f7c1b]))) 0-dict: data is NULL
> === snip ===
>
> Thank you in advance,
> Regards
> Hristo Hristov
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list