[Gluster-devel] Problem with GlusterFS quotas (Debian Wheezy/x86_64)
Hristo Hristov
id at yaht.net
Thu Apr 26 16:37:44 UTC 2012
Hello Raghavendra,
Thank you for your replay /and appropriate patch-set(s) :)/. Let me
summarize the results I achieved during the testing. Testing environment
was the same: two nodes, one replicated volume. Here is brief
description of my test-bet volume.
=== snip ===
muzzy:~# gluster volume info m2d
Volume Name: m2d
Type: Replicate
Volume ID: 4d29926d-38d7-4b41-b500-83875ff37c44
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: g0:/g0
Brick2: g1:/g1
Options Reconfigured:
auth.allow: 194.12.*,130.204.198.50,82.119.73.118
features.quota: on
features.limit-usage:
/dav/64mb:64MB,/cifs/64mb:64MB,/cifs/384mb:384MB,/cifs/352mb:352MB
muzzy:~# gluster volume quota m2d list
path limit_set size
----------------------------------------------------------------------------------
/dav/64mb 64MB 30.2MB
/cifs/64mb 64MB 42.2MB
/cifs/384mb 384MB 365.5MB
/cifs/352mb 352MB 335.3MB
g0:
muzzy:~# du -sh /g0/dav/64mb
31M /g0/dav/64mb
muzzy:~# du -sh /g0/cifs/64mb
43M /g0/cifs/64mb
muzzy:~# du -sh /g0/cifs/384mb
366M /g0/cifs/384mb
muzzy:~# du -sh /g0/cifs/352mb
336M /g0/cifs/352mb
g1:
root at g1:/# du -sh /g1/dav/64mb
31M /g1/dav/64mb
root at g1:/# du -sh /g1/cifs/64mb
43M /g1/cifs/64mb
root at g1:/# du -sh /g1/cifs/384mb
366M /g1/cifs/384mb
root at g1:/# du -sh /g1/cifs/352mb
336M /g1/cifs/352mb
=== snip ===
Basically I did again my quota testing in both (already described) ways,
i.e. with quota limit set on existing and on non-existing directory
entry. Everything went ok this time. Quota limits correctly take effect
and client sides are notified correctly.
Let me describe some more findings during the testing:
(o) patches applied to master glusterfs branch:
- a compile time issue: after execution of autogen.sh script, where was
successfully generated configure scrip. When I tried to use the script,
the following error occured:
=== snip ===
checking if libxml2 is present... ./configure: line 12771: syntax error
near unexpected token `LIBXML2,'
./configure: line 12771: `PKG_CHECK_MODULES(LIBXML2, libxml-2.0 >= 2.6.19,'
=== snip ===
After fixing broken check, I had successful compilation and
installation. During, my testing with master branch, I noticed the
following:
- there was a huge slow down of file transfer between g0 serving node
and my testing webdav client;
- there was a strange quota size variation during the upload;
- although these problems quota limit restriction successfully took place;
(o) patches applied to qa37 tar:
- no compile time issues;
- i did not notice any performance issues;
- quota lime worked like expected;
So, for now I'm stuck on 3.3.0qa37 + described patches :)
Please find attached glusterd logs from two nodes (g0 and g1) and native
client log from g0 node. All logs are taken with DEBUG option on.
I've planed more tests - with distributed and replicated volume. I'll
test again with patched 3.3.0qa37 version and I hope to achieve same
results :) during the testing.
Regards,
Hristo Hristov
On 4/25/12 5:14 AM, Raghavendra Gowdappa wrote:
> Hi,
>
> Can you apply following patch to latest master?
> http://review.gluster.com/3022
>
> and let us know, whether it solves your issue?
>
> If you want to stick to 3.3.0qa37, you need following patch along with the above:
> http://review.gluster.com/#change,3021
>
> regards,
> Raghavendra.
>
> ----- Original Message -----
>> From: "Hristo Hristov"<id at yaht.net>
>> To: "Vijay Bellur"<vbellur at redhat.com>
>> Cc: gluster-devel at nongnu.org, "Raghavendra Gowdappa"<rgowdapp at redhat.com>
>> Sent: Monday, April 23, 2012 7:25:35 PM
>> Subject: Re: [Gluster-devel] Problem with GlusterFS quotas (Debian Wheezy/x86_64)
>>
>> Hello Vijay,
>>
>> I made the testing against the version 3.3.0 qa 37 of gluster fs with
>> same cluster config. I'm pasting configuration info:
>>
>> === snip ===
>> muzzy:/home/lisp/work/fs/logs# gluster --version
>> glusterfs 3.3.0qa37 built on Apr 23 2012 13:55:08
>> Repository revision: git://git.gluster.com/glusterfs.git
>> Copyright (c) 2006-2011 Gluster Inc.<http://www.gluster.com>
>> GlusterFS comes with ABSOLUTELY NO WARRANTY.
>> You may redistribute copies of GlusterFS under the terms of the GNU
>> General Public License.
>>
>> muzzy:/home/lisp/work/fs/logs# gluster volume info m2d
>>
>> Volume Name: m2d
>> Type: Replicate
>> Volume ID: 752e6b5b-504b-46bc-be5e-498fc5f364d1
>> Status: Started
>> Number of Bricks: 1 x 2 = 2
>> Transport-type: tcp
>> Bricks:
>> Brick1: g0:/g0
>> Brick2: g1:/g1
>> Options Reconfigured:
>> features.limit-usage:
>> /dav/64mb:64MB,/cifs/64mb:64MB,/dav/92mb:92MB,/cifs/92mb:92MB,/cifs/128mb:128MB,/dav/128mb:128MB
>> features.quota: on
>> auth.allow: 194.12.*
>> muzzy:/home/lisp/work/fs/logs#
>>
>> muzzy:/home/lisp/work/fs/logs# gluster volume quota m2d list
>> path limit_set size
>> ----------------------------------------------------------------------------------
>> /dav/64mb 64MB 365.5MB
>> /cifs/64mb 64MB 365.5MB
>> /dav/92mb 92MB 365.5MB
>> /cifs/92mb 92MB 0Bytes
>> /cifs/128mb 128MB 126.9MB
>> /dav/128mb 128MB 83.6MB
>>
>>
>> mount -t glusterfs g0:/m2d /mnt/m2d -o
>> acl,log-level=DEBUG,log-file=/usr/local/var/log/glusterfs/g0-mount-client.log
>> === snip ===
>>
>> Unfortunately, gluster's quota management behaves exactly the same
>> way,
>> like in previous version. When I set a quota limit (no matter on
>> existing or not existing directory), quota limitation doesn't work,
>> until net next gluster volume remount (i.e. unmount and then mount)
>>
>> Please find attached log files from g0 node:
>>
>> (o) g0-daemon.log
>> log from glusterfs daemon, run in debug mode;
>>
>> (o) 0-mount-client.log.00.before-remounting
>> client log before remounting;
>>
>> (o) g0-mount-client.log.01.after-remounting
>> client log after unmount, mount (i.e. remounting) sequence; then
>> quota
>> restrictions work as expected (Entries '/cifs/128mb' and '/dav/128mb'
>> in
>> upper 'gluster volume quota m2d list' quota).
>>
>> All the log files are inside 3-3-0-qa-38-logs.tar.gz archive.
>>
>> One more thing, could you please point me out something like a
>> 'big-picture-of-glusterfs-architecture' page/doc @ gluster's web
>> site.
>> I'm constantly finding a
>> http://www.gluster.org/community/documentation/index.php/GlusterFS_Hackers_Guide
>> link, which is quite outdated :(
>>
>> Regards,
>> Hristo Hristov
>>
>> On 4/23/12 1:26 PM, Vijay Bellur wrote:
>>> On 04/23/2012 01:17 PM, Hristo Hristov wrote:
>>>> Thank you for your replay. I'm sending you requested files, please
>>>> correct me if there is a missing one :) /debian logs its services
>>>> in
>>>> debianish way, so I may be missing something:)/
>>>>
>>>> (o) glusterd log from brick 0:
>>>>
>>>> location: /var/log/glusterfs/bricks
>>>> name: g0.log
>>>>
>>>> (o) native client mount log
>>>>
>>>> name: gluster.log
>>>>
>>>> I'll definitely try v3.3.0 release of glusterfs with same test
>>>> configuration. Which version I should try (i.e. which one is more
>>>> recent):
>>>>
>>>> (o)
>>>> http://bits.gluster.com/pub/gluster/glusterfs/src/glusterfs-3.3.0qa37.tar.gz
>>>>
>>> qa37 is more recent. That would be a good release to try this out
>>> on.
>>>
>>> Thanks,
>>> Vijay
>>>
>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: qa37.patched.tar.gz
Type: application/x-gzip
Size: 348160 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20120426/f0708762/attachment-0003.gz>
More information about the Gluster-devel
mailing list