[Gluster-users] Old story - glusterfs memory usage

[Shehjar Tikoo]

Krzysztof Strasburger wrote:
> I upgraded today from 2.0.8 to 3.0.3.
> The first sad impression was that unify no longer works, but it is not that
> important. Not a surprise. Switched to dht. Tried my old "du" test:
> du /home
> (/home is a glusterfs filesystem, of course ;) and saw again the bad old
> unlimited increase of glusterfs clients memory usage. I hoped that the
> problem went to quick-read, but it didn't.

This is a known problem. See a previous email on the devel list about it 
at:
http://article.gmane.org/gmane.comp.file-systems.gluster.devel/1469

A bug is filed is at:
http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=545

For more info on the drop_caches mentioned in that gluster-devel thread, 
see:
http://www.linuxinsight.com/proc_sys_vm_drop_caches.html

Do let us know if dropping caches as shown in that thread helps.

Thanks
-Shehjar

> So switched back to 2.0.10rc1, to restore unify.
> And then did a simple test, to be sure that no translators are to be blamed.
> I created the following, trivial setup:
> 
> file loopback.vol:
> volume loopback
>  type storage/posix
>  option directory /root/loopback
> end-volume
> 
> And then mounted /root/loopback on /root/loop-test:
> 
> mount -t glusterfs loopback.vol /root/loop-test
> 
> Do you see? No translators, even no server and networking. Just a loopback
> to a directory on the local machine.
> 
> I copied then the /usr directory to /root/loop-test (c.a. 160000 files).
> And then ran "du /root/loop-test".
> Memory usage of respective glusterfs process went up from 16 MB to 50 MB.
> I could reproduce it perfectly by umounting /root/loop-test, mounting it
> again and re-running "du".
> More files touched mean more memory used by glusterfs.
> This is not a memory leak. Repeating this "du" does not cause memory
> usage go even a single byte up. Glusterfs client keeps somewhere an 
> information about _every file touched_, and keeps it _forever_. As the situation 
> did not improve since the times of glusterfs 1.3, I assume this behavior 
> to be a part of its design. IMHO a wrong one. This is not a feature,
> this is a bug. It costs memory and probably performace too, if it is a kind
> of cache (with millions of entries, potentially).
> I filed a bug report almost one year ago - no change. Dear other users of
> glusterfs - are you at least able to reproduce this memory consumption
> problem? If it occurs on my site only, then the reason is somewhere in my
> system setup, otherwise - let us do something about it, finally.
> It is not in translators, it is in the core of glusterfs.
> With regards
> Krzysztof Strasburger
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users