[Gluster-devel] io-cache exceeding cache-size value

[Amar Tumballi (bulde)]

Hi Gordan and Dan,

It would help me a lot if its possible for you to get the info as described
below,

compile glusterfs like

bash# make clean > /dev/null
bash# make CFLAGS="-g -O0 -DDEBUG" > /dev/null
bash# make install

run the process which consumes memory (mostly client process) like below:

bash# glusterfs <any argument you give generally> -N
<this process will run in foreground now>

Open another terminal

bash# ps aux | grep glusterfs
bash# kill -s SIGUSR1 <pid of glusterfs -N process>
<Check in other terminal for memory usage stats>

bash# <run your application over glusterfs as you do till you get high
memory usage of glusterfs.. >
bash# kill -s SIGUSR1 <pid of glusterfs -N process>
<Check the stat in another terminal>

bash# echo 3 > /proc/sys/vm/drop_caches
bash# kill -s SIGUSR1 <pid of glusterfs -N process>
<Check the stat in another terminal>

Even after dropping caches, if 'in use bytes =' in malloc stats shows a high
value, then it is a leak. If its showing less, but just 'system bytes = ' is
a high value, this means glusterfs is not really consuming high memory, but
the problem is really in the memory allocation segments.

Regards,
Amar

NOTE: 'malloc_stats' will be printed to 'stdout' if we enable -DDEBUG while
compiling glusterfs, as it hits performance badly otherwise.


2009/2/23 Gordan Bobic <gordan at bobich.net>

> Dan Parsons wrote:
>
>> I'm having an issue with glusterfs exceeding its cache-size value. Right
>> now I have it set to 4000MB and I've seen it climb as high as 4800MB. If I
>> set it to 5000, I've seen it go as high as 6000MB. This is a problem because
>> it causes me to set the value very low so that my apps don't get pushed into
>> swap. Is there any way to fix this? To get it to stick to the limit I set
>> and not exceed?
>>
>
> It's possible you are running into the same memory leak that I'm seeing,
> and I'm not using io-cache or any other performance translators at all. With
> rootfs on Gluster, doing a kernel compile (kernel source tree being on NFS,
> so this won't be contributing to the bloat, hopefully) makes the glusterfsd
> bloat by about 80MB per pass, and never frees it.
>
> Gordan
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>



-- 
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Super Storage!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20090223/348f19e0/attachment-0003.html>