[Gluster-devel] Excessive memory usage with 1.3.12

[Krishna Srinivas]

Dan, Lukas, Thomas,

Any updates on this thread? Shall we conclude that it is not a memory
leak and io-cache is working fine?

Regards
Krishna

On Sat, Nov 8, 2008 at 10:32 AM, Krishna Srinivas <krishna at zresearch.com> wrote:
> Dan, Lukas, Thomas,
>
> Internally io-cache limits the cache size to 120% of what is
> configured in the spec file. If you say 2GB, memory usage can go up to
> 2.4-2.5 GB. This is creating confusion, we will make the necessary
> change. Can you guys check if the memory usage goes beyond the 120%
> limit and keeps increasing?
>
> Dan, can you see if the memory usage can go to 3GB? It will be easier
> for you to configure cache-size as 512MB and see if glusterfs usage
> grows to 1GB.
>
> Thanks
> Krishna
>
> On Sat, Nov 8, 2008 at 1:12 AM, Dan Parsons <dparsons at nyip.net> wrote:
>> To be fair, I'm not even sure it's technically a "leak", it's just io-cache
>> not limiting itself to the cache-size setting. As in, io-cache is doing its
>> job, it's just doing it a bit .. too enthusiastically.
>>
>> First, here's my io-cache block:
>>
>> volume ioc
>>  type performance/io-cache
>>  subvolumes stripe0         # In this example it is 'client' you may have to
>> change it according to your spec file.
>>  option page-size 1MB      # 128KB is default
>>  option cache-size 2048MB    # 32MB is default
>>  option force-revalidate-timeout 5 # 1second is default
>>  option priority *.psiblast:3,*.seq:2,*:1
>> end-volume
>>
>> And for my test:
>>
>> [root at cpu26 ~]# cd /distfs
>> [root at cpu26 distfs]# ls -lah bigfile.img
>> -rw-r--r-- 1 root employees 6.3G Aug 26 00:21 bigfile.img
>>
>> Note the memory usage of a freshly started glusterfs client:
>> root     23972  0.0  0.0 110340  1540 ?        Ssl  11:37   0:00 [glusterfs]
>>
>> Now I do this:
>> [root at cpu26 distfs]# cat bigfile.img > /dev/null
>>
>> The glusterfs memory footprint grows at a nice rate. In just a few seconds
>> it's gone from 110kb to 2.5GB, way past my cache-size limit.
>>
>> root     23972  9.4 61.8 2590552 2503164 ?     Ssl  11:37   0:14 [glusterfs]
>>
>> This box has only 4GB RAM so i killed the 'cat' process before things got
>> out of hand. But, there's your test.
>>
>> [root at cpu26 ~]# glusterfs --version
>> glusterfs 1.3.11 built on Aug 21 2008 11:26:38
>> Repository revision: glusterfs--mainline--2.5--patch-795
>>
>>
>> Dan Parsons
>>
>>
>> On Nov 7, 2008, at 11:31 AM, Krishna Srinivas wrote:
>>
>>> Lukas, Dan,
>>>
>>> Are you sure that its a leak of io-cache? did you try by removing the
>>> translator and not observe the leak?
>>>
>>> What made you conclude that cache-size option was being ignored? It
>>> could be a memory leak by io-cache at some other place too.
>>>
>>> I tried to make io-cache memleak, but its not happening, what
>>> operations do you guys do to see the leak? can you see if some simple
>>> test case makes it leak? (so that it will be easy for me to reproduce
>>> the bug here and fix it)
>>>
>>> Thanks!
>>> Krishna
>>>
>>> On
>