[Gluster-users] Memory leak in glusterfs

Thu Jun 6 06:38:20 UTC 2019

Hi Nithya,

Here is the Setup details and test which we are doing as below:

One client, two gluster Server.
The client is writing and deleting one file each 15 minutes by script
test_v4.15.sh.

IP
Server side:
128.224.98.157 /gluster/gv0/
128.224.98.159 /gluster/gv0/

Client side:
128.224.98.160 /gluster_mount/

Server side:
gluster volume create gv0 replica 2 128.224.98.157:/gluster/gv0/
128.224.98.159:/gluster/gv0/ force
gluster volume start gv0

root at 128:/tmp/brick/gv0# gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: 7105a475-5929-4d60-ba23-be57445d97b5
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 128.224.98.157:/gluster/gv0
Brick2: 128.224.98.159:/gluster/gv0
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

exec script: ./ps_mem.py -p 605 -w 61 > log
root at 128:/# ./ps_mem.py -p 605
Private + Shared = RAM used Program
23668.0 KiB + 1188.0 KiB = 24856.0 KiB glusterfsd
---------------------------------
24856.0 KiB
=================================

Client side:
mount -t glusterfs -o acl -o resolve-gids 128.224.98.157:gv0 /gluster_mount

We are using the below script write and delete the file.

*test_v4.15.sh <http://test_v4.15.sh>*

Also the below script to see the memory increase whihle the script is above
script is running in background.

*ps_mem.py*

I am attaching the script files as well as the result got after testing the
scenario.

On Wed, Jun 5, 2019 at 7:23 PM Nithya Balachandran <nbalacha at redhat.com>
wrote:

> Hi,
>
> Writing to a volume should not affect glusterd. The stack you have shown
> in the valgrind looks like the memory used to initialise the structures
> glusterd uses and will free only when it is stopped.
>
> Can you provide more details to what it is you are trying to test?
>
> Regards,
> Nithya
>
>
> On Tue, 4 Jun 2019 at 15:41, ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
> wrote:
>
>> Hi Team,
>>
>> Please respond on the issue which I raised.
>>
>> Regards,
>> Abhishek
>>
>> On Fri, May 17, 2019 at 2:46 PM ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
>> wrote:
>>
>>> Anyone please reply....
>>>
>>> On Thu, May 16, 2019, 10:49 ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
>>> wrote:
>>>
>>>> Hi Team,
>>>>
>>>> I upload some valgrind logs from my gluster 5.4 setup. This is writing
>>>> to the volume every 15 minutes. I stopped glusterd and then copy away the
>>>> logs.  The test was running for some simulated days. They are zipped in
>>>> valgrind-54.zip.
>>>>
>>>> Lots of info in valgrind-2730.log. Lots of possibly lost bytes in
>>>> glusterfs and even some definitely lost bytes.
>>>>
>>>> ==2737== 1,572,880 bytes in 1 blocks are possibly lost in loss record
>>>> 391 of 391
>>>> ==2737== at 0x4C29C25: calloc (in
>>>> /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
>>>> ==2737== by 0xA22485E: ??? (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0xA217C94: ??? (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0xA21D9F8: ??? (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0xA21DED9: ??? (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0xA21E685: ??? (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0xA1B9D8C: init (in
>>>> /usr/lib64/glusterfs/5.4/xlator/mgmt/glusterd.so)
>>>> ==2737== by 0x4E511CE: xlator_init (in /usr/lib64/libglusterfs.so.0.0.1)
>>>> ==2737== by 0x4E8A2B8: ??? (in /usr/lib64/libglusterfs.so.0.0.1)
>>>> ==2737== by 0x4E8AAB3: glusterfs_graph_activate (in
>>>> /usr/lib64/libglusterfs.so.0.0.1)
>>>> ==2737== by 0x409C35: glusterfs_process_volfp (in /usr/sbin/glusterfsd)
>>>> ==2737== by 0x409D99: glusterfs_volumes_init (in /usr/sbin/glusterfsd)
>>>> ==2737==
>>>> ==2737== LEAK SUMMARY:
>>>> ==2737== definitely lost: 1,053 bytes in 10 blocks
>>>> ==2737== indirectly lost: 317 bytes in 3 blocks
>>>> ==2737== possibly lost: 2,374,971 bytes in 524 blocks
>>>> ==2737== still reachable: 53,277 bytes in 201 blocks
>>>> ==2737== suppressed: 0 bytes in 0 blocks
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>> Regards
>>>> Abhishek Paliwal
>>>>
>>>
>>
>> --
>>
>>
>>
>>
>> Regards
>> Abhishek Paliwal
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>
>

-- 

Regards
Abhishek Paliwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190606/0be379d2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ps_mem.py
Type: text/x-python
Size: 18465 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190606/0be379d2/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_v4.15.sh
Type: application/x-shellscript
Size: 660 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190606/0be379d2/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ps_mem_server1.log
Type: text/x-log
Size: 135168 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190606/0be379d2/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ps_mem_server2.log
Type: text/x-log
Size: 135168 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190606/0be379d2/attachment-0002.bin>