[Gluster-users] Memory leak in 3.6.9

[Tim]

There have been alot of fixes since 3.6.9.

Specifically, https://bugzilla.redhat.com/1311377 was fixed in 3.7.9.
re:
https://github.com/gluster/glusterfs/blob/release-3.7/doc/release-notes/3.7.9.md

And another memory bug was fixed in 3.7.10.


On 26/04/16 22:14, Alessandro Ipe wrote:
>
> Hi,
>
>  
>
>  
>
> Apparently, version 3.6.9 is suffering from a SERIOUS memory leak as
> illustrated in the following logs:
>
> 2016-04-26T11:54:27.971564+00:00 tsunami1 kernel: [698635.210069]
> glusterfsd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
>
> 2016-04-26T11:54:27.974133+00:00 tsunami1 kernel: [698635.210076] Pid:
> 28111, comm: glusterfsd Tainted: G W O 3.7.10-1.1-desktop #1
>
> 2016-04-26T11:54:27.974136+00:00 tsunami1 kernel: [698635.210077] Call
> Trace:
>
> 2016-04-26T11:54:27.974137+00:00 tsunami1 kernel: [698635.210090]
> [<ffffffff81004818>] dump_trace+0x88/0x300
>
> 2016-04-26T11:54:27.974137+00:00 tsunami1 kernel: [698635.210096]
> [<ffffffff8158b033>] dump_stack+0x69/0x6f
>
> 2016-04-26T11:54:27.974138+00:00 tsunami1 kernel: [698635.210101]
> [<ffffffff8158db39>] dump_header+0x70/0x200
>
> 2016-04-26T11:54:27.974139+00:00 tsunami1 kernel: [698635.210105]
> [<ffffffff81112ad4>] oom_kill_process+0x244/0x390
>
> 2016-04-26T11:54:28.113125+00:00 tsunami1 kernel: [698635.210111]
> [<ffffffff81113211>] out_of_memory+0x451/0x490
>
> 2016-04-26T11:54:28.113142+00:00 tsunami1 kernel: [698635.210116]
> [<ffffffff81118afe>] __alloc_pages_nodemask+0x8ae/0x9f0
>
> 2016-04-26T11:54:28.113143+00:00 tsunami1 kernel: [698635.210122]
> [<ffffffff81152fb7>] alloc_pages_current+0xb7/0x130
>
> 2016-04-26T11:54:28.113144+00:00 tsunami1 kernel: [698635.210127]
> [<ffffffff81111673>] filemap_fault+0x283/0x440
>
> 2016-04-26T11:54:28.113144+00:00 tsunami1 kernel: [698635.210131]
> [<ffffffff811345ee>] __do_fault+0x6e/0x560
>
> 2016-04-26T11:54:28.113145+00:00 tsunami1 kernel: [698635.210136]
> [<ffffffff81137cf7>] handle_pte_fault+0x97/0x490
>
> 2016-04-26T11:54:28.113145+00:00 tsunami1 kernel: [698635.210141]
> [<ffffffff8159af8b>] __do_page_fault+0x16b/0x4c0
>
> 2016-04-26T11:54:28.113562+00:00 tsunami1 kernel: [698635.210145]
> [<ffffffff815982f8>] page_fault+0x28/0x30
>
> 2016-04-26T11:54:28.113565+00:00 tsunami1 kernel: [698635.210158]
> [<00007fa9d8a8292b>] 0x7fa9d8a8292a
>
> 2016-04-26T11:54:28.120811+00:00 tsunami1 kernel: [698635.226243] Out
> of memory: Kill process 17144 (glusterfsd) score 694 or sacrifice child
>
> 2016-04-26T11:54:28.120811+00:00 tsunami1 kernel: [698635.226251]
> Killed process 17144 (glusterfsd) total-vm:8956384kB,
> anon-rss:6670900kB, file-rss:0kB
>
>  
>
> It makes this version completely useless in production. Bricks servers
> have 8 GB of RAM (but will be upgraded to 16 GB).
>
>  
>
> gluster volume info <VOLUME> returns:
>
> Volume Name: home
>
> Type: Distributed-Replicate
>
> Volume ID: 501741ed-4146-4022-af0b-41f5b1297766
>
> Status: Started
>
> Number of Bricks: 14 x 2 = 28
>
> Transport-type: tcp
>
> Bricks:
>
> Brick1: tsunami1:/data/glusterfs/home/brick1
>
> Brick2: tsunami2:/data/glusterfs/home/brick1
>
> Brick3: tsunami1:/data/glusterfs/home/brick2
>
> Brick4: tsunami2:/data/glusterfs/home/brick2
>
> Brick5: tsunami1:/data/glusterfs/home/brick3
>
> Brick6: tsunami2:/data/glusterfs/home/brick3
>
> Brick7: tsunami1:/data/glusterfs/home/brick4
>
> Brick8: tsunami2:/data/glusterfs/home/brick4
>
> Brick9: tsunami3:/data/glusterfs/home/brick1
>
> Brick10: tsunami4:/data/glusterfs/home/brick1
>
> Brick11: tsunami3:/data/glusterfs/home/brick2
>
> Brick12: tsunami4:/data/glusterfs/home/brick2
>
> Brick13: tsunami3:/data/glusterfs/home/brick3
>
> Brick14: tsunami4:/data/glusterfs/home/brick3
>
> Brick15: tsunami3:/data/glusterfs/home/brick4
>
> Brick16: tsunami4:/data/glusterfs/home/brick4
>
> Brick17: tsunami5:/data/glusterfs/home/brick1
>
> Brick18: tsunami6:/data/glusterfs/home/brick1
>
> Brick19: tsunami5:/data/glusterfs/home/brick2
>
> Brick20: tsunami6:/data/glusterfs/home/brick2
>
> Brick21: tsunami5:/data/glusterfs/home/brick3
>
> Brick22: tsunami6:/data/glusterfs/home/brick3
>
> Brick23: tsunami5:/data/glusterfs/home/brick4
>
> Brick24: tsunami6:/data/glusterfs/home/brick4
>
> Brick25: tsunami7:/data/glusterfs/home/brick1
>
> Brick26: tsunami8:/data/glusterfs/home/brick1
>
> Brick27: tsunami7:/data/glusterfs/home/brick2
>
> Brick28: tsunami8:/data/glusterfs/home/brick2
>
> Options Reconfigured:
>
> nfs.export-dir: /gerb-reproc/Archive
>
> nfs.volume-access: read-only
>
> cluster.ensure-durability: on
>
> features.quota: on
>
> performance.cache-size: 512MB
>
> performance.io-thread-count: 32
>
> performance.flush-behind: off
>
> performance.write-behind-window-size: 4MB
>
> performance.write-behind: off
>
> nfs.disable: off
>
> cluster.read-hash-mode: 2
>
> diagnostics.brick-log-level: CRITICAL
>
> cluster.lookup-unhashed: on
>
> server.allow-insecure: on
>
> auth.allow: localhost, <COUPLE OF IP ADDRESSES>
>
> cluster.readdir-optimize: on
>
> performance.readdir-ahead: on
>
> nfs.export-volumes: off
>
>  
>
> Are you aware if this issue ?
>
>  
>
>  
>
> Thanks,
>
>  
>
>  
>
> A.
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160427/3f4ef5e7/attachment.html>