[Gluster-users] Memory leak with glusterfs NFS on 3.2.6
d.a.bretherton at reading.ac.uk
Tue Jun 12 10:00:25 UTC 2012
On 06/12/2012 05:15 AM, gluster-users-request at gluster.org wrote:
> Date: Mon, 11 Jun 2012 22:00:17 +0200
> From: Philip Poten<philip.poten at gmail.com>
> Subject: [Gluster-users] Memory leak with glusterfs NFS on 3.2.6
> To: gluster-users at gluster.org
> <CAO3z7E6GXZMMfpJA2MTh6DkD44Vr5shc40V1mhUFti3Mh_PB4A at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> we're running a distributed-replicated setup for our images, and while
> we use a caching proxy for the hotset, quite a few requests land on
> glusterfs (3.2.6 on squeeze). Since glusterfs fuse client experiences
> regular hangs which require reboots (I couldnt yet find a solution to
> that), we run on NFS.
> NFS however, specifically the glusterfs process, eats memory like
> crazy, around 3-4GB a week, until we (have to) restart it.
> Does/did anybody experience this problem, and if so, did you find a
> way to mitigate it?
I wonder if this memory leak is the cause of the NFS performance
degradation I reported in April. I just went looking for a link to the
relevant thread, but links to mailing lists from the gluster.org web
site appear to be broken or are redirected to
http://www.gluster.org/about. The best I can do is this (which times
out at the time of writing but might be available later).
As I said in that discussion, I have to restart glusterd every day on
machines exporting NFS to avoid NFS becoming unusable after a few days.
To avoid restarting glusterd on the storage servers (which have other
important GlusterFS related things to do besides NFS), and to balance
the NFS load, my compute servers export NFS to themselves as described
in this Gluster community article.
Until recently I thought the daily glusterd restarts were having no
adverse side effects, but a couple of users recently reported
applications crashing for no apparent reason in the middle of the night
and some of my maintenance tasks (eg lengthy chmod and chown operations)
have been affected as well. Therefore I really would like to find a
better solution to this problem. If it's being caused by a memory leak
it should be relatively easy to fix I would have thought (not knowing
anything about the code...). It certainly should be easy to reproduce,
but as far as I know none of the developers have acknowledged that the
performance degradation problem exists.
More information about the Gluster-users