[Gluster-users] glusterFS memory leak?

[Raghavendra G]

what operations are you doing on mount point? how easy it is to reproduce
memory leak?

On Thu, Dec 24, 2009 at 5:08 AM, Larry Bates <larry.bates at vitalesafe.com>wrote:

>  I restarted the mirror and once again the glusterfs client process is
> just growing.  Echoing the 3 to /proc/sys/vm/drop_caches eems to shriink the
> memory footprint but only by a small amount.
>
> BTW - it is the client process that seems to grow infinitely.  Note: I have
> one machine that is acting as both a client and a server (which is where the
> client process is growing) and another machine that is a dedicated GFS
> server.
>
>
>
> Here are my configs:
>
>
>
> Client:
>
>
>
> #
>
> # Add client feature and attach to remote subvolumes of gfs001
>
> #
>
> volume gfs001brick1
>
>   type protocol/client
>
>   option transport-type tcp            # for TCP/IP transport
>
>   option remote-host gfs001            # IP address of the remote volume
>
>   option remote-subvolume vol1         # name of the remote volume
>
>   #option transport.socket.nodelay on   # undocumented option for speed
>
> end-volume
>
>
>
> volume gfs001brick2
>
>   type protocol/client
>
>   option transport-type tcp            # for TCP/IP transport
>
>   option remote-host gfs001            # IP address of the remote volume
>
>   option remote-subvolume vol2         # name of the remote volume
>
>   #option transport.socket.nodelay on   # undocumented option for speed
>
> end-volume
>
>
>
> volume coraidbrick13
>
>   type protocol/client
>
>   option transport-type tcp
>
>   option remote-host 10.0.0.71
>
>   option remote-subvolume coraidvol13
>
>   #option transport.socket.nodelay on
>
> end-volume
>
>
>
> volume coraidbrick14
>
>   type protocol/client
>
>   option transport-type tcp
>
>   option remote-host 10.0.0.71
>
>   option remote-subvolume coraidvol14
>
>   #option transport.socket.nodelay on
>
> end-volume
>
>
>
> #
>
> # Replicate volumes
>
> #
>
> volume afr-vol1
>
>   type cluster/replicate
>
>   subvolumes gfs001brick1 coraidbrick13
>
> end-volume
>
>
>
> volume afr-vol2
>
>   type cluster/replicate
>
>   subvolumes gfs001brick2 coraidbrick14
>
> end-volume
>
>
>
> #
>
> # Distribute files across bricks
>
> #
>
> volume dht-vol
>
>   type cluster/distribute
>
>   subvolumes afr-vol1 afr-vol2
>
>   option min-free-disk 2%               # 2% of 1.8Tb volumes is 36Gb
>
> end-volume
>
>
>
> #
>
> # Add quick-read for small files
>
> #
>
> volume quickread
>
>   type performance/quick-read
>
>   option cache-timeout 1         # default 1 second
>
>   option max-file-size 256KB     # default 64Kb
>
>   subvolumes dht-vol
>
> end-volume
>
>
>
> Server:
>
>
>
> volume coraidbrick13
>
>   type storage/posix                          # POSIX FS translator
>
>   option directory /mnt/glusterfs/e101.13     # Export this directory
>
>   option background-unlink yes                # unlink in background
>
> end-volume
>
>
>
> volume coraidbrick14
>
>   type storage/posix
>
>   option directory /mnt/glusterfs/e101.14
>
>   option background-unlink yes
>
> end-volume
>
>
>
> volume iot1
>
>   type performance/io-threads
>
>   option thread-count 4
>
>   subvolumes coraidbrick13
>
> end-volume
>
>
>
> volume iot2
>
>   type performance/io-threads
>
>   option thread-count 4
>
>   subvolumes coraidbrick14
>
> end-volume
>
>
>
> volume coraidvol13
>
>   type features/locks
>
>   subvolumes iot1
>
> end-volume
>
>
>
> volume coraidvol14
>
>   type features/locks
>
>   subvolumes iot2
>
> end-volume
>
>
>
> ## Add network serving capability to volumes
>
> volume server
>
>   type protocol/server
>
>   option transport-type tcp                    # For TCP/IP transport
>
>   subvolumes coraidvol13 coraidvol14           # Expose both volumes
>
>   option auth.addr.coraidvol13.allow 10.0.0.*
>
>   option auth.addr.coraidvol14.allow 10.0.0.*
>
> end-volume
>
>
>
> Thanks,
>
> Larry
>
>
>
>
>
>
>
> *From:* raghavendra.hg at gmail.com [mailto:raghavendra.hg at gmail.com] *On
> Behalf Of *Raghavendra G
> *Sent:* Wednesday, December 23, 2009 2:43 PM
> *To:* Larry Bates
> *Cc:* gluster-users
> *Subject:* Re: [Gluster-users] glusterFS memory leak?
>
>
>
> Hi Larry,
>
> What is the client and server configuration? Instead of killing glusterfs
> process, can you do "echo 3 > /proc/sys/vm/drop_caches" and check whether
> memory usage comes down?
>
> regards,
>
> On Wed, Dec 23, 2009 at 9:01 PM, Larry Bates <larry.bates at vitalesafe.com>
> wrote:
>
> I've successfully set up GlusterFS 3.0 with a single server.  I brought on
> a 2nd server and setup AFR and have been working through the mirroring
> process.  I started an "ls -alR" to trigger a complete mirror of all files
> between the two servers.  After running for about 16 hours I started getting
> kernel out of memory errors.  Looking at top I see:
>
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  2561 root      15   0 13.0g 8.9g  860 S    7 91.6  60:56.02 glusterfs
>
> Seems that the client has used up all my memory (RAM and SWAP).  Killing
> the process returned all the memory to the OS.
>
> BTW - I'm working on a 2.3Tb store that contains about 2.5 million files in
> 65K folders.
>
> Thoughts?
>
> Larry Bates
> vitalEsafe, Inc.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
>
>
> --
> Raghavendra G
>



-- 
Raghavendra G