[Gluster-users] gluster-fuse and proftpd/vsftpd issue

Barry Stetler barry at hivelocity.net
Fri Feb 14 12:26:00 UTC 2014


Thanks for the response. I have been having this problem for a while but 
now that I test again I cant reproduce it... If it comes backup I will 
run these commands thanks again..

On 2/11/14, 9:22 PM, Venky Shankar wrote:
> Could you provide this information from the server (also the 
> client/server logs):
>
> cat /proc/<pid>/task/*/stack
> ls -l /proc/<pid>/task/*/fd
>
> <pid> == process ID of glusterfsd
>
>
> On Fri, Feb 7, 2014 at 6:14 PM, Barry Stetler <barry at hivelocity.net 
> <mailto:barry at hivelocity.net>> wrote:
>
>     We started using gluster in a custom solution recently.  Basically
>     we setup a gluster volume that we mount at /home. Then we give
>     users their own home directory with a gluster quota. They have
>     FTP, SFTP and SSHMOUNT access. I first used vsftpd and notice that
>     ftp connections will not close and the load goes up ever couple of
>     days the funny thing is the load does not seem to slow down the
>     server. The load went up to 20 in the last two months but the
>     server was not slow at all... At first I thought it was vsftpd so
>     I changed to proftpd then the issue came back.  If I reboot the
>     server it goes away for a few days depending on how many ftp
>     connections. I noticed in the dmesg log this making me think the
>     issue was some kind of leak in gluster-fuse
>
>     This is on CentOS 6.5 64 bit.
>
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>     INFO: task proftpd:7831 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
>     message.
>     proftpd       D 0000000000000000     0  7831   4384 0x00000004
>      ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
>      ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
>      ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
>     Call Trace:
>      [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
>      [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
>      [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
>      [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
>      [<ffffffff8117e0bc>] filp_close+0x3c/0x90
>      [<ffffffff8117e1b5>] sys_close+0xa5/0x100
>      [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>
>
>     I appreciate any help I can get on this...
>
>     Thank you
>
>     -- 
>     Barry Stetler
>     Hivelocity Hosting
>     http://hivelocity.net
>     _______________________________________________
>     Gluster-users mailing list
>     Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>     http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>

-- 
Barry Stetler
Hivelocity Hosting
http://hivelocity.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140214/fac5457f/attachment.html>


More information about the Gluster-users mailing list