[Gluster-users] gluster-fuse and proftpd/vsftpd issue
Barry Stetler
barry at hivelocity.net
Fri Feb 14 12:26:00 UTC 2014
Thanks for the response. I have been having this problem for a while but
now that I test again I cant reproduce it... If it comes backup I will
run these commands thanks again..
On 2/11/14, 9:22 PM, Venky Shankar wrote:
> Could you provide this information from the server (also the
> client/server logs):
>
> cat /proc/<pid>/task/*/stack
> ls -l /proc/<pid>/task/*/fd
>
> <pid> == process ID of glusterfsd
>
>
> On Fri, Feb 7, 2014 at 6:14 PM, Barry Stetler <barry at hivelocity.net
> <mailto:barry at hivelocity.net>> wrote:
>
> We started using gluster in a custom solution recently. Basically
> we setup a gluster volume that we mount at /home. Then we give
> users their own home directory with a gluster quota. They have
> FTP, SFTP and SSHMOUNT access. I first used vsftpd and notice that
> ftp connections will not close and the load goes up ever couple of
> days the funny thing is the load does not seem to slow down the
> server. The load went up to 20 in the last two months but the
> server was not slow at all... At first I thought it was vsftpd so
> I changed to proftpd then the issue came back. If I reboot the
> server it goes away for a few days depending on how many ftp
> connections. I noticed in the dmesg log this making me think the
> issue was some kind of leak in gluster-fuse
>
> This is on CentOS 6.5 64 bit.
>
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
> INFO: task proftpd:7831 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> proftpd D 0000000000000000 0 7831 4384 0x00000004
> ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
> ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
> ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
> Call Trace:
> [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
> [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
> [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
> [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
> [<ffffffff8117e0bc>] filp_close+0x3c/0x90
> [<ffffffff8117e1b5>] sys_close+0xa5/0x100
> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
>
>
> I appreciate any help I can get on this...
>
> Thank you
>
> --
> Barry Stetler
> Hivelocity Hosting
> http://hivelocity.net
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
--
Barry Stetler
Hivelocity Hosting
http://hivelocity.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140214/fac5457f/attachment.html>
More information about the Gluster-users
mailing list