[Gluster-users] gluster-fuse and proftpd/vsftpd issue

Barry Stetler barry at hivelocity.net
Fri Feb 7 12:44:04 UTC 2014


We started using gluster in a custom solution recently.  Basically we 
setup a gluster volume that we mount at /home. Then we give users their 
own home directory with a gluster quota. They have FTP, SFTP and 
SSHMOUNT access. I first used vsftpd and notice that ftp connections 
will not close and the load goes up ever couple of days the funny thing 
is the load does not seem to slow down the server. The load went up to 
20 in the last two months but the server was not slow at all... At first 
I thought it was vsftpd so I changed to proftpd then the issue came 
back.  If I reboot the server it goes away for a few days depending on 
how many ftp connections. I noticed in the dmesg log this making me 
think the issue was some kind of leak in gluster-fuse

This is on CentOS 6.5 64 bit.

INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task proftpd:7831 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
proftpd       D 0000000000000000     0  7831   4384 0x00000004
  ffff88031f3bde38 0000000000000082 0000000000000000 ffffffff81051439
  ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8 ffff880335936040
  ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88 ffff8803365cfab8
Call Trace:
  [<ffffffff81051439>] ? __wake_up_common+0x59/0x90
  [<ffffffffa0213075>] fuse_request_send+0xe5/0x290 [fuse]
  [<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40
  [<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]
  [<ffffffff8117e0bc>] filp_close+0x3c/0x90
  [<ffffffff8117e1b5>] sys_close+0xa5/0x100
  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b


I appreciate any help I can get on this...

Thank you

-- 
Barry Stetler
Hivelocity Hosting
http://hivelocity.net



More information about the Gluster-users mailing list