[Gluster-users] [Gluster-devel] gluster 10.3: task glfs_fusenoti blocked for more than 120 seconds
revirii at googlemail.com
Fri May 5 14:03:38 UTC 2023
strange, after the latest kernel update (5.10.0-22-amd64 #1 Debian
5.10.178-3) and with gluster 10.4 running, i got this message again
today. Result: for a certain time period e.g. 'df -h' hangs. And as
gluster is the only thing (at least that i'm aware of) that uses FUSE
i thought there might... ;-)
Maybe i'll have to check if there's anything that can be done on
kernel/system side. Searching... :-)
Am Di., 2. Mai 2023 um 14:28 Uhr schrieb Mohit Agrawal <moagrawa at redhat.com>:
> I don't think the issue is on gluster side, it seems the issue is on kernel side (possible deadlock in fuse_reverse_inval_entry)
> On Tue, May 2, 2023 at 5:48 PM Hu Bert <revirii at googlemail.com> wrote:
>> Good morning,
>> we've recently had some strange message in /var/log/syslog.
>> debian bullseye, kernel 5.10.0-21-amd64 and 5.10.0-22-amd64
>> gluster 10.3
>> The message look like:
>> Apr 27 13:30:18 piggy kernel: [24287.715229] INFO: task
>> glfs_fusenoti:2787 blocked for more than 120 seconds.
>> Apr 27 13:30:18 piggy kernel: [24287.715327] Not tainted
>> 5.10.0-22-amd64 #1 Debian 5.10.178-3
>> Apr 27 13:30:18 piggy kernel: [24287.715419] "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Apr 27 13:30:18 piggy kernel: [24287.715575] task:glfs_fusenoti
>> state:D stack: 0 pid: 2787 ppid: 1 flags:0x00000000
>> Apr 27 13:30:18 piggy kernel: [24287.715734] Call Trace:
>> Apr 27 13:30:18 piggy kernel: [24287.715847] __schedule+0x282/0x870
>> Apr 27 13:30:18 piggy kernel: [24287.715959] schedule+0x46/0xb0
>> Apr 27 13:30:18 piggy kernel: [24287.716073]
>> Apr 27 13:30:18 piggy kernel: [24287.716194]
>> fuse_reverse_inval_entry+0x3b/0x1e0 [fuse]
>> etc. Full excerpt here: https://pastebin.com/6gDHgh16
>> This may cause our application to "hang". In the latest release i read
>> something about fuse:
>> "Fix fuse concurrency problems"
>> but i checked the tickets and wasn't able to find something.
>> However, i upgraded gluster client 10.3 -> 10.4, hoping that this
>> fixes the hang-issue.
>> Has anyone seen these messages before?
>> Best regards,
>> Community Meeting Calendar:
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://meet.google.com/cpu-eiue-hvk
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
More information about the Gluster-users