[Gluster-devel] locking race condition

[Anand Avati]

On Sun, Aug 19, 2012 at 6:25 PM, Emmanuel Dreyfus <manu at netbsd.org> wrote:

> Hi
>
> I hit this bug, on a 2x2 replicated:distributes volume: It does not
> reproduce
> 100% off the time, but works (well, breaks) most of the time:
>
> # echo "xxx"|cat -l > test9
> cat: stdout: No such file or directory
>
> FUSE log (2 is ENOENT)
> 1345425065.650284521 LOOKUP "tmp" path = "test9" error = 2 elapsed =
> 0.003035340
> 1345425065.653437220 CREATE "tmp" path = "test9" error = 0 elapsed =
> 0.006853510
> 1345425065.663216217 SETLKW "test9"  error = 2 elapsed = 0.449751521
> 1345425066.114117361 SETLK "test9"  error = 2 elapsed = 0.002567192
>
> client log:
> [2012-08-20 03:11:06.116305] W [client3_1-fops.c:2186:client3_1_lk_cbk]
>   0-gfs33-client-0: remote operation failed: No such file or directory
> [2012-08-20 03:11:06.116605] W [fuse-bridge.c:3196:fuse_setlk_cbk]
>   0-glusterfs-fuse: 4828311: ERR => -1 (No such file or directory)
>
> client-0 brick log:
> [2012-08-20 03:11:06.115077] I [server3_1-fops.c:203:server_lk_cbk]
>   0-gfs33-server: 3755325: LK -2 (--) ==> -1 (No such file or directory)
>

This line is the point to debug deeper. Can you put a breakpoint here in
server_lk_cbk and print the backtrace why errno is ENOENT? The rest of all
the observations are cascaded effects.

Avati



> [2012-08-20 03:11:06.119141] I [server3_1-fops.c:203:server_lk_cbk]
>   0-gfs33-server: 3755326: LK -2 (--) ==> -1 (No such file or directory)
>
> client0 glusterfsd kernel trace:
>  20304     56 glusterfsd CALL  __lstat50(0xb15ff470,0xb15ff8ec)
>  20304     56 glusterfsd NAMI
> "/export/wd3a/.glusterfs/f9/11/f9111a97-03ed-4409-976b-325d05240edd"
>  20304     56 glusterfsd RET   __lstat50 -1 errno 2 No such file or
> directory
> [it retries 5 times, then logs the error]
>
>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu at netbsd.org
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20120819/01fe8a54/attachment-0003.html>