[Gluster-devel] Issues with replacing hard links with symlinks in the .glusterfs directory

Niels de Vos ndevos at redhat.com
Sun Oct 25 08:53:23 UTC 2015


On Sun, Oct 25, 2015 at 04:16:14AM -0400, Marty Rosenberg wrote:
> ok, now stopped at fuse_setattr_cbk, here is a bit of my session with gdb:
> (gdb) p op_ret
> $1 = 0
> (gdb) p op_errno
> $2 = 0
> (gdb) p *statpost
> $3 = {ia_ino = 9708385387769469648, ia_gfid =
> "\r(H\266\237\347H*\206\273\035\022\254\227\232\320", ia_dev = 1206432852,
> ia_type = IA_IFLNK, ia_prot = {suid = 0 '\000', sgid = 0 '\000', sticky = 0
> '\000',
>     owner = {read = 1 '\001', write = 1 '\001', exec = 1 '\001'}, group =
> {read = 1 '\001', write = 1 '\001', exec = 1 '\001'}, other = {read = 1
> '\001', write = 1 '\001', exec = 1 '\001'}}, ia_nlink = 0,
>   ia_uid = 0, ia_gid = 0, ia_rdev = 1099511562495, ia_size = 71, ia_blksize
> = 4096, ia_blocks = 1, ia_atime = 1445755399, ia_atime_nsec = 635070168,
> ia_mtime = 1444602630, ia_mtime_nsec = 578330000,
>   ia_ctime = 1444602468, ia_ctime_nsec = 558067808}
> (gdb) p *statp
> $4 = {ia_ino = 9708385387769469648, ia_gfid =
> "\r(H\266\237\347H*\206\273\035\022\254\227\232\320", ia_dev = 1206432852,
> ia_type = IA_IFLNK, ia_prot = {suid = 0 '\000', sgid = 0 '\000', sticky = 0

If this was returned on executing "chmod +x some_file", than this
statpost structure is incorrect. This seems to be the stat/iatt from the
symlink. Probably because you replaced the .glusterfs/ hardlink with
symlinks in your environment, iiuc. The POSIX xlator that handled the
setattr call should also get the post-operation attributes of the
modified file, not the attributes of the symlink (stat() vs lstat()?).

Maybe FUSE (in the kernel) has a check for unexpected changes, like
ia_type/ia_ino/.. and that causes the EIO error?

This is the first possibility that I can think of, hope it helps,
Niels


> '\000',
>     owner = {read = 1 '\001', write = 1 '\001', exec = 1 '\001'}, group =
> {read = 1 '\001', write = 1 '\001', exec = 1 '\001'}, other = {read = 1
> '\001', write = 1 '\001', exec = 1 '\001'}}, ia_nlink = 0,
>   ia_uid = 0, ia_gid = 0, ia_rdev = 1099511562495, ia_size = 71, ia_blksize
> = 4096, ia_blocks = 1, ia_atime = 1445755086, ia_atime_nsec = 307072106,
> ia_mtime = 1444602630, ia_mtime_nsec = 578330000,
>   ia_ctime = 1444602468, ia_ctime_nsec = 558067808}
> 
> 
> In order to track this down, I tried setting printing out the error value
> every time a message is written to the kernel.  it was never non-zero.
> 
> On Fri, Oct 23, 2015 at 2:04 PM, Marty Rosenberg <marty.rosenberg at gmail.com>
> wrote:
> 
> > Sure, I think I've set a breakpoint there before, and nothing looked awry,
> > but I'll do that when I get home.
> >
> > On Fri, Oct 23, 2015 at 1:57 PM, Vijay Bellur <vbellur at redhat.com> wrote:
> >
> >> On Friday 23 October 2015 07:28 AM, Marty Rosenberg wrote:
> >>
> >>> First off: I've based my work off of the release of 3.7.3, since it was
> >>> the most recent release when I started this project, and I couldn't get
> >>> HEAD to build on freebsd. (I'm using a freebsd server, and linux clients)
> >>>
> >>> I realize that many things will be broken by doing this (renaming open
> >>> files, deleting open files, possibly some other stuff), but I can live
> >>> with those limitations.
> >>>
> >>> What I've done:
> >>>   I've modified the code to failback to a symlink if making a hardlink
> >>> fails (which it will do somewhat frequently due to being on a different
> >>> filesystem).
> >>> I created an extended property on symlinks that are emulating hard links
> >>> changed the setattr code to check this before it tries to set the
> >>> attributs, and if it is set, it dereferences the link, then proceeds
> >>> with the setattr
> >>>
> >>> To test this, I made a file, and ran chmod +x on it
> >>> the good: attributes were correctly set on the file!
> >>> the bad: chmod says it failed with EIO
> >>> my issue: I have no clue where this EIO is coming from.  Under the
> >>> hood,s chmod is calling fchmodat
> >>>
> >>> After no luck with printf debugging, I just ran gluster under gdb, and
> >>> set a breakpoint on send_fuse_iov.  Here's the backtrace:
> >>> #0  send_fuse_iov (this=0x63a150, finh=0x7fffe0005fe0,
> >>> iov_out=0x7ffff08e7500, count=2) at fuse-bridge.c:158
> >>> #1  0x00007ffff550fcfd in send_fuse_data (this=0x63a150,
> >>> finh=0x7fffe0005fe0, data=0x7ffff08e75a0, size=104) at fuse-bridge.c:197
> >>> #2  0x00007ffff5511be1 in fuse_attr_cbk (frame=0x7fffe000145c,
> >>> cookie=0x7fffe000616c, this=0x63a150, op_ret=0, op_errno=117,
> >>> buf=0x7fffe0006734, xdata=0x0) at fuse-bridge.c:734
> >>>
> >>
> >>
> >> Can you break at fuse_setattr_cbk() and check the backtrace? That could
> >> provide more relevant information for the chmod failure.
> >>
> >> Regards,
> >> Vijay
> >>
> >>
> >

> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20151025/a237af70/attachment.sig>


More information about the Gluster-devel mailing list