[Gluster-devel] NFS re-export works for me
Raghavendra G
raghavendra.hg at gmail.com
Mon Mar 9 11:04:46 UTC 2009
Hi,
starting glusterfs with direct-io disabled (glusterfs --disable-direct-io)
ensures working of NFS re-export of glusterfs.
regards,
On Fri, Apr 27, 2007 at 12:47 AM, gpvcs at tiscali.fr <gpvcs at tiscali.fr> wrote:
> Hi Avati,
>
> I first thought that I found the error when suddenly it was
> working on my machine but I never managed to reproduce it.
> Then I realized that in cluster mode, one of my machine was
> running using the default fuse but it did it again on my
> standalone machine.
> One think for sure is that with default fuse (API 7.6)
> NFS mount fail always. You have to use a latter fuse.
>
> After many try I started to trace NFS and Fuse in kernel mode:
>
> cd /proc/sys/sunrpc
> echo 32767 > nfs_debug
> echo 32767 > nfsd_debug
>
> and did my test again:
>
> exportfs:
> localhost:/gfs/fs1 on /gfs/d610.nfsV3 type nfs
> (rw,nfsvers=3,noac,hard,intr,tcp,lock,posix,addr=127.0.0.1)
>
> cd /gfs/nfsV3
> ls > 11
>
>
> The trace between a successfull writing other NFS and error
> change here:
> Apr 26 16:21:06 fturi kernel: NFS call write 6 @ 0
> Apr 26 16:21:06 fturi kernel: nfsd_dispatch: vers 3 proc 7
> Apr 26 16:21:06 fturi kernel: nfsd: WRITE(3) 24: 02010001
> 00000000 00000003 00000000 00000001 00000000 6 bytes at 0 stable
> Apr 26 16:21:06 fturi kernel: nfsd: fh_verify(24: 02010001
> 00000000 00000003 00000000 00000001 00000000)
>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> Apr 26 16:21:06 fturi kernel: # NO PERM 531 nfsd: write
> complete err=-1
> Apr 26 16:21:06 fturi kernel: NFS reply write: -1
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>
> The NO PERM message comes from this function in fuse
> 2.6.3/kernel/file.c
>
> static int fuse_get_user_pages(struct fuse_req *req, const
> char __user *buf,
> unsigned nbytes, int write)
> {
> unsigned long user_addr = (unsigned long) buf;
> unsigned offset = user_addr & ~PAGE_MASK;
> int npages;
>
> /* This doesn't work with nfsd */
> if (!current->mm)
> {printk(KERN_CRIT "# NO PERM
> %d",__LINE__);return -EPERM;};
>
> nbytes = min(nbytes, (unsigned)
> FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT);
> npages = (nbytes + offset + PAGE_SIZE - 1) >>
> PAGE_SHIFT;
> npages = max(npages, 1);
> npages = min(npages, FUSE_MAX_PAGES_PER_REQ);
> down_read(¤t->mm->mmap_sem);
> npages = get_user_pages(current, current->mm,
> user_addr, npages, write,
> 0, req->pages, NULL);
> up_read(¤t->mm->mmap_sem);
> if (npages < 0)
> return npages;
>
> req->num_pages = npages;
> req->page_offset = offset;
> return 0;
> }
>
>
> I Just added the dprintk to make sure it was there. There is
> a comment about: /* doesnt work with nfsd */
>
> So, its look like its fuse refusing (error=-1 = NOPERM) the
> writting other NFS.
> I'm puzzled by the fact it works under another linux. maybe
> the fuse_get_user_pages is not called in your case of the
> structure "current->mm" is filled up in your case.
>
> This is the trace collected by the debug mode of server.vol
> and client.vol ( debug on top of the brick):
>
> glusterfsd is empty
> glusterfs contains:
> [Apr 26 16:21:02] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:21:02] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/11)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=64356, st_
> mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
> st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
> 15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
> 15:54:32]})
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/11)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=64356, st_
> mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
> st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
> 15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
> 15:54:32]})
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:21:04] [DEBUG/trace.c:967/trace_open()]
> trace:trace_open (*this=0x8395650, path=/11, flags=32769,
> mode=0)
> [Apr 26 16:21:04] [DEBUG/trace.c:88/trace_open_cbk()]
> trace:trace_open_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *ctx=0x8396f58), *buf=0x8396c60 {st_dev=64768,
> st_ino=
> 64356, st_mode=33206, st_nlink=1, st_uid=99, st_gid=99,
> st_rdev=0, st_size=0, st_blksize=4096, st_blocks=8,
> st_atime=[Apr 26 15:48:19], st_mtime=[Apr 26 15:54:32], st_cti
> me=[Apr 26 15:54:32]})
> [Apr 26 16:21:04] [DEBUG/trace.c:1089/trace_release()]
> trace:trace_release (*this=0x8395650, *ctx=0x8396f58)
> [Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/11)
>
> [Apr 26 16:21:04] [DEBUG/trace.c:386/trace_release_cbk()]
> trace:trace_release_cbk (*this=0x8395650, op_ret=0, op_errno=95)
> [Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8397098 {st_dev=64768, st_ino=64356, st_
> mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
> st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
> 15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
> 15:54:32]})
> [Apr 26 16:28:28] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:28] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:28] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:28] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
> trace:trace_getattr (*this=0x8395650, path=/)
>
> [Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
> trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
> op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
> mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
> st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
> 26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
> r 26 09:42:58]})
> [Apr 26 16:28:31] [DEBUG/trace.c:1155/trace_getxattr()]
> trace:trace_getxattr (*this=0x8395650, path=/,
> name=system.posix_acl_access, size=0)
> [Apr 26 16:28:31] [DEBUG/trace.c:513/trace_getxattr_cbk()]
> trace:trace_getxattr_cbk (*this=0x8395650, op_ret=-1,
> op_errno=95)
>
>
> Cdt FT
>
>
>
>
>
> ---------- Initial Header -----------
>
> From : Anand Avati <avati at zresearch.com>
> To : gpvcs at tiscali.fr
> Cc : gluster-devel at nongnu.org
> Date : Wed, 18 Apr 2007 08:55:26 -0700
> Subject : Re: Re:[Gluster-devel] NFS re-export works for me
>
> I was able to successfully mount a glusterfs export via NFS
> too. I am
> using slackware 10.2 and did nothing special apart from using
> fuse-2.6.0. Maybe your problem could be because of the AIX
> client.
> please try from a GNU/Linux client and see if it helps.
> Attaching the
> log file by loading debug/trace will help get a better picture.
>
> regards,
> avati
>
> On Wed, Apr 18, 2007 at 11:25:49AM -0400, Brent A Nelson wrote:
> > rw,fsid=10 are the export options.
> >
> > 2.6.15 is the kernel (it's a Ubuntu Dapper with its
> kernel, probably
> > closer to 2.6.16). Fuse 2.6.3 was added from an extra
> repository.
> >
> > I used no options at all on the Ubuntu Breezy NFS client
> mount. Looking
> > at the mtab, no special options are listed there, either,
> so the options
> > are whatever is default for this platform.
> >
> > Thanks,
> >
> > Brent
> >
> > On Wed, 18 Apr 2007, gpvcs at tiscali.fr wrote:
> >
> > >Hi Brent,
> > >
> > >That's a very good news for me. Could you please tell me
> > >which linux kernel version are you using (uname -a) and
> > >/etc/exports line and mount command you used ?
> > >I'm not lucky with NFS and will very appreciate thoses
> details.
> > >
> > >Have a nice day, Francois
> > >---------- Initial Header -----------
> > >
> > >From : gluster-devel-bounces+gpvcs=tiscali.fr at nongnu.org
> > >To : gluster-devel at nongnu.org
> > >Cc :
> > >Date : Wed, 18 Apr 2007 00:51:51 -0400 (EDT)
> > >Subject : [Gluster-devel] NFS re-export works for me
> > >
> > >I tried a quick NFS re-export to another Linux box (with the
> > >fsid= option
> > >and fuse 2.6.3 on the NFS server, of course). I didn't test
> > >it much, but
> > >copies to and from the NFS filesystem worked just fine. I
> > >think writes
> > >were very slow, but they didn't give me any trouble.
> > >
> > >No special mount options (such as NFS version or transport)
> > >were needed.
> > >
> > >Thanks,
> > >
> > >Brent
> > >
> > >
> > >_______________________________________________
> > >Gluster-devel mailing list
> > >Gluster-devel at nongnu.org
> > >http://lists.nongnu.org/mailman/listinfo/gluster-devel
> > >
> >
> > ------------------------ ALICE C'EST ENCORE MIEUX AVEC
> CANAL+ LE BOUQUET !
> > ---------------
> > Découvrez vite l'offre exclusive ALICEBOX et CANAL+ LE
> BOUQUET, en cliquant
> > ici http://alicebox.fr
> > Soumis à conditions.
> > >
> > >
>
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org
> > http://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>
> --
> ultimate_answer_t
> deep_thought (void)
> {
> sleep (years2secs (7500000));
> return 42;
> }
>
>
> ------------------------ ALICE C'EST ENCORE MIEUX AVEC CANAL+ LE BOUQUET !
> ---------------
> Découvrez vite l'offre exclusive ALICEBOX et CANAL+ LE BOUQUET, en cliquant
> ici http://alicebox.fr
> Soumis à conditions.
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>
--
Raghavendra G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20090309/e0859867/attachment-0003.html>
More information about the Gluster-devel
mailing list