Re: Re:[Gluster-devel] NFS re-export works for me
gpvcs@tiscali.fr
gpvcs at tiscali.fr
Thu Apr 26 20:47:00 UTC 2007
Hi Avati,
I first thought that I found the error when suddenly it was
working on my machine but I never managed to reproduce it.
Then I realized that in cluster mode, one of my machine was
running using the default fuse but it did it again on my
standalone machine.
One think for sure is that with default fuse (API 7.6)
NFS mount fail always. You have to use a latter fuse.
After many try I started to trace NFS and Fuse in kernel mode:
cd /proc/sys/sunrpc
echo 32767 > nfs_debug
echo 32767 > nfsd_debug
and did my test again:
exportfs:
localhost:/gfs/fs1 on /gfs/d610.nfsV3 type nfs
(rw,nfsvers=3,noac,hard,intr,tcp,lock,posix,addr=127.0.0.1)
cd /gfs/nfsV3
ls > 11
The trace between a successfull writing other NFS and error
change here:
Apr 26 16:21:06 fturi kernel: NFS call write 6 @ 0
Apr 26 16:21:06 fturi kernel: nfsd_dispatch: vers 3 proc 7
Apr 26 16:21:06 fturi kernel: nfsd: WRITE(3) 24: 02010001
00000000 00000003 00000000 00000001 00000000 6 bytes at 0 stable
Apr 26 16:21:06 fturi kernel: nfsd: fh_verify(24: 02010001
00000000 00000003 00000000 00000001 00000000)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Apr 26 16:21:06 fturi kernel: # NO PERM 531 nfsd: write
complete err=-1
Apr 26 16:21:06 fturi kernel: NFS reply write: -1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
The NO PERM message comes from this function in fuse
2.6.3/kernel/file.c
static int fuse_get_user_pages(struct fuse_req *req, const
char __user *buf,
unsigned nbytes, int write)
{
unsigned long user_addr = (unsigned long) buf;
unsigned offset = user_addr & ~PAGE_MASK;
int npages;
/* This doesn't work with nfsd */
if (!current->mm)
{printk(KERN_CRIT "# NO PERM
%d",__LINE__);return -EPERM;};
nbytes = min(nbytes, (unsigned)
FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT);
npages = (nbytes + offset + PAGE_SIZE - 1) >>
PAGE_SHIFT;
npages = max(npages, 1);
npages = min(npages, FUSE_MAX_PAGES_PER_REQ);
down_read(¤t->mm->mmap_sem);
npages = get_user_pages(current, current->mm,
user_addr, npages, write,
0, req->pages, NULL);
up_read(¤t->mm->mmap_sem);
if (npages < 0)
return npages;
req->num_pages = npages;
req->page_offset = offset;
return 0;
}
I Just added the dprintk to make sure it was there. There is
a comment about: /* doesnt work with nfsd */
So, its look like its fuse refusing (error=-1 = NOPERM) the
writting other NFS.
I'm puzzled by the fact it works under another linux. maybe
the fuse_get_user_pages is not called in your case of the
structure "current->mm" is filled up in your case.
This is the trace collected by the debug mode of server.vol
and client.vol ( debug on top of the brick):
glusterfsd is empty
glusterfs contains:
[Apr 26 16:21:02] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:21:02] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/11)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=64356, st_
mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
15:54:32]})
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/11)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=64356, st_
mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
15:54:32]})
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:21:04] [DEBUG/trace.c:967/trace_open()]
trace:trace_open (*this=0x8395650, path=/11, flags=32769,
mode=0)
[Apr 26 16:21:04] [DEBUG/trace.c:88/trace_open_cbk()]
trace:trace_open_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *ctx=0x8396f58), *buf=0x8396c60 {st_dev=64768,
st_ino=
64356, st_mode=33206, st_nlink=1, st_uid=99, st_gid=99,
st_rdev=0, st_size=0, st_blksize=4096, st_blocks=8,
st_atime=[Apr 26 15:48:19], st_mtime=[Apr 26 15:54:32], st_cti
me=[Apr 26 15:54:32]})
[Apr 26 16:21:04] [DEBUG/trace.c:1089/trace_release()]
trace:trace_release (*this=0x8395650, *ctx=0x8396f58)
[Apr 26 16:21:04] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/11)
[Apr 26 16:21:04] [DEBUG/trace.c:386/trace_release_cbk()]
trace:trace_release_cbk (*this=0x8395650, op_ret=0, op_errno=95)
[Apr 26 16:21:04] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8397098 {st_dev=64768, st_ino=64356, st_
mode=33206, st_nlink=1, st_uid=99, st_gid=99, st_rdev=0,
st_size=0, st_blksize=4096, st_blocks=8, st_atime=[Apr 26
15:48:19], st_mtime=[Apr 26 15:54:32], st_ctime=[Apr 26
15:54:32]})
[Apr 26 16:28:28] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:28] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:28] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:28] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:688/trace_getattr()]
trace:trace_getattr (*this=0x8395650, path=/)
[Apr 26 16:28:31] [DEBUG/trace.c:109/trace_getattr_cbk()]
trace:trace_getattr_cbk (*this=0x8395650, op_ret=0,
op_errno=95, *buf=0x8396c60 {st_dev=64768, st_ino=68494, st_
mode=16895, st_nlink=2, st_uid=0, st_gid=500, st_rdev=0,
st_size=4096, st_blksize=4096, st_blocks=16, st_atime=[Apr
26 16:19:02], st_mtime=[Apr 26 09:42:58], st_ctime=[Ap
r 26 09:42:58]})
[Apr 26 16:28:31] [DEBUG/trace.c:1155/trace_getxattr()]
trace:trace_getxattr (*this=0x8395650, path=/,
name=system.posix_acl_access, size=0)
[Apr 26 16:28:31] [DEBUG/trace.c:513/trace_getxattr_cbk()]
trace:trace_getxattr_cbk (*this=0x8395650, op_ret=-1,
op_errno=95)
Cdt FT
---------- Initial Header -----------
From : Anand Avati <avati at zresearch.com>
To : gpvcs at tiscali.fr
Cc : gluster-devel at nongnu.org
Date : Wed, 18 Apr 2007 08:55:26 -0700
Subject : Re: Re:[Gluster-devel] NFS re-export works for me
I was able to successfully mount a glusterfs export via NFS
too. I am
using slackware 10.2 and did nothing special apart from using
fuse-2.6.0. Maybe your problem could be because of the AIX
client.
please try from a GNU/Linux client and see if it helps.
Attaching the
log file by loading debug/trace will help get a better picture.
regards,
avati
On Wed, Apr 18, 2007 at 11:25:49AM -0400, Brent A Nelson wrote:
> rw,fsid=10 are the export options.
>
> 2.6.15 is the kernel (it's a Ubuntu Dapper with its
kernel, probably
> closer to 2.6.16). Fuse 2.6.3 was added from an extra
repository.
>
> I used no options at all on the Ubuntu Breezy NFS client
mount. Looking
> at the mtab, no special options are listed there, either,
so the options
> are whatever is default for this platform.
>
> Thanks,
>
> Brent
>
> On Wed, 18 Apr 2007, gpvcs at tiscali.fr wrote:
>
> >Hi Brent,
> >
> >That's a very good news for me. Could you please tell me
> >which linux kernel version are you using (uname -a) and
> >/etc/exports line and mount command you used ?
> >I'm not lucky with NFS and will very appreciate thoses
details.
> >
> >Have a nice day, Francois
> >---------- Initial Header -----------
> >
> >From : gluster-devel-bounces+gpvcs=tiscali.fr at nongnu.org
> >To : gluster-devel at nongnu.org
> >Cc :
> >Date : Wed, 18 Apr 2007 00:51:51 -0400 (EDT)
> >Subject : [Gluster-devel] NFS re-export works for me
> >
> >I tried a quick NFS re-export to another Linux box (with the
> >fsid= option
> >and fuse 2.6.3 on the NFS server, of course). I didn't test
> >it much, but
> >copies to and from the NFS filesystem worked just fine. I
> >think writes
> >were very slow, but they didn't give me any trouble.
> >
> >No special mount options (such as NFS version or transport)
> >were needed.
> >
> >Thanks,
> >
> >Brent
> >
> >
> >_______________________________________________
> >Gluster-devel mailing list
> >Gluster-devel at nongnu.org
> >http://lists.nongnu.org/mailman/listinfo/gluster-devel
> >
>
> ------------------------ ALICE C'EST ENCORE MIEUX AVEC
CANAL+ LE BOUQUET !
> ---------------
> Découvrez vite l'offre exclusive ALICEBOX et CANAL+ LE
BOUQUET, en cliquant
> ici http://alicebox.fr
> Soumis à conditions.
> >
> >
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
--
ultimate_answer_t
deep_thought (void)
{
sleep (years2secs (7500000));
return 42;
}
------------------------ ALICE C'EST ENCORE MIEUX AVEC CANAL+ LE BOUQUET ! ---------------
Découvrez vite l'offre exclusive ALICEBOX et CANAL+ LE BOUQUET, en cliquant ici http://alicebox.fr
Soumis à conditions.
More information about the Gluster-devel
mailing list