[Gluster-devel] gluster doesn't like Oracle's FSINFO RPC call

[Niels de Vos]

On Mon, Apr 08, 2013 at 01:57:55PM -0400, Michael Brown wrote:
> I'm trying to get Oracle's DNFS working against gluster's internal NFS
> and I've run into a snag. After Oracle mounts the exported NFS
> filesystem the FSINFO call fails.
> 
> Let's look with wireshark:
...
> *Remote Procedure Call, Type:Call XID:0x47349478*
>     Program: NFS (100003)
>     Program Version: 3
>     Procedure: FSINFO (19)
> Network File System, FSINFO Call DH:0x10650fe6
>     [Program Version: 3]
>     [V3 Procedure: FSINFO (19)]
>     object
>         length: 34
>         [hash (CRC-32): 0x10650fe6]
>         [Name: 192.168.10.1:/gv0/fleming3/db0/ALTUS_data]
>         filehandle:
> 3a4f20117b487f884f169490a0349afacf71331965f573144e93b289a395148edfe5

Bytes of the object parameter:

	    < length  > <         10 bytes          >
00a0  .. .. 00 00 00 22 3a 4f 20 11 7b 48 7f 88 4f 16   .....":O .{H..O.
....  <               16 bytes                      >
00b0  94 90 a0 34 9a fa cf 71 33 19 65 f5 73 14 4e 93   ...4...q3.e.s.N.
....  <      8 bytes        >
00c0  b2 89 a3 95 14 8e df e5                           ........

Makes a total of a 34 byte sized fhandle.

...
> Network File System, FSINFO Call DH:0xb2ae682f
>     [Program Version: 3]
>     [V3 Procedure: FSINFO (19)]
>     object
>         length: 34
>         [hash (CRC-32): 0xb2ae682f]
>         [Name: 192.168.10.1:/gv0/fleming1/db0/ALTUS_data]
>         filehandle:
> 3a4f20117b487f884f169490a0349afacf71e8bd0e2198c34cb88a0231dbeb071035

Bytes of the object parameter:

			< length  > <    6 bytes    >
00b0  .. .. .. .. .. .. 00 00 00 22 3a 4f 20 11 7b 48   .........":O .{H
....  <               16 bytes                      >
00c0  7f 88 4f 16 94 90 a0 34 9a fa cf 71 e8 bd 0e 21   ..O....4...q...!
....  <               12 bytes          > < ? >
00d0  98 c3 4c b8 8a 02 31 db eb 07 10 35 00 00         ..L...1....5..

Note the two 0-bytes at the end. Thinking about that, the following may 
be the case:

XDR (http://tools.ietf.org/html/rfc4506, the encoding used for the RPC 
protocol) uses 'blocks' for alignment. A fhandle byte array that is 
34-bytes long, needs to be 34 / 4 + 1 = 36 bytes in size. The 'length' 
given in the structure tells the consumer to ignore the two tailing 
bytes.

The NFSv3 specification (http://tools.ietf.org/html/rfc1813#page-21) 
defines the nfs_fh3 as a opaque (not bytes) structure.

> So for some reason, gluster is happy with Linux's request but not 
> Oracle's.

My guess is that this (untested) change would fix it, can you try that?

--- a/rpc/xdr/src/xdr-nfs3.c
+++ b/rpc/xdr/src/xdr-nfs3.c
@@ -184,7 +184,7 @@ xdr_specdata3 (XDR *xdrs, specdata3 *objp)
 bool_t
 xdr_nfs_fh3 (XDR *xdrs, nfs_fh3 *objp)
 {
-	 if (!xdr_bytes (xdrs, (char **)&objp->data.data_val, (u_int *) &objp->data.data_len, NFS3_FHSIZE))
+	 if (!xdr_opaque (xdrs, &objp, (u_int *) &objp->data.data_len))
 		 return FALSE;
 	return TRUE;
 }


HTH,
Niels

> 
> All I get out of gluster is:
> [2013-04-08 12:54:32.206312] E [nfs3.c:4741:nfs3svc_fsinfo] 0-nfs-nfsv3:
> Error decoding arguments
> 
> 
> I've attached abridged packet captures and text explanations of the
> packets (thanks to wireshark).
> 
> Can someone please look at this and determine if it's gluster's parsing
> of the RPC call to blame, or if it's Oracle?
> 
> This is the same setup on which I reported the NFS race condition bug.
> It does have that patch applied.
> Details:
> http://lists.gnu.org/archive/html/gluster-devel/2013-04/msg00014.html
> 
> Thanks,
> 
> Michael
> 
> -- 
> Michael Brown               | `One of the main causes of the fall of
> Systems Consultant          | the Roman Empire was that, lacking zero,
> Net Direct Inc.             | they had no way to indicate successful
> ?: +1 519 883 1172 x5106    | termination of their C programs.' - Firth
> 





> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel


-- 
Niels de Vos
Sr. Software Maintenance Engineer
Support Engineering Group
Red Hat Global Support Services