[Gluster-users] NFS Client issues with Gluster Server 3.6.9

Christian Rice crice at pandora.com
Wed Mar 9 09:25:09 UTC 2016


setting server.manage-gids to on fixed remote i/o error messages for me, accessing over nfs, gluster 3.6.2-2 on Debian Wheezy.  worth a shot.

From: <gluster-users-bounces at gluster.org<mailto:gluster-users-bounces at gluster.org>> on behalf of Mark Selby <mselby at unseelie.name<mailto:mselby at unseelie.name>>
Date: Monday, March 7, 2016 at 8:11 AM
To: Jiffin Tony Thottan <jthottan at redhat.com<mailto:jthottan at redhat.com>>, "gluster-users at gluster.org<mailto:gluster-users at gluster.org>" <gluster-users at gluster.org<mailto:gluster-users at gluster.org>>
Subject: Re: [Gluster-users] NFS Client issues with Gluster Server 3.6.9

Here are the logs that you requested

Please let me know if I can send you anything else.

I really appreciate you taking a look at this - thanks!


root at dc1strg001x /root 547# gluster vol info backups

Volume Name: backups
Type: Replicate
Volume ID: 71a26ea6-632d-4a1d-8610-e782ce2a5100
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: dc1strg001x:/zfspool/glusterfs/backups/data
Brick2: dc1strg002x:/zfspool/glusterfs/backups/data
Options Reconfigured:
nfs.disable: off


root at dc1strg001x /var/log/glusterfs 551# cat nfs.log
[2016-03-07 16:03:14.257919] I [MSGID: 100030] [glusterfsd.c:2318:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.6
(args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p
/var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S
/var/run/gluster/ad38be3bd1baece29e1b672e6659ae60.socket)
[2016-03-07 16:03:14.267862] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
[2016-03-07 16:03:14.273283] I
[rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service:
Configured rpc.outstanding-rpc-limit with value 16
[2016-03-07 16:03:14.284154] W [MSGID: 112153]
[mount3.c:3929:mnt3svc_init] 0-nfs-mount: Exports auth has been disabled!
[2016-03-07 16:03:14.306163] I [rpc-drc.c:694:rpcsvc_drc_init]
0-rpc-service: DRC is turned OFF
[2016-03-07 16:03:14.306216] I [MSGID: 112110] [nfs.c:1494:init] 0-nfs:
NFS service started
[2016-03-07 16:03:14.312901] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 2
[2016-03-07 16:03:14.314078] W [graph.c:357:_log_if_unknown_option]
0-nfs-server: option 'rpc-auth.auth-glusterfs' is not recognized
[2016-03-07 16:03:14.314137] W [graph.c:357:_log_if_unknown_option]
0-nfs-server: option 'rpc-auth-allow-insecure' is not recognized
[2016-03-07 16:03:14.314185] W [graph.c:357:_log_if_unknown_option]
0-nfs-server: option 'transport-type' is not recognized
[2016-03-07 16:03:14.314270] I [MSGID: 114020] [client.c:2118:notify]
0-backups-client-0: parent translators are ready, attempting connect on
transport
[2016-03-07 16:03:14.315341] I [MSGID: 114020] [client.c:2118:notify]
0-backups-client-1: parent translators are ready, attempting connect on
transport
[2016-03-07 16:03:14.315923] I [rpc-clnt.c:1847:rpc_clnt_reconfig]
0-backups-client-0: changing port to 49152 (from 0)
Final graph:
+------------------------------------------------------------------------------+
   1: volume backups-client-0
   2:     type protocol/client
   3:     option ping-timeout 42
   4:     option remote-host dc1strg001x
   5:     option remote-subvolume /zfspool/glusterfs/backups/data
   6:     option transport-type socket
   7:     option username 42fa7a62-1420-4169-ad00-53c3481dbe5b
   8:     option password b71b3c88-51e0-464c-8b09-14b661fdb4d3
   9:     option send-gids true
  10: end-volume
  11:
  12: volume backups-client-1
  13:     type protocol/client
  14:     option ping-timeout 42
  15:     option remote-host dc1strg002x
[2016-03-07 16:03:14.317412] I [MSGID: 114057]
[client-handshake.c:1437:select_server_supported_programs]
0-backups-client-0: Using Program GlusterFS 3.3, Num (1298437), Version
(330)
  16:     option remote-subvolume /zfspool/glusterfs/backups/data
  17:     option transport-type socket
  18:     option username 42fa7a62-1420-4169-ad00-53c3481dbe5b
  19:     option password b71b3c88-51e0-464c-8b09-14b661fdb4d3
  20:     option send-gids true
  21: end-volume
  22:
  23: volume backups-replicate-0
  24:     type cluster/replicate
  25:     subvolumes backups-client-0 backups-client-1
  26: end-volume
  27:
  28: volume backups-dht
  29:     type cluster/distribute
  30:     subvolumes backups-replicate-0
  31: end-volume
  32:
  33: volume backups-write-behind
  34:     type performance/write-behind
  35:     subvolumes backups-dht
  36: end-volume
  37:
  38: volume backups
  39:     type debug/io-stats
  40:     option latency-measurement off
  41:     option count-fop-hits off
  42:     subvolumes backups-write-behind
  43: end-volume
  44:
  45: volume nfs-server
  46:     type nfs/server
  47:     option rpc-auth.auth-glusterfs on
  48:     option rpc-auth.auth-unix on
  49:     option rpc-auth.auth-null on
  50:     option rpc-auth.ports.insecure on
  51:     option rpc-auth-allow-insecure on
  52:     option transport-type socket
  53:     option transport.socket.listen-port 2049
  54:     option nfs.dynamic-volumes on
  55:     option nfs.nlm on
  56:     option nfs.drc off
  57:     option rpc-auth.addr.backups.allow *
  58:     option nfs3.backups.volume-id 71a26ea6-632d-4a1d-8610-e782ce2a5100
  59:     option nfs.backups.disable off
  60:     option nfs.logs.disable off
  61:     option nfs.users.disable off
  62:     subvolumes backups
  63: end-volume
  64:
+------------------------------------------------------------------------------+
[2016-03-07 16:03:14.318157] I [MSGID: 114046]
[client-handshake.c:1213:client_setvolume_cbk] 0-backups-client-0:
Connected to backups-client-0, attached to remote volume
'/zfspool/glusterfs/backups/data'.
[2016-03-07 16:03:14.318276] I [MSGID: 114047]
[client-handshake.c:1224:client_setvolume_cbk] 0-backups-client-0:
Server and Client lk-version numbers are not same, reopening the fds
[2016-03-07 16:03:14.318400] I [MSGID: 108005]
[afr-common.c:3841:afr_notify] 0-backups-replicate-0: Subvolume
'backups-client-0' came back up; going online.
[2016-03-07 16:03:14.318470] I [MSGID: 114035]
[client-handshake.c:193:client_set_lk_version_cbk] 0-backups-client-0:
Server lk version = 1
[2016-03-07 16:03:14.496642] I [rpc-clnt.c:1847:rpc_clnt_reconfig]
0-backups-client-1: changing port to 49152 (from 0)
[2016-03-07 16:03:14.498394] I [MSGID: 114057]
[client-handshake.c:1437:select_server_supported_programs]
0-backups-client-1: Using Program GlusterFS 3.3, Num (1298437), Version
(330)
[2016-03-07 16:03:14.505580] I [MSGID: 114046]
[client-handshake.c:1213:client_setvolume_cbk] 0-backups-client-1:
Connected to backups-client-1, attached to remote volume
'/zfspool/glusterfs/backups/data'.
[2016-03-07 16:03:14.505627] I [MSGID: 114047]
[client-handshake.c:1224:client_setvolume_cbk] 0-backups-client-1:
Server and Client lk-version numbers are not same, reopening the fds
[2016-03-07 16:03:14.506210] I [MSGID: 114035]
[client-handshake.c:193:client_set_lk_version_cbk] 0-backups-client-1:
Server lk version = 1
[2016-03-07 16:03:14.507836] I [MSGID: 108031]
[afr-common.c:1782:afr_local_discovery_cbk] 0-backups-replicate-0:
selecting local read_child backups-client-0




On 3/6/16 9:13 PM, Jiffin Tony Thottan wrote:


On 05/03/16 07:12, Mark Selby wrote:
I am trying to use GlusterFS as a general purpose NFS file server. I
have tried using the FUSE client but the performance fall off vs NFS
is quite large

Both the client and the server are Ubuntu 14.04.

I am using Gluster 3.6.9 because of the FUSE performance issues that
have been reported with 3.7.8 (see
https://bugzilla.redhat.com/show_bug.cgi?id=1309462)

I am having serious issues with a generic NFS client as shown by the
issues below. Basically most FOPs are giving me a Remote I/O error.

I would not think I was 1st person to see these issues - but my
Google Fu is not working.

Any and all help would be much appreciated

BTW - These operation against a plain Linux NFS server work fine.


root at dc1strg001x /var/log 448# gluster volume status
Status of volume: backups
Gluster process Port    Online  Pid
------------------------------------------------------------------------------

Brick dc1strg001x:/zfspool/glusterfs/backups/data 49152   Y 6462
Brick dc1strg002x:/zfspool/glusterfs/backups/data 49152   Y 6382
NFS Server on localhost 2049    Y       6619
Self-heal Daemon on localhost N/A     Y       6626
NFS Server on dc1strg002x 2049    Y       6502
Self-heal Daemon on dc1strg002x N/A     Y       6509


root at vc1test001 /root 735# mount -o vers=3 -t nfs
dc1strg001x:/backups /mnt/backups_nfs

root at vc1test001 /mnt/backups_nfs 737# dd if=/dev/zero of=testfile
bs=16k count=16384
16384+0 records in
16384+0 records out
268435456 bytes (268 MB) copied, 2.46237 s, 109 MB/s

root at vc1test001 /mnt/backups_nfs 738# rm testfile

root at vc1test001 /mnt/backups_nfs 739# dd if=/dev/zero of=testfile
bs=16k count=16384
dd: failed to open ~testfile~: Remote I/O error

root at vc1test001 /var/tmp 743# rsync -av testfile /mnt/backups_nfs/
sending incremental file list
testfile
rsync: mkstemp "/mnt/backups_nfs/.testfile.bzg47C" failed: Remote I/O
error (121)

sent 1,074,004,056 bytes  received 121 bytes 165,231,411.85 bytes/sec
total size is 1,073,741,824  speedup is 1.00
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1183) [sender=3.1.0]


Can you please provide the volume configuration(gluster vol info )
and log file for nfs server which u mounted (/var/log/glusterfs)

--
Jiffin
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org<mailto:Gluster-users at gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-users


_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org<mailto:Gluster-users at gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160309/959eedef/attachment.html>


More information about the Gluster-users mailing list