[Gluster-users] No NFS connection due to GlusterFS CPU load

[te-yamauchi at usen.co.jp]

When executing the load test with the FIO tool, execute the following job from the client
When executed, the load of 2 cores is high for the CPU. Up to 100%.
At that time, if another client is performing NFS mounting, the df command
I can not connect NFS without coming back. The log will continue to be output below.
I believe that if the CPU utilization is distributed, the load will be eliminated.

Will not improve by tuning the following parameters?
Most of the parameters remain as defaults.

server.event-threads : 1
client.event-threads : 2
server.outstanding-rpc-limit : 64
nfs.outstanding-rpc-limit : 16
performance.io-thread-count : 16

/var/log/glusterfs/nfs.log
[2017-06-14 10:02:03.964405] I [MSGID: 108006] [afr-common.c:4941:afr_local_init] 0-gvol01-replicate-0: no subvolumes up
[2017-06-14 10:02:04.026299] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f2729b3ae8b] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f27299018ee] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f27299019fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7f2729903170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7f2729903c20] ))))) 0-gvol01-client-1: forced unwinding frame type(GlusterFS 3.3) op(WRITE(13)) called at 2017-06-14 09:58:00.107164 (xid=0x8dc455)
[2017-06-14 10:02:06.967780] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f2729b3ae8b] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f27299018ee] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f27299019fe] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7f2729903170] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7f2729903c20] ))))) 0-gvol01-client-0: forced unwinding frame type(GlusterFS 3.3) op(FSYNC(16)) called at 2017-06-14 09:55:20.342725 (xid=0x663a73)
The message "I [MSGID: 108006] [afr-common.c:4941:afr_local_init] 0-gvol01-replicate-0: no subvolumes up" repeated 37 times between [2017-06-14 10:02:03.964405] and [2017-06-14 10:02:06.880464]
[2017-06-14 10:02:06.967820] W [MSGID: 114031] [client-rpc-fops.c:972:client3_3_fsync_cbk] 0-gvol01-client-0: remote operation failed [Communication end point is not connected]
[2017-06-14 10:02:06.967890] W [MSGID: 108035] [afr-transaction.c:2243:afr_changelog_fsync_cbk] 0-gvol01-replicate-0: fsync(08ed1905-d81e-4ad3-9de2-1395f2c4667e) failed on subvolume gvol01-client-0. Transaction was WRITE [Communication end point is not connected]