[Gluster-users] GlusterFS Speed

TomK tomkcpr at mdevsys.com
Fri Oct 4 03:18:45 UTC 2019


Wanted to run some stats by you guy's to see if this is the sort of IO 
expected off the GlusterFS w/ oVirt that I have:

[root at mdskvm-p01 master]# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59108
[  5] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59110
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   108 MBytes   905 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   939 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   939 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   939 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   939 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   938 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   938 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   939 Mbits/sec
[  5]   8.00-9.00   sec   112 MBytes   940 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   939 Mbits/sec
[  5]  10.00-10.04  sec  4.25 MBytes   939 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.04  sec  1.09 GBytes   935 Mbits/sec 
receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59126
[  5] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59128
iperf3: the client has unexpectedly closed the connection
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.0.39, port 59164
[  6] local 192.168.0.60 port 5201 connected to 192.168.0.39 port 59166
[ ID] Interval           Transfer     Bandwidth
[  6]   0.00-1.00   sec   107 MBytes   898 Mbits/sec
[  6]   1.00-2.00   sec   102 MBytes   858 Mbits/sec
[  6]   2.00-3.00   sec   107 MBytes   901 Mbits/sec
[  6]   3.00-4.00   sec   107 MBytes   896 Mbits/sec
[  6]   4.00-5.00   sec  94.5 MBytes   793 Mbits/sec
[  6]   5.00-6.00   sec  93.9 MBytes   788 Mbits/sec
[  6]   6.00-7.00   sec  78.1 MBytes   655 Mbits/sec
[  6]   7.00-8.00   sec  65.8 MBytes   552 Mbits/sec
[  6]   8.00-9.00   sec  50.6 MBytes   425 Mbits/sec
[  6]   9.00-10.00  sec  52.8 MBytes   443 Mbits/sec
[  6]  10.00-10.07  sec  4.28 MBytes   491 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  6]   0.00-10.07  sec  0.00 Bytes  0.00 bits/sec                  sender
[  6]   0.00-10.07  sec   864 MBytes   719 Mbits/sec 
receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
^Ciperf3: interrupt - the server has terminated
[root at mdskvm-p01 master]#


Then the client:


[root at mdskvm-p02 master]# iperf3 -c mdskvm-p01.nix.mds.xyz
Connecting to host mdskvm-p01.nix.mds.xyz, port 5201
[  4] local 192.168.0.39 port 59110 connected to 192.168.0.60 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   114 MBytes   955 Mbits/sec    0    358 KBytes
[  4]   1.00-2.00   sec   112 MBytes   938 Mbits/sec    0    358 KBytes
[  4]   2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    362 KBytes
[  4]   3.00-4.00   sec   111 MBytes   935 Mbits/sec    0    362 KBytes
[  4]   4.00-5.00   sec   112 MBytes   940 Mbits/sec    0    362 KBytes
[  4]   5.00-6.00   sec   112 MBytes   942 Mbits/sec    0    362 KBytes
[  4]   6.00-7.00   sec   111 MBytes   935 Mbits/sec    0    363 KBytes
[  4]   7.00-8.00   sec   112 MBytes   941 Mbits/sec    0    363 KBytes
[  4]   8.00-9.00   sec   112 MBytes   937 Mbits/sec    0    363 KBytes
[  4]   9.00-10.00  sec   112 MBytes   942 Mbits/sec    0    363 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   940 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   939 Mbits/sec 
receiver

iperf Done.
[root at mdskvm-p02 master]#



[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]# iperf3 -c 
mdskvm-p01.nix.mds.xyz -F e13fca65-65c5-4665-8817-9668f81ad6d4
Connecting to host mdskvm-p01.nix.mds.xyz, port 5201
[  4] local 192.168.0.39 port 59166 connected to 192.168.0.60 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   113 MBytes   945 Mbits/sec    0    373 KBytes
[  4]   1.00-2.00   sec   102 MBytes   859 Mbits/sec    0    373 KBytes
[  4]   2.00-3.00   sec   108 MBytes   905 Mbits/sec    0    373 KBytes
[  4]   3.00-4.00   sec   107 MBytes   896 Mbits/sec    0    373 KBytes
[  4]   4.00-5.00   sec  94.5 MBytes   793 Mbits/sec    0    373 KBytes
[  4]   5.00-6.05   sec  88.4 MBytes   710 Mbits/sec    0    373 KBytes
[  4]   6.05-7.01   sec  78.6 MBytes   687 Mbits/sec    0    373 KBytes
[  4]   7.01-8.01   sec  68.3 MBytes   573 Mbits/sec    0    373 KBytes
[  4]   8.01-9.00   sec  52.4 MBytes   442 Mbits/sec    0    373 KBytes
[  4]   9.00-10.04  sec  52.1 MBytes   422 Mbits/sec    0    379 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.04  sec   864 MBytes   722 Mbits/sec    0             sender
         Sent  864 MByte / 7.24 GByte (11%) of 
e13fca65-65c5-4665-8817-9668f81ad6d4
[  4]   0.00-10.04  sec   864 MBytes   722 Mbits/sec 
receiver

iperf Done.
[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]# ls -altri
total 7588869
13832606062331202506 -rw-rw----.  1 vdsm kvm    1048576 Oct  1 07:11 
e13fca65-65c5-4665-8817-9668f81ad6d4.lease
  9777876818032195061 drwxr-xr-x. 10 vdsm kvm       4096 Oct  1 07:12 ..
  9787392456663466582 -rw-rw----.  1 vdsm kvm 7769948160 Oct  1 07:21 
e13fca65-65c5-4665-8817-9668f81ad6d4
12966769610620992411 -rw-r--r--.  1 vdsm kvm        357 Oct  1 07:21 
e13fca65-65c5-4665-8817-9668f81ad6d4.meta
10878339854193785066 drwxr-xr-x.  2 vdsm kvm        146 Oct  1 07:21 .
[root at mdskvm-p02 28fcc382-f469-4f8b-af0f-6b6e048dfcca]#



A bit about the storage.  Cheap LAB hardware presenting a SAN LUN to the 
physicals running the GlusterFS.  The raw storage itself can deliver 
@300MB/s .   Network is 1GB/s.

Guessing the numbers are fine given the raw specs?

-- 
Thx,
TK.


More information about the Gluster-users mailing list