[Gluster-users] 40 gig ethernet

[Bryan Whitehead]

GigE is slower. Here is ping from same boxes but using the 1GigE cards:

[root at node0.cloud ~]# ping -c 10 10.100.0.11
PING 10.100.0.11 (10.100.0.11) 56(84) bytes of data.
64 bytes from 10.100.0.11: icmp_seq=1 ttl=64 time=0.628 ms
64 bytes from 10.100.0.11: icmp_seq=2 ttl=64 time=0.283 ms
64 bytes from 10.100.0.11: icmp_seq=3 ttl=64 time=0.307 ms
64 bytes from 10.100.0.11: icmp_seq=4 ttl=64 time=0.275 ms
64 bytes from 10.100.0.11: icmp_seq=5 ttl=64 time=0.313 ms
64 bytes from 10.100.0.11: icmp_seq=6 ttl=64 time=0.278 ms
64 bytes from 10.100.0.11: icmp_seq=7 ttl=64 time=0.309 ms
64 bytes from 10.100.0.11: icmp_seq=8 ttl=64 time=0.197 ms
64 bytes from 10.100.0.11: icmp_seq=9 ttl=64 time=0.267 ms
64 bytes from 10.100.0.11: icmp_seq=10 ttl=64 time=0.187 ms

--- 10.100.0.11 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9000ms
rtt min/avg/max/mdev = 0.187/0.304/0.628/0.116 ms

Note: The Infiniband interfaces have a constant load of traffic from
glusterfs. The Nic cards comparatively have very little traffic.

On Fri, Jun 14, 2013 at 12:40 PM, Stephan von Krawczynski
<skraw at ithnet.com> wrote:
> On Fri, 14 Jun 2013 12:13:53 -0700
> Bryan Whitehead <driver at megahappy.net> wrote:
>
>> I'm using 40G Infiniband with IPoIB for gluster. Here are some ping
>> times (from host 172.16.1.10):
>>
>> [root at node0.cloud ~]# ping -c 10 172.16.1.11
>> PING 172.16.1.11 (172.16.1.11) 56(84) bytes of data.
>> 64 bytes from 172.16.1.11: icmp_seq=1 ttl=64 time=0.093 ms
>> 64 bytes from 172.16.1.11: icmp_seq=2 ttl=64 time=0.113 ms
>> 64 bytes from 172.16.1.11: icmp_seq=3 ttl=64 time=0.163 ms
>> 64 bytes from 172.16.1.11: icmp_seq=4 ttl=64 time=0.125 ms
>> 64 bytes from 172.16.1.11: icmp_seq=5 ttl=64 time=0.125 ms
>> 64 bytes from 172.16.1.11: icmp_seq=6 ttl=64 time=0.125 ms
>> 64 bytes from 172.16.1.11: icmp_seq=7 ttl=64 time=0.198 ms
>> 64 bytes from 172.16.1.11: icmp_seq=8 ttl=64 time=0.171 ms
>> 64 bytes from 172.16.1.11: icmp_seq=9 ttl=64 time=0.194 ms
>> 64 bytes from 172.16.1.11: icmp_seq=10 ttl=64 time=0.115 ms
>>
>> --- 172.16.1.11 ping statistics ---
>> 10 packets transmitted, 10 received, 0% packet loss, time 8999ms
>> rtt min/avg/max/mdev = 0.093/0.142/0.198/0.035 ms
>
> What you like to say is that there is no significant difference compared to
> GigE, right?
> Anyone got a ping between two kvm-qemu virtio-net cards at hand?
>
> --
> Regards,
> Stephan
>