[Gluster-users] Switch recommendations

[John Lauro]

The latency shouldn't be that bad to really be noticeable unless the links
are really saturated.  One thing I don't like about the Dell 5424 switches
(but otherwise great switches, and you can design around the flaw if you
are aware of it) is the buffers are a little small and they will send
pause packets fairly easily.  Make sure you don't have anything that does
a moderate amount of traffic on 100mbps port, as it can easily backup all
the other ports, especially if it's on a different switch and the switch
to switch port is paused...

Run some ping tests between servers on the same switch, and servers that
are going over your switch to switch link.  It could be you just have a
bad cable somewhere...

Generally I use the following for ping tests:
ping serverip -s 16000 -c 1000 -q -A               
PING xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) 8000(8028) bytes of data.

--- xxx.xxx.xxx.xxx ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 1035ms
rtt min/avg/max/mdev = 0.631/0.834/5.110/0.235 ms, ipg/ewma 1.036/0.802 ms


Here are the options, in case you are using something other than Linux as
they are not completely standard cross platforms.

The -s 16000 is big enough that it requires packet fragmentation, so it
ensures that MTU handling is working correctly.  This is especially
important if you have some devices on jumbo frames, and others not.  It is
also small enough that most things should handle it, although if you have
any low end devices, like the switches themselves, you may have to drop
down to 1470 if they don't handle fragmentation.

The -c 1000 is so it sends enough to get a good test.  1000 overside
packets is generally a reasonable test, and fairly quick if everything is
working well.  If not, you should be able to hit <CTRL>C and get stats on
how many were sent/received after you are tired of waiting for it to
return.

The -q is so you don't have to see the result of every packet, which you
really don't want to see if you are sending out thousands of packets...

The -A does an adapative ping.  Basically it will only send one packet out
at a time, and so the faster the network, the faster they are sent out.



Any chance you have your internet uplink on a 100mbps port and servers on
a different switch send traffic to it?  That could force your switch to
switch link to pause fairly often.