[Gluster-users] Switch recommendations

[Jeff Anderson-Lee]

Note: for significant fabric-related benefit, the *clients* need to be 
on the same fabric as the server nodes, not just the servers. We have a 
handful of servers, but hundreds of clients, so InfiniBand was out for 
us on economic grounds. And with just 1GbT (even with a high-end Cisco 
switch) we don't get the performance we desired on small files. If you 
have small files, many clients, and a small budget: beware.

On 1/30/2012 11:05 AM, Peter Linder wrote:
> Putting all machines on the same switch will at least save a 
> switch-hop, cutting latency by 1/3 in the ideal case which is an 
> improvement. If it will matter for your application, I do not know :)
>
> On 1/30/2012 6:45 PM, Dan Bretherton wrote:
>> Thanks for the advice Peter,
>>> You could get a cisco switch that supports cut through instead of
>>> store-and-forward, for lower latency.
>> Interesting option, but those cost ~£10K and are out of our price 
>> range unfortunately.  The "cut through" switching technology is also 
>> available in Dell's new Force 10 range I believe.
>>
>>> Other than that, compare the port to port forwarding times and see if
>>> there is a difference between the switches you are looking at (probably
>>> not) and make your decision based on that.
>> You're right, there isn't much of a difference between the forwarding 
>> rate of the 5548 and the 6248, both are about 100Mpps.  They also 
>> have a similar bandwidth of about 180Gbps.  However the 7048 does 
>> better on both measures, with forwarding rate of 160Mpps and a 
>> bandwidth of 224Gbps.  Unfortunately the 7048 costs five times as 
>> much as the 5548, and I don't know if the users would notice any 
>> difference at all.  I expect some would and some wouldn't.
>>
>> -Dan.
>>
>> On 01/27/2012 01:48 PM, gluster-users-request at gluster.org wrote:
>>> Date: Fri, 27 Jan 2012 14:16:36 +0100
>>> From: Peter Linder<peter.linder at fiberdirekt.se>
>>> Subject: Re: [Gluster-users] Switch recommendations
>>> To:gluster-users at gluster.org
>>> Message-ID:<4F22A3B4.8000008 at fiberdirekt.se>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> You could get a cisco switch that supports cut through instead of
>>> store-and-forward, for lower latency.
>>>
>>> Other than that, compare the port to port forwarding times and see if
>>> there is a difference between the switches you are looking at (probably
>>> not) and make your decision based on that. Consider connecting
>>> everything to two switches, for failover in case a switch breaks?
>>>
>>> On 1/27/2012 2:04 PM, Dan Bretherton wrote:
>>>> >  Dear All,
>>>> >  I need to buy a bigger GigE switch for my GlusterFS cluster and 
>>>> I am
>>>> >  trying to decide whether or not a much more expensive one would be
>>>> >  justified.  I have limited experience with networking so I don't 
>>>> know
>>>> >  if it would be appropriate to spend ?500, ?1500 or ?3500 for a 
>>>> 48-port
>>>> >  switch.  Those rough costs are based on a comparison of 3 Dell
>>>> >  Powerconnect switches: the 5548 (bigger version of what we have 
>>>> now),
>>>> >  the 6248 and the 7048.  The servers in the cluster are nothing 
>>>> special
>>>> >  - mostly Supermicro with SATA drives and 1GigE network adapters.  I
>>>> >  can only justify spending more than ~?500 if I can be sure that 
>>>> users
>>>> >  would notice the difference.  Some of the users' applications do 
>>>> lots
>>>> >  of small reads and writes, and they do run much more slowly if 
>>>> all the
>>>> >  servers are not connected to the same switch, as is the case now 
>>>> while
>>>> >  I don't have a big enough switch.  Any advice or comments would be
>>>> >  much appreciated.
>>>> >
>>>> >  Regards
>>>> >  Dan.
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users