[Gluster-users] Problem: Gluster performance too high..

[Harry Mangalam]

On Thursday 29 March 2012 12:34:10 Jeff Darcy wrote:
> On 03/29/2012 03:30 PM, Harry Mangalam wrote:
> > I'm doing some perf tests on a small gluster filesystem - 5
> > bricks on 4 servers, all single-homed on the private net.
> > 
> > I've spawned up to 70 simultaneous jobs on our cluster nodes
> > writing files of various sizes from /dev/zero to the gluster fs
> > to see what the effect on the aggregate bandwith and the data is
> > slightly unbelievable in that it seems to exceed the theoretical
> > max of the network. (I used /dev/zero instead of /dev/urandom
> > since /dev/urandom couldn't generate data fast enough.
> > 
> > The 35,000 files of the right size do hit the filesystem (of
> > course they're all zero's) but the speed at which they transfer
> > exceeds (by quite a bit) the theoretical max of a 1 Gb network.
> > 
> > Does gluster (or anything else) do transparent compression? What
> > else would explain this oddity?
> 
> How do you define "theoretical max of a 1Gb network"?  If it's a
> switched network, the actual maximum throughput depends on the
> capabilities of the switch but is likely to be far in excess of
> 1Gb/s.  Could that be it?  Could you give more detail about the
> actual traffic patterns and results?
> 
> BTW, this is my favorite message title ever.  Thanks for that.  :)

So something good has come out of this.. :)

You make a good point - I'm more of a biologist gone to to the dark 
side than a formally trained network guy.  The cluster is in fact a 
distributed one - some of the cluster workers are in a remote building 
VLAN'ed into the main group, and the rest are on the main cluster 
network. From my understanding of the cluster networking, there are a 
number of 24- and 48- port stacked switches providing ports to the 
worker nodes, as well as to the gluster servers.  

My assumption was that at some point there would be a 1Gb bottleneck, 
but if the packets were switched all the way thru, there would be a 
theoretical max related the number of gluster-server Gb ports (4).  So 
until I approach 4Gb/s, I guess I wouldn't necessarily see this 
bottleneck.  Is that correct?

Harry


-- 
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  Google Voice Multiplexer: (949) 478-4487
415 South Circle View Dr, Irvine, CA, 92697 [shipping]
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120329/bac21aee/attachment.html>