[Gluster-devel] Spurious disconnections / connectivity loss

Gordan Bobic gordan at bobich.net
Mon Feb 1 11:10:16 UTC 2010


Stephan von Krawczynski wrote:
> On Sun, 31 Jan 2010 13:37:49 +0000
> Gordan Bobic <gordan at bobich.net> wrote:
> 
>> Stephan von Krawczynski wrote:
>>> On Sun, 31 Jan 2010 00:29:55 +0000
>>> Gordan Bobic <gordan at bobich.net> wrote:
> 
>>> Slightly offtopic I would like to ask if you, too, experienced glusterfs using
>>> a lot more bandwith than a comparable nfs connection on the server network
>>> side. It really looks a bit like a waste of resources to me...
>> I haven't noticed bandwidth going "missing", if that's what you mean. I 
>> do my replication server-side, so the server replicates the writes n-1 
>> times for n servers, and my cacti graphs are broadly in line with the 
>> bandwidth usage expected. If I disconnect all the mirrors except the 
>> server I'm connecting to,the bandwidth usage between the client and the 
>> server is similar to NFS.
>>
>> What bandwidth "leakage" are you observing?
> 
> My replication is done on client-side, because this is the only way to have it
> redundantly access the data if one server goes down (in theory).
> If I compare the bandwith used by glusterfs and the bandwith used by nfs for
> the same client it is obvious that nfs uses far less bandwith than glusterfs
> (comparing use of only one server of course). Interestingly incoming and
> outgoing server traffic is merely the same, whereas nfs has far less incoming
> traffic (server side), obviously because the client writes a lot less than it
> reads.

That's hardly unexpected. If you are using client-side replicate, I'd 
expect to see the bandwidth requirements multiply with the number of 
replicas. For all clustered configurations (not limited to glfs) I use a 
separate LAN for cluster communication to ensure best possible 
throughput/latencies, and specifically in case of glfs, I do server side 
replicate so that the replicate traffic gets offloaded to that private 
cluster LAN, so the bandwidth requirements to the clients can be kept 
down to sane levels.

Gordan





More information about the Gluster-devel mailing list