[Gluster-users] is glusterfs DHT really distributed?
mark at mark.mielke.cc
Mon Sep 28 14:40:43 UTC 2009
On 09/28/2009 10:35 AM, Wei Dong wrote:
> Hi All,
> I noticed a very weird phenomenon when I'm copying data (200KB image
> files) to our glusterfs storage. When I run only run client, it
> copies roughly 20 files per second and as soon as I start a second
> client on another machine, the copy rate of the first client
> immediately degrade to 5 files per second. When I stop the second
> client, the first client will immediately speed up to the original 20
> files per second. When I run 15 clients, the aggregate throughput is
> about 8 files per second, much worse than running only one client.
> Neither CPU nor network is saturated. My volume file is attached.
> The servers are running on a 66 node cluster and the clients are a
> 15-node cluster.
> We have 33x2 servers and at most 15 separate machines, with each
> server serving < 0.5 clients on average. I cannot think of a reason
> for a distributed system to behave like this. There must be some kind
> of central access point.
Although there is probably room for the GlusterFS folk to optimize...
You should consider directory write operations to involve the whole
cluster. Creating a file is a directory write operation. Think of how it
might have to do self-heal across the cluster, make sure the name is
right and not already in use across the cluster, and such things.
Once you get to reads and writes for a particular file, it should be
Mark Mielke<mark at mielke.cc>
More information about the Gluster-users