[Gluster-users] small write speed problem on EBS, distributed replica

[karol skocik]

Hi,
  I am in the process of evaluation of Gluster for major BI company,
but I was surprised by very small write performance on Amazon EBS.
Our setup is Gluster 3.1.2, distributed replica 2x2 on 64-bit m1.large
instances. Every server node has 1 EBS volume attached to it.
The configuration of the distributed replica is a default one, my
small attemps to improve performance (io-threads, disabled io-stats
and latency-measurement):

volume EBSVolume-posix
    type storage/posix
    option directory /mnt/ebs
end-volume

volume EBSVolume-access-control
    type features/access-control
    subvolumes EBSVolume-posix
end-volume

volume EBSVolume-locks
    type features/locks
    subvolumes EBSVolume-access-control
end-volume

volume EBSVolume-io-threads
    type performance/io-threads
    option thread-count 4
    subvolumes EBSVolume-locks
end-volume

volume /mnt/ebs
    type debug/io-stats
    option log-level NONE
    option latency-measurement off
    subvolumes EBSVolume-io-threads
end-volume

volume EBSVolume-server
    type protocol/server
    option transport-type tcp
    option auth.addr./mnt/ebs.allow *
    subvolumes /mnt/ebs
end-volume

In our test, all clients starts writing to different 1GB file at the same time.
The measured write bandwidth, with 2x2 servers:

1 client: 6.5 MB/s
2 clients: 4.1 MB/s
3 clients: 2.4 MB/s
4 clients: 4.3 MB/s

This is not acceptable for our needs. With PVFS2 (I know it's
stripping which is very different from replica) we can get up to 35
MB/s.
2-3 times slower than that would be understandable. But 5-15 times
slower is not, and I would like to know whether there is something we
could try out.

Could anybody publish their write speeds on similar setup, and tips
how to achieve better performance?

Thank you,
  Karol