[Gluster-users] slow writes using replicate with 2.0rc1

[Krishna Srinivas]

Ray,
We will get back on this.
Krishna

On Wed, Feb 4, 2009 at 4:58 AM, Ray Speth <speth at mit.edu> wrote:
> I am wondering if anyone knows how to achieve good performance when
> writing to a volume using cluter/replicate. I am getting write speeds of
> about 4 MB/s over Gigabit Ethernet when transferring (single, large)
> files to such a volume. When I remove replication, I get transfer rates
> using "cp" of 21 MB/s. I can get much higher rates using "dd" with large
> block sizes (at least 128KB) -- 34 MB/s for the replicated volume and
> 117 MB/s without replication. Read speeds are over 100 MB/s in all
> cases. The transfer rates for cp are comparable to those using dd with a
> block size between 4KB and 8KB.
>
> I am using Gluster 2.0.0rc1 on Ubuntu 8.10 (Intrepid) systems, running
> the stock 2.6.27 kernel. I have tried a range of things with the various
> performance translators, but have not seen much change in performance. I
>  tried applying the FUSE patch, but it appears that the patch for FUSE
> 2.7.3 modifies sections of the kernel module that have changed in the
> 2.6.27 kernel.
>
> Does anyone know how I can improve the write performance in this type of
> setup? Even the 21 MB/s that I get without replicate would be
> acceptable, even if it's not really making use of all the available
> bandwidth.
>
> My server and client configuration files are below. These include
> cluster/distribute as well, but I don't think that has any effect when
> copying a single file.
>
> Thanks,
> Ray
>
> *** glusterfs-server.vol ***
> volume brick0
>   type storage/posix
>   option directory /scratch/gluster-brick
> end-volume
>
> volume brick1
>   type features/locks
>   subvolumes brick0
> end-volume
>
> volume brick
>   type performance/io-threads
>   option thread-count 8
>   subvolumes brick1
> end-volume
>
> volume server
>   type protocol/server
>   option transport-type tcp
>   option auth.addr.brick.allow *
>   subvolumes brick
> end-volume
>
> *** glusterfs-client.vol ***
>
> volume node01
>   type protocol/client
>   option transport-type tcp
>   option remote-host node01
>   option remote-subvolume brick
> end-volume
>
> <snip: etc. for node02...node08>
>
> volume rep1
>   type cluster/replicate
>   subvolumes node01 node02
> end-volume
>
> < snip: etc. for rep2, rep3, rep4 on the remaining nodes>
>
> volume distribute
>   type cluster/distribute
>   subvolumes rep1 rep2 rep3 rep4
> end-volume
>
> volume writebehind
>   type performance/write-behind
>   option block-size 1MB
>   option cache-size 4MB
>   option flush-behind on
>   subvolumes distribute
> end-volume
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>