[Gluster-users] Strange performance issues on Debian/Lenny compared to Ubuntu

[Jeremy Enos]

I too am searching for a way to get Gluster to use a cache effectively.  
So far, no performance translator, variation on translator parameters, 
or locating translator on server or clients side has made any 
difference.  :(

     Jeremy


On 3/31/2010 3:23 AM, Olivier Le Cam wrote:
> Hi -
>
>> * Can you test dd performance with only write-behind translator 
>> (along with
>> protocol/client) in client volume configuration?
>
> That makes no difference with either only write-behind translator or 
> all of default translator configurated by genvol. Without it, results 
> are abt 10 times worse.
>
>> * Also can you add "debug/trace" translator above and below 
>> write-behind,
>> start glusterfs with Log-level trace and send the log files?
>
> * Debian/Lenny (kernel 2.6.32-trunk-amd64, same with 2.6.26)
>    ~# dd if=/dev/zero of=/mnt/test_file count=262144 bs=1024
>    262144+0 enregistrements lus
>    262144+0 enregistrements écrits
>    268435456 bytes (268 MB) copied, 117,14 s, 2,3 MB/s
>
>    Log file size is 283MB. Do you want to get it entirely?
>    First 10000 lines of /var/log/glusterfs/mnt.log uploaded to:
>    http://olecam.online.fr/glusterfs/lenny_log.txt
>
>    ~# cat /var/log/glusterfs/mnt.log |wc -l
>    1792172
>
>
> * Ubuntu/Hardy (kernel 2.6.24-23-server x86_64)
>     ~# dd if=/dev/zero of=/mnt/test_file count=262144 bs=1024
>     262144+0 records in
>     262144+0 records out
>     268435456 bytes (268 MB) copied, 17,1914 s, 15,6 MB/s
>
>     Log file size is 115MB. Do you want to get it entirely?
>     First 10000 lines of /var/log/glusterfs/mnt.log uploaded to:
>     http://olecam.online.fr/glusterfs/ubuntu_log.txt
>
>     ~# cat /var/log/glusterfs/mnt.log |wc -l
>     553600
>
> * Addition notes:
>
> - Using tcpdump, I have observed that dd'ing from the Debian/Lenny 
> generates abt 10 times more frames on the network. Frame size sent by 
> the Ubuntu client are generally larger, this can explain why the 
> transfer rate is so terrible from my Lenny client.
>
>  - tcpdump on Lenny client shows: "278648 packets captured"
>  - tcpdump on Ubuntu client shows: "28120 packets captured" (abt 10x 
> less frames traffic!!)
>
>  tcpdump -v outputs also show that packet size sent the Ubuntu client 
> is usually abt 40-60 KBytes large. For example:
>  10:06:34.795101 IP (tos 0x0, ttl 64, id 38433, offset 0, flags [DF], 
> proto TCP (6), length 50732)
>  This is never the case from the Debian client(always < 10KBytes).
>
> As observed by Paul who has better results than mine with an Lenny 
> client, this might be related to some kernel options or 
> "optimizations" in recent versions. I haven't been able to give a test 
> with 2.6.18, the only one I have under hand is a xen domU (under an 
> Etch dom0) and I cannot get the fuse loading properly on it.
>
> Still available for anything else I can do.
>
> Thanks and best regards,