[Gluster-users] Performance problem, Network traffic, ASCII/BINARY protocol

[Francis GASCHET]

Hi all,

I installed glusterFS on 2 computers under Mandriva 2008.
Connection type : Ethernet 100 mbits/S (mii-tool result: negotiated
100baseTx-FD flow-control)

Hereafter is my configuration (same on both PCs):

*glusterfs-server.vol:*

volume dir_main
   type storage/posix                                        # POSIX FS
translator
   option directory /main                                    # Export
this directory
end-volume

volume locks_main
     type features/posix-locks
     subvolumes dir_main
end-volume

volume main
     type protocol/server
     option transport-type tcp/server                          # For
TCP/IP transport
    subvolumes locks_main
     option auth.ip.main.allow 127.0.0.1,172.16.1.*            # Allow
access to "brick" volume
end-volume


*client.vol:*

volume main_loc
     type protocol/client
     option transport-type tcp/client
     option remote-host localhost
     option remote-subvolume main
end-volume

volume main_dist
     type protocol/client
     option transport-type tcp/client
     option remote-host other
     option remote-subvolume main
end-volume

volume raid_main_afr
     type cluster/afr
     subvolumes main_loc main_dist
     option read-subvolume main_loc
end-volume

volume raid_main_ra
     type performance/read-ahead
     option page-size 128kB
     option page-count 4
     option force-atime-update off
     subvolumes raid_main_afr
end-volume

volume raid_main_wb
     type performance/write-behind
     option aggregate-size 1MB
     option flush-behind on
     subvolumes raid_main_ra
end-volume

volume raid_main
     type performance/io-cache
     option cache-size 512MB
     option page-size 1MB
     option priority *:0                 # *.html:2,*:1
     option force-revalidate-timeout 2   # default is 1
     subvolumes raid_main_wb
end-volume


It works fine, but slowly !
I'm a newbie in glusterFS, so may be some option isn't adequate. Please
advise.

Due to the option "read-subvolume main_loc" I didn't expect network
traffic when I just list files or read them, but actually, even with a
simple ls, I see a lot of network traffic.
A "ls -R" takes 7 to 8 seconds for less than 8000 files. If I do it
locally, I get the result instantly.
*Question 1:* Is this traffic normal on read only operation ?

*Question 2:* in the documentation, I read that there is 2 protocols :
ASCII protocol and binary protocol. Currently, according to what I see
with tcpdump, my glusterFS uses the ASCII protocol. I guess it's not the
best for performance ! What is the way to enforce it using the binary
protocol ?

Thank's for any help.

Best regards,

-- 
Francis GASCHET / NUMLOG
http://www.numlog.fr
Tel.: +33 (0) 130 791 616
Fax.: +33 (0) 130 819 286

NUMLOG recrute sur LOLIX :
http://fr.lolix.org/