[Gluster-users] Really slow performance

[Matt Lawrence]

I am seeing extremely slow performance with glusterfs.

OS: CentOS 5
glusterfs version: glusterfs-1.3.9-1

Server configuration:

##############################################
###  GlusterFS Server Volume Specification  ##
##############################################

#### CONFIG FILE RULES:
### "#" is comment character.
### - Config file is case sensitive
### - Options within a volume block can be in any order.
### - Spaces or tabs are used as delimitter within a line.
### - Multiple values to options will be : delimitted.
### - Each option should end within a line.
### - Missing or commented fields will assume default values.
### - Blank/commented lines are allowed.
### - Sub-volumes should already be defined above before referring.

volume shelf-1
  type storage/posix
  option directory /glusterfs/shelf-1
end-volume

volume shelf-1-readahead
  type performance/read-ahead
  subvolumes shelf-1
end-volume

volume shelf-1-writebehind
  type performance/write-behind
  option aggregate-size 1MB
  option window-size    4MB
  subvolumes shelf-1-readahead
end-volume

### Add network serving capability to above brick.
volume server
  type protocol/server
  option transport-type tcp/server     # For TCP/IP transport
  subvolumes shelf-1-writebehind
  option auth.ip.shelf-1-writebehind.allow 127.0.0.1
end-volume


Local client configuration:

##############################################
###  GlusterFS Client Volume Specification  ##
##############################################

volume storage
  type protocol/client
  option transport-type tcp/client
  option remote-host 127.0.0.1
  option remote-subvolume shelf-1-writebehind
end-volume

Writing directly to the underlying filesystem:

-bash-3.2# dd if=/dev/zero of=/glusterfs/shelf-1/testing count=20M

20971520+0 records in

20971520+0 records out

10737418240 bytes (11 GB) copied, 230.862 seconds, 46.5 MB/s


Writing through glusterfs with no readahead/writebehind:

-bash-3.2# dd if=/dev/zero of=/storage/testing count=20M

20971520+0 records in

20971520+0 records out

10737418240 bytes (11 GB) copied, 3429.89 seconds, 3.1 MB/s


Using the above configuration is even slower:

-bash-3.2# dd if=/dev/zero of=/storage/testing count=20M
20971520+0 records in
20971520+0 records out
10737418240 bytes (11 GB) copied, 4379.16 seconds, 2.5 MB/s


What should I be trying to improve performance?  I really need help!

-- Matt