[Gluster-users] Write performance in a replicated/distributed setup with KVM?
harald.hannelius at arcada.fi
Fri Mar 2 09:04:02 UTC 2012
This has probably been discussed before, but since I'm new on the list I hope
You have patience with me.
I have a four brick distributed/replicated setup. The computers are multi-core
16GB memory and 2*2.0TB in raid1 SATA-disks locally. The nodes are connected by
1 GB ethernet. All nodes have glusterfs 3.3beta2 installed and they are running
debian 6 64bit. The underlying filesystems are xfs.
I have setup a volume like so;
gluster volume create virtuals replica 2 transport tcp \
adraste:/data/brick alcippe:/data/brick aethra:/data/brick helen:/data/brick
Which resulted in a nice volume;
# gluster volume info virtuals
Volume Name: virtuals
Number of Bricks: 2 x 2 = 4
All seems OK so far, but write performance seems very slow. When writing to
localhost:/virtuals I get single-digit MB/s performance which isn't really what
I had expected. I know that the write has to go to at least two (?) nodes at
the same time, but still?
A single scp of a 1GB file from a node to another gives something like
A copy of a virtual image took 17 minutes;
# time cp debtest.raw /gluster/debtest.img
# ls -lah /gluster/debtest.img
-rw------- 1 root root 20G Mar 1 12:35 /gluster/debtest.img
# du -ah /gluster/debtest.img
I noted that the processlist shows that direct-io-mode is disabled. Default
should be on, or should it?
Any help is really appreciated!
Harald Hannelius | harald.hannelius/a\arcada.fi | +358 50 594 1020
More information about the Gluster-users