[Gluster-users] tuning glusterfs?

[Jiann-Ming Su]

On Wed, Jan 20, 2010 at 5:49 PM, pkoelle <pkoelle at gmail.com> wrote:
>
> This looks suspicious. Is this a gfs2 mount? what kind of performance do you
> get wen running bonnie++ directly against /mnt/gluster/gfs2?
>

Bonnie++ command:  ./bonnie++ -u 99:99 -d /mnt/gluster/gfs2 -s 2000

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.loc 2000M    74  99 39432  12 27308  10   836  99 65104   9 351.5  11
Latency               179ms    1207ms     429ms     109ms     109ms     215ms
Version  1.96       ------Sequential Create------ --------Random Create--------
localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
             files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                16 23991  81 +++++ +++ +++++ +++ 17007  57 +++++ +++ +++++ +++
Latency              1276us    1189us     787us    1183us     438us    2813us


>
> Can you leave out some of the performance/* translators and also try a
> "real" network interface with lower MTU values to see of that helps? I had
> terrible results with bigger MTU (6500) and bigger values for
> net.core.(r/w)mem_max in /etc/sysctl.conf. IOPS where in the range of 5/sec.
>

GlusterFS server and client are running on the same system, so there's
no network latency involved at all. I don't see how introducing the
network layer into this would help.  Blame it on gluster or fuse, but
the reality is glusterfs read performance, in a simple base config, is
1/10th of the native file system.  In order to have this be worthwhile
at all, I'd have to stripe the data across at least 10 servers to get
the same read performance.  This may be a legitimate setup, but it's
not what I need out of gluster.  I need the replication capabilities
out of gluster and the ability for the node to read the local copy
reasonably fast.


-- 
Jiann-Ming Su
"I have to decide between two equally frightening options.
 If I wanted to do that, I'd vote." --Duckman
"The system's broke, Hank.  The election baby has peed in
the bath water.  You got to throw 'em both out."  --Dale Gribble
"Those who vote decide nothing.
Those who count the votes decide everything.”  --Joseph Stalin