[Gluster-devel] Performance question.

[Kevan Benson]

Have you tried running the tests with a single server exporting a 
filesystem without stripe or unify?  You could run it from the same NFS 
server to get a better comparison.  If glusterfs performs well as a 
simple share with one volume (maybe a few perf translators in there), 
you could add on translators and change the layout as you wanted to see 
what's causing your slowdown.

Chris Johnson wrote:
>      Hi, again,
> 
>      I asked about stack building philosophy.  Apparently there isn't
> one.  So I tried a few things.  The configs are down the end here.
> 
>      Two systems, CentOS5, both running fuse-devel-2.7.0-1 gluster
> enhanced, glusterfs-1.3.5-2.  Both have gigabit ethernet, server runs
> a SATABeast.  Currently I ge the following from from iozone.
> 
> iozone -aN -r 32k -s 131072k -f /mnt/glusterfs/sdm1/junknstuff
> 
>                                                             random  
> random    bkwd  record  stride
>               KB  reclen   write rewrite    read    reread    read   
> write    read rewrite    read   fwrite frewrite   fread  freread
>           131072      32     589     587      345      343     818     
> 621     757     624     845      592      591     346      366
> 
> Now, a similar test using NFS on a CentOS4.4 system running a 3ware RAID 
> card gives this
> 
> iozone -aN -r 32k -s 131072k -f /space/sake/5/admin/junknstuff
> 
>                                                             random  
> random    bkwd  record  stride
>               KB  reclen   write rewrite    read    reread    read   
> write    read rewrite    read   fwrite frewrite   fread  freread
>           131072      32      27      26      292       11      11      
> 24     542       9     539       30       28     295       11
> 
> And you can see that the NFS system is faster.  Is this because of the
> hardware 3ware RAID or is NFS really that much faster here?  Is there
> a better way to stack this that would improve things?  And I tried with
> and without striping.  No noticable difference in gluster performance.
> 
>      Help appreciated.
> 
> ============  server config
> 
> volume brick1
>   type storage/posix
>   option directory /home/sdm1
> end-volume
> 
> volume brick2
>   type storage/posix
>   option directory /home/sdl1
> end-volume
> 
> volume brick3
>   type storage/posix
>   option directory /home/sdk1
> end-volume
> 
> volume brick4
>   type storage/posix
>   option directory /home/sdk1
> end-volume
> 
> volume ns-brick
>   type storage/posix
>   option directory /home/sdk1
> end-volume
> 
> volume stripe1
>  type cluster/stripe
>  subvolumes brick1 brick2
> # option block-size *:10KB, end-volume
> 
> volume stripe2
>  type cluster/stripe
>  subvolumes brick3 brick4
> # option block-size *:10KB, end-volume
> 
> volume unify0
>  type cluster/unify
>  subvolumes stripe1 stripe2
>  option namespace ns-brick
>  option scheduler rr # option rr.limits.min-disk-free 5
> end-volume
> 
> volume iot
>  type performance/io-threads
>  subvolumes unify0
>  option thread-count 8
> end-volume
> 
> volume writebehind
>   type performance/write-behind
>   option aggregate-size 131072 # in bytes
>   subvolumes iot
> end-volume
> 
> volume readahead
>   type performance/read-ahead
> #  option page-size 65536 ### in bytes
>   option page-size 128kb ### in bytes
> #  option page-count 16 ### memory cache size is page-count x
> page-size per file
>   option page-count 2 ### memory cache size is page-count x page-size
> per file
>   subvolumes writebehind
> end-volume
> 
> volume server
>   type protocol/server
>   subvolumes readahead
>   option transport-type tcp/server     # For TCP/IP transport
> #  option client-volume-filename /etc/glusterfs/glusterfs-client.vol
>   option auth.ip.readahead.allow *
> end-volume
> 
> 
> ============  client config
> 
> volume client
>   type protocol/client
>   option transport-type tcp/client
>   option remote-host xxx.xxx.xxx.xxx
>   option remote-subvolume readahead
> end-volume
> 
> ------------------------------------------------------------------------------- 
> Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
> Systems Administrator       |Web:      
> http://www.nmr.mgh.harvard.edu/~johnson
> NMR Center                  |Voice:    617.726.0949
> Mass. General Hospital      |FAX:      617.726.7422
> 149 (2301) 13th Street      |A compromise is a solution nobody is happy 
> with.
> Charlestown, MA., 02129 USA |     Observation, Unknown
> ------------------------------------------------------------------------------- 
> 
> 
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
> .
> 


-- 

-Kevan Benson
-A-1 Networks