[Gluster-users] Performance

[Todd Daugherty]

I have run iozone on the same machine with GlusterFS and without
GlusterFS. (Local storage and Local storage via GlusterFS). I also
included the results from a remote machine. The remote machine and the
local glusterfs are the same. I am getting around 200 megabytes per
second on reads. My local storage will do around 1400 megabytes per
second. Any ideas why?

Todd


Without GlusterFS

iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.315 $
		Compiled for 64 bit mode.
		Build: linux

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

	Run began: Mon Jun  7 12:54:52 2010

	Auto Mode
	File size set to 8388608 KB
	Record Size 16384 KB
	Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.23908.tmp
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                            random
random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read
write    read  rewrite     read   fwrite frewrite   fread  freread
         8388608   16384  961438 1053043  1471859  1496125

iozone test complete.


With Glusterfs

iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.315 $
		Compiled for 64 bit mode.
		Build: linux

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

	Run began: Mon Jun  7 12:50:02 2010

	Auto Mode
	File size set to 8388608 KB
	Record Size 16384 KB
	Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.23908.tmp
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                            random
random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read
write    read  rewrite     read   fwrite frewrite   fread  freread
         8388608   16384  344149  578252   223183   201110

iozone test complete.

GlusterFS on remote machine.

iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.283 $
		Compiled for 64 bit mode.
		Build: linux

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
	             Erik Habbinga, Kris Strecker, Walter Wong.

	Run began: Mon Jun  7 12:59:19 2010

	Auto Mode
	File size set to 8388608 KB
	Record Size 16384 KB
	Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.14702.tmp
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                            random
random    bkwd  record  stride
              KB  reclen   write rewrite    read    reread    read
write    read rewrite    read   fwrite frewrite   fread  freread
         8388608   16384  596662  724340   227773   209839

iozone test complete.


## file auto generated by /usr/bin/glusterfs-volgen (mount.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name neuron --transport ib-verbs
192.168.0.117:/node01/storage

# TRANSPORT-TYPE ib-verbs
volume 192.168.0.117-1
    type protocol/client
    option transport-type ib-verbs
    option remote-host 192.168.0.117
    option transport.ib-verbs.port 1
    option transport.remote-port 6997
    option remote-subvolume brick1
end-volume

volume writebehind
    type performance/write-behind
    option cache-size 4MB
    subvolumes 192.168.0.117-1
end-volume

volume readahead
    type performance/read-ahead
    option page-count 4
    subvolumes writebehind
end-volume

volume iocache
    type performance/io-cache
    option cache-size 1GB
    option cache-timeout 1
    subvolumes readahead
end-volume

volume quickread
    type performance/quick-read
    option cache-timeout 1
    option max-file-size 64kB
    subvolumes iocache
end-volume

volume statprefetch
    type performance/stat-prefetch
    subvolumes quickread
end-volume

## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name neuron --transport ib-verbs
192.168.0.117:/node01/storage

volume posix1
  type storage/posix
  option directory /node01/storage
end-volume

volume locks1
    type features/locks
    subvolumes posix1
end-volume

volume brick1
    type performance/io-threads
    option thread-count 8
    subvolumes locks1
end-volume

volume server-ib-verbs
    type protocol/server
    option transport-type ib-verbs
    option auth.addr.brick1.allow *
    option transport.ib-verbs.listen-port 6997
    option transport.ib-verbs.port 1
    subvolumes brick1
end-volume