[Gluster-users] No performance difference using libgfapi?

[Dan Lambright]

A possible reason that you do not see a performance difference is the buffer cache on the client. This cache is unavailable to libgfapi, but may be utilized if you use FUSE.

Take a look at the mount option: fopen-keep-cache. If you have the source, you can find that option in fuse-bridge.c. By default, this is enabled. Gluster will invalidate cache entries (using FUSE interfaces) when it detects changes using STAT.

You could try to disable that mount option to see what difference it makes.

----- Original Message -----
From: "Humble Devassy Chirammal" <humble.devassy at gmail.com>
To: "Dave Christianson" <davidchristianson3 at gmail.com>
Cc: gluster-users at gluster.org
Sent: Friday, April 4, 2014 3:05:20 AM
Subject: Re: [Gluster-users] No performance difference using libgfapi?

Hi David, 

Regarding hdparm: 

'hdparm' has to be used against SATA/IDE device. 

--snip-- 
hdparm - get/set SATA/IDE device parameters 

hdparm provides a command line interface to various kernel interfaces supported by the Linux SATA/PATA/SAS "libata" subsystem and the older 
IDE driver subsystem. Many newer (2008 and later) USB drive enclosures now also support "SAT" (SCSI-ATA Command Translation) and therefore 
may also work with hdparm. E.g. recent WD "Passport" models and recent NexStar-3 enclosures. Some options may work correctly only with 
the latest kernels. 

--/snip-- 

Here in your guest , its 'virtio' disk ( /dev/vd{a,b,c..} which uses 'virtio' bus and virtio-blk is not ATA, so this looks to be an incorrect way of using 'hdparm'. 

More or less, now the virt software allows you to use "virtio-scsi" ( the disk shown inside the guest will be sd{a,b,..}, where most of featureset is respected from scsi protocol point of view.. you may look into that as well. 

--Humble 





On Thu, Apr 3, 2014 at 3:35 PM, Dave Christianson < davidchristianson3 at gmail.com > wrote: 



Good Morning, 

In my earlier experience invoking a VM using qemu/libgfapi, I reported that it was noticeably faster than the same VM invoked from libvirt using a FUSE mount; however, this was erroneous as the qemu/libgfapi-invoked image was created using 2x the RAM and cpu's... 

So, invoking the image using both methods using consistent specs of 2GB RAM and 2 cpu's, I attempted to check drive performance using the following commands: 

(For regular FUSE mount I have the gluster volume mounted at /var/lib/libvirt/images.) 

(For libgfapi I specify -disk file=gluster://gfs-00/gfsvol/tester1/img.) 

Using libvirt/FUSE mount: 
[root at tester1 ~]# hdparm -Tt /dev/vda1 
/dev/vda1: 
Timing cached reads: 11346 MB in 2.00 seconds = 5681.63 MB/sec 
Timing buffered disk reads: 36 MB in 3.05 seconds = 11.80 MB/sec 
[root at tester1 ~]# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output 
10240+0 records in 
10240+0 records out 
41943040 bytes (42MB) copied, 0.0646241 s, 649 MB/sec 

Using qemu/libgfapi: 
[root at tester1 ~]# hdparm -Tt /dev/vda1 
/dev/vda1: 
Timing cached reads: 11998 MB in 2.00 seconds = 6008.57 MB/sec 
Timing buffered disk reads: 36 MB in 3.03 seconds = 11.89 MB/sec 
[root at tester1 ~]# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output 
10240+0 records in 
10240+0 records out 
41943040 bytes (42MB) copied, 0.0621412 s, 675 MB/sec 

Should I be seeing a bigger difference, or am I doing something wrong? 

I'm also curious whether people have found that the performance difference is greater as the size of the gluster cluster scales up. 

Thanks, 
David 


_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://supercolony.gluster.org/mailman/listinfo/gluster-users 


_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users