[Gluster-devel] Fw: Re: Corvid gluster testing

Thu Aug 7 20:32:27 UTC 2014

I could give you the size of the data done with each "write" statement in our fortran code if that is helpful. It is a fortran code with write statements inside a do-loop. I can give you the size of the writes along with the number if writes if that is helpful. 

> You could also enable the io-stats xlator on the client side just below FUSE (before reaching write-behind), and extract data using setfattr.

Happy to do any testing you want. I have no idea how to do the above. If you can tell me what to do, I will test when I get back Monday. 

David  (Sent from mobile)

===============================
David F. Robinson, Ph.D. 
President - Corvid Technologies
704.799.6944 x101 [office]
704.252.1310      [cell]
704.799.7974      [fax]
David.Robinson at corvidtec.com
http://www.corvidtechnologies.com

> On Aug 7, 2014, at 2:05 PM, Anand Avati <avati at gluster.org> wrote:
> 
> David,
> Is it possible to profile the app to understand the block sizes used for performing write() (using strace, source code inspection etc)? The block sizes reported by gluster volume profile is measured on the server side and is subject to some aggregation by the client side write-behind xlator. Typically the biggest hurdle for small block writes is FUSE context switches which happens even before reaching the client side write-behind xlator.
> 
> You could also enable the io-stats xlator on the client side just below FUSE (before reaching write-behind), and extract data using setfattr.
> 
> 
> 
>> On Wed, Aug 6, 2014 at 10:00 AM, David F. Robinson <david.robinson at corvidtec.com> wrote:
>> My apologies.  I did some additional testing and realized that my timing wasn't right.  I believe that after I do the write, NFS caches the data and until I close and flush the file, the timing isn't correct.
>> I believe the appropriate timing is now 38-seconds for NFS and 60-seconds for gluster.  I played around with some of the parameters and got it down to 52-seconds with gluster by setting:
>> 
>> performance.write-behind-window-size: 128MB
>> performance.cache-size: 128MB
>> 
>> I couldn't get it closer to the NFS timing on the writes, although the read speads were slightly better than NFS.  I am not sure if this is reasonable, or if I should be able to get write speeds that are more comparable to the NFS mount...
>> 
>> Sorry for the confusion I might have caused with my first email... It isn't 25x slower.  It is roughly 30% slower for the writes...
>> 
>> 
>> David
>> 
>> 
>> ------ Original Message ------
>> From: "Vijay Bellur" <vbellur at redhat.com>
>> To: "David F. Robinson" <david.robinson at corvidtec.com>; gluster-devel at gluster.org
>> Sent: 8/6/2014 12:48:09 PM
>> Subject: Re: [Gluster-devel] Fw: Re: Corvid gluster testing
>> 
>>>> On 08/06/2014 12:11 AM, David F. Robinson wrote:
>>>> I have been testing some of the fixes that Pranith incorporated into the
>>>> 3.5.2-beta to see how they performed for moderate levels of i/o. All of
>>>> the stability issues that I had seen in previous versions seem to have
>>>> been fixed in 3.5.2; however, there still seem to be some significant
>>>> performance issues. Pranith suggested that I send this to the
>>>> gluster-devel email list, so here goes:
>>>> I am running an MPI job that saves a restart file to the gluster file
>>>> system. When I use the following in my fstab to mount the gluster
>>>> volume, the i/o time for the 2.5GB file is roughly 45-seconds.
>>>> / gfsib01a.corvidtec.com:/homegfs /homegfs glusterfs
>>>> transport=tcp,_netdev 0 0
>>>> /
>>>> When I switch this to use the NFS protocol (see below), the i/o time is
>>>> 2.5-seconds.
>>>> / gfsib01a.corvidtec.com:/homegfs /homegfs nfs
>>>> vers=3,intr,bg,rsize=32768,wsize=32768 0 0/
>>>> The read-times for gluster are 10-20% faster than NFS, but the write
>>>> times are almost 20x slower.
>>> 
>>> What is the block size of the writes that are being performed? You can expect better throughput and lower latency with block sizes that are close to or greater than 128KB.
>>> 
>>> -Vijay
>> 
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140807/24bca192/attachment-0001.html>