[Gluster-devel] Reduce memcpy in glfs read and write

Mohammed Rafi K C rkavunga at redhat.com
Wed Jun 22 06:56:03 UTC 2016


Hi Sachin,

Good to see you again in Gluster-devel.

I had implemented those API which I mentioned for a POC. In fact I
couldn't push into master.

Regarding your questions, my comments are inline.

On 06/22/2016 05:50 AM, Sachin Pandit wrote:
>
> Hey Pranith, I am good, I hope you are doing good too.
>
> Please find the comments inline.
>
>  
>
> *From:*Pranith Kumar Karampuri [mailto:pkarampu at redhat.com]
> *Sent:* Tuesday, June 21, 2016 5:58 AM
> *To:* Sachin Pandit <spandit at commvault.com>
> *Cc:* gluster-devel at gluster.org
> *Subject:* Re: [Gluster-devel] Reduce memcpy in glfs read and write
>
>  
>
> Hey!!
>
>        Hope you are doing good. I took a look at the bt. So when flush
> comes write-behind has to flush all the writes down. I see the
> following frame hung in iob_unref:
> Thread 7 (Thread 0x7fa601a30700 (LWP 16218)):
> #0  0x00007fa60cc55225 in pthread_spin_lock () from
> /lib64/libpthread.so.0  <<---- Does it always hang there?
>
> ---------------------------------
>
> >>It does always hang here.
>
> ---------------------------------
> #1  0x00007fa60e1f373e in iobref_unref (iobref=0x19dc7e0) at iobuf.c:907
> #2  0x00007fa60e246fb2 in args_wipe (args=0x19e70ec) at
> default-args.c:1593
> #3  0x00007fa60e1ea534 in call_stub_wipe_args (stub=0x19e709c) at
> call-stub.c:2466
> #4  0x00007fa60e1ea5de in call_stub_destroy (stub=0x19e709c) at
> call-stub.c:2482
>
> Is this on top of master branch? It seems like we missed an unlock of
> the spin-lock or the iobref has junk value which gives the feeling
> that it is in locked state (May be double free?). Do you have any
> extra patches you have in your repo which make changes in iobuf?
>
> ----------------------------------
>
> >>I have implemented a method to reduce memcpy in libgfapi (My patch is
> on top of master branch), by making use of buffer from iobuf pool and
> passing the buffer to application. However, I have not made any
> changes in iobuf core feature. I don’t  think double free is happening
> anywhere in the code (I did check this using logs)
>
>  
>
>      Method that I have implemented:
>
> 1)      Application asks for a buffer of specific size, and the buffer
> is allocated from the iobuf pool.
>
> 2)      Buffer is passed on to application, and the application writes
> the data into that buffer.
>
> 3)      Buffer with data in it is passed from application to libgfapi
> and the underlying translators (no memcpy in glfs_write)
>
>  
>
> I have couple of questions, and observations:
>
>  
>
> Observations:
>
> ------------------
>
> 1)      For every write if I get a fresh buffer then I don’t see any
> problem. All the writes are going through.
>
> 2)      If I try to make use of buffer for consecutive writes, then I
> am seeing the hang in flush.
>
>  
>
> Question1: Is it fine if I reuse the buffer for consecutive writes??
>

The answer is no, If IO-cache and write-behind are enabled. In case of
IO-cache, It just take a ref on the buffer to store into the cache,
which means the io-cache is still using the buffer,

In case of write-behind, if it decided to aggregate multiple write
request , in this case also it will take a ref on the iobuf and will lie
to the appluca


> Question2: Is it always ensured that the data is written to the file
> when I get a response from syncop_writev.
>

As I explained the write-behind may prevent this.


>  
>
> Thanks,
>
> Sachin Pandit.
>
>  
>
> ----------------------------------
>
>  
>
> On Tue, Jun 21, 2016 at 4:07 AM, Sachin Pandit <spandit at commvault.com
> <mailto:spandit at commvault.com>> wrote:
>
>     Hi all,
>
>      
>
>     I bid adieu to you all with the hope of crossing path again, and
>     the time has come rather quickly. It feels great to work on
>     GlusterFS again.
>
>                    
>
>     Currently we are trying to write data backed up by Commvault
>     Simpana to glusterfs volume (Disperse volume). To improve the
>     performance, I have implemented the proposal put forward my Rafi
>      K C [1]. I have some questions regarding libgfapi and iobuf pool.
>
>      
>
>     To reduce an extra level of copy in glfs read and write, I have
>     implemented few APIs to request a buffer (similar to the one
>     represented in  [1]) from iobuf pool which can be used by the
>     application to write data to. With this implementation, when I try
>     to reuse the buffer for consecutive writes, I could see a hang in
>     syncop_flush of glfs_close (BT of the hang can be found in [2]). I
>     wanted to know if reusing the buffer is recommended. If not, do we
>     need to request buffer for each writes?
>
>      
>
>     Setup : Distributed-Disperse ( 4 * (2+1)). Bricks scattered over 3
>     nodes.
>
>      
>
>     [1]
>     http://www.gluster.org/pipermail/gluster-devel/2015-February/043966.html
>
>     [2] Attached file -  bt.txt
>
>      
>
>     Thanks & Regards,
>
>     Sachin Pandit.
>
>      
>
>     ***************************Legal Disclaimer***************************
>
>     "This communication may contain confidential and privileged material for the
>
>     sole use of the intended recipient. Any unauthorized review, use or distribution
>
>     by others is strictly prohibited. If you have received the message by mistake,
>
>     please advise the sender by reply email and delete the message. Thank you."
>
>     **********************************************************************
>
>
>     _______________________________________________
>     Gluster-devel mailing list
>     Gluster-devel at gluster.org <mailto:Gluster-devel at gluster.org>
>     http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
>
> -- 
>
> Pranith
>
> ***************************Legal Disclaimer***************************
> "This communication may contain confidential and privileged material for the
> sole use of the intended recipient. Any unauthorized review, use or distribution
> by others is strictly prohibited. If you have received the message by mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **********************************************************************
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160622/46da336b/attachment-0001.html>


More information about the Gluster-devel mailing list