<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
        {font-family:宋体;
        panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:DengXian;
        panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:"Segoe UI";
        panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
        {font-family:"\@宋体";
        panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
        {font-family:"\@等线";
        panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:宋体;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
        {mso-style-name:msonormal;
        mso-margin-top-alt:auto;
        margin-right:0cm;
        mso-margin-bottom-alt:auto;
        margin-left:0cm;
        font-size:12.0pt;
        font-family:宋体;}
span.EmailStyle18
        {mso-style-type:personal-reply;
        font-family:DengXian;
        color:windowtext;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-family:DengXian;}
@page WordSection1
        {size:612.0pt 792.0pt;
        margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Ok ,another question, why priv-&gt;ssl_sbio = BIO_new_socket(priv-&gt;sock, BIO_NOCLOSE); use NOCLOSE mode instead of BIO_CLOSE?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">cynthia<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Milind Changire &lt;mchangir@redhat.com&gt;
<br>
<b>Sent:</b> Monday, April 22, 2019 2:21 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;cynthia.zhou@nokia-sbell.com&gt;<br>
<b>Cc:</b> Atin Mukherjee &lt;amukherj@redhat.com&gt;; gluster-devel@gluster.org<br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Looks like using BIO_free() is not right. Here's what the SSL_set_bio() man page says ...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="background:#FFF2CC">SSL_set_bio() is similar to SSL_set0_rbio() and SSL_set0_wbio() except that it connects both the rbio and the wbio at the same time, and
<b><span style="color:red">transfers the ownership of rbio and wbio to ssl</span></b> according to the following set of rules:</span><span lang="EN-US"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">So, I think you were right about SSL_free() doing the job for the bio. However, SSL_free() has no reason to set the priv-&gt;ssl_sbio pointer to NULL. I think priv-&gt;ssl_sbio should be set to NULL immediately after the call
 to SSL_set_bio() is successful. And we need to add a comment while setting priv-&gt;ssl_sbio to NULL that the ownership of the bio has now been transferred to SSL and SSL will free the related memory appropriately.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p>&nbsp;</o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">On Mon, Apr 22, 2019 at 11:37 AM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com">cynthia.zhou@nokia-sbell.com</a>&gt; wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">I tried to print priv-&gt;ssl_sbio after SSL_free() find this pointer is not null, so I add free ssl_sbio with BIO_free,
 however this cause glusterfd coredump</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">(gdb) bt</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#0&nbsp; 0x00007f3047867f9b in raise () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#1&nbsp; 0x00007f3047869351 in abort () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#2&nbsp; 0x00007f30478aa8c7 in __libc_message () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#3&nbsp; 0x00007f30478b0e6a in malloc_printerr () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#4&nbsp; 0x00007f30478b2835 in _int_free () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#5&nbsp; 0x00007f3047c5bbbd in CRYPTO_free () from /lib64/libcrypto.so.10</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#6&nbsp; 0x00007f3047d07582 in BIO_free () from /lib64/libcrypto.so.10</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#7&nbsp; 0x00007f3043f9ba4b in __socket_reset (this=0x7f303c1ae710) at socket.c:1032</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#8&nbsp; 0x00007f3043f9c4aa in socket_event_poll_err (this=0x7f303c1ae710, gen=1, idx=17) at socket.c:1232</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#9&nbsp; 0x00007f3043fa1b7d in socket_event_handler (fd=26, idx=17, gen=1, data=0x7f303c1ae710, poll_in=1, poll_out=0,
 poll_err=0) at socket.c:2669</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#10 0x00007f3049307984 in event_dispatch_epoll_handler (event_pool=0x1035610, event=0x7f3043b14e84) at event-epoll.c:587</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#11 0x00007f3049307c5b in event_dispatch_epoll_worker (data=0x107e3e0) at event-epoll.c:663</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#12 0x00007f30480535da in start_thread () from /lib64/libpthread.so.0</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">#13 0x00007f3047929eaf in clone () from /lib64/libc.so.6</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">@@ -1019,7 &#43;1019,20 @@ static void __socket_reset(rpc_transport_t *this) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; memset(&amp;priv-&gt;incoming, 0, sizeof(priv-&gt;incoming));</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;event_unregister_close(this-&gt;ctx-&gt;event_pool, priv-&gt;sock, priv-&gt;idx);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">-</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_INFO,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;sock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_INFO,&quot;priv-&gt;ssl_sbio of socket(%d)is %p &quot;,priv-&gt;sock,priv-&gt;ssl_sbio);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; if(priv-&gt;ssl_sbio != NULL)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; BIO_free(priv-&gt;ssl_sbio);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_sbio = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; }</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;sock = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;idx = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;connected = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">@@ -4238,6 &#43;4251,20 @@ void fini(rpc_transport_t *this) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_mutex_destroy(&amp;priv-&gt;out_lock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_mutex_destroy(&amp;priv-&gt;cond_lock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_cond_destroy(&amp;priv-&gt;cond);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_INFO,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;sock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_INFO,&quot;priv-&gt;ssl_sbio of socket(%d)is %p &quot;,priv-&gt;sock,priv-&gt;ssl_sbio);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; if(priv-&gt;ssl_sbio != NULL)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; BIO_free(priv-&gt;ssl_sbio);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_sbio = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; }</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; if (priv-&gt;ssl_private_key) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; GF_FREE(priv-&gt;ssl_private_key);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Milind
 Changire &lt;<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>&gt;
<br>
<b>Sent:</b> Monday, April 22, 2019 1:35 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;<br>
<b>Cc:</b> Atin Mukherjee &lt;<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>&gt;;
<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">This probably went unnoticed until now.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">On Mon, Apr 22, 2019 at 10:45 AM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;
 wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid windowtext 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Why there is no bio_free called in ssl_teardown_connection then?
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">cynthia</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Milind
 Changire &lt;</span><span lang="EN-US"><a href="mailto:mchangir@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">mchangir@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;
<br>
<b>Sent:</b> Monday, April 22, 2019 10:21 AM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;</span><span lang="EN-US"><a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">cynthia.zhou@nokia-sbell.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;<br>
<b>Cc:</b> Atin Mukherjee &lt;</span><span lang="EN-US"><a href="mailto:amukherj@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">amukherj@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;;
</span><span lang="EN-US"><a href="mailto:gluster-devel@gluster.org" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">gluster-devel@gluster.org</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">After patch
<a href="https://review.gluster.org/c/glusterfs/&#43;/22334" target="_blank">22334</a>, the
</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">priv-&gt;ssl_ctx</span><span lang="EN-US"> is now maintained per socket connection and is no longer shared.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">So you might want to
</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">SSL_free(priv-&gt;ssl_ctx)</span><span lang="EN-US"> as well and set
</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">priv-&gt;ssl_ctx</span><span lang="EN-US"> to
</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">NULL</span><span lang="EN-US">.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">There might be some strings that are duplicated (</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">gf_strdup()</span><span lang="EN-US">) via the
</span><span lang="EN-US" style="font-family:&quot;Courier New&quot;">socket_init()</span><span lang="EN-US"> code path. Please take a look at those as well.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Sorry about that. I missed it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">On Mon, Apr 22, 2019 at 7:25 AM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;
 wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid windowtext 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Hi,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">From my code study it seems priv-&gt;ssl_ssl is not properly released, I made a patch and the glusterfsd memory leak
 is alleviated with my patch, but some otherwhere is still leaking, I have no clue about the other leak points.</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">--- a/rpc/rpc-transport/socket/src/socket.c</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&#43;&#43; b/rpc/rpc-transport/socket/src/socket.c</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">@@ -1019,7 &#43;1019,16 @@ static void __socket_reset(rpc_transport_t *this) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; memset(&amp;priv-&gt;incoming, 0, sizeof(priv-&gt;incoming));</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;event_unregister_close(this-&gt;ctx-&gt;event_pool, priv-&gt;sock, priv-&gt;idx);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">-</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_TRACE,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;sock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; }</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;sock = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;idx = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp; priv-&gt;connected = -1;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">@@ -4238,6 &#43;4250,16 @@ void fini(rpc_transport_t *this) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_mutex_destroy(&amp;priv-&gt;out_lock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_mutex_destroy(&amp;priv-&gt;cond_lock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; pthread_cond_destroy(&amp;priv-&gt;cond);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp; if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; gf_log(this-&gt;name, GF_LOG_TRACE,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; priv-&gt;sock);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp;&nbsp;&nbsp; priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&#43;&nbsp; }</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; if (priv-&gt;ssl_private_key) {</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; GF_FREE(priv-&gt;ssl_private_key);</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;&nbsp;&nbsp;&nbsp; }</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0cm 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Zhou,
 Cynthia (NSB - CN/Hangzhou) <br>
<b>Sent:</b> Thursday, April 18, 2019 5:31 PM<br>
<b>To:</b> 'Atin Mukherjee' &lt;</span><span lang="EN-US"><a href="mailto:amukherj@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">amukherj@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;<br>
<b>Cc:</b> 'Raghavendra Gowdappa' &lt;</span><span lang="EN-US"><a href="mailto:rgowdapp@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">rgowdapp@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;;
 '</span><span lang="EN-US"><a href="mailto:gluster-devel@gluster.org" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">gluster-devel@gluster.org</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">'
 &lt;</span><span lang="EN-US"><a href="mailto:gluster-devel@gluster.org" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">gluster-devel@gluster.org</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;<br>
<b>Subject:</b> RE: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">We scan it use memory-leak tool, there are following prints. We doubt some open ssl lib malloc is is not properly
 freed by glusterfs code.</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">er&#43;0x2af [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libglusterfs.so.0.0.1]\n\t\tstart_thread&#43;0xda</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">
 [</span><span lang="EN-US"><a href="http://libpthread-2.27.so" target="_blank"><span style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif">libpthread-2.27.so</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">13580</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 175 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">232904</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 14 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
[15:41:56] Top 10 stacks with outstanding allocations:<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">8792</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 14 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">9408</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 42 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_realloc&#43;0x4d [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">9723</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 14 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">10696</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 21 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">11319</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 602 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">11431</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 518 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">11704</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A"> bytes in 371 allocations from stack<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;b'CRYPTO_malloc&#43;0x58 [</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue">libcrypto.so.1.0.2p]\n\t\t[unknown</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:#1A1A1A">]'</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">cynthia</span><span lang="EN-US"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0cm 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Zhou,
 Cynthia (NSB - CN/Hangzhou) <br>
<b>Sent:</b> Thursday, April 18, 2019 5:25 PM<br>
<b>To:</b> 'Atin Mukherjee' &lt;</span><span lang="EN-US"><a href="mailto:amukherj@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">amukherj@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;<br>
<b>Cc:</b> Raghavendra Gowdappa &lt;</span><span lang="EN-US"><a href="mailto:rgowdapp@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">rgowdapp@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;;
</span><span lang="EN-US"><a href="mailto:gluster-devel@gluster.org" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">gluster-devel@gluster.org</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"><br>
<b>Subject:</b> RE: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">I</span><span style="font-size:10.5pt;font-family:DengXian">’<span lang="EN-US">ve test on glusterfs3.12.15 and
 glusterfs5.5 all have this issue, after enable tls ssl socket, when execute gluster v heal &lt;vol-name&gt; info, will trigger glfshel to connect glusterfsd process, and cause glusterfsd process memory leak. Could you please try in your env?
</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">cynthia</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">&nbsp;</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> Atin
 Mukherjee &lt;</span><span lang="EN-US"><a href="mailto:amukherj@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">amukherj@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;
<br>
<b>Sent:</b> Thursday, April 18, 2019 1:19 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;</span><span lang="EN-US"><a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">cynthia.zhou@nokia-sbell.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;<br>
<b>Cc:</b> Raghavendra Gowdappa &lt;</span><span lang="EN-US"><a href="mailto:rgowdapp@redhat.com" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">rgowdapp@redhat.com</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">&gt;;
</span><span lang="EN-US"><a href="mailto:gluster-devel@gluster.org" target="_blank"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">gluster-devel@gluster.org</span></a></span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">On Wed, 17 Apr 2019 at 10:53, Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;
 wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid windowtext 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">In my recent test, I found that there are very severe glusterfsd memory leak when enable socket ssl option<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">What gluster version are you testing? Would you be able to continue your investigation and share the root cause?&nbsp;<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">&nbsp;<o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">--
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">- Atin (atinm)<o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><br clear="all">
<br>
-- <o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="EN-US">Milind<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><br clear="all">
<br>
-- <o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="EN-US">Milind<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-US"><br clear="all">
<br>
-- <o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US">Milind<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>