<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jun 10, 2019 at 10:43 AM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com">cynthia.zhou@nokia-sbell.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">





<div lang="ZH-CN">
<div class="gmail-m_-6931486356439643519WordSection1">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Hi,
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">How about this patch? I see there is a failed test, is that related to my change?</span></p></div></div></blockquote><div><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Quite likely. Have you looked at the failure? It produces a stack which looks close to where your patch is:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">01:02:58.118 Thread 1 (Thread 0x7efe40930700 (LWP 17150)):<br>01:02:58.118 #0  0x00007efe4dfd359c in free () from /lib64/libc.so.6<br>01:02:58.118 No symbol table info available.<br>01:02:58.118 #1  0x00007efe4e38970d in CRYPTO_free () from /lib64/libcrypto.so.10<br>01:02:58.118 No symbol table info available.<br>01:02:58.118 #2  0x00007efe4e4400e7 in sk_free () from /lib64/libcrypto.so.10<br>01:02:58.118 No symbol table info available.<br>01:02:58.118 #3  0x00007efe4e4863de in x509_verify_param_zero () from /lib64/libcrypto.so.10<br>01:02:58.118 No symbol table info available.<br>01:02:58.118 #4  0x00007efe4e48644e in X509_VERIFY_PARAM_free () from /lib64/libcrypto.so.10<br>01:02:58.118 No symbol table info available.<br>01:02:58.118 #5  0x00007efe42a107d9 in SSL_CTX_free () from /lib64/libssl.so.10<br>01:02:58.120 No symbol table info available.<br>01:02:58.120 #6  0x00007efe42a12cc0 in SSL_free () from /lib64/libssl.so.10<br>01:02:58.122 No symbol table info available.<br>01:02:58.122 #7  0x00007efe42c463eb in __socket_reset (this=0x7efe34001240) at /home/jenkins/root/workspace/centos7-regression/rpc/rpc-transport/socket/src/socket.c:1170<br>01:02:58.123         priv = 0x7efe340017a0<br>01:02:58.123         __FUNCTION__ = &quot;__socket_reset&quot;<br>01:02:58.123 #8  0x00007efe42c46e43 in socket_event_poll_err (this=0x7efe34001240, gen=4, idx=2) at /home/jenkins/root/workspace/centos7-regression/rpc/rpc-transport/socket/src/socket.c:1383<br>01:02:58.123         priv = 0x7efe340017a0<br>01:02:58.123         socket_closed = false<br>01:02:58.123         __FUNCTION__ = &quot;socket_event_poll_err&quot;<br>01:02:58.123 #9  0x00007efe42c4d056 in socket_event_handler (fd=6, idx=2, gen=4, data=0x7efe34001240, poll_in=1, poll_out=0, poll_err=16, event_thread_died=0 &#39;\000&#39;) at /home/jenkins/root/workspace/centos7-regression/rpc/rpc-transport/socket/src/socket.c:3037</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="ZH-CN"><div class="gmail-m_-6931486356439643519WordSection1"><p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">cynthia<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Raghavendra Gowdappa &lt;<a href="mailto:rgowdapp@redhat.com" target="_blank">rgowdapp@redhat.com</a>&gt;
<br>
<b>Sent:</b> Thursday, May 09, 2019 12:13 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;<br>
<b>Cc:</b> Amar Tumballi Suryanarayan &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;; <a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">Thanks!!<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">On Thu, May 9, 2019 at 8:34 AM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt; wrote:<u></u><u></u></span></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Hi,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Ok, It is posted to
</span><span lang="EN-US"><a href="https://review.gluster.org/#/c/glusterfs/+/22687/" target="_blank">https://review.gluster.org/#/c/glusterfs/+/22687/</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Raghavendra
 Gowdappa &lt;<a href="mailto:rgowdapp@redhat.com" target="_blank">rgowdapp@redhat.com</a>&gt;
<br>
<b>Sent:</b> Wednesday, May 08, 2019 7:35 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;<br>
<b>Cc:</b> Amar Tumballi Suryanarayan &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;;
<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">On Wed, May 8, 2019 at 1:29 PM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;
 wrote:<u></u><u></u></span></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Hi &#39;Milind Changire&#39; ,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">The leak is getting more and more clear to me now. the unsolved memory leak is because of in gluterfs version 3.12.15
 (in my env)the ssl context is a shared one, while we do ssl_acept, ssl will allocate some read/write buffer to ssl object, however, ssl_free in socket_reset or fini function of socket.c, the buffer is returened back to ssl context free list instead of completely
 freed.</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Thanks Cynthia for your efforts in identifying and fixing the leak. If you post a patch to gerrit, I&#39;ll be happy to merge it and get the fix into the codebase.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">So following patch is able to fix the memory leak issue completely.(created for gluster master branch)</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">--- a/rpc/rpc-transport/socket/src/socket.c<br>
+++ b/rpc/rpc-transport/socket/src/socket.c<br>
@@ -446,6 +446,7 @@ ssl_setup_connection_postfix(rpc_transport_t *this)<br>
     gf_log(this-&gt;name, GF_LOG_DEBUG,<br>
            &quot;SSL verification succeeded (client: %s) (server: %s)&quot;,<br>
            this-&gt;peerinfo.identifier, this-&gt;myinfo.identifier);<br>
+    X509_free(peer);<br>
     return gf_strdup(peer_CN);<br>
<br>
     /* Error paths. */<br>
@@ -1157,7 +1158,21 @@ __socket_reset(rpc_transport_t *this)<br>
     memset(&amp;priv-&gt;incoming, 0, sizeof(priv-&gt;incoming));<br>
<br>
     event_unregister_close(this-&gt;ctx-&gt;event_pool, priv-&gt;sock, priv-&gt;idx);<br>
-<br>
+    if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)<br>
+    {<br>
+      gf_log(this-&gt;name, GF_LOG_TRACE,<br>
+             &quot;clear and reset for socket(%d), free ssl &quot;,<br>
+             priv-&gt;sock);<br>
+               if(priv-&gt;ssl_ctx)<br>
+                 {<br>
+                       SSL_CTX_free(priv-&gt;ssl_ctx);<br>
+                       priv-&gt;ssl_ctx = NULL;<br>
+                 }<br>
+      SSL_shutdown(priv-&gt;ssl_ssl);<br>
+      SSL_clear(priv-&gt;ssl_ssl);<br>
+      SSL_free(priv-&gt;ssl_ssl);<br>
+      priv-&gt;ssl_ssl = NULL;<br>
+    }<br>
     priv-&gt;sock = -1;<br>
     priv-&gt;idx = -1;<br>
     priv-&gt;connected = -1;<br>
@@ -4675,6 +4690,21 @@ fini(rpc_transport_t *this)<br>
         pthread_mutex_destroy(&amp;priv-&gt;out_lock);<br>
         pthread_mutex_destroy(&amp;priv-&gt;cond_lock);<br>
         pthread_cond_destroy(&amp;priv-&gt;cond);<br>
+               if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)<br>
+               {<br>
+                 gf_log(this-&gt;name, GF_LOG_TRACE,<br>
+                                &quot;clear and reset for socket(%d), free ssl &quot;,<br>
+                                priv-&gt;sock);<br>
+                 if(priv-&gt;ssl_ctx)<br>
+                 {<br>
+                       SSL_CTX_free(priv-&gt;ssl_ctx);<br>
+                       priv-&gt;ssl_ctx = NULL;<br>
+                 }<br>
+                 SSL_shutdown(priv-&gt;ssl_ssl);<br>
+                 SSL_clear(priv-&gt;ssl_ssl);<br>
+                 SSL_free(priv-&gt;ssl_ssl);<u></u><u></u></span></p>
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Zhou,
 Cynthia (NSB - CN/Hangzhou) <br>
<b>Sent:</b> Monday, May 06, 2019 2:12 PM<br>
<b>To:</b> &#39;Amar Tumballi Suryanarayan&#39; &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;<br>
<b>Cc:</b> &#39;Milind Changire&#39; &lt;<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>&gt;; &#39;<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>&#39; &lt;<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>&gt;<br>
<b>Subject:</b> RE: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Hi,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">From our test valgrind and libleak all blame ssl3_accept</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">///////////////////////////from valgrind attached to glusterfds///////////////////////////////////////////</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">==16673== 198,720 bytes in 12 blocks are definitely lost in loss record 1,114 of 1,123<br>
==16673==    at 0x4C2EB7B: malloc (vg_replace_malloc.c:299)<br>
==16673==    by 0x63E1977: CRYPTO_malloc (in /usr/lib64/<u><span style="color:blue">libcrypto.so.1.0.2p</span></u>)<br>
==16673==    by 0xA855E0C: ssl3_setup_write_buffer (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA855E77: ssl3_setup_buffers (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA8485D9: <span style="color:red">ssl3_accept</span> (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA610DDF: ssl_complete_connection (socket.c:400)<br>
==16673==    by 0xA617F38: ssl_handle_server_connection_attempt (socket.c:2409)<br>
==16673==    by 0xA618420: socket_complete_connection (socket.c:2554)<br>
==16673==    by 0xA618788: socket_event_handler (socket.c:2613)<br>
==16673==    by 0x4ED6983: event_dispatch_epoll_handler (event-epoll.c:587)<br>
==16673==    by 0x4ED6C5A: event_dispatch_epoll_worker (event-epoll.c:663)<br>
==16673==    by 0x615C5D9: start_thread (in /usr/lib64/<u><span style="color:blue"><a href="http://libpthread-2.27.so" target="_blank">libpthread-2.27.so</a></span></u>)<br>
==16673==<br>
==16673== 200,544 bytes in 12 blocks are definitely lost in loss record 1,115 of 1,123<br>
==16673==    at 0x4C2EB7B: malloc (vg_replace_malloc.c:299)<br>
==16673==    by 0x63E1977: CRYPTO_malloc (in /usr/lib64/<u><span style="color:blue">libcrypto.so.1.0.2p</span></u>)<br>
==16673==    by 0xA855D12: ssl3_setup_read_buffer (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA855E68: ssl3_setup_buffers (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA8485D9: <span style="color:red">ssl3_accept</span> (in /usr/lib64/<u><span style="color:blue">libssl.so.1.0.2p</span></u>)<br>
==16673==    by 0xA610DDF: ssl_complete_connection (socket.c:400)<br>
==16673==    by 0xA617F38: ssl_handle_server_connection_attempt (socket.c:2409)<br>
==16673==    by 0xA618420: socket_complete_connection (socket.c:2554)<br>
==16673==    by 0xA618788: socket_event_handler (socket.c:2613)<br>
==16673==    by 0x4ED6983: event_dispatch_epoll_handler (event-epoll.c:587)<br>
==16673==    by 0x4ED6C5A: event_dispatch_epoll_worker (event-epoll.c:663)<br>
==16673==    by 0x615C5D9: start_thread (in /usr/lib64/<u><span style="color:blue"><a href="http://libpthread-2.27.so" target="_blank">libpthread-2.27.so</a></span></u>)<br>
==16673==<br>
valgrind --leak-check=f<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">////////////////////////////////////with libleak attached to glusterfsd/////////////////////////////////////////</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">callstack[2419] expires. count=1 size=224/224 alloc=362 free=350<br>
    /home/robot/libleak/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libleak.so(malloc+0x25</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f1460604065]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(CRYPTO_malloc+0x58</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ecd9978]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(EVP_DigestInit_ex+0x2a9</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed95749]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libssl.so.10(ssl3_digest_cached_records+0x11d</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145abb6ced]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libssl.so.10(</span></u><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:red" lang="EN-US">ssl3_accept</span></u><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">+0xc8f</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145abadc4f]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(ssl_complete_connection+0x5e</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae00f3a]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc16d</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae0816d]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc68a</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae0868a]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc9f2</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae089f2]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libglusterfs.so.0(+0x9b96f</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f146038596f]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libglusterfs.so.0(+0x9bc46</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f1460385c46]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libpthread.so.0(+0x75da</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145f0d15da]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libc.so.6(clone+0x3f</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145e9a7eaf]</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">callstack[2432] expires. count=1 size=104/104 alloc=362 free=0<br>
    /home/robot/libleak/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libleak.so(malloc+0x25</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f1460604065]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(CRYPTO_malloc+0x58</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ecd9978]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(BN_MONT_CTX_new+0x17</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed48627]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(BN_MONT_CTX_set_locked+0x6d</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ed489fd]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(+0xff4d9</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed6a4d9]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(int_rsa_verify+0x1cd</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed6d41d]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(RSA_verify+0x32</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed6d972]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(+0x107ff5</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed72ff5]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libcrypto.so.10(EVP_VerifyFinal+0x211</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145ed9dd51]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libssl.so.10(ssl3_get_cert_verify+0x5bb</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145abac06b]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libssl.so.10(</span></u><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:red" lang="EN-US">ssl3_accept</span></u><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">+0x988</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145abad948]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(ssl_complete_connection+0x5e</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae00f3a]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc16d</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae0816d]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc68a</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae0868a]<br>
    /usr/lib64/glusterfs/3.12.15/rpc-transport/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">socket.so(+0xc9f2</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">)
 [0x7f145ae089f2]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libglusterfs.so.0(+0x9b96f</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f146038596f]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libglusterfs.so.0(+0x9bc46</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f1460385c46]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libpthread.so.0(+0x75da</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145f0d15da]<br>
    /lib64/</span><u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:blue" lang="EN-US">libc.so.6(clone+0x3f</span></u><span style="font-size:10pt;font-family:&quot;Segoe UI&quot;,sans-serif;color:rgb(26,26,26)" lang="EN-US">) [0x7f145e9a7eaf]</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt;font-family:DengXian" lang="EN-US">one interesting thing is that the memory goes up to about 300m then it stopped  increasing !!!</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt;font-family:DengXian" lang="EN-US">I am wondering if this is caused by open-ssl library? But when I search from openssl community, there is no such
 issue reported before.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:14pt;font-family:DengXian" lang="EN-US">Is glusterfs using ssl_accept correctly?</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">cynthia</span><span lang="EN-US"><u></u><u></u></span></p>
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Zhou,
 Cynthia (NSB - CN/Hangzhou) <br>
<b>Sent:</b> Monday, May 06, 2019 10:34 AM<br>
<b>To:</b> &#39;Amar Tumballi Suryanarayan&#39; &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;<br>
<b>Cc:</b> Milind Changire &lt;<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>&gt;;
<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> RE: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Hi,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Sorry, I am so busy with other issues these days, could you help me to submit my patch for review? It is based on
 glusterfs3.12.15 code. But even with this patch , memory leak still exists, from memory leak tool it should be related with ssl_accept, not sure if it is because of openssl library or because improper use of ssl interfaces.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">--- a/rpc/rpc-transport/socket/src/socket.c</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+++ b/rpc/rpc-transport/socket/src/socket.c</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">@@ -1019,7 +1019,16 @@ static void __socket_reset(rpc_transport_t *this) {</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">   memset(&amp;priv-&gt;incoming, 0, sizeof(priv-&gt;incoming));</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">   event_unregister_close(this-&gt;ctx-&gt;event_pool, priv-&gt;sock, priv-&gt;idx);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">-</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+  if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+  {</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    gf_log(this-&gt;name, GF_LOG_INFO,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+           &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+           priv-&gt;sock);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+  }</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">   priv-&gt;sock = -1;</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">   priv-&gt;idx = -1;</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">   priv-&gt;connected = -1;</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">@@ -4238,6 +4250,16 @@ void fini(rpc_transport_t *this) {</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">     pthread_mutex_destroy(&amp;priv-&gt;out_lock);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">     pthread_mutex_destroy(&amp;priv-&gt;cond_lock);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">     pthread_cond_destroy(&amp;priv-&gt;cond);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+     if(priv-&gt;use_ssl&amp;&amp; priv-&gt;ssl_ssl)</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+  {</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    gf_log(this-&gt;name, GF_LOG_INFO,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+           &quot;clear and reset for socket(%d), free ssl &quot;,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+           priv-&gt;sock);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_shutdown(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_clear(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    SSL_free(priv-&gt;ssl_ssl);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+    priv-&gt;ssl_ssl = NULL;</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">+  }</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">     if (priv-&gt;ssl_private_key) {</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">       GF_FREE(priv-&gt;ssl_private_key);</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">     }</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Amar
 Tumballi Suryanarayan &lt;<a href="mailto:atumball@redhat.com" target="_blank">atumball@redhat.com</a>&gt;
<br>
<b>Sent:</b> Wednesday, May 01, 2019 8:43 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;<br>
<b>Cc:</b> Milind Changire &lt;<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>&gt;;
<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">Hi Cynthia Zhou,<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Can you post the patch which fixes the issue of missing free? We will continue to investigate the leak further, but would really appreciate getting the patch
 which is already worked on land into upstream master.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">-Amar<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">On Mon, Apr 22, 2019 at 1:38 PM Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;
 wrote:<u></u><u></u></span></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Ok, I am clear now.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">I</span><span style="font-size:10.5pt;font-family:DengXian">’<span lang="EN-US">ve added ssl_free in socket reset
 and socket finish function, though glusterfsd memory leak is not that much, still it is leaking, from source code I can not find anything else,</span></span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Could you help to check if this issue exists in your env? If not I may have a try to merge your patch .</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">Step</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="gmail-m_-6931486356439643519gmail-m3980449368408844904gmail-m-1109814905353783812gmail-m3903278274190134145msolistparagraph" style="margin-left:18pt">
<span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">1&gt;</span><span style="font-size:7pt;font-family:&quot;Times New Roman&quot;,serif" lang="EN-US">  
</span><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">while true;do gluster v heal &lt;vol-name&gt; info,</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="gmail-m_-6931486356439643519gmail-m3980449368408844904gmail-m-1109814905353783812gmail-m3903278274190134145msolistparagraph" style="margin-left:18pt">
<span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">2&gt;</span><span style="font-size:7pt;font-family:&quot;Times New Roman&quot;,serif" lang="EN-US">  
</span><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">check the vol-name glusterfsd memory usage, it is obviously increasing.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US">cynthia</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:DengXian" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US">From:</span></b><span style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif" lang="EN-US"> Milind
 Changire &lt;<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>&gt;
<br>
<b>Sent:</b> Monday, April 22, 2019 2:36 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) &lt;<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>&gt;<br>
<b>Cc:</b> Atin Mukherjee &lt;<a href="mailto:amukherj@redhat.com" target="_blank">amukherj@redhat.com</a>&gt;;
<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-devel] glusterfsd memory leak issue found after enable ssl</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">According to BIO_new_socket() man page ...<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style="background:rgb(255,242,204) none repeat scroll 0% 0%" lang="EN-US">If the close flag is set then the socket is shut down and closed when the BIO is freed.</span></b><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">For Gluster to have more control over the socket shutdown, the BIO_NOCLOSE flag is set. Otherwise, SSL takes control of socket shutdown whenever BIO is freed.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US">_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><u></u><u></u></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-US"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">--
<u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Amar Tumballi (amarts)<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12pt"><span lang="EN-US">_______________________________________________<br>
<br>
Community Meeting Calendar:<br>
<br>
APAC Schedule -<br>
Every 2nd and 4th Tuesday at 11:30 AM IST<br>
Bridge: <a href="https://bluejeans.com/836554017" target="_blank">https://bluejeans.com/836554017</a><br>
<br>
NA/EMEA Schedule -<br>
Every 1st and 3rd Tuesday at 01:00 PM EDT<br>
Bridge: <a href="https://bluejeans.com/486278655" target="_blank">https://bluejeans.com/486278655</a><br>
<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><u></u><u></u></span></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>

_______________________________________________<br>
<br>
Community Meeting Calendar:<br>
<br>
APAC Schedule -<br>
Every 2nd and 4th Tuesday at 11:30 AM IST<br>
Bridge: <a href="https://bluejeans.com/836554017" rel="noreferrer" target="_blank">https://bluejeans.com/836554017</a><br>
<br>
NA/EMEA Schedule -<br>
Every 1st and 3rd Tuesday at 01:00 PM EDT<br>
Bridge: <a href="https://bluejeans.com/486278655" rel="noreferrer" target="_blank">https://bluejeans.com/486278655</a><br>
<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><br>
<br>
</blockquote></div></div>