<div dir="ltr"><div>I would still be worried about the Invalid read/write. IMO whether
an illegal access causes a crash depends on whether the page is
currently mapped.<br></div><div>So, it could so happen that there is a
use after free / use outside of bounds happening in the code and it
turns out that this location gets mapped in a different (unmapped) page
when IO threads is not loaded.<br><br></div>Could you please share the valgrind logs as well.</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 7, 2017 at 8:22 PM, Kinglong Mee <span dir="ltr"><<a href="mailto:kinglongmee@gmail.com" target="_blank">kinglongmee@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">After deleting io-threads from the vols, quota operates (list/set/modify) lets glusterfsd crash.<br>
I use it at CentOS 7 (CentOS Linux release 7.3.1611) with glusterfs 3.8.12.<br>
It seems the stack corrupt, when testing with the following diff, glusterfsd runs correctly.<br>
<br>
There are two questions as,<br>
1. When using valgrind, it shows there are many "Invalid read/write" when with io-threads.<br>
Why glusterfsd runs correctly with io-threads? but crash without io-threads?<br>
<br>
2. With the following diff, valgrind also shows many "Invalid read/write" when without io-threads?<br>
but no any crash.<br>
<br>
Any comments are welcome.<br>
<br>
Revert <a href="http://review.gluster.org/11499" rel="noreferrer" target="_blank">http://review.gluster.org/<wbr>11499</a> seems better than the diff.<br>
<br>
diff --git a/xlators/features/marker/src/<wbr>marker-quota.c b/xlators/features/marke<br>
index 902b8e5..f3d2507 100644<br>
--- a/xlators/features/marker/src/<wbr>marker-quota.c<br>
+++ b/xlators/features/marker/src/<wbr>marker-quota.c<br>
@@ -1075,7 +1075,7 @@ mq_synctask1 (xlator_t *this, synctask_fn_t task, gf_boole<br>
}<br>
<br>
if (spawn) {<br>
- ret = synctask_new1 (this->ctx->env, 1024 * 16, task,<br>
+ ret = synctask_new1 (this->ctx->env, 0, task,<br>
mq_synctask_cleanup, NULL, args);<br>
if (ret) {<br>
gf_log (this->name, GF_LOG_ERROR, "Failed to spawn "<br>
<br>
------------------------------<wbr>-----test steps ------------------------------<wbr>----<br>
1. gluster volume create gvtest node1:/test/ node2:/test/<br>
2. gluster volume start gvtest<br>
3. gluster volume quota enable gvtest<br>
<br>
4. "deletes io-threads from all vols"<br>
5. reboot node1 and node2.<br>
6. sh quota-set.sh<br>
<br>
# cat quota-set.sh<br>
gluster volume quota gvtest list<br>
gluster volume quota gvtest limit-usage / 10GB<br>
gluster volume quota gvtest limit-usage /1234 1GB<br>
gluster volume quota gvtest limit-usage /hello 1GB<br>
gluster volume quota gvtest limit-usage /test 1GB<br>
gluster volume quota gvtest limit-usage /xyz 1GB<br>
gluster volume quota gvtest list<br>
gluster volume quota gvtest remove /hello<br>
gluster volume quota gvtest remove /test<br>
gluster volume quota gvtest list<br>
gluster volume quota gvtest limit-usage /test 1GB<br>
gluster volume quota gvtest remove /xyz<br>
gluster volume quota gvtest list<br>
<br>
-----------------------<wbr>glusterfsd crash without the diff--------------------------<wbr>------<br>
<br>
/usr/local/lib/libglusterfs.<wbr>so.0(_gf_msg_backtrace_nomem+<wbr>0xf5)[0x7f6e1e950af1]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(gf_print_trace+0x21f)[<wbr>0x7f6e1e956943]<br>
/usr/local/sbin/glusterfsd(<wbr>glusterfsd_print_trace+0x1f)[<wbr>0x409c83]<br>
/lib64/libc.so.6(+0x35250)[<wbr>0x7f6e1d025250]<br>
/lib64/libc.so.6(gsignal+0x37)<wbr>[0x7f6e1d0251d7]<br>
/lib64/libc.so.6(abort+0x148)[<wbr>0x7f6e1d0268c8]<br>
/lib64/libc.so.6(+0x74f07)[<wbr>0x7f6e1d064f07]<br>
/lib64/libc.so.6(+0x7baf5)[<wbr>0x7f6e1d06baf5]<br>
/lib64/libc.so.6(+0x7c3e6)[<wbr>0x7f6e1d06c3e6]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(__gf_free+0x311)[<wbr>0x7f6e1e981327]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(synctask_destroy+0x82)[<wbr>0x7f6e1e995c20]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(synctask_done+0x25)[<wbr>0x7f6e1e995c47]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(synctask_switchto+0xcf)[<wbr>0x7f6e1e996585]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(syncenv_processor+0x60)[<wbr>0x7f6e1e99663d]<br>
/lib64/libpthread.so.0(+<wbr>0x7dc5)[0x7f6e1d7a2dc5]<br>
/lib64/libc.so.6(clone+0x6d)[<wbr>0x7f6e1d0e773d]<br>
<br>
or<br>
<br>
package-string: glusterfs 3.8.12<br>
/usr/local/lib/libglusterfs.<wbr>so.0(_gf_msg_backtrace_nomem+<wbr>0xf5)[0x7fa15e623af1]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(gf_print_trace+0x21f)[<wbr>0x7fa15e629943]<br>
/usr/local/sbin/glusterfsd(<wbr>glusterfsd_print_trace+0x1f)[<wbr>0x409c83]<br>
/lib64/libc.so.6(+0x35250)[<wbr>0x7fa15ccf8250]<br>
/lib64/libc.so.6(gsignal+0x37)<wbr>[0x7fa15ccf81d7]<br>
/lib64/libc.so.6(abort+0x148)[<wbr>0x7fa15ccf98c8]<br>
/lib64/libc.so.6(+0x74f07)[<wbr>0x7fa15cd37f07]<br>
/lib64/libc.so.6(+0x7dd4d)[<wbr>0x7fa15cd40d4d]<br>
/lib64/libc.so.6(__libc_<wbr>calloc+0xb4)[0x7fa15cd43a14]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(__gf_calloc+0xa7)[<wbr>0x7fa15e653a5f]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(iobref_new+0x2b)[<wbr>0x7fa15e65875a]<br>
/usr/local/lib/glusterfs/3.8.<wbr>12/rpc-transport/socket.so(+<wbr>0xa98c)[0x7fa153a8398c]<br>
/usr/local/lib/glusterfs/3.8.<wbr>12/rpc-transport/socket.so(+<wbr>0xacbc)[0x7fa153a83cbc]<br>
/usr/local/lib/glusterfs/3.8.<wbr>12/rpc-transport/socket.so(+<wbr>0xad10)[0x7fa153a83d10]<br>
/usr/local/lib/glusterfs/3.8.<wbr>12/rpc-transport/socket.so(+<wbr>0xb2a7)[0x7fa153a842a7]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(+0x97ea9)[0x7fa15e68eea9]<br>
/usr/local/lib/libglusterfs.<wbr>so.0(+0x982c6)[0x7fa15e68f2c6]<br>
/lib64/libpthread.so.0(+<wbr>0x7dc5)[0x7fa15d475dc5]<br>
/lib64/libc.so.6(clone+0x6d)[<wbr>0x7fa15cdba73d]<br>
<br>
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br></div>