<div dir="ltr">I did a typo during the initial setup but corrected it. I do see a few files in the .trashcan folder that were created. However, gluster stopped about 3 hours after I enabled the feature. I don't have anything else in the logs than these entries.<div><br></div><div>Ludwig</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 20, 2017 at 10:15 AM, Anoop C S <span dir="ltr"><<a href="mailto:anoopcs@redhat.com" target="_blank">anoopcs@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tue, 2017-06-20 at 08:52 -0400, Ludwig Gamache wrote:<br>
> All,<br>
><br>
> I currently have 2 bricks running Gluster 3.10.1. This is a Centos installation. On Friday last<br>
> week, I enabled the trashcan feature on one of my volumes:<br>
> gluster volume set date01 features.trash on<br>
<br>
</span>I think you misspelled the volume name. Is it data01 or date01?<br>
<div><div class="h5"><br>
> I also limited the max file size to 500MB:<br>
> gluster volume set data01 features.trash-max-filesize 500MB<br>
> 3 hours after that I enabled this, this specific gluster volume went down:<br>
> [2017-06-16 16:08:14.410905] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: quotad already stopped<br>
> [2017-06-16 16:08:14.412027] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: quotad service is stopped<br>
> [2017-06-16 16:08:14.412217] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: bitd already stopped<br>
> [2017-06-16 16:08:14.412243] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: bitd service is stopped<br>
> [2017-06-16 16:08:14.412422] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: scrub already stopped<br>
> [2017-06-16 16:08:14.412451] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: scrub service is stopped<br>
> [2017-06-16 16:08:14.437968] I [run.c:191:runner_log] (<br>
> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S30samba-set.sh --volname=intermediate01 -o features.trash=on<br>
> --gd-workdir=/var/lib/glusterd<br>
> [2017-06-16 16:08:14.453290] I [run.c:191:runner_log] (<br>
> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S32gluster_enable_shared_<wbr>storage.sh --volname=intermediate01 -o<br>
> features.trash=on --gd-workdir=/var/lib/glusterd<br>
> [2017-06-16 16:08:22.416477] E [MSGID: 106525] [glusterd-op-sm.c:4188:<wbr>glusterd_dict_set_volid] 0-<br>
> management: Volume date01 does not exist<br>
<br>
</div></div>See the above log entry ^^<br>
<div><div class="h5"><br>
> [2017-06-16 16:08:22.416525] E [MSGID: 106289] [glusterd-syncop.c:1927:gd_<wbr>sync_task_begin] 0-<br>
> management: Failed to build payload for operation 'Volume Set'<br>
> [2017-06-16 16:08:28.452042] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: bitd service is stopped<br>
> [2017-06-16 16:08:28.452204] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: scrub already stopped<br>
> [2017-06-16 16:08:28.452258] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: scrub service is stopped<br>
> [2017-06-16 16:08:28.475740] I [run.c:191:runner_log] (<br>
> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S30samba-set.sh --volname=data01 -o features.trash=on --gd-<br>
> workdir=/var/lib/glusterd<br>
> [2017-06-16 16:08:28.490047] I [run.c:191:runner_log] (<br>
> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S32gluster_enable_shared_<wbr>storage.sh --volname=data01 -o<br>
> features.trash=on --gd-workdir=/var/lib/glusterd<br>
> [2017-06-16 16:08:28.451421] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: quotad already stopped<br>
> [2017-06-16 16:08:28.451839] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: quotad service is stopped<br>
> [2017-06-16 16:08:28.452030] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: bitd already stopped<br>
> [2017-06-16 16:09:30.589316] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: quotad already stopped<br>
> [2017-06-16 16:09:30.589366] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: quotad service is stopped<br>
> [2017-06-16 16:09:30.589562] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: bitd already stopped<br>
> [2017-06-16 16:09:30.589589] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: bitd service is stopped[2017-06-16 16:09:30.589778] I [MSGID: 106132] [glusterd-proc-<br>
> mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped<br>
> [2017-06-16 16:09:30.589800] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-<br>
> management: scrub service is stopped[2017-06-16 16:09:30.614742] I [run.c:191:runner_log] (<br>
> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
</div></div>> /var/lib/glusterd/hooks/1/set/<wbr>post/S30samba-set.sh --volname=data01 -o features.trash-max-<br>
<span class="">> filesize=500MB --gd-workdir=/var/lib/<wbr>glusterd[2017-06-16 16:09:30.630514] I [run.c:191:runner_log]<br>
> (-->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S32gluster_enable_shared_<wbr>storage.sh --volname=data01 -o<br>
> features.trash-max-filesize=<wbr>500MB --gd-workdir=/var/lib/<wbr>glusterd[2017-06-16 16:09:40.293734] I<br>
> [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-management: bitd service is stopped<br>
> [2017-06-16 16:09:40.293903] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: scrub already stopped[2017-06-16 16:09:40.318753] I [run.c:191:runner_log] (<br>
</span><span class="">> -->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
</span>> /var/lib/glusterd/hooks/1/set/<wbr>post/S30samba-set.sh --volname=intermediate01 -o features.trash-max-<br>
<span class="">> filesize=500MB --gd-workdir=/var/lib/<wbr>glusterd[2017-06-16 16:09:40.336530] I [run.c:191:runner_log]<br>
> (-->/usr/lib64/glusterfs/3.10.<wbr>1/xlator/mgmt/glusterd.so(+<wbr>0xdb46a) [0x7f30460bc46a] --<br>
> >/usr/lib64/glusterfs/3.10.1/<wbr>xlator/mgmt/glusterd.so(+<wbr>0xdaf2d) [0x7f30460bbf2d] --<br>
> >/lib64/libglusterfs.so.0(<wbr>runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:<br>
> /var/lib/glusterd/hooks/1/set/<wbr>post/S32gluster_enable_shared_<wbr>storage.sh --volname=intermediate01 -o<br>
> features.trash-max-filesize=<wbr>500MB --gd-workdir=/var/lib/<wbr>glusterd[2017-06-16 16:10:23.007199] W<br>
> [socket.c:593:__socket_rwv] 0-management: readv on<br>
> /var/run/gluster/<wbr>6c1a3e8424c8fe80032e1777c4f549<wbr>fc.socket failed (No data available)[2017-06-16<br>
> 16:09:40.293957] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-management: scrub<br>
> service is stopped[2017-06-16 16:10:23.007696] I [MSGID: 106005] [glusterd-<br>
> handler.c:5691:__glusterd_<wbr>brick_rpc_notify] 0-management: Brick 192.168.186.11:/mnt/AIDATA/<wbr>data<br>
> has disconnected from glusterd.[2017-06-16 16:09:40.293532] I [MSGID: 106132] [glusterd-proc-<br>
> mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped[2017-06-16 16:09:40.293550] I<br>
> [MSGID: 106568] [glusterd-svc-mgmt.c:228:<wbr>glusterd_svc_stop] 0-management: quotad service is<br>
> stopped<br>
> [2017-06-16 16:09:40.293723] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:<wbr>glusterd_proc_stop] 0-<br>
> management: bitd already st<br>
><br>
> I now disabled the trashcan feature. I remounted the volumes on all clients and it started to<br>
> work.<br>
><br>
> Why would the trashcan feature have crashed/disabled my volume?<br>
<br>
</span>Is there a crash related back trace that you could see in brick logs? If so that would be helpful.<br>
<span class=""><br>
> Regards,<br>
><br>
> Ludwig<br>
><br>
> -- <br>
> Ludwig Gamache<br>
> IT Director - Element AI<br>
> 4200 St-Laurent, suite 1200<br>
> <a href="tel:514-704-0564" value="+15147040564">514-704-0564</a><br>
</span>> ______________________________<wbr>_________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><span style="color:rgb(136,136,136)">Ludwig Gamache</span><br><div style="color:rgb(136,136,136)"><div dir="ltr"><div dir="ltr"><div>IT Director - Element AI</div><div>4200 St-Laurent, suite 1200</div><div>514-704-0564</div></div></div></div></div></div></div></div>
</div>