[Gluster-users] trash can feature, crashed???

Anoop C S anoopcs at redhat.com
Tue Jun 20 14:15:13 UTC 2017 

On Tue, 2017-06-20 at 08:52 -0400, Ludwig Gamache wrote:
> All,
> 
> I currently have 2 bricks running Gluster 3.10.1. This is a Centos installation. On Friday last
> week, I enabled the trashcan feature on one of my volumes:
> gluster volume set date01 features.trash on

I think you misspelled the volume name. Is it data01 or date01?

> I also limited the max file size to 500MB:
> gluster volume set data01 features.trash-max-filesize 500MB
> 3 hours after that I enabled this, this specific gluster volume went down:
> [2017-06-16 16:08:14.410905] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: quotad already stopped
> [2017-06-16 16:08:14.412027] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: quotad service is stopped
> [2017-06-16 16:08:14.412217] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: bitd already stopped
> [2017-06-16 16:08:14.412243] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: bitd service is stopped
> [2017-06-16 16:08:14.412422] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: scrub already stopped
> [2017-06-16 16:08:14.412451] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: scrub service is stopped
> [2017-06-16 16:08:14.437968] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh --volname=intermediate01 -o features.trash=on
> --gd-workdir=/var/lib/glusterd
> [2017-06-16 16:08:14.453290] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh --volname=intermediate01 -o
> features.trash=on --gd-workdir=/var/lib/glusterd
> [2017-06-16 16:08:22.416477] E [MSGID: 106525] [glusterd-op-sm.c:4188:glusterd_dict_set_volid] 0-
> management: Volume date01 does not exist

See the above log entry ^^

> [2017-06-16 16:08:22.416525] E [MSGID: 106289] [glusterd-syncop.c:1927:gd_sync_task_begin] 0-
> management: Failed to build payload for operation 'Volume Set'
> [2017-06-16 16:08:28.452042] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: bitd service is stopped
> [2017-06-16 16:08:28.452204] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: scrub already stopped
> [2017-06-16 16:08:28.452258] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: scrub service is stopped
> [2017-06-16 16:08:28.475740] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh --volname=data01 -o features.trash=on --gd-
> workdir=/var/lib/glusterd
> [2017-06-16 16:08:28.490047] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh --volname=data01 -o
> features.trash=on --gd-workdir=/var/lib/glusterd
> [2017-06-16 16:08:28.451421] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: quotad already stopped
> [2017-06-16 16:08:28.451839] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: quotad service is stopped
> [2017-06-16 16:08:28.452030] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: bitd already stopped
> [2017-06-16 16:09:30.589316] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: quotad already stopped
> [2017-06-16 16:09:30.589366] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: quotad service is stopped
> [2017-06-16 16:09:30.589562] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: bitd already stopped
> [2017-06-16 16:09:30.589589] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: bitd service is stopped[2017-06-16 16:09:30.589778] I [MSGID: 106132] [glusterd-proc-
> mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
> [2017-06-16 16:09:30.589800] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-
> management: scrub service is stopped[2017-06-16 16:09:30.614742] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh --volname=data01 -o features.trash-max-
> filesize=500MB --gd-workdir=/var/lib/glusterd[2017-06-16 16:09:30.630514] I [run.c:191:runner_log]
> (-->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh --volname=data01 -o
> features.trash-max-filesize=500MB --gd-workdir=/var/lib/glusterd[2017-06-16 16:09:40.293734] I
> [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: bitd service is stopped
> [2017-06-16 16:09:40.293903] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: scrub already stopped[2017-06-16 16:09:40.318753] I [run.c:191:runner_log] (
> -->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh --volname=intermediate01 -o features.trash-max-
> filesize=500MB --gd-workdir=/var/lib/glusterd[2017-06-16 16:09:40.336530] I [run.c:191:runner_log]
> (-->/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdb46a) [0x7f30460bc46a] --
> >/usr/lib64/glusterfs/3.10.1/xlator/mgmt/glusterd.so(+0xdaf2d) [0x7f30460bbf2d] --
> >/lib64/libglusterfs.so.0(runner_log+0x115) [0x7f3051532255] ) 0-management: Ran script:
> /var/lib/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh --volname=intermediate01 -o
> features.trash-max-filesize=500MB --gd-workdir=/var/lib/glusterd[2017-06-16 16:10:23.007199] W
> [socket.c:593:__socket_rwv] 0-management: readv on
> /var/run/gluster/6c1a3e8424c8fe80032e1777c4f549fc.socket failed (No data available)[2017-06-16
> 16:09:40.293957] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: scrub
> service is stopped[2017-06-16 16:10:23.007696] I [MSGID: 106005] [glusterd-
> handler.c:5691:__glusterd_brick_rpc_notify] 0-management: Brick 192.168.186.11:/mnt/AIDATA/data
> has disconnected from glusterd.[2017-06-16 16:09:40.293532] I [MSGID: 106132] [glusterd-proc-
> mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped[2017-06-16 16:09:40.293550] I
> [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: quotad service is
> stopped
> [2017-06-16 16:09:40.293723] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-
> management: bitd already st
> 
> I now disabled the trashcan feature. I remounted the volumes on all clients and it started to
> work.
> 
> Why would the trashcan feature have crashed/disabled my volume?

Is there a crash related back trace that you could see in brick logs? If so that would be helpful.

> Regards,
> 
> Ludwig
> 
> -- 
> Ludwig Gamache
> IT Director - Element AI
> 4200 St-Laurent, suite 1200
> 514-704-0564
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users

More information about the Gluster-users mailing list