[Gluster-users] One file incessant self heal
Pranith Kumar Karampuri
pkarampu at redhat.com
Thu Jul 12 04:35:07 UTC 2012
Homer,
Could you give the output of
getfattr -d -m . -e hex /export/data10/.glusterfs/d9/b0/d9b0c350-33ba-4090-ab08-f91f30dd661f
getfattr -d -m . -e hex /export/data11/.glusterfs/d9/b0/d9b0c350-33ba-4090-ab08-f91f30dd661f
and also 'stat' of these files.
Pranith.
----- Original Message -----
From: "Homer Li" <01jay.ly at gmail.com>
To: "gluster-users" <gluster-users at gluster.org>
Sent: Thursday, July 12, 2012 7:54:24 AM
Subject: [Gluster-users] One file incessant self heal
Hello ;
I found many self-heal triggered log in every 10 minutes.
Only one file , it 's gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f.
Heal-failed and split-brain have not display anything.
Does any problem in this file ?
GlusterFS config:
OS: 2.6.32-220.17.1.el6.x86_64 Scientific Linux release 6.2 (Carbon)
# rpm -qa | grep glusterfs
glusterfs-3.3.0-2.el6.x86_64
glusterfs-devel-3.3.0-2.el6.x86_64
glusterfs-fuse-3.3.0-2.el6.x86_64
glusterfs-geo-replication-3.3.0-2.el6.x86_64
glusterfs-rdma-3.3.0-2.el6.x86_64
glusterfs-server-3.3.0-2.el6.x86_64
glusterfs-debuginfo-3.3.0-2.el6.x86_64
gluster> volume info
Volume Name: gvol1
Type: Distributed-Replicate
Volume ID: a7d8ffdf-7296-404b-aeab-824ee853ec59
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: 172.30.1.125:/export/data00
Brick2: 172.30.1.125:/export/data01
Brick3: 172.30.1.125:/export/data10
Brick4: 172.30.1.125:/export/data11
Options Reconfigured:
features.limit-usage: /source:500GB
features.quota: on
performance.cache-refresh-timeout: 30
performance.io-thread-count: 32
nfs.disable: off
cluster.min-free-disk: 5%
performance.cache-size: 128MB
gluster volume heal gvol1 info
Heal operation on volume gvol1 has been successful
Brick 172.30.1.125:/export/data00
Number of entries: 0
Brick 172.30.1.125:/export/data01
Number of entries: 0
Brick 172.30.1.125:/export/data10
Number of entries: 1
/fs126/Graphite-monitor_vdb.qcow2
Brick 172.30.1.125:/export/data11
Number of entries: 1
/fs126/Graphite-monitor_vdb.qcow2
# gluster volume heal gvol1 info heal-failed
Heal operation on volume gvol1 has been successful
Brick 172.30.1.125:/export/data00
Number of entries: 0
Brick 172.30.1.125:/export/data01
Number of entries: 0
Brick 172.30.1.125:/export/data10
Number of entries: 0
Brick 172.30.1.125:/export/data11
Number of entries: 0
gluster volume heal gvol1 info split-brain
Heal operation on volume gvol1 has been successful
Brick 172.30.1.125:/export/data00
Number of entries: 0
Brick 172.30.1.125:/export/data01
Number of entries: 0
Brick 172.30.1.125:/export/data10
Number of entries: 0
Brick 172.30.1.125:/export/data11
Number of entries: 0
Log detail:
[2012-07-12 09:13:11.666417] I
[afr-self-heald.c:282:_remove_stale_index] 0-gvol1-replicate-0:
Removing stale index for e6087bf7-ae55-441b-8f88-a7b17475caea on
gvol1-client-0
[2012-07-12 09:13:11.666998] W
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-gvol1-client-0: remote
operation failed: No such file or directory
[2012-07-12 09:13:11.667132] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:13:11.667141] E
[afr-self-heald.c:287:_remove_stale_index] 0-gvol1-replicate-0:
e6087bf7-ae55-441b-8f88-a7b17475caea: Failed to remove index on
gvol1-client-0 - No such file or directory
[2012-07-12 09:13:11.667589] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:13:11.668581] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:13:11.669322] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:13:11.669775] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:13:11.670408] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:23:11.770994] I
[afr-self-heald.c:282:_remove_stale_index] 0-gvol1-replicate-0:
Removing stale index for e6087bf7-ae55-441b-8f88-a7b17475caea on
gvol1-client-0
[2012-07-12 09:23:11.771358] W
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-gvol1-client-0: remote
operation failed: No such file or directory
[2012-07-12 09:23:11.771416] E
[afr-self-heald.c:287:_remove_stale_index] 0-gvol1-replicate-0:
e6087bf7-ae55-441b-8f88-a7b17475caea: Failed to remove index on
gvol1-client-0 - No such file or directory
[2012-07-12 09:23:11.771898] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:23:11.772059] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:23:11.773074] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:23:11.773686] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:23:11.774094] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:23:11.774652] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:33:11.874474] I
[afr-self-heald.c:282:_remove_stale_index] 0-gvol1-replicate-0:
Removing stale index for e6087bf7-ae55-441b-8f88-a7b17475caea on
gvol1-client-0
[2012-07-12 09:33:11.874919] W
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-gvol1-client-0: remote
operation failed: No such file or directory
[2012-07-12 09:33:11.874978] E
[afr-self-heald.c:287:_remove_stale_index] 0-gvol1-replicate-0:
d9b0c350-33ba-4090-ab08-f91f30dd661f: Failed to remove index on
gvol1-client-0 - No such file or directory
[2012-07-12 09:33:11.875505] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:33:11.875676] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:33:11.876613] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:33:11.877244] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:33:11.877646] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:33:11.878191] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:43:11.971727] I
[afr-self-heald.c:282:_remove_stale_index] 0-gvol1-replicate-0:
Removing stale index for e6087bf7-ae55-441b-8f88-a7b17475caea on
gvol1-client-0
[2012-07-12 09:43:11.972004] W
[client3_1-fops.c:592:client3_1_unlink_cbk] 0-gvol1-client-0: remote
operation failed: No such file or directory
[2012-07-12 09:43:11.972066] E
[afr-self-heald.c:287:_remove_stale_index] 0-gvol1-replicate-0:
e6087bf7-ae55-441b-8f88-a7b17475caea: Failed to remove index on
gvol1-client-0 - No such file or directory
[2012-07-12 09:43:11.972635] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:43:11.972924] I
[afr-common.c:1340:afr_launch_self_heal] 0-gvol1-replicate-1:
background data self-heal triggered. path:
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>, reason: lookup detected
pending operations
[2012-07-12 09:43:11.973885] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:43:11.974451] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:43:11.974965] I
[afr-self-heal-data.c:712:afr_sh_data_fix] 0-gvol1-replicate-1: no
active sinks for performing self-heal on file
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
[2012-07-12 09:43:11.975772] I
[afr-self-heal-common.c:2159:afr_self_heal_completion_cbk]
0-gvol1-replicate-1: background data self-heal completed on
<gfid:d9b0c350-33ba-4090-ab08-f91f30dd661f>
--
Best Regards
Homer Li
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list