[Gluster-users] Gluster log files being flooded with messages.

[Koby, Bradley]

Hello,

I am new to Gluster, having recently been made admin of this after a sudden
departure of a fellow co-worker.   There does not seem to be any visible
degradation of Gluster, and replication seems to be happening as well.
 However, there are two log files that are filling up so fast, that the
previous administrator had added some cron jobs to purge the files every
20min.

The running version is 3.3.0.


Any help or ideas would be greatly appreciated!

Thanks.


millions of these type of entries every 20 min in glustershd.log

[2014-08-12 09:28:23.309811] E [afr-self-heald.c:287:_remove_stale_index]
0-NewRHS-replicate-0: fd1abea4-472d-4db2-8ffd-13e4cb4325fe: Failed to
remove index on NewRHS-client-0 - No such file or directory
[2014-08-12 09:28:23.310745] E [afr-self-heald.c:287:_remove_stale_index]
0-NewRHS-replicate-0: d8e4419f-1ba5-4a42-9c4d-1058e4bcbaf3: Failed to
remove index on NewRHS-client-0 - No such file or directory
[2014-08-12 09:28:23.311651] E [afr-self-heald.c:287:_remove_stale_index]
0-NewRHS-replicate-0: 7de3320a-eff7-43f6-ab88-67ce765df6a4: Failed to
remove index on NewRHS-client-0 - No such file or directory

and

about 4 million of these type of entries every 20min in
mnt-gluster-NewRHS-start.log

[2014-08-12 09:34:54.492495] E [posix.c:223:posix_stat] 0-NewRHS-posix:
lstat on
/mnt/gluster/NewRHS/start/.glusterfs/a0/79/a079b4de-2998-48ea-9dca-835858ae2ff1
failed: No such file or directory
[2014-08-12 09:34:54.492528] I [server3_1-fops.c:1085:server_unlink_cbk]
0-NewRHS-server: 715465019: UNLINK
<gfid:7c0d26c5-07b2-4ff4-8b8c-e2dc2e6eba47>/34c985ca-2273-4c00-ae42-3c2df7ecd40a
(a079b4de-2998-48ea-9dca-835858ae2ff1) ==> -1 (No such file or directory)
[2014-08-12 09:34:54.493213] W
[marker-quota.c:2047:mq_inspect_directory_xattr] 0-NewRHS-marker: cannot
add a new contribution node
[2014-08-12 09:34:54.493322] E [posix.c:223:posix_stat] 0-NewRHS-posix:
lstat on
/mnt/gluster/NewRHS/start/.glusterfs/11/74/117433f7-ea58-486f-9305-fdebc98e0f0a
failed: No such file or directory
[2014-08-12 09:34:54.493355] I [server3_1-fops.c:1085:server_unlink_cbk]
0-NewRHS-server: 715465021: UNLINK
<gfid:7c0d26c5-07b2-4ff4-8b8c-e2dc2e6eba47>/79493dd2-a267-4dcb-b813-a9de10833de7
(117433f7-ea58-486f-9305-fdebc98e0f0a) ==> -1 (No such file or directory)


Some basic info.......

gluster> volume info NewRHS

Volume Name: NewRHS
Type: Replicate
Volume ID: 6a63d285-d14c-4ac8-8405-3db771d25e34
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: mualglup01.mcs.muohio.edu:/mnt/gluster/NewRHS/start
Brick2: mualglup02.mcs.muohio.edu:/mnt/gluster/NewRHS/start
Options Reconfigured:
auth.allow:
10.0.72.135,10.0.93.*,192.168.251.*,127.0.0.1,10.0.72.132,10.0.72.133
nfs.disable: off
features.quota: on
nfs.register-with-portmap: 1
diagnostics.client-log-level: ERROR


gluster> peer status
Number of Peers: 1

Hostname: mualglup02
Uuid: 33ab9b77-b49b-43af-8edd-f7c39b19841b
State: Peer in Cluster (Connected)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140812/6b62a9a9/attachment.html>