[Bugs] [Bug 1250009] New: Dist-geo-rep: Too many "remote operation failed: No such file or directory" warning messages in auxilary mount log on slave while executing "rm -rf"

bugzilla at redhat.com bugzilla at redhat.com
Tue Aug 4 10:47:39 UTC 2015


https://bugzilla.redhat.com/show_bug.cgi?id=1250009

            Bug ID: 1250009
           Summary: Dist-geo-rep: Too many "remote operation failed: No
                    such file or directory" warning messages in auxilary
                    mount log on slave while executing "rm -rf"
           Product: GlusterFS
           Version: mainline
         Component: geo-replication
          Assignee: bugs at gluster.org
          Reporter: khiremat at redhat.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com



Description of problem:

NOTE: BZ 1210562 is not completely fixed, hence opening a new bug

I was running rm -rf * on the master mount point and saw lots of Warning
messages in the auxiliary mount logs in the slave.

The messages were 

[2013-07-31 11:11:59.688460] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:59.688548] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:59.690008] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:59.691900] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:59.691959] I [afr-lk-common.c:1085:afr_lock_blocking]
0-hosa_slave-replicate-1: unable to lock on even one child
[2013-07-31 11:11:59.691993] I
[afr-transaction.c:1063:afr_post_blocking_inodelk_cbk]
0-hosa_slave-replicate-1: Blocking inodelks failed.
[2013-07-31 11:11:59.693990] W [client-rpc-fops.c:2523:client3_3_opendir_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory.
Path: <gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)
[2013-07-31 11:11:59.694328] W [client-rpc-fops.c:2523:client3_3_opendir_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory.
Path: <gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)


Version-Release number of selected component (if applicable):
mailine

How reproducible:
Hit once. Not sure about reproducibility.

Steps to Reproduce:
1. Create and start a geo-rep session between master and slave.
2. Untar the linux kernel twice in 2 different locations on the master
mountpoint.
3. Now run from master mount point run, rm -rf *

Actual results:
 hosa_slave-client-2
[2013-07-31 11:11:58.042628] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:58.042946] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:58.087397] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-0: remote operation failed: No such file or directory
[2013-07-31 11:11:58.088347] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-1: remote operation failed: No such file or directory
[2013-07-31 11:11:58.088422] W [fuse-bridge.c:1624:fuse_unlink_cbk]
0-glusterfs-fuse: 1528415: RMDIR()
<gfid:69f85589-6525-4cdb-a308-9dae4d6c6935>/vhost_test => -1 (No such file or
directory)
[2013-07-31 11:11:58.291267] E
[afr-self-heal-entry.c:2325:afr_sh_post_nonblocking_entry_cbk]
0-hosa_slave-replicate-1: Non Blocking entrylks failed for
<gfid:69f85589-6525-4cdb-a308-9dae4d6c6935>/virtio-trace.
[2013-07-31 11:11:58.291409] E
[afr-self-heal-common.c:2722:afr_log_self_heal_completion_status]
0-hosa_slave-replicate-1: foreground gfid or missing entry self heal  is not
attempted, medatadata self heal  is not attempted, data self heal  is not
attempted, entry self heal  failed on 
<gfid:69f85589-6525-4cdb-a308-9dae4d6c6935>/virtio-trace
[2013-07-31 11:11:58.296013] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:58.296128] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:58.324498] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-0: remote operation failed: No such file or directory
[2013-07-31 11:11:58.325157] W [client-rpc-fops.c:695:client3_3_rmdir_cbk]
0-hosa_slave-client-1: remote operation failed: No such file or directory
[2013-07-31 11:11:58.325237] W [fuse-bridge.c:1624:fuse_unlink_cbk]
0-glusterfs-fuse: 1581366: RMDIR()
<gfid:69f85589-6525-4cdb-a308-9dae4d6c6935>/virtio-trace => -1 (No such file or
directory)
[2013-07-31 11:11:59.613599] I [dht-layout.c:636:dht_layout_normalize]
0-hosa_slave-dht: found anomalies in
<gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools. holes=1 overlaps=1 missing=1
down=0 misc=0
[2013-07-31 11:11:59.674199] W [client-rpc-fops.c:1983:client3_3_setattr_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:59.674292] W [client-rpc-fops.c:1983:client3_3_setattr_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:59.675293] W [client-rpc-fops.c:1744:client3_3_xattrop_cbk]
0-hosa_slave-client-2: remote operation failed: Success. Path:
<gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)
[2013-07-31 11:11:59.675997] W [client-rpc-fops.c:1744:client3_3_xattrop_cbk]
0-hosa_slave-client-3: remote operation failed: Success. Path:
<gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)
[2013-07-31 11:11:59.688460] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:59.688548] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:59.690008] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory
[2013-07-31 11:11:59.691900] W [client-rpc-fops.c:1529:client3_3_inodelk_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory
[2013-07-31 11:11:59.691959] I [afr-lk-common.c:1085:afr_lock_blocking]
0-hosa_slave-replicate-1: unable to lock on even one child
[2013-07-31 11:11:59.691993] I
[afr-transaction.c:1063:afr_post_blocking_inodelk_cbk]
0-hosa_slave-replicate-1: Blocking inodelks failed.
[2013-07-31 11:11:59.693990] W [client-rpc-fops.c:2523:client3_3_opendir_cbk]
0-hosa_slave-client-2: remote operation failed: No such file or directory.
Path: <gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)
[2013-07-31 11:11:59.694328] W [client-rpc-fops.c:2523:client3_3_opendir_cbk]
0-hosa_slave-client-3: remote operation failed: No such file or directory.
Path: <gfid:ec6152e9-e1e9-498c-9e05-46edb9d0bc07>/tools
(dce9238f-e5c7-49c1-9741-73388f92f050)


Expected results:
Not so much of warning and error logs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.


More information about the Bugs mailing list