[Bugs] [Bug 1652548] Error reading some files - Stale file handle - distribute 2 - replica 3 volume with sharding

Mon Nov 26 12:43:40 UTC 2018

https://bugzilla.redhat.com/show_bug.cgi?id=1652548


--- Comment #5 from marcoc at prismatelecomtesting.com ---
Hi,
I tried running dd from the VM:

dd if=/dev/vda of=/dev/null BS=1M status=progress
13519000000 (14GB) copied, 14.056719s, 962 MB/s

VM paused

host
/var/log/glusterfs/rhev-data-center-mnt-glusterSD-s20gfs.ovirt.prisma\:_VOL__VMDATA.log


[2018-11-26 12:26:04.176267] I [MSGID: 109069]
[dht-common.c:1474:dht_lookup_unlink_stale_linkto_cbk] 0-VOL_VMDATA-dht:
Returned with op_ret 0 and op_errno 0 for
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27
[2018-11-26 12:26:04.177778] I [MSGID: 109069]
[dht-common.c:1474:dht_lookup_unlink_stale_linkto_cbk] 0-VOL_VMDATA-dht:
Returned with op_ret -1 and op_errno 2 for
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27
[2018-11-26 12:26:04.207594] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.208249] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.208668] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-1, gfid local = 7b295e4a-7a48-48ab-94ad-14fecb3c96db, gfid
node = 421b2564-332f-4761-a85c-b1b86f9f23c7
[2018-11-26 12:26:04.208700] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.208722] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819774: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.209947] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.210225] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-1, gfid local = 7b295e4a-7a48-48ab-94ad-14fecb3c96db, gfid
node = 421b2564-332f-4761-a85c-b1b86f9f23c7
[2018-11-26 12:26:04.210265] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819772: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "W [MSGID: 109009] [dht-common.c:2210:dht_lookup_linkfile_cbk]
0-VOL_VMDATA-dht: /.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid
different on data file on VOL_VMDATA-replicate-0, gfid local =
00000000-0000-0000-0000-000000000000, gfid node =
7b295e4a-7a48-48ab-94ad-14fecb3c96db " repeated 2 times between [2018-11-26
12:26:04.207594] and [2018-11-26 12:26:04.213769]
[2018-11-26 12:26:04.214709] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.214767] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819773: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "E [MSGID: 133010] [shard.c:1724:shard_common_lookup_shards_cbk]
0-VOL_VMDATA-shard: Lookup on shard 27 failed. Base file gfid =
62e1c5d8-8533-4e6b-826e-030680043011 [Stale file handle]" repeated 2 times
between [2018-11-26 12:26:04.208700] and [2018-11-26 12:26:04.214761]
[2018-11-26 12:26:04.214801] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.215431] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.215436] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.215483] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819771: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.215482] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.215957] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.216003] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819751: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.216307] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819750: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "E [MSGID: 133010] [shard.c:1724:shard_common_lookup_shards_cbk]
0-VOL_VMDATA-shard: Lookup on shard 27 failed. Base file gfid =
62e1c5d8-8533-4e6b-826e-030680043011 [Stale file handle]" repeated 2 times
between [2018-11-26 12:26:04.215436] and [2018-11-26 12:26:04.216301]
[2018-11-26 12:26:04.218010] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.218610] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.218616] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.218651] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819753: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.219400] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.219923] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.219999] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819756: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.219990] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.220589] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.221134] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.221140] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.221174] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819742: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.221921] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.222407] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.222458] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819754: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.222451] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.223214] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.223747] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.223753] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.223789] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819755: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.224431] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.224905] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.224954] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819757: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.224948] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.225654] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
The message "W [MSGID: 109009] [dht-common.c:1949:dht_lookup_everywhere_cbk]
0-VOL_VMDATA-dht: /.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs
on subvolume VOL_VMDATA-replicate-1, gfid local =
7b295e4a-7a48-48ab-94ad-14fecb3c96db, gfid node =
421b2564-332f-4761-a85c-b1b86f9f23c7" repeated 2 times between [2018-11-26
12:26:04.210225] and [2018-11-26 12:26:04.226192]
[2018-11-26 12:26:04.226202] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.226283] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819758: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.227461] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819759: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.228805] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819760: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.229964] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819761: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.231246] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819762: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.232471] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819763: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.233704] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819766: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.234984] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819767: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
[2018-11-26 12:26:04.236171] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819768: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "W [MSGID: 109009] [dht-common.c:1949:dht_lookup_everywhere_cbk]
0-VOL_VMDATA-dht: /.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs
on subvolume VOL_VMDATA-replicate-0, gfid local =
421b2564-332f-4761-a85c-b1b86f9f23c7, gfid node =
7b295e4a-7a48-48ab-94ad-14fecb3c96db" repeated 10 times between [2018-11-26
12:26:04.224905] and [2018-11-26 12:26:04.237655]
[2018-11-26 12:26:04.237795] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-1, gfid local = 7b295e4a-7a48-48ab-94ad-14fecb3c96db, gfid
node = 421b2564-332f-4761-a85c-b1b86f9f23c7
[2018-11-26 12:26:04.237846] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819765: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "W [MSGID: 109009] [dht-common.c:2210:dht_lookup_linkfile_cbk]
0-VOL_VMDATA-dht: /.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid
different on data file on VOL_VMDATA-replicate-0, gfid local =
00000000-0000-0000-0000-000000000000, gfid node =
7b295e4a-7a48-48ab-94ad-14fecb3c96db " repeated 10 times between [2018-11-26
12:26:04.225654] and [2018-11-26 12:26:04.239028]
[2018-11-26 12:26:04.239518] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.239569] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819769: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "E [MSGID: 133010] [shard.c:1724:shard_common_lookup_shards_cbk]
0-VOL_VMDATA-shard: Lookup on shard 27 failed. Base file gfid =
62e1c5d8-8533-4e6b-826e-030680043011 [Stale file handle]" repeated 10 times
between [2018-11-26 12:26:04.226202] and [2018-11-26 12:26:04.239562]
[2018-11-26 12:26:04.241144] W [MSGID: 109009]
[dht-common.c:2210:dht_lookup_linkfile_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid different on data file on
VOL_VMDATA-replicate-0, gfid local = 00000000-0000-0000-0000-000000000000, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db 
[2018-11-26 12:26:04.241697] W [MSGID: 109009]
[dht-common.c:1949:dht_lookup_everywhere_cbk] 0-VOL_VMDATA-dht:
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27: gfid differs on subvolume
VOL_VMDATA-replicate-0, gfid local = 421b2564-332f-4761-a85c-b1b86f9f23c7, gfid
node = 7b295e4a-7a48-48ab-94ad-14fecb3c96db
[2018-11-26 12:26:04.241704] E [MSGID: 133010]
[shard.c:1724:shard_common_lookup_shards_cbk] 0-VOL_VMDATA-shard: Lookup on
shard 27 failed. Base file gfid = 62e1c5d8-8533-4e6b-826e-030680043011 [Stale
file handle]
[2018-11-26 12:26:04.241743] W [fuse-bridge.c:2318:fuse_readv_cbk]
0-glusterfs-fuse: 237819764: READ => -1
gfid=62e1c5d8-8533-4e6b-826e-030680043011 fd=0x7f5d8c14e680 (Stale file handle)
The message "I [MSGID: 109069]
[dht-common.c:1474:dht_lookup_unlink_stale_linkto_cbk] 0-VOL_VMDATA-dht:
Returned with op_ret -1 and op_errno 2 for
/.shard/62e1c5d8-8533-4e6b-826e-030680043011.27" repeated 2 times between
[2018-11-26 12:26:04.177778] and [2018-11-26 12:26:04.180779]

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.