[Bugs] [Bug 1254437] New: tiering: rename fails with "Device or resource busy" error message

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1254437

            Bug ID: 1254437
           Summary: tiering: rename fails with "Device or resource busy"
                    error message
           Product: GlusterFS
           Version: 3.7.3
         Component: tiering
          Keywords: Triaged
          Severity: high
          Priority: high
          Assignee: bugs at gluster.org
          Reporter: rkavunga at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org
        Depends On: 1248306



+++ This bug was initially created as a clone of Bug #1248306 +++

Description of problem:

On a tiered volume after a file promoted/demoted, then unlink on that file will
fail with EBUSY.

Version-Release number of selected component (if applicable):

mainline.

How reproducible:

100%

Steps to Reproduce:
1.create a distributed-replicated volume.
2.attach a distributed-replicated volume.
3.mount the volume and create some files on the mount point
4.let the files demote.
5.Rename any any of the demoted files.

Actual results:

rename fails

Expected results:

rename should succeed.

Additional info:

--- Additional comment from Mohammed Rafi KC on 2015-07-30 01:04:04 EDT ---

RCA:

A newly created files will always hashed to hot-tier for a tiered volume. When
the files was in hot tier, then hashed and cached subvolume will be hot-tier.
Once the file is migrated from hot-tier to cold-tier, then subsequent lookup
will send a revalidate lookup to hot-tier and it will find out that the file
was
actually moved and there is only a link file in the cached subvolume. So dht
will return an ESTALE to fuse. Upon receiving ESTALE for a lookup, fuse
will create a new inode and sent a fresh lookup. This lookup will be
successful, and it will locate the file properly. Then fuse try to link
the inode, but the older inode was already there in inmemory inode cache
with same gfid and that is also shared with fuse kernal. So inode_link
will return the older ionode itself. So the subsequent rename fop will
come to gluster with the older inode. From dht_rename, we will take a
lock on the inode and after successful inodelk on inode dht will send
lookup before creating a link. this lookup will again find out that the
file is a link file, and then dht will think that file is
migrating/migrated during the course, and will send a EBUSY error.

--- Additional comment from Anand Avati on 2015-08-05 09:53:53 EDT ---

REVIEW: http://review.gluster.org/11768 (dht/tier :unlink fails with EBUSY)
posted (#2) for review on master by mohammed rafi  kc (rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-08-06 02:48:36 EDT ---

REVIEW: http://review.gluster.org/11768 (dht/tier :rename fails with EBUSY)
posted (#3) for review on master by mohammed rafi  kc (rkavunga at redhat.com)

--- Additional comment from Anand Avati on 2015-08-13 10:32:28 EDT ---

COMMIT: http://review.gluster.org/11768 committed in master by Dan Lambright
(dlambrig at redhat.com) 
------
commit 0ad26041fbf65ab36856a0ad178c32e51bf87319
Author: Mohammed Rafi KC <rkavunga at redhat.com>
Date:   Wed Aug 5 19:20:04 2015 +0530

    dht/tier :rename fails with EBUSY

    When the files was in hot tier and the look up was done already, then
    hashed and cached subvolume will be hot-tier. Once the file is moved
    from hot-tier to cold-tier, then subsequent lookup will send a
    revalidate lookup to hot-tier and it will find out that the file was
    actually moved and there is only link in the cached subvolume. So dht
    will return an ESTALE to fuse. Upon receiving ESTALE for a lookup, fuse
    will create a new inode and sent a fresh lookup. This lookup will be
    successful, and it will locate the file properly. Then fuse try to link
    the inode, but the older inode was already there in inmemory inode cache
    with same gfid and that is also shared with fuse kernal. So inode_link
    will return the older ionode itself. So the subsequent rename fop will
    come to gluster with the older inode. From dht_rename, we will take a
    lock on the inode and after successful inodelk on inode dht will send
    lookup before creating a link. this lookup will again find out that the
    file is a link file, and then dht will think that file is
    migrating/migrated in the mean time, and will send EBUSY.

    Change-Id: Ib3a01e5b1d7f64514b04bb6234026d049f082679
    BUG: 1248306
    Signed-off-by: Mohammed Rafi KC <rkavunga at redhat.com>
    Reviewed-on: http://review.gluster.org/11768
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Raghavendra G <rgowdapp at redhat.com>
    Reviewed-by: Dan Lambright <dlambrig at redhat.com>
    Tested-by: NetBSD Build System <jenkins at build.gluster.org>
    Tested-by: Dan Lambright <dlambrig at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1248306
[Bug 1248306] tiering: rename fails with "Device or resource busy" error
message
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.