[Bugs] [Bug 1466037] Fuse mount crashed with continuous dd on a file and reading the file in parallel

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1466037



--- Comment #4 from Worker Ant <bugzilla-bot at gluster.org> ---
COMMIT: https://review.gluster.org/17644 committed in master by Pranith Kumar
Karampuri (pkarampu at redhat.com) 
------
commit 97defef2375b911c7b6a3924c242ba8ef4593686
Author: Pranith Kumar K <pkarampu at redhat.com>
Date:   Wed Jun 28 09:10:53 2017 +0530

    features/shard: Remove ctx from LRU in shard_forget

    Problem:
    There is a race when the following two commands are executed on the mount
in
    parallel from two different terminals on a sharded volume,
    which leads to use-after-free.

    Terminal-1:
    while true; do dd if=/dev/zero of=file1 bs=1M count=4; done

    Terminal-2:
    while true; do cat file1 > /dev/null; done

    In the normal case this is the life-cycle of a shard-inode
    1) Shard is added to LRU when it is first looked-up
    2) For every operation on the shard it is moved up in LRU
    3) When "unlink of the shard"/"LRU limit is hit" happens it is removed from
LRU

    But we are seeing a race where the inode stays in Shard LRU even after it
is
    forgotten which leads to Use-after-free and then some memory-corruptions.

    These are the steps:
    1) Shard is added to LRU when it is first looked-up
    2) For every operation on the shard it is moved up in LRU

    Reader-handler                                    Truncate-handler
    1) Reader handler needs shard-x to be read.     1) Truncate has just
deleted shard-x
    2) In shard_common_resolve_shards(), it does
       inode_resolve() and that leads to
       a hit in LRU, so it is going to call
    __shard_update_shards_inode_list() to move the
       inode to top of LRU
                            2) shard-x gets unlinked from the itable
                               and inode_forget(inode, 0) is called
                               to make sure the inode can be purged
                               upon last unref
    3) when __shard_update_shards_inode_list() is
       called it finds that the inode is not in LRU
       so it adds it back to the LRU-list

    Both these operations complete and call inode_unref(shard-x) which leads to
the inode
    getting freed and forgotten, even when it is in Shard LRU list. When more
inodes are
    added to LRU, use-after-free will happen and it leads to undefined
behaviors.

    Fix:
    I see that the inode can be removed from LRU even by the protocol layers
like gfapi/gNFS
    when LRU limit is reached. So it is better to add a check in shard_forget()
to remove itself
    from LRU list if it exists.

    BUG: 1466037
    Change-Id: Ia79c0c5c9d5febc56c41ddb12b5daf03e5281638
    Signed-off-by: Pranith Kumar K <pkarampu at redhat.com>
    Reviewed-on: https://review.gluster.org/17644
    Smoke: Gluster Build System <jenkins at build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
    Reviewed-by: Krutika Dhananjay <kdhananj at redhat.com>

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.