[Bugs] [Bug 1438423] [Ganesha + EC] : Input/ Output Error while creating LOTS of smallfiles

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1438423



--- Comment #3 from Worker Ant <bugzilla-bot at gluster.org> ---
COMMIT: https://review.gluster.org/16987 committed in release-3.10 by
Shyamsundar Ranganathan (srangana at redhat.com) 
------
commit 560c92862f72128ccd962192b9c4d728386c7bb1
Author: Pranith Kumar K <pkarampu at redhat.com>
Date:   Thu Mar 30 14:58:38 2017 +0530

    cluster/dht: Modify local->loc.gfid in thread safe manner

        Backport of https://review.gluster.org/16986

    Problem:
    local->loc.gfid in dht_lookup_directory() will be null-gfid for a fresh
lookup.
    dht_lookup_dir_cbk() updates local->loc.gfid while in other thread
dht_lookup_directory()
    is still winding lookup calls to subvolumes so there is a chance of partial
gfid being
    seen by EC.

    We saw in 12x(4+2) volume, ec is receiving an loc where the gfid has last
10 bytes matching
    with the gfid of the directory and the first 4 bytes are all-zeros. This is
leading to EC
    erroring out the lookup with EINVAL which leads to NFS failing lookup with
EIO.

    snip from gdb:
    $37 = (dht_local_t *) 0x7fde5de5b3cc
    (gdb) p /x $37->loc.gfid
    $39 = {0x3b, 0x82, 0x10, 0x5e, 0x40, 0x65, 0x43, 0x14, 0xa0, 0xc6, 0x8,
0xf5,
    0x6c, 0x2c, 0xb8, 0x56}
    (gdb) fr 7
    state=<optimized out>) at ec-generic.c:837
    837                    ec_lookup_rebuild(fop->xl->private, fop, cbk);
    (gdb) p /x fop->loc[0].gfid
    $40 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x43, 0x14, 0xa0, 0xc6, 0x8, 0xf5,
0x6c,
    0x2c, 0xb8, 0x56}

    snip from log:
    [2017-01-29 03:22:30.132328] W [MSGID: 122019]
    [ec-helpers.c:354:ec_loc_gfid_check] 0-butcher-disperse-4: Mismatching
GFID's
    in loc [2017-01-29 03:22:30.132709] W [MSGID: 112199]
    [nfs3-helpers.c:3515:nfs3_log_newfh_res] 0-nfs-nfsv3:
    /linux-4.9.5/Documentation => (XID: b27b9474, MKDIR: NFS: 5(I/O error),
POSIX:
    5(Input/output error)), FH: exportid 00000000-0000-0000-0000-000000000000,
gfid
    00000000-0000-0000-0000-000000000000, mountid
    00000000-0000-0000-0000-000000000000 [Invalid argument]

    Fix:
    update local->loc.gfid in last-call to make sure there are no races.

     >BUG: 1438411
     >Change-Id: Ifcb7e911568c1f1f83123da6ff0cf742b91800a0
     >Signed-off-by: Pranith Kumar K <pkarampu at redhat.com>

    BUG: 1438423
    Change-Id: I804822a1d50215301881ac18318282c1a6951cfb
    Signed-off-by: Pranith Kumar K <pkarampu at redhat.com>
    Reviewed-on: https://review.gluster.org/16987
    Smoke: Gluster Build System <jenkins at build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins at build.gluster.org>
    Reviewed-by: Raghavendra G <rgowdapp at redhat.com>

-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=WZT5LFgohO&a=cc_unsubscribe