[Bugs] [Bug 1447571] New: RFE: Enhance handleops readdirplus operation to return handles along with dirents

Wed May 3 08:50:25 UTC 2017

https://bugzilla.redhat.com/show_bug.cgi?id=1447571

            Bug ID: 1447571
           Summary: RFE: Enhance handleops readdirplus operation to return
                    handles along with dirents
           Product: GlusterFS
           Version: 3.11
         Component: libgfapi
          Keywords: Triaged
          Severity: high
          Assignee: bugs at gluster.org
          Reporter: skoduri at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org
        Depends On: 1442950

+++ This bug was initially created as a clone of Bug #1442950 +++

Description of problem:

Any application using glfs handles (like NFS-Ganesha) may need to do lookup on
each dirent returned by readdir/readdirp operation to create handles. But since
the lookup in glusterfs is very costly operation (as need to be sent to all
replica bricks), as the directory gets larger or the replica count increases,
the time taken for single readdir operation to complete can take hours of time.

To avoid that we need an extended readdirp API which can return handles along
with dirent stat as part of its reply.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:

Expected results:

Additional info:

--- Additional comment from Worker Ant on 2017-04-18 04:54:17 EDT ---

REVIEW: https://review.gluster.org/15663 (WIP/gfapi/handleops: Introducing
READDIRPLUS_EXT() fop for handleops) posted (#6) for review on master by soumya
k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-21 08:42:06 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#7) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-21 08:45:48 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#8) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-26 03:44:21 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#9) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-27 05:08:30 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#10) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-27 05:43:45 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#11) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-04-28 07:55:10 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_h_xreaddirplus_r() fop for handleops) posted (#12) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-05-01 22:55:09 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_xreaddirplus_r() fop for handleops) posted (#13) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-05-01 23:09:29 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_xreaddirplus_r() fop for handleops) posted (#14) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-05-02 01:49:10 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_xreaddirplus_r() fop for handleops) posted (#15) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-05-02 11:54:05 EDT ---

REVIEW: https://review.gluster.org/15663 (gfapi/handleops: Introducing
glfs_xreaddirplus_r() fop for handleops) posted (#16) for review on master by
soumya k (skoduri at redhat.com)

--- Additional comment from Worker Ant on 2017-05-02 19:33:16 EDT ---

COMMIT: https://review.gluster.org/15663 committed in master by Jeff Darcy
(jeff at pl.atyp.us) 
------
commit 41000cd0b57a81c4ace4a1d3da0fcc352a11f146
Author: Soumya Koduri <skoduri at redhat.com>
Date:   Fri Apr 21 16:30:20 2017 +0530

    gfapi/handleops: Introducing glfs_xreaddirplus_r() fop for handleops

    Its known that readdirplus operation fetches stat as well for each of the
    dirents. But often applications may need extra information, like for eg.,
    NFS-Ganesha which operates on handles needs handles for each of those
    dirents returned. So this would require extra calls to the backend, in this
    case LOOKUP (which is very expensive operation) resulting in very low
    readdir performance.

    To address that introducing this new API using which applications can
    make request for any extra information to be returned as part of
    readdirplus response.

    Currently this new api returns stat and handles as demanded by application.
    The synopsis of the API is noted in glfs.h.

    @todo:
    * Enhance test script using this new API

    Below were the perf results on single brick volume with and without
    these changes -

    Dataset used -
    10*100 directories and each directory containing 100 empty files.

    I used NFS-Ganesha application to test these changes -
    >for i in {1..5}; do systemctl restart nfs-ganesha; sleep 10; mount -t nfs
-o vers=4 localhost:/brick_vol /mnt; cd /mnt; echo "ITERATION$i"; date; find .
> tmp-nfs.log; date; cd /; umount /mnt; sleep 2; done;

    Without these changes -
    ITERATION1
    Mon Mar 20 17:22:26 IST 2017
    Mon Mar 20 17:23:18 IST 2017
    ITERATION2
    Mon Mar 20 17:23:39 IST 2017
    Mon Mar 20 17:24:28 IST 2017
    ITERATION3
    Mon Mar 20 17:24:49 IST 2017
    Mon Mar 20 17:25:36 IST 2017
    ITERATION4
    Mon Mar 20 17:30:57 IST 2017
    Mon Mar 20 17:31:37 IST 2017
    ITERATION5
    Mon Mar 20 17:31:57 IST 2017
    Mon Mar 20 17:32:40 IST 2017
    [root at dhcp35-197 /]#

    On an average ~46.2 sec

    With these changes applied -
    ITERATION1
    Mon Mar 20 17:35:03 IST 2017
    Mon Mar 20 17:35:15 IST 2017
    ITERATION2
    Mon Mar 20 17:35:36 IST 2017
    Mon Mar 20 17:35:46 IST 2017
    ITERATION3
    Mon Mar 20 17:36:06 IST 2017
    Mon Mar 20 17:36:17 IST 2017
    ITERATION4
    Mon Mar 20 17:41:38 IST 2017
    Mon Mar 20 17:41:49 IST 2017
    ITERATION5
    Mon Mar 20 17:42:10 IST 2017
    Mon Mar 20 17:42:20 IST 2017

    On an average ~10.8 sec

    Updates #174
    BUG: 1442950
    Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876
    Signed-off-by: Soumya Koduri <skoduri at redhat.com>
    Reviewed-on: https://review.gluster.org/15663
    Smoke: Gluster Build System <jenkins at build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins at build.gluster.org>
    Reviewed-by: Niels de Vos <ndevos at redhat.com>
    CentOS-regression: Gluster Build System <jenkins at build.gluster.org>

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1442950
[Bug 1442950] RFE: Enhance handleops readdirplus operation to return
handles along with dirents
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.