[Bugs] [Bug 1214248] New: Persist file size and block count of sharded files in the form of xattrs

Wed Apr 22 09:57:02 UTC 2015

https://bugzilla.redhat.com/show_bug.cgi?id=1214248

            Bug ID: 1214248
           Summary: Persist file size and block count of sharded files in
                    the form of xattrs
           Product: GlusterFS
           Version: 3.7.0
         Component: sharding
          Keywords: Triaged
          Assignee: bugs at gluster.org
          Reporter: kdhananj at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org
        Depends On: 1207603
            Blocks: 1199352 (glusterfs-3.7.0)

+++ This bug was initially created as a clone of Bug #1207603 +++

Description of problem:
Metadata read fops like stat, fstat, lookup, readdirp etc return the file size
and block count in their response. This means that for shard'ed files, it is
necessary to look up each of the shards associated with the file and aggregate
their block count and block_size before returning it to the translators above.
But that would involve lot of network fops, increasing the latency of these
fops. To get around this problem, the size and block count can be kept up to
data in the form of an xattr on the first shard with each inode write operation
(like writev, truncate, ftruncate, zerofill, discard, fallocate etc).

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:

Expected results:

Additional info:

--- Additional comment from Anand Avati on 2015-03-31 06:11:22 EDT ---

REVIEW: http://review.gluster.org/10066 (features/shard: Introduce file size
xattr) posted (#1) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-01 07:50:38 EDT ---

REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size
xattr) posted (#1) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-01 07:50:41 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#1) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-07 02:40:51 EDT ---

REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size
xattr) posted (#2) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-07 06:35:52 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#2) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-07 06:35:55 EDT ---

REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size
xattr) posted (#3) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-07 11:44:13 EDT ---

REVIEW: http://review.gluster.org/10097 (features/shard: Introduce file size
xattr) posted (#4) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-07 21:08:00 EDT ---

REVIEW: http://review.gluster.org/10158 (storage/posix: Introduce xattr_fill
capability in posix_stat) posted (#1) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-08 02:11:26 EDT ---

COMMIT: http://review.gluster.org/10158 committed in master by Pranith Kumar
Karampuri (pkarampu at redhat.com) 
------
commit 6a465e6e7e46940e00a387089cc995464975b53d
Author: Krutika Dhananjay <kdhananj at redhat.com>
Date:   Tue Apr 7 21:45:49 2015 +0530

    storage/posix: Introduce xattr_fill capability in posix_stat

    Change-Id: I2b6503ad9333f445ebdcd9fa660da20b861b985f
    BUG: 1207603
    Signed-off-by: Krutika Dhananjay <kdhananj at redhat.com>
    Reviewed-on: http://review.gluster.org/10158
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu at redhat.com>
    Tested-by: Pranith Kumar Karampuri <pkarampu at redhat.com>

--- Additional comment from Anand Avati on 2015-04-08 11:24:42 EDT ---

COMMIT: http://review.gluster.org/10097 committed in master by Vijay Bellur
(vbellur at redhat.com) 
------
commit 4be65bb376e2fffd7175f579724aae4c5718d57c
Author: Krutika Dhananjay <kdhananj at redhat.com>
Date:   Wed Apr 1 16:00:05 2015 +0530

    features/shard: Introduce file size xattr

    With each inode write FOP, the size and block count of the file will be
    updated within the xattr. There are two 64 byte fields that are
    intentionally left blank for now for future use when consistency
    guarantee is introduced later in sharding.

    Change-Id: I40a2e700150c1f199a6bf87909f063c84ab7bb43
    BUG: 1207603
    Signed-off-by: Krutika Dhananjay <kdhananj at redhat.com>
    Reviewed-on: http://review.gluster.org/10097
    Reviewed-by: Pranith Kumar Karampuri <pkarampu at redhat.com>
    Tested-by: Pranith Kumar Karampuri <pkarampu at redhat.com>
    Tested-by: Gluster Build System <jenkins at build.gluster.com>

--- Additional comment from Anand Avati on 2015-04-09 10:38:09 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#3) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-10 06:25:18 EDT ---

REVIEW: http://review.gluster.org/10180 (storage/posix: Introduce xattr-fill on
fds) posted (#2) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-13 02:35:28 EDT ---

COMMIT: http://review.gluster.org/10180 committed in master by Pranith Kumar
Karampuri (pkarampu at redhat.com) 
------
commit f5220016c6525a6e166b83bcc24a036c5db0498b
Author: Krutika Dhananjay <kdhananj at redhat.com>
Date:   Wed Apr 8 14:48:32 2015 +0530

    storage/posix: Introduce xattr-fill on fds

    ... with some of the code borrowed from http://review.gluster.org/#/c/3904/

    Change-Id: I4901ef14d6f843d8d69f102d43d21b60ba298092
    BUG: 1207603
    Signed-off-by: Krutika Dhananjay <kdhananj at redhat.com>
    Reviewed-on: http://review.gluster.org/10180
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu at redhat.com>

--- Additional comment from Anand Avati on 2015-04-13 05:39:22 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#4) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-15 04:27:22 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#5) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-15 05:49:26 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#6) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-17 05:10:32 EDT ---

REVIEW: http://review.gluster.org/10098 (features/shard: Consume size and block
count in metadata read ops) posted (#7) for review on master by Krutika
Dhananjay (kdhananj at redhat.com)

--- Additional comment from Anand Avati on 2015-04-22 01:38:25 EDT ---

COMMIT: http://review.gluster.org/10098 committed in master by Vijay Bellur
(vbellur at redhat.com) 
------
commit f1bdc3f186576107fda4f3665c809c791b4cbe95
Author: Krutika Dhananjay <kdhananj at redhat.com>
Date:   Fri Mar 27 16:03:20 2015 +0530

    features/shard: Consume size and block count in metadata read ops

    Metadata read fops like lookup, stat etc will now fetch the xattr that
    holds the size and block count information, extract the size and block
    count fields and set them in respective stbuf before unwinding the
    resultant iatt to the parent xlator.

    Change-Id: I881be8955092fa6b75f8b0e4f3deb01344cb638e
    BUG: 1207603
    Signed-off-by: Krutika Dhananjay <kdhananj at redhat.com>
    Reviewed-on: http://review.gluster.org/10098
    Reviewed-by: Pranith Kumar Karampuri <pkarampu at redhat.com>
    Tested-by: NetBSD Build System
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Vijay Bellur <vbellur at redhat.com>

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1199352
[Bug 1199352] GlusterFS 3.7.0 tracker
https://bugzilla.redhat.com/show_bug.cgi?id=1207603
[Bug 1207603] Persist file size and block count of sharded files in the
form of xattrs
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.