[Bugs] [Bug 1251106] New: sharding - Renames on non-sharded files failing with ENOMEM

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1251106

            Bug ID: 1251106
           Summary: sharding - Renames on non-sharded files failing with
                    ENOMEM
           Product: GlusterFS
           Version: 3.7.3
         Component: sharding
          Keywords: Triaged
          Assignee: bugs at gluster.org
          Reporter: kdhananj at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org
        Depends On: 1250855



+++ This bug was initially created as a clone of Bug #1250855 +++

Description of problem:

Same as the subject.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create a plain distribute volume, start it, mount it using FUSE and create a
few directories and a few files under them.
2. Enable sharding on the volume.
3. cd into one of the directories.
4. Perform ls (=> readdirp)
5. In a loop, rename all of the files under the directory.

Actual results:
Renames on some of them fails with ENOMEM.
Also seen are logs of the following kind in <mount>.log per failed rename:

[2015-08-06 07:15:08.652746] E [shard.c:2191:shard_rename] 2-dis-shard: Failed
to get block size from inode ctx of c3bce9d1-37f4-4f95-8257-23a5d223149d
[2015-08-06 07:15:08.652780] W [fuse-bridge.c:1725:fuse_rename_cbk]
0-glusterfs-fuse: 59652: /bin/alsa-info -> /bin/alsa-info-sharded => -1 (Cannot
allocate memory)

Expected results:


Additional info:

I tried the same experiment with one change: stat-prefetch set to off at (1)
and volume mounted with entry-timeout and attribute-timeout equal to 0 and was
not able to hit this issue.

Turns out this is due to shard translator expecting inode ctx to be populated
for each linked inode in most fop paths, failing which it would fail the
operation with ENOMEM. And the only place where shard translator initialises
the inode ctx is LOOKUP. Just after the graph switch in step 2, the `ls` in
step 4 could cause the entries to be fetched and linked in fuse-bridge via
readdirp(). This could prevent LOOKUPs on the entries from being called or
reaching shard translator before RENAMEs are wound. And a failure to GET the
inode ctx in memory will cause shard translator to fail the fop with ENOMEM.

The solution would be to initialise the inode ctx in shard_readdir_cbk() if it
doesn't exist already.

--- Additional comment from Anand Avati on 2015-08-06 09:09:51 EDT ---

REVIEW: http://review.gluster.org/11854 (features/shard: Fill inode ctx in
readdir(p) callback too) posted (#1) for review on master by Krutika Dhananjay
(kdhananj at redhat.com)


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1250855
[Bug 1250855] sharding - Renames on non-sharded files failing with ENOMEM
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.