[Bugs] [Bug 1379976] New: [NFS Ganesha + tiering]: IO hangs for a while during attach tier

bugzilla at redhat.com bugzilla at redhat.com
Wed Sep 28 09:55:33 UTC 2016


https://bugzilla.redhat.com/show_bug.cgi?id=1379976

            Bug ID: 1379976
           Summary: [NFS Ganesha + tiering]: IO hangs for a while during
                    attach tier
           Product: Red Hat Gluster Storage
           Version: 3.2
         Component: nfs-ganesha
          Keywords: Triaged
          Severity: high
          Assignee: kkeithle at redhat.com
          Reporter: sraj at redhat.com
        QA Contact: sraj at redhat.com
                CC: bugs at gluster.org, jthottan at redhat.com,
                    kkeithle at redhat.com, kramdoss at redhat.com,
                    ndevos at redhat.com, rhs-bugs at redhat.com,
                    skoduri at redhat.com, sraj at redhat.com,
                    storage-qa-internal at redhat.com
        Depends On: 1374229



+++ This bug was initially created as a clone of Bug #1374229 +++

Description of problem:
IO hangs when a hot tier is attached to a disperse volume. Although IO resumes
after a while, the duration of IO hang increases in proportion to the
configuration (2x2 vs 4x2) of hot tier and the IO load.

IO hangs for 2 minutes (avg) with 2x2 hot tier and kernel untar IO load. With
4x2 hot tier, IO hangs for more than 5 minutes. 

Volume Name: krk-vol
Type: Tier
Volume ID: 1bf70c56-56e1-4b9e-aeeb-e94a1fc42a28
Status: Started
Number of Bricks: 16
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick1: 10.70.37.142:/bricks/brick4/ht1
Brick2: 10.70.37.153:/bricks/brick4/ht1
Brick3: 10.70.37.194:/bricks/brick4/ht1
Brick4: 10.70.37.182:/bricks/brick4/ht1
Cold Tier:
Cold Tier Type : Distributed-Disperse
Number of Bricks: 2 x (4 + 2) = 12
Brick5: 10.70.37.182:/bricks/brick0/v1
Brick6: 10.70.37.194:/bricks/brick0/v1
Brick7: 10.70.37.153:/bricks/brick0/v1
Brick8: 10.70.37.142:/bricks/brick0/v1
Brick9: 10.70.37.114:/bricks/brick0/v1
Brick10: 10.70.37.86:/bricks/brick0/v1
Brick11: 10.70.37.182:/bricks/brick1/v1
Brick12: 10.70.37.194:/bricks/brick1/v1
Brick13: 10.70.37.153:/bricks/brick1/v1
Brick14: 10.70.37.142:/bricks/brick1/v1
Brick15: 10.70.37.114:/bricks/brick1/v1
Brick16: 10.70.37.86:/bricks/brick1/v1
Options Reconfigured:
cluster.tier-mode: cache
features.ctr-enabled: on
features.quota-deem-statfs: on
features.inode-quota: on
features.quota: on
ganesha.enable: on
features.cache-invalidation: on
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
nfs-ganesha: enable
cluster.enable-shared-storage: enable


Version-Release number of selected component (if applicable):
[root at dhcp37-114 ~]# rpm -qa | grep 'gluster'
glusterfs-cli-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
glusterfs-server-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
python-gluster-3.8.3-0.1.git2ea32d9.el7.centos.noarch
glusterfs-client-xlators-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
glusterfs-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
glusterfs-fuse-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
nfs-ganesha-gluster-next.20160813.2f47e8a-1.el7.centos.x86_64
glusterfs-libs-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
glusterfs-api-3.8.3-0.1.git2ea32d9.el7.centos.x86_64
glusterfs-ganesha-3.8.3-0.1.git2ea32d9.el7.centos.x86_64


How reproducible:
Always

Steps to Reproduce:
1. create a distributed-disperse volume
2. enable quota and set limits
3. start IO and attach hot tier

Actual results:
Attach tier succeeds, IO hangs for a while

Expected results:
No IO hang should be seen with attach tier operation

Additional info:

--- Additional comment from Niels de Vos on 2016-09-12 01:36:54 EDT ---

All 3.8.x bugs are now reported against version 3.8 (without .x). For more
information, see
http://www.gluster.org/pipermail/gluster-devel/2016-September/050859.html

--- Additional comment from Soumya Koduri on 2016-09-13 08:14:45 EDT ---

Could you please collect pkt trace and logs when IO hang is seen.


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1374229
[Bug 1374229] [NFS Ganesha + tiering]: IO hangs for a while during attach
tier
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=I4mCPV2bQ5&a=cc_unsubscribe


More information about the Bugs mailing list