[Bugs] [Bug 1274595] New: Data Tiering:getting failed to fsync on germany-hot-dht (Structure needs cleaning) warning

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1274595

            Bug ID: 1274595
           Summary: Data Tiering:getting failed to fsync on
                    germany-hot-dht (Structure needs cleaning) warning
           Product: Red Hat Gluster Storage
           Version: 3.1
         Component: glusterfs
     Sub Component: tiering
          Severity: high
          Assignee: rhs-bugs at redhat.com
          Reporter: vagarwal at redhat.com
        QA Contact: nchilaka at redhat.com
                CC: bugs at gluster.org, nbalacha at redhat.com,
                    nchilaka at redhat.com, vagarwal at redhat.com
        Depends On: 1272036
            Blocks: 1260923, 1273372



+++ This bug was initially created as a clone of Bug #1272036 +++

Description of problem:
======================

I am seeing getting this error when there is a failure to migrate file to hot
tier


In my case following is the way I probably hit it.

I had the following setup:

a hot tier 2x2 setup with each 1GB bricks
over a cold tier 2x2 of 500GB space

I then created files f1 of 700MB and then after f1 got demoted i created f2 of
700MB. When even f2 got demoted, I touched both files to get them promoted to
hot brick, and see what happens as it can accomodate only 1GB data.

I then saw  logs and found this error.





Version-Release number of selected component (if applicable):
==============================================================
glusterfs-server-3.7.5-0.18.gitb8ba012.el7.centos.x86_64




[2015-10-14 12:12:04.867032] W [dht-rebalance.c:1247:dht_migrate_file]
0-germany-tier-dht: /berlin.7: failed to fsync on germany-hot-dht (Structure
needs cleaning)
[2015-10-14 13:14:08.219263] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.10: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:28:25.074476] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.2: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:30:26.091045] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.10: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:34:26.703553] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.2: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:34:51.579771] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.3: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:38:26.134830] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.4: failed to fsync on greece-hot-dht (Structure
needs cleaning)
[2015-10-14 13:44:24.459860] W [dht-rebalance.c:1247:dht_migrate_file]
0-greece-tier-dht: /athens.3: failed to fsync on greece-hot-dht (Structure
needs cleaning)



attaching sos report:
Kindly refer to "greece" volume tier logs

--- Additional comment from nchilaka on 2015-10-15 06:43:00 EDT ---

sosreprots

rhsqe-repo.lab.eng.blr.redhat.com:/home/repo/sosreports/bug.1272036
[root at rhsqe-repo sosreports]#

--- Additional comment from Vijay Bellur on 2015-10-21 07:19:52 EDT ---

REVIEW: http://review.gluster.org/12408 (cluster/dht : op_ret not set correctly
in dht_fsync_cbk) posted (#1) for review on release-3.7 by N Balachandran
(nbalacha at redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-21 07:21:37 EDT ---

REVIEW: http://review.gluster.org/12409 (cluster/dht : op_ret not set correctly
in dht_fsync_cbk) posted (#1) for review on release-3.7 by N Balachandran
(nbalacha at redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-21 11:17:55 EDT ---

REVIEW: http://review.gluster.org/12409 (cluster/dht : op_ret not set correctly
in dht_fsync_cbk) posted (#2) for review on release-3.7 by N Balachandran
(nbalacha at redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-21 12:58:06 EDT ---

REVIEW: http://review.gluster.org/12409 (cluster/dht : op_ret not set correctly
in dht_fsync_cbk) posted (#3) for review on release-3.7 by N Balachandran
(nbalacha at redhat.com)


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1260923
[Bug 1260923] Tracker for tiering in 3.1.2
https://bugzilla.redhat.com/show_bug.cgi?id=1272036
[Bug 1272036] Data Tiering:getting failed to fsync on germany-hot-dht
(Structure needs cleaning) warning
https://bugzilla.redhat.com/show_bug.cgi?id=1273372
[Bug 1273372] Data Tiering:getting failed to fsync on germany-hot-dht
(Structure needs cleaning) warning
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=EEfIqkWdlR&a=cc_unsubscribe