[Bugs] [Bug 1249921] New: [upgrade] After upgrade from 3.5 to 3.6 onwards version, bumping up op-version failed

bugzilla at redhat.com bugzilla at redhat.com
Tue Aug 4 07:18:21 UTC 2015


https://bugzilla.redhat.com/show_bug.cgi?id=1249921

            Bug ID: 1249921
           Summary: [upgrade] After upgrade from 3.5 to 3.6 onwards
                    version, bumping up op-version failed
           Product: GlusterFS
           Version: 3.7.3
         Component: glusterd
          Keywords: Triaged
          Severity: high
          Assignee: bugs at gluster.org
          Reporter: amukherj at redhat.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com,
                    nlevinki at redhat.com, rhs-bugs at redhat.com,
                    sasundar at redhat.com, vbellur at redhat.com
        Depends On: 1247947, 1248298



+++ This bug was initially created as a clone of Bug #1248298 +++

+++ This bug was initially created as a clone of Bug #1247947 +++

Description of problem:
------------------------
Upgraded 3.5 nodes to 3.6/3.7.
After upgrade, bumping up op-version to 30703 failed

Version-Release number of selected component (if applicable):
--------------------------------------------------------------
mainline

How reproducible:
------------------
Always

Steps to Reproduce:
--------------------
1. Upgrade 3.5 Nodes to 3.6/3.7
2. After upgrade bump up op-version to 30703

Actual results:
---------------
Bumping up op-version failed

Expected results:
-----------------
Bumping up op-version should succeed

Additional info:
----------------
[2015-07-29 11:50:31.860731]  : volume set all cluster.op-version 30703 :
FAILED :

[root@ ~]# gluster volume get drvol op-version
Option Value
------ -----
cluster.op-version                      30703

Following are the logs from 2 nodes.

NODE-1
----------
[2015-07-29 11:50:31.860355] E [MSGID: 106116]
[glusterd-mgmt.c:134:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed
on dhcp37-126.lab.eng.blr.redhat.com. Please check log file for details.
[2015-07-29 11:50:31.860493] E [MSGID: 106152]
[glusterd-syncop.c:1562:gd_unlock_op_phase] 0-management: Failed to unlock on
some peer(s)
[2015-07-29 11:50:31.860587] E [MSGID: 106025]
[glusterd-locks.c:641:glusterd_mgmt_v3_unlock] 0-management: name is null.
[Invalid argument]
[2015-07-29 11:50:31.860666] E [MSGID: 106118]
[glusterd-syncop.c:1588:gd_unlock_op_phase] 0-management: Unable to release
lock for (null)
[2015-07-29 11:50:31.875251] I [run.c:190:runner_log] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x7fdcd220c5e0] (-->
/usr/lib64/libglusterfs.so.0(runner_log+0x105)[0x7fdcd225ff95] (-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_hooks_run_hooks+0x4cc)[0x7fdcc6cac10c]
(-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(+0xed422)[0x7fdcc6cac422]
(--> /lib64/libpthread.so.0(+0x3429c07a51)[0x7fdcd12f3a51] ))))) 0-management:
Ran script: /var/lib/glusterd/hooks/1/set/post/S30samba-set.sh --volname=all -o
cluster.op-version=30703 --gd-workdir=/var/lib/glusterd
[2015-07-29 11:50:31.893561] I [run.c:190:runner_log] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x7fdcd220c5e0] (-->
/usr/lib64/libglusterfs.so.0(runner_log+0x105)[0x7fdcd225ff95] (-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_hooks_run_hooks+0x4cc)[0x7fdcc6cac10c]
(-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(+0xed422)[0x7fdcc6cac422]
(--> /lib64/libpthread.so.0(+0x3429c07a51)[0x7fdcd12f3a51] ))))) 0-management:
Ran script:
/var/lib/glusterd/hooks/1/set/post/S32gluster_enable_shared_storage.sh
--volname=all -o cluster.op-version=30703 --gd-workdir=/var/lib/glusterd


NODE-2
-----------

[2015-07-29 11:50:31.622533] E [MSGID: 106118]
[glusterd-op-sm.c:3619:glusterd_op_ac_unlock] 0-management: Unable to release
lock for all
[2015-07-29 11:50:31.622788] E [MSGID: 106376]
[glusterd-op-sm.c:7286:glusterd_op_sm] 0-management: handler returned: -1

--- Additional comment from SATHEESARAN on 2015-07-29 07:32:24 EDT ---

The volume set fails, but the op-version actually got bumped up.
There are no problems functionally

--- Additional comment from Anand Avati on 2015-07-30 00:20:01 EDT ---

REVIEW: http://review.gluster.org/11798 (glusterd: fix op-version bump up flow)
posted (#1) for review on master by Atin Mukherjee (amukherj at redhat.com)

--- Additional comment from Anand Avati on 2015-08-04 00:26:01 EDT ---

COMMIT: http://review.gluster.org/11798 committed in master by Kaushal M
(kaushal at redhat.com) 
------
commit b467b97e4c4546b7f870a3ac624d56c62bfa5cf9
Author: Atin Mukherjee <amukherj at redhat.com>
Date:   Thu Jul 30 09:40:24 2015 +0530

    glusterd: fix op-version bump up flow

    If a cluster is upgraded from 3.5 to latest version, gluster volume set all
    cluster.op-version <VERSION> will throw an error message back to the user
saying
    unlocking failed. This is because of trying to release a volume wise lock
in
    unlock phase as the lock was taken cluster wide. The problem surfaced
because
    the op-version is updated in commit phase and unlocking works in the v3
    framework where it should have used cluster unlock.

    Fix is to decide which lock/unlock is to be followed before invoking lock
phase

    Change-Id: Iefb271a058431fe336a493c24d240ed833f279c5
    BUG: 1248298
    Signed-off-by: Atin Mukherjee <amukherj at redhat.com>
    Reviewed-on: http://review.gluster.org/11798
    Reviewed-by: Avra Sengupta <asengupt at redhat.com>
    Tested-by: NetBSD Build System <jenkins at build.gluster.org>
    Reviewed-by: Anand Nekkunti <anekkunt at redhat.com>
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Kaushal M <kaushal at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1247947
[Bug 1247947] [upgrade] After in-service software upgrade from RHGS 2.1 to
RHGS 3.1, bumping up op-version failed
https://bugzilla.redhat.com/show_bug.cgi?id=1248298
[Bug 1248298] [upgrade] After upgrade from 3.5 to 3.6 onwards version,
bumping up op-version failed
-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.


More information about the Bugs mailing list