[Bugs] [Bug 1262341] New: Database locking due to write contention between CTR sql connection and tier migrator sql connection

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1262341

            Bug ID: 1262341
           Summary: Database locking due to write contention between CTR
                    sql connection and tier migrator sql connection
           Product: GlusterFS
           Version: 3.7.5
         Component: tiering
          Keywords: Triaged
          Assignee: bugs at gluster.org
          Reporter: dlambrig at redhat.com
        QA Contact: bugs at gluster.org
                CC: bugs at gluster.org, dlambrig at redhat.com,
                    josferna at redhat.com
        Depends On: 1260730
            Blocks: 1261039, 1261140



+++ This bug was initially created as a clone of Bug #1260730 +++

Description of problem:
The DB on the brick is been accessed by CTR, for write and
tier migrator, for read and write. The write from tier migrator is reseting
the heat counters after a cycle. Since we are using sqlite, two connections
trying to write would cause a db lock contention. As a result CTR used to fail
to update the db.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Vijay Bellur on 2015-09-07 10:27:02 EDT ---

REVIEW: http://review.gluster.org/12031 (tier/ctr: Solving DB Lock issue due to
write contention from db connections) posted (#5) for review on master by
Joseph Fernandes

--- Additional comment from Vijay Bellur on 2015-09-08 08:13:06 EDT ---

COMMIT: http://review.gluster.org/12031 committed in master by Dan Lambright
(dlambrig at redhat.com) 
------
commit 96af474045c9ba5ab74ca76daa823d91a0a0c610
Author: Joseph Fernandes <josferna at redhat.com>
Date:   Thu Aug 27 17:23:07 2015 +0530

    tier/ctr: Solving DB Lock issue due to write contention from db connections

    Problem: The DB on the brick is been accessed by CTR, for write and
    tier migrator, for read and write. The write from tier migrator is reseting
    the heat counters after a cycle. Since we are using sqlite, two connections
    trying to write would cause a db lock contention. As a result CTR used to
fail
    to update the db.

    Solution: Using the same db connection of CTR for reseting the heat
counters.
    1) Introducted a new IPC FOP for CTR
    2) After the query do a ipc syncop to the underlying client xlator
associated
       to the brick.
    3) CTR in brick will catch the IPC FOP and cleat the heat counters.

    Change-Id: I53306bfc08dcdba479deb4ccc154896521336150
    BUG: 1260730
    Signed-off-by: Joseph Fernandes <josferna at redhat.com>
    Reviewed-on: http://review.gluster.org/12031
    Tested-by: NetBSD Build System <jenkins at build.gluster.org>
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Dan Lambright <dlambrig at redhat.com>
    Tested-by: Dan Lambright <dlambrig at redhat.com>

--- Additional comment from Dan Lambright on 2015-09-10 15:23:47 EDT ---

reopening in order to fix freebsd bug 


> /usr/home/jenkins/root/workspace/freebsd-smoke/libglusterfs/src/gfdb/gfdb_sqlite3.h:15:10: fatal error: 'sqlite3.h' file not found
> 
> https://build.gluster.org/job/freebsd-smoke/9859/console
>

--- Additional comment from Dan Lambright on 2015-09-10 15:23:47 EDT ---

reopening in order to fix freebsd bug 


> /usr/home/jenkins/root/workspace/freebsd-smoke/libglusterfs/src/gfdb/gfdb_sqlite3.h:15:10: fatal error: 'sqlite3.h' file not found
> 
> https://build.gluster.org/job/freebsd-smoke/9859/console
>

--- Additional comment from Vijay Bellur on 2015-09-10 15:28:00 EDT ---

REVIEW: http://review.gluster.org/12155 (cluster/tier fix bug with sql includes
introduced by 12031) posted (#1) for review on master by Dan Lambright
(dlambrig at redhat.com)

--- Additional comment from Vijay Bellur on 2015-09-11 08:28:50 EDT ---

REVIEW: http://review.gluster.org/12155 (cluster/tier fix bug with sql includes
introduced by 12031) posted (#2) for review on master by Dan Lambright
(dlambrig at redhat.com)


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1260730
[Bug 1260730] Database locking due to write contention between CTR sql
connection and tier migrator sql connection
https://bugzilla.redhat.com/show_bug.cgi?id=1261039
[Bug 1261039] Database locking due to write contention between CTR sql
connection and tier migrator sql connection
https://bugzilla.redhat.com/show_bug.cgi?id=1261140
[Bug 1261140] Database locking due to write contention between CTR sql
connection and tier migrator sql connection
-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
You are the assignee for the bug.