[Bugs] [Bug 1782200] New: glusterd restart failing to start.

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1782200

            Bug ID: 1782200
           Summary: glusterd restart failing to start.
           Product: GlusterFS
           Version: mainline
          Hardware: x86_64
                OS: Linux
            Status: NEW
         Component: glusterd
          Severity: medium
          Assignee: bugs at gluster.org
          Reporter: kiyer at redhat.com
                CC: bugs at gluster.org
  Target Milestone: ---
    Classification: Community



Description of problem:
glusterd restart seems to be failing when I try to run the testcase
test_snap_status_glusterd_restart on the latest builds with "Job for
glusterd.service failed because start of the service was attempted too often.
See "systemctl status glusterd.service" and "journalctl -xe" for details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again."

################################################################################
Test run output:
################################################################################
17:26:09
functional/snapshot/test_snap_status_glusterd_restart.py::SnapshotGlusterdrestart_cplex_dispersed_glusterfs::test_snap_status_glusterd_restart
PASSED [ 20%]
17:26:49
functional/snapshot/test_snap_status_glusterd_restart.py::SnapshotGlusterdrestart_cplex_distributed-replicated_glusterfs::test_snap_status_glusterd_restart
PASSED [ 40%]
17:27:30
functional/snapshot/test_snap_status_glusterd_restart.py::SnapshotGlusterdrestart_cplex_replicated_glusterfs::test_snap_status_glusterd_restart
FAILED [ 60%]
17:27:45
functional/snapshot/test_snap_status_glusterd_restart.py::SnapshotGlusterdrestart_cplex_replicated_glusterfs::test_snap_status_glusterd_restart
ERROR [ 60%]
################################################################################


What does the testcase do?

1. Create volume of any type and start it.
2. Create two snapshots with description
3. Check snapshot status information with snap name, volume name and
   without snap name/vol name.
4. Restart glusterd on all nodes
5. Follow step3 again and validate snapshot.



################################################################################
glustomain.log
################################################################################
2019-12-11 11:57:42,525 INFO (run_async) root at 172.19.2.188 (cp): service
glusterd restart
2019-12-11 11:57:42,525 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.188
2019-12-11 11:57:42,530 INFO (run_async) root at 172.19.2.80 (cp): service
glusterd restart
2019-12-11 11:57:42,531 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.80
2019-12-11 11:57:42,536 INFO (run_async) root at 172.19.2.75 (cp): service
glusterd restart
2019-12-11 11:57:42,537 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.75
2019-12-11 11:57:42,542 INFO (run_async) root at 172.19.2.79 (cp): service
glusterd restart
2019-12-11 11:57:42,542 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.79
2019-12-11 11:57:42,547 INFO (run_async) root at 172.19.2.81 (cp): service
glusterd restart
2019-12-11 11:57:42,548 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.81
2019-12-11 11:57:42,553 INFO (run_async) root at 172.19.2.74 (cp): service
glusterd restart
2019-12-11 11:57:42,553 DEBUG (_get_ssh_connection) Retrieved connection from
cache: root at 172.19.2.74
2019-12-11 11:57:42,631 INFO (_log_results) RETCODE (root at 172.19.2.79): 1
2019-12-11 11:57:42,631 INFO (_log_results) STDERR (root at 172.19.2.79)...
Redirecting to /bin/systemctl restart glusterd.service
Job for glusterd.service failed because start of the service was attempted too
often. See "systemctl status glusterd.service" and "journalctl -xe" for
details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again.

2019-12-11 11:57:42,641 INFO (_log_results) RETCODE (root at 172.19.2.81): 1
2019-12-11 11:57:42,641 INFO (_log_results) STDERR (root at 172.19.2.81)...
Redirecting to /bin/systemctl restart glusterd.service
Job for glusterd.service failed because start of the service was attempted too
often. See "systemctl status glusterd.service" and "journalctl -xe" for
details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again.

2019-12-11 11:57:42,641 INFO (_log_results) RETCODE (root at 172.19.2.80): 1
2019-12-11 11:57:42,641 INFO (_log_results) STDERR (root at 172.19.2.80)...
Redirecting to /bin/systemctl restart glusterd.service
Job for glusterd.service failed because start of the service was attempted too
often. See "systemctl status glusterd.service" and "journalctl -xe" for
details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again.

2019-12-11 11:57:42,641 INFO (_log_results) RETCODE (root at 172.19.2.74): 1
2019-12-11 11:57:42,642 INFO (_log_results) STDERR (root at 172.19.2.74)...
Redirecting to /bin/systemctl restart glusterd.service
Job for glusterd.service failed because start of the service was attempted too
often. See "systemctl status glusterd.service" and "journalctl -xe" for
details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again.

2019-12-11 11:57:42,642 INFO (_log_results) RETCODE (root at 172.19.2.75): 1
2019-12-11 11:57:42,642 INFO (_log_results) STDERR (root at 172.19.2.75)...
Redirecting to /bin/systemctl restart glusterd.service
Job for glusterd.service failed because start of the service was attempted too
often. See "systemctl status glusterd.service" and "journalctl -xe" for
details.
To force a start use "systemctl reset-failed glusterd.service" followed by
"systemctl start glusterd.service" again.

2019-12-11 11:57:43,017 INFO (_log_results) RETCODE (root at 172.19.2.188): 0
2019-12-11 11:57:43,017 INFO (_log_results) STDERR (root at 172.19.2.188)...
Redirecting to /bin/systemctl restart glusterd.service
################################################################################


Version-Release number of selected component (if applicable):
Whatever is the latest upstream versioin

How reproducible:
Always.

Steps to Reproduce:
https://review.gluster.org/#/c/glusto-tests/+/20150/16/tests/functional/snapshot/test_snap_status_glusterd_restart.py

Actual results:
glusterd restart fails.

Expected results:
glusterd restart shouldn't fail.

Additional info:

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.