[Gluster-users] VM failed to start | Bad volume specification
Punit Dambiwal
hypunit at gmail.com
Thu Mar 19 08:25:18 UTC 2015
Hi Michal,
The Storage domain is up and running and mounted on all the host nodes...as
i updated before that it was working perfectly before but just after reboot
can not make the VM poweron...
[image: Inline image 1]
[image: Inline image 2]
[root at cpu01 log]# gluster volume info
Volume Name: ds01
Type: Distributed-Replicate
Volume ID: 369d3fdc-c8eb-46b7-a33e-0a49f2451ff6
Status: Started
Number of Bricks: 48 x 2 = 96
Transport-type: tcp
Bricks:
Brick1: cpu01:/bricks/1/vol1
Brick2: cpu02:/bricks/1/vol1
Brick3: cpu03:/bricks/1/vol1
Brick4: cpu04:/bricks/1/vol1
Brick5: cpu01:/bricks/2/vol1
Brick6: cpu02:/bricks/2/vol1
Brick7: cpu03:/bricks/2/vol1
Brick8: cpu04:/bricks/2/vol1
Brick9: cpu01:/bricks/3/vol1
Brick10: cpu02:/bricks/3/vol1
Brick11: cpu03:/bricks/3/vol1
Brick12: cpu04:/bricks/3/vol1
Brick13: cpu01:/bricks/4/vol1
Brick14: cpu02:/bricks/4/vol1
Brick15: cpu03:/bricks/4/vol1
Brick16: cpu04:/bricks/4/vol1
Brick17: cpu01:/bricks/5/vol1
Brick18: cpu02:/bricks/5/vol1
Brick19: cpu03:/bricks/5/vol1
Brick20: cpu04:/bricks/5/vol1
Brick21: cpu01:/bricks/6/vol1
Brick22: cpu02:/bricks/6/vol1
Brick23: cpu03:/bricks/6/vol1
Brick24: cpu04:/bricks/6/vol1
Brick25: cpu01:/bricks/7/vol1
Brick26: cpu02:/bricks/7/vol1
Brick27: cpu03:/bricks/7/vol1
Brick28: cpu04:/bricks/7/vol1
Brick29: cpu01:/bricks/8/vol1
Brick30: cpu02:/bricks/8/vol1
Brick31: cpu03:/bricks/8/vol1
Brick32: cpu04:/bricks/8/vol1
Brick33: cpu01:/bricks/9/vol1
Brick34: cpu02:/bricks/9/vol1
Brick35: cpu03:/bricks/9/vol1
Brick36: cpu04:/bricks/9/vol1
Brick37: cpu01:/bricks/10/vol1
Brick38: cpu02:/bricks/10/vol1
Brick39: cpu03:/bricks/10/vol1
Brick40: cpu04:/bricks/10/vol1
Brick41: cpu01:/bricks/11/vol1
Brick42: cpu02:/bricks/11/vol1
Brick43: cpu03:/bricks/11/vol1
Brick44: cpu04:/bricks/11/vol1
Brick45: cpu01:/bricks/12/vol1
Brick46: cpu02:/bricks/12/vol1
Brick47: cpu03:/bricks/12/vol1
Brick48: cpu04:/bricks/12/vol1
Brick49: cpu01:/bricks/13/vol1
Brick50: cpu02:/bricks/13/vol1
Brick51: cpu03:/bricks/13/vol1
Brick52: cpu04:/bricks/13/vol1
Brick53: cpu01:/bricks/14/vol1
Brick54: cpu02:/bricks/14/vol1
Brick55: cpu03:/bricks/14/vol1
Brick56: cpu04:/bricks/14/vol1
Brick57: cpu01:/bricks/15/vol1
Brick58: cpu02:/bricks/15/vol1
Brick59: cpu03:/bricks/15/vol1
Brick60: cpu04:/bricks/15/vol1
Brick61: cpu01:/bricks/16/vol1
Brick62: cpu02:/bricks/16/vol1
Brick63: cpu03:/bricks/16/vol1
Brick64: cpu04:/bricks/16/vol1
Brick65: cpu01:/bricks/17/vol1
Brick66: cpu02:/bricks/17/vol1
Brick67: cpu03:/bricks/17/vol1
Brick68: cpu04:/bricks/17/vol1
Brick69: cpu01:/bricks/18/vol1
Brick70: cpu02:/bricks/18/vol1
Brick71: cpu03:/bricks/18/vol1
Brick72: cpu04:/bricks/18/vol1
Brick73: cpu01:/bricks/19/vol1
Brick74: cpu02:/bricks/19/vol1
Brick75: cpu03:/bricks/19/vol1
Brick76: cpu04:/bricks/19/vol1
Brick77: cpu01:/bricks/20/vol1
Brick78: cpu02:/bricks/20/vol1
Brick79: cpu03:/bricks/20/vol1
Brick80: cpu04:/bricks/20/vol1
Brick81: cpu01:/bricks/21/vol1
Brick82: cpu02:/bricks/21/vol1
Brick83: cpu03:/bricks/21/vol1
Brick84: cpu04:/bricks/21/vol1
Brick85: cpu01:/bricks/22/vol1
Brick86: cpu02:/bricks/22/vol1
Brick87: cpu03:/bricks/22/vol1
Brick88: cpu04:/bricks/22/vol1
Brick89: cpu01:/bricks/23/vol1
Brick90: cpu02:/bricks/23/vol1
Brick91: cpu03:/bricks/23/vol1
Brick92: cpu04:/bricks/23/vol1
Brick93: cpu01:/bricks/24/vol1
Brick94: cpu02:/bricks/24/vol1
Brick95: cpu03:/bricks/24/vol1
Brick96: cpu04:/bricks/24/vol1
Options Reconfigured:
diagnostics.count-fop-hits: on
diagnostics.latency-measurement: on
nfs.disable: on
user.cifs: enable
auth.allow: 10.10.0.*
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
storage.owner-uid: 36
storage.owner-gid: 36
server.allow-insecure: on
network.ping-timeout: 100
[root at cpu01 log]#
-----------------------------------------
[root at cpu01 log]# gluster volume status
Status of volume: ds01
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick cpu01:/bricks/1/vol1 49152 Y
33474
Brick cpu02:/bricks/1/vol1 49152 Y
40717
Brick cpu03:/bricks/1/vol1 49152 Y
18080
Brick cpu04:/bricks/1/vol1 49152 Y
40447
Brick cpu01:/bricks/2/vol1 49153 Y
33481
Brick cpu02:/bricks/2/vol1 49153 Y
40724
Brick cpu03:/bricks/2/vol1 49153 Y
18086
Brick cpu04:/bricks/2/vol1 49153 Y
40453
Brick cpu01:/bricks/3/vol1 49154 Y
33489
Brick cpu02:/bricks/3/vol1 49154 Y
40731
Brick cpu03:/bricks/3/vol1 49154 Y
18097
Brick cpu04:/bricks/3/vol1 49154 Y
40460
Brick cpu01:/bricks/4/vol1 49155 Y
33495
Brick cpu02:/bricks/4/vol1 49155 Y
40738
Brick cpu03:/bricks/4/vol1 49155 Y
18103
Brick cpu04:/bricks/4/vol1 49155 Y
40468
Brick cpu01:/bricks/5/vol1 49156 Y
33502
Brick cpu02:/bricks/5/vol1 49156 Y
40745
Brick cpu03:/bricks/5/vol1 49156 Y
18110
Brick cpu04:/bricks/5/vol1 49156 Y
40474
Brick cpu01:/bricks/6/vol1 49157 Y
33509
Brick cpu02:/bricks/6/vol1 49157 Y
40752
Brick cpu03:/bricks/6/vol1 49157 Y
18116
Brick cpu04:/bricks/6/vol1 49157 Y
40481
Brick cpu01:/bricks/7/vol1 49158 Y
33516
Brick cpu02:/bricks/7/vol1 49158 Y
40759
Brick cpu03:/bricks/7/vol1 49158 Y
18122
Brick cpu04:/bricks/7/vol1 49158 Y
40488
Brick cpu01:/bricks/8/vol1 49159 Y
33525
Brick cpu02:/bricks/8/vol1 49159 Y
40766
Brick cpu03:/bricks/8/vol1 49159 Y
18130
Brick cpu04:/bricks/8/vol1 49159 Y
40495
Brick cpu01:/bricks/9/vol1 49160 Y
33530
Brick cpu02:/bricks/9/vol1 49160 Y
40773
Brick cpu03:/bricks/9/vol1 49160 Y
18137
Brick cpu04:/bricks/9/vol1 49160 Y
40502
Brick cpu01:/bricks/10/vol1 49161 Y
33538
Brick cpu02:/bricks/10/vol1 49161 Y
40780
Brick cpu03:/bricks/10/vol1 49161 Y
18143
Brick cpu04:/bricks/10/vol1 49161 Y
40509
Brick cpu01:/bricks/11/vol1 49162 Y
33544
Brick cpu02:/bricks/11/vol1 49162 Y
40787
Brick cpu03:/bricks/11/vol1 49162 Y
18150
Brick cpu04:/bricks/11/vol1 49162 Y
40516
Brick cpu01:/bricks/12/vol1 49163 Y
33551
Brick cpu02:/bricks/12/vol1 49163 Y
40794
Brick cpu03:/bricks/12/vol1 49163 Y
18157
Brick cpu04:/bricks/12/vol1 49163 Y
40692
Brick cpu01:/bricks/13/vol1 49164 Y
33558
Brick cpu02:/bricks/13/vol1 49164 Y
40801
Brick cpu03:/bricks/13/vol1 49164 Y
18165
Brick cpu04:/bricks/13/vol1 49164 Y
40700
Brick cpu01:/bricks/14/vol1 49165 Y
33566
Brick cpu02:/bricks/14/vol1 49165 Y
40809
Brick cpu03:/bricks/14/vol1 49165 Y
18172
Brick cpu04:/bricks/14/vol1 49165 Y
40706
Brick cpu01:/bricks/15/vol1 49166 Y
33572
Brick cpu02:/bricks/15/vol1 49166 Y
40815
Brick cpu03:/bricks/15/vol1 49166 Y
18179
Brick cpu04:/bricks/15/vol1 49166 Y
40714
Brick cpu01:/bricks/16/vol1 49167 Y
33579
Brick cpu02:/bricks/16/vol1 49167 Y
40822
Brick cpu03:/bricks/16/vol1 49167 Y
18185
Brick cpu04:/bricks/16/vol1 49167 Y
40722
Brick cpu01:/bricks/17/vol1 49168 Y
33586
Brick cpu02:/bricks/17/vol1 49168 Y
40829
Brick cpu03:/bricks/17/vol1 49168 Y
18192
Brick cpu04:/bricks/17/vol1 49168 Y
40727
Brick cpu01:/bricks/18/vol1 49169 Y
33593
Brick cpu02:/bricks/18/vol1 49169 Y
40836
Brick cpu03:/bricks/18/vol1 49169 Y
18201
Brick cpu04:/bricks/18/vol1 49169 Y
40735
Brick cpu01:/bricks/19/vol1 49170 Y
33600
Brick cpu02:/bricks/19/vol1 49170 Y
40843
Brick cpu03:/bricks/19/vol1 49170 Y
18207
Brick cpu04:/bricks/19/vol1 49170 Y
40741
Brick cpu01:/bricks/20/vol1 49171 Y
33608
Brick cpu02:/bricks/20/vol1 49171 Y
40850
Brick cpu03:/bricks/20/vol1 49171 Y
18214
Brick cpu04:/bricks/20/vol1 49171 Y
40748
Brick cpu01:/bricks/21/vol1 49172 Y
33614
Brick cpu02:/bricks/21/vol1 49172 Y
40858
Brick cpu03:/bricks/21/vol1 49172 Y
18222
Brick cpu04:/bricks/21/vol1 49172 Y
40756
Brick cpu01:/bricks/22/vol1 49173 Y
33621
Brick cpu02:/bricks/22/vol1 49173 Y
40864
Brick cpu03:/bricks/22/vol1 49173 Y
18227
Brick cpu04:/bricks/22/vol1 49173 Y
40762
Brick cpu01:/bricks/23/vol1 49174 Y
33626
Brick cpu02:/bricks/23/vol1 49174 Y
40869
Brick cpu03:/bricks/23/vol1 49174 Y
18234
Brick cpu04:/bricks/23/vol1 49174 Y
40769
Brick cpu01:/bricks/24/vol1 49175 Y
33631
Brick cpu02:/bricks/24/vol1 49175 Y
40874
Brick cpu03:/bricks/24/vol1 49175 Y
18239
Brick cpu04:/bricks/24/vol1 49175 Y
40774
Self-heal Daemon on localhost N/A Y
33361
Self-heal Daemon on cpu05 N/A Y 2353
Self-heal Daemon on cpu04 N/A Y
40786
Self-heal Daemon on cpu02 N/A Y
32442
Self-heal Daemon on cpu03 N/A Y
18664
Task Status of Volume ds01
------------------------------------------------------------------------------
Task : Rebalance
ID : 5db24b30-4b9f-4b65-8910-a7a0a6d327a4
Status : completed
[root at cpu01 log]#
[root at cpu01 log]# gluster pool list
UUID Hostname State
626c9360-8c09-480f-9707-116e67cc38e6 cpu02 Connected
dc475d62-b035-4ee6-9006-6f03bf68bf24 cpu05 Connected
41b5b2ff-3671-47b4-b477-227a107e718d cpu03 Connected
c0afe114-dfa7-407d-bad7-5a3f97a6f3fc cpu04 Connected
9b61b0a5-be78-4ac2-b6c0-2db588da5c35 localhost Connected
[root at cpu01 log]#
[image: Inline image 3]
Thanks,
Punit
On Thu, Mar 19, 2015 at 2:53 PM, Michal Skrivanek <
michal.skrivanek at redhat.com> wrote:
>
> On Mar 19, 2015, at 03:18 , Punit Dambiwal <hypunit at gmail.com> wrote:
>
> > Hi All,
> >
> > Is there any one have any idea about this problem...it seems it's bug
> either in Ovirt or Glusterfs...that's why no one has the idea about
> it....please correct me if i am wrong….
>
> Hi,
> as I said, storage access times out; so it seems to me as a gluster setup
> problem, the storage domain you have your VMs on is not working…
>
> Thanks,
> michal
>
> >
> > Thanks,
> > Punit
> >
> > On Wed, Mar 18, 2015 at 5:05 PM, Punit Dambiwal <hypunit at gmail.com>
> wrote:
> > Hi Michal,
> >
> > Would you mind to let me know the possible messedup things...i will
> check and try to resolve it....still i am communicating gluster community
> to resolve this issue...
> >
> > But in the ovirt....gluster setup is quite straight....so how come it
> will be messedup with reboot ?? if it can be messedup with reboot then it
> seems not good and stable technology for the production storage....
> >
> > Thanks,
> > Punit
> >
> > On Wed, Mar 18, 2015 at 3:51 PM, Michal Skrivanek <
> michal.skrivanek at redhat.com> wrote:
> >
> > On Mar 18, 2015, at 03:33 , Punit Dambiwal <hypunit at gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Is there any one from community can help me to solve this issue...??
> > >
> > > Thanks,
> > > Punit
> > >
> > > On Tue, Mar 17, 2015 at 12:52 PM, Punit Dambiwal <hypunit at gmail.com>
> wrote:
> > > Hi,
> > >
> > > I am facing one strange issue with ovirt/glusterfs....still didn't
> find this issue is related with glusterfs or Ovirt....
> > >
> > > Ovirt :- 3.5.1
> > > Glusterfs :- 3.6.1
> > > Host :- 4 Hosts (Compute+ Storage)...each server has 24 bricks
> > > Guest VM :- more then 100
> > >
> > > Issue :- When i deploy this cluster first time..it work well for
> me(all the guest VM created and running successfully)....but suddenly one
> day my one of the host node rebooted and none of the VM can boot up
> now...and failed with the following error "Bad Volume Specification"
> > >
> > > VMId :- d877313c18d9783ca09b62acf5588048
> > >
> > > VDSM Logs :- http://ur1.ca/jxabi
> >
> > you've got timeouts while accessing storage…so I guess something got
> messed up on reboot, it may also be just a gluster misconfiguration…
> >
> > > Engine Logs :- http://ur1.ca/jxabv
> > >
> > > ------------------------
> > > [root at cpu01 ~]# vdsClient -s 0 getVolumeInfo
> e732a82f-bae9-4368-8b98-dedc1c3814de 00000002-0002-0002-0002-000000000145
> 6d123509-6867-45cf-83a2-6d679b77d3c5 9030bb43-6bc9-462f-a1b9-f6d5a02fb180
> > > status = OK
> > > domain = e732a82f-bae9-4368-8b98-dedc1c3814de
> > > capacity = 21474836480
> > > voltype = LEAF
> > > description =
> > > parent = 00000000-0000-0000-0000-000000000000
> > > format = RAW
> > > image = 6d123509-6867-45cf-83a2-6d679b77d3c5
> > > uuid = 9030bb43-6bc9-462f-a1b9-f6d5a02fb180
> > > disktype = 2
> > > legality = LEGAL
> > > mtime = 0
> > > apparentsize = 21474836480
> > > truesize = 4562972672
> > > type = SPARSE
> > > children = []
> > > pool =
> > > ctime = 1422676305
> > > ---------------------
> > >
> > > I opened same thread earlier but didn't get any perfect answers to
> solve this issue..so i reopen it...
> > >
> > > https://www.mail-archive.com/users@ovirt.org/msg25011.html
> > >
> > > Thanks,
> > > Punit
> > >
> > >
> > >
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150319/dd62238a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 15357 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150319/dd62238a/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 7573 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150319/dd62238a/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 4766 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150319/dd62238a/attachment-0002.png>
More information about the Gluster-users
mailing list