[Gluster-users] Gluster + Cinder

[Derek Yarnell]

I am trying to integrate Gluster (3.4.0-8) with a RDO deployment (Havana
pre-release) of OpenStack with Cinder.  I have a simple 3 node
depolyment (admin host, network host, hypervisor host).  The
cinder-volume is running on the admin host and has mounted the glusterfs
volume successfully.

Excuse me if I am confused on some of these points but the documentation
on Gluster and RHOS doesn't seem to cover the exact specifics of all
these components interactions.

So I can get Cinder and Gluster integrated enough to be able to create
volumes, delete volumes etc.  My understanding is that from the
cinder-volume node it will take that volume and export it to the
instance via iSCSI using the tgtd daemons.  The thing is that when I
attach the volume I get a local libvirt error below[1].  It seems that
libvirt is trying to attach a locally mounted volume file.  This is not
what I expected and I was wondering if anyone has done this integration
that might share some insights.

[1]
2013-09-26 20:03:43.320 8718 ERROR nova.openstack.common.rpc.amqp
[req-acb8d4fa-ab55-45e6-934f-5e723a958acd
92741ec5808647f787a7ea8317d1ca43 064d1904dec8428da2e30e3726b9b943]
Exception during message handling
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
Traceback (most recent call last):
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
line 461, in _process_data
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
**args)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py",
line 172, in dispatch
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/exception.py", line 89, in wrapped
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
payload)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
return f(self, context, *args, **kw)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in
decorated_function
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp     pass
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in
decorated_function
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
return function(self, context, *args, **kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in
decorated_function
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp     e,
sys.exc_info())
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in
decorated_function
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
return function(self, context, *args, **kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3575,
in attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
context, instance, mountpoint)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3570,
in attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
mountpoint, instance)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3612,
in _attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
connector)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3602,
in _attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
mountpoint)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line
1072, in attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
disk_dev)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line
1059, in attach_volume
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
virt_dom.attachDeviceFlags(conf.to_xml(), flags)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
result = proxy_call(self._autowrap, f, *args, **kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in
proxy_call
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp     rv
= execute(f,*args,**kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp     rv
= meth(*args,**kwargs)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib64/python2.6/site-packages/libvirt.py", line 419, in
attachDeviceFlags
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp     if
ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed',
dom=self)
2013-09-26 20:03:43.320 8718 TRACE nova.openstack.common.rpc.amqp
libvirtError: cannot open file
'/var/lib/nova/mnt/0fb9dadec76b71926e1cdb9ca60e1dfe/volume-da84a50f-c0f3-4f55-91e4-6cd3bb5ddc6a':
No such file or directory


-- 
---
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies