[Gluster-devel] Glusterfs 3.7.11 with LibGFApi in Qemu on Ubuntu Xenial does not work

[Stephen Howell]

I would like to follow up on a previous thread. I have here 3 machines
running Ubuntu. All were running 14.04 LTS and of these two have been
upgraded to 16.04. They all run QEMU with a shared GlusterFS mount for
storing VM images. Libgfapi was configured and running on all hosts with
14.04 but has stopped recently with 16.04.

I can see exactly the same problems as mentioned here on the 16.04
machines, with these packages:

ii  glusterfs-client                             3.7.14-ubuntu1~xenial1
                            amd64        clustered file-system (client
package)
ii  glusterfs-common                             3.7.14-ubuntu1~xenial1
                            amd64        GlusterFS common libraries and
translator modules
ii  glusterfs-server                             3.7.14-ubuntu1~xenial1
                            amd64        clustered file-system (server
package)
ii  qemu-block-extra:amd64
1:2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1           amd64        extra
block backend modules for qemu-system and qemu-utils
ii  qemu-kvm
1:2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1           amd64        QEMU
Full virtualization
ii  qemu-system-common
1:2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1           amd64        QEMU
full system emulation binaries (common files)
ii  qemu-system-x86
 1:2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1           amd64        QEMU
full system emulation binaries (x86)
ii  qemu-utils
1:2.5+dfsg-5ubuntu10.2glusterfs3.7.14xenial1           amd64        QEMU
utilities

Packages on the 14.04 instance:

ii  glusterfs-client                       3.7.13-ubuntu1~trusty1
            amd64        clustered file-system (client package)
ii  glusterfs-common                       3.7.13-ubuntu1~trusty1
            amd64        GlusterFS common libraries and translator modules
ii  glusterfs-server                       3.7.13-ubuntu1~trusty1
            amd64        clustered file-system (server package)
ii  qemu-common
 2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 all          dummy
transitional package from qemu-common to qemu-keymaps
ii  qemu-keymaps
2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 all          QEMU keyboard maps
ii  qemu-kvm
2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 amd64        QEMU Full
virtualization
ii  qemu-system-common
2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 amd64        QEMU full system
emulation binaries (common files)
ii  qemu-system-x86
 2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 amd64        QEMU full system
emulation binaries (x86)
ii  qemu-utils
2.0.0+dfsg-2ubuntu1.24glusterfs3.7.12trusty1 amd64        QEMU utilities


As you can see I am using Andre (monotek)'s packages to employ the Gluster
protocol in QEMU under Ubuntu (not compiled by default). The versions of
Gluster are similar and have indeed been identical in my prior testing. The
relevant volume options were set, apparmour rules were added and the setup
worked without issue serving block devices over libgfapi for 10+ VMs.
However on upgrading to 16.04 there are issues relating to QEMU being
unable to access the Gluster volume file when starting a VM. I can use
qemu-img to create a blank file using the Gluster profocol but I cannot
then start a VM using that file.

Error message:

 [MSGID: 104007] [glfs-mgmt.c:637:glfs_mgmt_getspec_cbk] 0-glfs-mgmt:
failed to fetch volume file (key:VM) [Invalid argument] [2016-08-20
11:28:02.985483] E [MSGID: 104024] [glfs-mgmt.c:738:mgmt_rpc_notify]
0-glfs-mgmt: failed to connect with remote-host: 127.0.0.1 (Permission
denied) [Permission denied] 2016-08-20T11:28:03.979968Z qemu-system-x86_64:
-drive file=gluster://
127.0.0.1/VM/vm1.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none:
Gluster connection failed for server=127.0.0.1 port=0 volume=VM
image=vm1.qcow2 transport=tcp: Permission denied

Any assistance on changes to permissions or apparmour in 16.04 would be
greatly appreciated.

thanks
Stephen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160820/f281ebfd/attachment.html>