[Gluster-devel] Xen on gluster 1.3.x report

[Jordi Moles Blanco]

Hi,

i just want to report back after my experience with 1.3.x from tla 
repositories and also ask for you opinion about a couple of issues I've 
faced.

The thing is that i had an already running Xen 3.2 machine, with some 
virtual machines in it. Then i prepared a gluster environment with 2TB 
to store all these machines and a lot of new ones.

What i needed to do to make old machines work:

-I moved the machines into the gluster mount point and updated the 
configuration files to point at the new path into gluster.
-At first, they wouldn't start. Then i read that if you use Xen you need 
the "--disable-direct-io-mode" when you mount gluster on Xen's side, and 
so i did and all the old machines would then start.
-Then i saw that, even though the machines seemed to be working, they 
would freeze mysteriously. I read a little more and i saw an option you 
need to include in the vm's config file:

extra       = 'xencons=tty'

so... be aware if you use Xen 3.2, you may need this option for the vm 
to work properly.

-Then i went through the log files on both, node's and Xen's side, and 
everything was ok, except that i saw this on Xen's side:

************
2008-12-23 14:03:32 E [afr.c:919:afr_setxattr_cbk] grup2: 
(path=/xen-cfg/xen-game11.cfg child=espai2) op_ret=-1 op_errno=95
2008-12-23 14:03:32 E [afr.c:919:afr_setxattr_cbk] grup2: 
(path=/xen-cfg/xen-game11.cfg child=espai6) op_ret=-1 op_errno=95
2008-12-23 14:03:33 E [afr.c:919:afr_setxattr_cbk] grup2: 
(path=/xen-cfg/xen-game11.cfg child=espai4) op_ret=-1 op_errno=95
************

and this on node's side:

************
2008-12-23 14:03:33 W [posix.c:1524:posix_setxattr] esp: setxattr on 
/xen-cfg/xen-game11.cfg with key (system.posix_acl_access): Operation 
not supported
2008-12-23 14:03:33 W [posix.c:1524:posix_setxattr] esp: setxattr on 
/xen-cfg/xen-game11.cfg with key (system.posix_acl_access): Operation 
not supported
************

it seems that when Xen creates a virtual machine, it sets some extended 
attributes on that file, and when i moved the file into glusterfs, this 
was omitted. Is there any reason for that? Is it possible that gluster 
only excepts extra attibutes related to glusterfs itself? Or may be is 
there a list of supported attributes? So far, machines are working just 
fine, but i don't know if some time in the future i will need this extra 
attributes.

I would also require your help on another issue, this far more important 
than the setxattr one.
The problem is that when i create machines directly into a glusterfs 
mount point, the machine won't work, without any chance of fixing it, 
it's totally corrupted.
Those machines which I already had worked fine when i moved them into 
gluster, but now, if i want to create new ones i can't. So far, my 
"workaround" has been to create them "locally", let's say, in /tmp 
directory, and then move the machine into /mnt/gluster.
However, i don't think this is ideal. Has anyone experienced this? Any 
idea why could i be having such problem?

As for everything else.... .i have to say that it's been working great 
for days, no error o warning messages, more than acceptable performance, 
and nobody using the virtual machines has complained about any aspect of 
their machines, so i guess that everything is just working fine.

Thanks.