[Gluster-users] Migrating a VM makes its gluster storage inaccessible

[Bernhard Glomm]

Hi Paul,
thanks a lot!
Sounds good.
Will try to confirm your findings ASAP
Regards
Bernhard
> Hi again,
> 
> First the good news: I found a way to make live migrations work again.
> 
> As quoted below, libvirt changes the ownership of the guest image, 
> unless it detects that the image is on a shared filesystem. After 
> looking at the code for libvirt, they have code to detect NFS, GFS2 and 
> SMB/CIFS, but not Gluster. As libvirt does not detect that the storage 
> is on a shared file system, the originating host will perform a chown 
> back to root:root at the end of a successful migration, whereas the 
> destination host will do a chown to libvirt-qemu:kvm. This is in fact a 
> race condition, so the difference in behaviour between 3.4.0 and 3.4.1 
> could be down to timing differences.
> 
> Workaround: stop your guests, then stop libvirt, and edit 
> /etc/libvirt/qemu.conf - this contains a commented out entry 
> 'dynamic_ownership=1', which is the default. Change this to 0, and 
> remove the comment. Then do a chown to libvirt-qemu:kvm for all your 
> stopped images. Then you can start the service libvirt-bin again, and 
> bring up the guests. Repeat on the other half of your cluster, and test 
> a live migration. For me, they work again.
> 
> The workaround seems fine with me, but now you have to take care of 
> properly setting the ownership of a guest image yourself (presumably 
> only once when you create it).
> 
> Other possible solutions:
> 
> JoeJulian suggested using libgfapi, giving libvirt direct access without 
> having to go through the filesystem. This is the preferred setup for 
> libvirt+gluster and should also result in better I/O performance. I 
> haven't tested this yet, but it's high on my to-do list.
> 
> Submit a patch to libvirt so it can detect that the filesystem is 
> Gluster. statfs() will only show 'FUSE", but we could then use getxattr 
> to see if there is a gluster-specific attribute set (suggested by 
> kkeithley). This could be trusted.glusterfs.volume-id, e.g.
> 
> Regards, Paul Boven.
> 
> 
> On 01/28/2014 01:57 PM, Paul Boven wrote:
> > Hi everyone,
> > 
> > On the libvirt Wiki, I found the text below which might well apply to
> > our live-migration issue:
> > 
> > "The directory used for storing disk images has to be mounted from
> > shared storage on both hosts. Otherwise, the domain may lose access to
> > its disk images during migration because source libvirtd may change the
> > owner, permissions, and SELinux labels on the disk images once it
> > successfully migrates the domain to its destination. Libvirt avoids
> > doing such things if it detects that the disk images are mounted from a
> > shared storage. "
> > 
> > So perhaps libvirtd fails to recognize that it is on shared storage, and
> > it is the originating libvirt that throws a wrench in the wheels by
> > changing the ownership?
> > 
> > http://wiki.libvirt.org/page/Migration_fails_because_disk_image_cannot_be_found
> > 
> > 
> > Regards, Paul Boven.

> 
> -- 
> Paul Boven <> boven at jive.nl> > +31 (0)521-596547
> Unix/Linux/Networking specialist
> Joint Institute for VLBI in Europe - www.jive.nl
> VLBI - It's a fringe science
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users




-- 


  
         
    
        
      
        
          
            Bernhard Glomm

            IT Administration


          
            
                  Phone:
                
                
                  +49 (30) 86880 134
                
              
                  Fax:
                
                
                  +49 (30) 86880 100
                
              
                  Skype:
                
                
                  bernhard.glomm.ecologic
                
              
        
      
    
            
            
            
            
            
            
            
            
      
    
        
          Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany

          GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464

          Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH
        
      
    
         

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140128/461762cc/attachment.html>