[Gluster-devel] [SOLVED] "failed to fetch volume file (key:dpkvol)" error , when tried as non-root (vdsm) user

[Deepak C Shetty]

After doing many experiments and inputs I got from vijay on IRC, this is 
what finally worked.

1) volume set <volname> server.allow-insecure on
2) option rpc-auth-allow-insecure on in glusterd.vol

If either of the above is not present, i get the same "unable to get 
volfile" error.
I have tried setting virsh (actually libvirt) to spawn qemu process as 
'root', 'vdsm' and 'qemu' user
and it fails for all the 3 if either of the above is NOT set.

Only when both of the above is set, it works for me and I am able to 
start a VM via virsh.

Strangely enuf, on Bharata's machine, none of the above options are set, 
but it still works for him !
How why is the difference is  not clear.

But for now, i think i am good to go for my vdsm work, as this part of 
the stack is now working.

thanx,
deepak



On 12/13/2012 12:21 PM, Vijay Bellur wrote:
> On 12/13/2012 11:50 AM, Deepak C Shetty wrote:
>> On 12/13/2012 11:26 AM, Vijay Bellur wrote:
>>> On 12/13/2012 10:20 AM, Deepak C Shetty wrote:
>>>> Here is the qemu process spawned by virsh ( via libvirt )...
>>>>
>>>> root      2912     1  1 10:16 ?        00:00:00
>>>> /usr/local/bin/qemu-system-x86_64 -name virsh-vm-backed-by-gluster 
>>>> -S -M
>>>> pc -cpu qemu64,-svm -enable-kvm -m 1024 -smp
>>>> 1,sockets=1,cores=1,threads=1 -uuid 
>>>> bdbae806-c272-4b87-ae69-274fd4d57c5f
>>>> -smbios type=1,manufacturer=oVirt,product=oVirt
>>>> Node,version=17-1,serial=762589AD-3D52-42C3-6F65-D682277D5B37_52:54:00:c7:66:ec,uuid=bdbae806-c272-4b87-ae69-274fd4d57c5f 
>>>>
>>>>
>>>> -no-user-config -nodefaults -chardev
>>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/virsh-vm-backed-by-gluster.monitor,server,nowait 
>>>>
>>>>
>>>> -mon chardev=charmonitor,id=monitor,mode=control -rtc
>>>> base=2012-12-13T04:46:48,driftfix=slew -no-shutdown -device
>>>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
>>>> virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x3
>>>> -drive
>>>> file=gluster+tcp://vm-vdsm-de-1/dpkvol/debian_lenny_i386_standard.qcow2,if=none,id=drive-ide0-0-0 
>>>>
>>>>
>>>> -device
>>>> ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
>>>> -chardev pty,id=charconsole0 -device
>>>> virtconsole,chardev=charconsole0,id=console0 -vnc 0:0,password -vga
>>>> cirrus
>>>>
>>>> With this also, i still get the same error which says "unabel to fetch
>>>> volfile"
>>>>
>>>
>>> Did you happen to restart glusterd after setting allow-insecure to ON
>>> in glusterd.vol? Were the results same before and after setting the
>>> option?
>> Yes, today i came in and started glusterd from scratch so that should
>> have covered your point.
>> Also setting allow-insecure in glsuterd.vol ( as you suggested
>> yesterday) doesn't help either, in fact i get a warning msg in the
>> glusterd log saying that option is incorrect and hence ignoring.
>
> Please set rpc-auth-allow-insecure to on. You can always lookup the 
> relevant keys of translator options by looking at the definition of 
> struct volume_options within the translator code.
>
>>
>> Yes in short, with and without that insecure option, i see the same 
>> result.
>
> Since the option has not been recognized, you haven't still seen it 
> being effective.
>
>>
>> I am curious to know how insecure option can be related to glusterd not
>> able to find volfile ?
>
> The problem is about not being able to fetch volume file. This can 
> happen mostly due to:
>
> 1. A rpc failure between the client and glusterd
>
> 2. glusterd unable to locate the volume
>
> In your case, I suspect it is a rpc failure because of an unprivileged 
> client trying to access glusterd.
>
>>
>> Can you help list all the possible reason why glusterd can ever complain
>> abt not able to fetch volfile ? Maybe that can help me debug my env
>>
>
> It would be primarily one of the above. You can follow the code path 
> from server_getspec in xlators/mgmt/glusterd/src/glusterd-handshake.c 
> to determine where it is failing after the unprivileged issue is 
> sorted out.
>
> Regards,
> Vijay
>
>
>