[Gluster-users] [Nfs-ganesha-devel] Problems in /usr/libexec/ganesha/dbus-send.sh and ganesha dbus interface when disabling exports from gluster

Thu Jun 18 14:06:04 UTC 2015

Hi Allesandro,

I need the following output from you,

1. After you execute ganesha.enable on command.
ps aux | grep ganesha
showmount -e localhost
cat /etc/ganesha/ganesha.conf
cat "/etc/ganesha/exports/export.VOLNAME.conf" or "/usr/..../export.VOLNAME.conf"
tail /var/log/ganesha.log or wherever the ganesha log is.

2. gluster vol set VOLNAME ganesha.enable off

Output of the same as above.

Thanks for your perseverance :)

Meghana

----- Original Message -----
From: "Alessandro De Salvo" <Alessandro.DeSalvo at roma1.infn.it>
To: "Meghana Madhusudhan" <mmadhusu at redhat.com>
Cc: gluster-users at gluster.org, nfs-ganesha-devel at lists.sourceforge.net, "Soumya Koduri" <skoduri at redhat.com>
Sent: Thursday, June 18, 2015 7:24:55 PM
Subject: Re: [Nfs-ganesha-devel] Problems in /usr/libexec/ganesha/dbus-send.sh and ganesha dbus interface when disabling exports from gluster

Hi Meghana,

> Il giorno 18/giu/2015, alle ore 07:04, Meghana Madhusudhan <mmadhusu at redhat.com> ha scritto:
> 
> 
> 
> 
> On 06/17/2015 10:57 PM, Alessandro De Salvo wrote:
>> Hi,
>> when disabling exports from gluster 3.7.1, by using gluster vol set <volume> ganesha.enable off, I always get the following error:
>> 
>> Error: Dynamic export addition/deletion failed. Please see log file for details
>> 
>> This message is produced by the failure of /usr/libexec/ganesha/dbus-send.sh, and in fact if I manually perform the command to remove the share I see:
> you got it wrong. '/usr/libexec/ganesha/dbus-send.sh' is used by 
> Gluster-CLI to unexport the volume "gluster volume set <volname> 
> ganesha.enable off" which rightly deletes the export file too while 
> un-exporting the volume.
> 
>> 
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>> Error org.freedesktop.DBus.Error.NoReply: Message did not receive a reply (timeout by message bus)
>> 
>> So, there is a timeout and it fails completely.
> Check if nfs-ganesha is still running. There was a bug in unexporting 
> the volume. Its been fixed recently in V2.3-dev, yet to be back-ported 
> to V2.2-stable branch.
> https://review.gerrithub.io/#/c/236129/
> 
> Thanks,
> Soumya
> 
>> In this case I think there is a bug in /usr/libexec/ganesha/dbus-send.sh, since it blindly deletes the share config if the RemoveExport fails (function check_cmd_status()), but leaves the %include inside ganesha.conf as in the check_cmd_status() there is a runaway condition and the other removal statements are then not executed. I believe the logic should be fixed here, otherwise even a restart of the service will fail due to the bad configuration.
> 
> Yes. I see that the "sed -i /$VOL.conf/d $CONF" is placed after the check_cmd_status. I shall send a fix upstream in a related bug. But dynamic export removal
> will fail in three cases, 
> 1. nfs-ganesha is not running.

no, it was running

> 2. The export file that is particular to that volume is somehow deleted before you perform the removal. It does depend on that file to get the export ID.

I tried to comment out the rm in check_cmd_status to avoid this race condition, but it did not solve the problems.

> 3. The bug that Soumya pointed out. 

This might well be the real cause!

> 
> If it is failing consistently, there could be something that you are missing. If you can send the exact sequence of sequence of steps that you have executed,
> I can help you with it.

Yes, it’s failing consistently, unless as I said I do a DisplayExport before the RemoveExport, in which case it always works.

> 
> Ideally after exporting a particular volume, you'll see an entry in the /etc/ganesha/ganesha.conf file and the export file in "/etc/ganesha/exports" dir.

And this works perfectly, I see them correctly.

> If you have this in place and nfs-ganesha running, then dynamic export removal should work just fine.

But this is not, at least in my case.
The command I’m using are just the following:

gluster vol set <volume> ganesha.enable on
gluster vol set <volume> ganesha.enable off

It normally wait a few seconds between the two commands, to give time to ganesha to actually export the volume.
The export is always failing as described, unless I add the DisplayExport in dbus-send.sh before RemoveExport.
Many thanks for the help,

	Alessandro

> 
> Meghana
> 
> 
>> 
>> What’s more worrying is the problem with the dbus. Issuing a DisplayExport before the RemoveExport apparently fixes the problem, so something like this always works:
>> 
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:2
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>> 
>> So, it’s like the DisplayExport is forcing someway a refresh that is needed by the RemoveExport. Any idea why?
>> I’m using the latest version of ganesha 2.2.0, i.e. 2.2.0-3.
>> Thanks,
>> 
>> 	Alessandro
>> 
>> PS: sorry for reporting so many things in a few days :-)
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> 
>> 
>> 
>> _______________________________________________
>> Nfs-ganesha-devel mailing list
>> Nfs-ganesha-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>> 
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
> 