[Gluster-devel] Gluster CLI for setting options for glusterd.vol
Deepak C Shetty
deepakcs at linux.vnet.ibm.com
Thu Jun 27 09:00:27 UTC 2013
On 06/27/2013 12:34 PM, Kanagaraj M wrote:
>
>
> On Thu, Jun 27, 2013 at 11:20 AM, Deepak C Shetty
> <deepakcs at linux.vnet.ibm.com <mailto:deepakcs at linux.vnet.ibm.com>> wrote:
>
> On 06/24/2013 06:21 PM, Vijay Bellur wrote:
>
> On 06/20/2013 07:28 PM, M. Mohan Kumar wrote:
>
> Vijay Bellur <vbellur at redhat.com
> <mailto:vbellur at redhat.com>> writes:
>
> On 06/19/2013 09:51 PM, M. Mohan Kumar wrote:
>
> Hello,
>
> When qemu is invoked by a non-root user with -drive
> file=gluster://server/volname/imagename option,
> unprivileged port is
> used for gluster rpc and by default glusterd and
> gluster brick process
> deny the request if the request is from a
> unprivileged port. The option
> "rpc-auth-allow-insecure" needs to be enabled in
> glusterd.vol so that
> non privileged ports can be used to access Gluster
> volumes.
>
> In a typical environment VDSM might want to enable
> rpc-auth-allow-insecure
> option and the administrator has to edit the
> glusterd.vol manually and
> restart glusterd process.
>
> CLI options available to enable volume specific
> options to work with
> unprivileged ports by using gluster volume set
> <volname> <option>
> <value>. For example per volume
> server.allow-insecure option can be
> enabled so that unprivileged users can mount a
> GlusterFS volume.
>
> But as of now there is no CLI option available to
> set glusterd.vol
> options. How about adding a gluster CLI set option
> to configure
> glusterd.vol options? Can following CLI command
> line 'gluster volume set
> all <glusterd.option> <value>" be used for setting
> glusterd options?
> IIUC "all" is a reserved volume name and we can
> use this reserved name
> for setting glusterd option.
>
>
> 'volume set all' is mostly used for options that are
> applicable to all
> volumes. Since glusterd options are beyond the scope
> of a volume, tying
> them to the peer entity might be a good idea. We can
> introduce 'peer set
> all <key> <value>' which sets a particular option on
> all peers.
>
>
> You mean by 'gluster peer set all rpc-auth-allow-insecure
> on' will
> enable insecured port access to all glusterds in the peer
> environment?
>
>
>
> Yes.
>
>
> This still doesn't help the VDSM usecase, when VDSM host ( aka
> hypervisor host ) is not part of gluster peer.
> One of the goal here was to provide a cli way to modify glusterd
> options so that VDSM can exploit it, when Gluster volume is used
> as a storage domain, and VDSM needs rpc-auth-allow-insecure to be
> ON as VMs accessing Gluster volume natively via libgfapi will be
> running as non-root.
>
> On the same lines.. how does oVirt Engine 'Volumes' GUI manage
> Gluster volumes.... when the oVirt host is not part of the Gluster
> peer ? Just wondering....
>
>
> Why oVirt Engine host needs to be a gluster peer to be able to
> communicate with gluster(through vdsm)? As super-vdsm is running as
> root in the gluster node and it should be able to communicate with
> underlying gluster cli and can respond to oVirt engine.
Right, I forgot the fact that gluster clis are executed thru super-vdsm,
so non-priv / insecure issue won't be present. But the VDSM usecase
still is a issue... when VDSM host (hypervisor node) is not a gluster
peer, we need a way to set the glusterd option... currently one has to
do that manually, which is not a good choice.
>
> If not, are you talking about the scenario where gluster node doesn't
> have vdsm installed also the storage cluster is not managed through oVirt.
So, as i see it, there are couple of scenarios.... from VDSM perspective...
For the sake of discussion, lets say *VDSM (aka hypervisor host) is
hostA & Gluster node/host is hostB**.*
*Case 1:* hostA is not a gluster peer. hostB is a gluster peer
(obviously) , part of Gluster volume which is managed by oVirt Engine.
hostA will access Gluster volume as non-root (as part of VM running
on Gluster storage domain, using libgfapi). Since hostA is not a gluster
peer but since the Gluster volume is managed by oVirt, it should be
possible for the engine to first execute 'gluster peer set all
rpc-auth-allow-insecure on the gluster volume, then ask VDSM to create
the gluster storage domain. From the perspective of "User
friendliness", its better to have User click on something like "Allow
insecure access" or "Make this gluster volume work with VDSM Gluster
storage domain" which will execute 'gluster peer set all ..insecure.. '
option, so that User is aware of enabling the insecure option, instead
of engine doing it w/o User's knowledge as part of gluster storage
domain flow.
This should work and for this to happen seamless, the CLI support for
setting glusterd is needed, so this usecase is covered.
*Case 2:* hostA is not a gluster peer. hostB is a gluster peer
(obviously), but the Gluster volume is not managed by oVirt.
The same as Case 1 applies, with the exception that, since the
Gluster volume is not managed by oVirt, engine cannot do 'gluster peer
set all rpc-auth-allow-insecure'. Ther are again 2 sub-cases here ...
*2a)* Have the user import the gluster volume into oVirt, post which it
becomes like Case 1... problem solved.
*2b)* For some reason, user doesn't want to import the gluster volume
into oVirt.. in which case, the only way is for someone to _manually_ do
'gluster peer set all rpc-auth-allow-insecure' from one of the gluster
peers, which is not a good thing since it involved manual intervention.
One can argue that hostA can use --remote-host option and do 'gluster
peer set all...', which will work today, but that would be insecure way
of doing it, since you are allowing a non-peer to set insecure option
for all peers. Also Mohan's another patch
(http://review.gluster.org/#/c/5092/) adds support for IP based access
control, which also disables any 'set' kind of actions using
--remote-host, so --remote-host anyways won't be possible in VDSM once
that patch is thru. Net, there is no way in VDSM to enable glusterd
insecure option for this usecase.
*Case 3: *hostA is a gluster peer. hostB is a gluster peer (obviously) ,
part of Gluster volume which is managed by oVirt Engine.
Since oVirt is managing is gluster volume, the same as Case 1 applies
*
**Case 4: *hostA is a gluster peer. hostB is a gluster peer (obviously)
, but the Gluster volume is not managed by oVirt.
(Someone manually set up the Gluster volume, using hypervisor hosts
as peers, for using the un-used disk space on hyp host for storage)
This is similar to Case 2, but since hostA and hostB are part of
gluster peer, VDSM can do 'gluster peer set all ...insecure..' before
creating storage domain. Problem happens, when new hosts are added to
the datacentre which are not part of gluster peer. Note in a virt env.
hosts can be dynamically added / remove. So when a VM backed by gluster
storage domain migrates to a host that is not a gluster peer, this
usecase is broken like in Case 2b
*To Summarize:*
1) 'gluster peer set all ...insecure...' cli option will definitely help
and it will help more if the oVirt Volumes tab provides an UI way of
invoking this on User request. This will cover some usecases as desc above
2) Case 2b & 4 - how to handle this scenario ?
thanx,
deepak
>
> Thanks,
> Kanagaraj
>
> thanx,
> deepak
>
>
>
>
> IIUC glusterd.info <http://glusterd.info> file can
> be used to store about these parameters
> similar to how volume specific options are stored
> in vols/<volname>/info
> file?
>
>
> We can persist this in glusterd.vol referred by the
> respective glusterd
> instance.
>
>
> So glusterd.vol is not [re]generated during glusterd init?
>
>
>
> No, glusterd.vol does not get altered during init.
>
> -Vijay
>
>
> -Vijay
>
>
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130627/8f09a6c9/attachment-0001.html>
More information about the Gluster-devel
mailing list