[Gluster-users] User-serviceable snapshots design

[Anand Subramanian]

Answers inline.


On 05/07/2014 10:52 PM, Jeff Darcy wrote:
>> Attached is a basic write-up of the user-serviceable snapshot feature
>> design (Avati's). Please take a look and let us know if you have
>> questions of any sort...
> A few.
>
> The design creates a new type of daemon: snapview-server.
>
> * Where is it started?  One server (selected how) or all?
snapview-server is the same of the server side xlator. snapd is the 
(glusterfsd) daemon that when started and is running, looks like this:
  /usr/local/sbin/glusterfsd -s localhost --volfile-id snapd/vol2 -p 
/var/lib/glusterd/vols/vol2/run/snapd.pid -l 
/usr/local/var/log/glusterfs/snapd.log --xlator-option 
*-posix.glusterd-uuid=bd3b0111-33db-499c-8497-f455db729394 --brick-name 
vol2-server -S /var/run/2301b86236cdb9bd09c7f3988ac5c29f.socket 
--brick-port 49164 --xlator-option vol2-server.listen-port=49164

where vol2 is the name of the glusterfs vol in question.

The snapd is started by nodesvc start (as of today) and is started when 
the vol is started. It includes the protocol-server, io-threads and the 
snapview-server xlators.

>
> * How do clients find it?  Are we dynamically changing the client
>    side graph to add new protocol/client instances pointing to new
>    snapview-servers, or is snapview-client using RPC directly?  Are
>    the snapview-server ports managed through the glusterd portmapper
>    interface, or patched in some other way?

As Varun mentioned in his response, snapd gets a port through the 
glusterd pmap_registry calls. Client side xlator as usual does the pmap 
client query to find out the port it needs to connect to. Nothing 
different from the norm here.

>
> * Since a snap volume will refer to multiple bricks, we'll need
>    more brick daemons as well.  How are *those* managed?

This is infra handled by the "core" snapshot functionality/feature. When 
a snap is created, it is treated not only as a lvm2 thin-lv but as a 
glusterfs volume as well. The snap volume is activated and mounted and 
made available for regular use through the native fuse-protocol client. 
Management of these is not part of the USS feature. But handled as part 
of the core snapshot implementation. USS (mainly snapview-server xlator) 
talks to the snapshot volumes (and hence the bricks) through the glfs_t 
*, and passing a glfs_object pointer. Each of these glfs instances 
represents the gfapi world for an individual snapshotted volume - 
accessing any of the snap vol bricks etc. is handled within the gfapi 
world. So to that extent, snapview-server xlator is yet another consumer 
of the handle-based gfapi calls like nfs-ganesha.

> * How does snapview-server manage user credentials for connecting
>    to snap bricks?  What if multiple users try to use the same
>    snapshot at the same time?  How does any of this interact with
>    on-wire or on-disk encryption?

No interaction with on-disk or on-wire encryption. Multiple users can 
always access the same snapshot (volume) at the same time. Why do you 
see any restrictions there? Maybe you want to know if userA can access 
the .snaps directory of another user userB? Today the credentials are 
passed as is ie. snapview-server does not play around with user-creds. 
If the snapshot volume access allows it, it goes through. Let me get 
back with more details on this one and thanks for bringing it up.

Remember that the snapshot vol is read-only. And snapview-server has 
many glfs_t *  pointers, each one pointing to each of the snapshot 
volumes (through the gfapi world).

>
> I'm sure I'll come up with more later.  Also, next time it might
> be nice to use the upstream feature proposal template *as it was
> designed* to make sure that questions like these get addressed
> where the whole community can participate in a timely fashion.
Umm...this wasn't exactly a feature "proposal" was it ;-) ? But that 
said, point taken. Will do that. Next time.

Please do send all questions you come up with and thanks for these. 
Certainly helped clarify several imp points here.


Anand