[Gluster-devel] Corner-Case during snapshot handshake.

[Avra Sengupta]

*Re-formatting*

Hi,

In a cluster of 5 nodes a volumes V1 is present, which hosts 4 bricks on
nodes 1 through 4

       N1       N2      N3       N4       N5
     ------   ------  ------   ------   ------
V1    N1       N2      N3       N4       X

Now a Snapshot is taken S1 and snap volfiles will be created on nodes

       N1       N2       N3       N4       N5
     ------   ------   ------   ------   ------
S1   S1VOL    S1VOL    S1VOL    S1VOL    S1VOL

At this point N5 goes down, and after that we issue a delete on S1. The
delete will be successful, and will not log any missed entries because
all nodes hosting snap bricks are up and we delete the snap volfiles in
those nodes

       N1       N2       N3       N4       N5
     ------   ------   ------   ------   ------
S1    X        X        X        X        X

At this point we create another snap with the name S1, but it will be a
different snap with a different snap-uuid

       N1       N2       N3       N4       N5
     ------   ------   ------   ------   ------
S1   S1VOL    S1VOL    S1VOL    S1VOL    S1VOL(old)

At this point N5 node comes up, and during handshake it receives S1 snap
volume details from other node, which are for a different snap(have
different snap-id).

The challenge is to make the node aware of the right snap vol, without
resorting to a peer reject. This scenario will occur only with a node,
that is not hosting a snap-brick for the said snapshot because the
missed_snap_list keeps track of the missed creates, deletes and
restores, and will handle if any brick misses any of these ops.

What I propose is for the node to check if the incoming new Snap ID
features in the missed snap-deletes list for the node it's handshaking
with. If so, then its a no-op. If not, it should check, if any
snap-bricks for the said snap-volume are hosted in the peer, that it's
handshaking with. If no, then again it will be a no-op. Only in a case
where the node it is handshaking with, has no missed snap-deletes for
the particular snap-id and is hosting a brick for the same, is when it
will take precendence over the current node(N5). and the S1VOL files
will be overwriten.

So the flowchart for any peer having a name conflict during handshake
will be.

Step 1: Am I hosting any snap-bricks for the conflicted snap. If yes
goto step 5

Step 2: Is the node I am handshaking with, has any missed deletes for
the same snap-id. If yes then goto step 5

Step 3: Is the node I am handshaking with, hosting any snap-bricks for
the conflicted snap, If no, then goto step 5

Step 4: The other node takes precedence, and I should delte my snap-vol
files. Move on to the next snap

Step 5. It's a no-op. Move on to the next snap.

Regards,
Avra

On 04/10/2014 07:52 PM, Avra Sengupta wrote:
> Hi,
>
> In a cluster of 5 nodes a volumes V1 is present, which hosts 4 bricks 
> on nodes 1 through 4
>
>              N1               N2               N3 N4                  N5
>           ---------       -----------        -------- 
> ------------          ---------
> V1         B1               B2               B3 B4                   X
>
> Now a Snapshot is taken S1 and snap volfiles will be created on nodes
>
>              N1               N2               N3 N4                  N5
>           ---------       -----------        -------- 
> ------------          ---------
> S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL
>
> At this point N5 goes down, and after that we issue a delete on S1. 
> The delete will be successful, and will not log any missed entries 
> because all nodes hosting snap bricks are up and we delete the snap 
> volfiles in those nodes
>
>              N1               N2               N3 N4                  N5
>           ---------       -----------        -------- 
> ------------          ---------
> S1          X                  X                 X X                 
> S1VOL
>
> At this point we create another snap with the name S1, but it will be 
> a different snap with a different snap-uuid
>
>              N1               N2               N3 N4                  N5
>           ---------       -----------        -------- 
> ------------          ---------
> S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL(old)
>
> At this point N5 node comes up, and during handshake it receives S1 
> snap volume details from other node, which are for a different 
> snap(have different snap-id).
>
> The challenge is to make the node aware of the right snap vol, without 
> resorting to a peer reject. This scenario will occur only with a node, 
> that is not hosting a snap-brick for the said snapshot because the 
> missed_snap_list keeps track of the missed creates, deletes and 
> restores, and will handle if any brick misses any of these ops.
>
> What I propose is for the node to check if the incoming new Snap ID 
> features in the missed snap-deletes list for the node it's handshaking 
> with. If so, then its a no-op. If not, it should check, if any 
> snap-bricks for the said snap-volume are hosted in the peer, that it's 
> handshaking with. If no, then again it will be a no-op. Only in a case 
> where the node it is handshaking with, has no missed snap-deletes for 
> the particular snap-id and is hosting a brick for the same, is when it 
> will take precendence over the current node(N5). and the S1VOL files 
> will be overwriten.
>
>
> So the flowchart for any peer having a name conflict during handshake 
> will be.
>
> Step 1: Am I hosting any snap-bricks for the conflicted snap. If yes 
> goto step 5
>
> Step 2: Is the node I am handshaking with, has any missed deletes for 
> the same snap-id. If yes then goto step 5
>
> Step 3: Is the node I am handshaking with, hosting any snap-bricks for 
> the conflicted snap, If no, then goto step 5
>
> Step 4: The other node takes precedence, and I should delte my 
> snap-vol files. Move on to the next snap
>
> Step 5. It's a no-op. Move on to the next snap.
>
> Regards,
> Avra
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140410/d8fb726c/attachment-0001.html>