[Gluster-devel] Corner-Case during snapshot handshake.

[Avra Sengupta]

Hi,

In a cluster of 5 nodes a volumes V1 is present, which hosts 4 bricks on 
nodes 1 through 4

              N1               N2               N3 N4                  N5
           ---------       -----------        -------- 
------------          ---------
V1         B1               B2               B3 B4                   X

Now a Snapshot is taken S1 and snap volfiles will be created on nodes

              N1               N2               N3 N4                  N5
           ---------       -----------        -------- 
------------          ---------
S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL

At this point N5 goes down, and after that we issue a delete on S1. The 
delete will be successful, and will not log any missed entries because 
all nodes hosting snap bricks are up and we delete the snap volfiles in 
those nodes

              N1               N2               N3 N4                  N5
           ---------       -----------        -------- 
------------          ---------
S1          X                  X                 X X                 S1VOL

At this point we create another snap with the name S1, but it will be a 
different snap with a different snap-uuid

              N1               N2               N3 N4                  N5
           ---------       -----------        -------- 
------------          ---------
S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL(old)

At this point N5 node comes up, and during handshake it receives S1 snap 
volume details from other node, which are for a different snap(have 
different snap-id).

The challenge is to make the node aware of the right snap vol, without 
resorting to a peer reject. This scenario will occur only with a node, 
that is not hosting a snap-brick for the said snapshot because the 
missed_snap_list keeps track of the missed creates, deletes and 
restores, and will handle if any brick misses any of these ops.

What I propose is for the node to check if the incoming new Snap ID 
features in the missed snap-deletes list for the node it's handshaking 
with. If so, then its a no-op. If not, it should check, if any 
snap-bricks for the said snap-volume are hosted in the peer, that it's 
handshaking with. If no, then again it will be a no-op. Only in a case 
where the node it is handshaking with, has no missed snap-deletes for 
the particular snap-id and is hosting a brick for the same, is when it 
will take precendence over the current node(N5). and the S1VOL files 
will be overwriten.


So the flowchart for any peer having a name conflict during handshake 
will be.

Step 1: Am I hosting any snap-bricks for the conflicted snap. If yes 
goto step 5

Step 2: Is the node I am handshaking with, has any missed deletes for 
the same snap-id. If yes then goto step 5

Step 3: Is the node I am handshaking with, hosting any snap-bricks for 
the conflicted snap, If no, then goto step 5

Step 4: The other node takes precedence, and I should delte my snap-vol 
files. Move on to the next snap

Step 5. It's a no-op. Move on to the next snap.

Regards,
Avra