[Gluster-devel] Corner-Case during snapshot handshake.

[Avra Sengupta]

On 04/10/2014 08:57 PM, Atin Mukherjee wrote:
>
> On 04/10/2014 08:05 PM, Avra Sengupta wrote:
>> *Re-formatting*
>>
>> Hi,
>>
>> In a cluster of 5 nodes a volumes V1 is present, which hosts 4 bricks on
>> nodes 1 through 4
>>
>>        N1       N2      N3       N4       N5
>>      ------   ------  ------   ------   ------
>> V1    N1       N2      N3       N4       X
>>
>> Now a Snapshot is taken S1 and snap volfiles will be created on nodes
>>
>>        N1       N2       N3       N4       N5
>>      ------   ------   ------   ------   ------
>> S1   S1VOL    S1VOL    S1VOL    S1VOL    S1VOL
>>
>> At this point N5 goes down, and after that we issue a delete on S1. The
>> delete will be successful, and will not log any missed entries because
>> all nodes hosting snap bricks are up and we delete the snap volfiles in
>> those nodes
>>
>>        N1       N2       N3       N4       N5
>>      ------   ------   ------   ------   ------
>> S1    X        X        X        X     S1VOL(old)

It's a typo, my bad. The S1 Vol files won't be deleted in N5, coz during the delete operation
N5 was down. So with the new snap create(of the same name S1), the rest of the nodes will have
a new snapshot(and respective volume files), leading to the condition below.

>> At this point we create another snap with the name S1, but it will be a
>> different snap with a different snap-uuid
>>
>>        N1       N2       N3       N4       N5
>>      ------   ------   ------   ------   ------
>> S1   S1VOL    S1VOL    S1VOL    S1VOL    S1VOL(old)
> I may not have the complete sketch of snapshot design in my mind,
> however just wondering why would N5 have an old S1VOL as you have
> mentioned that its been already deleted. Am I missing any trick here?
> Once I get this clarity, then only the further paragraph makes sense for
> reading :)
> --Atin
>> At this point N5 node comes up, and during handshake it receives S1 snap
>> volume details from other node, which are for a different snap(have
>> different snap-id).
>>
>> The challenge is to make the node aware of the right snap vol, without
>> resorting to a peer reject. This scenario will occur only with a node,
>> that is not hosting a snap-brick for the said snapshot because the
>> missed_snap_list keeps track of the missed creates, deletes and
>> restores, and will handle if any brick misses any of these ops.
>>
>> What I propose is for the node to check if the incoming new Snap ID
>> features in the missed snap-deletes list for the node it's handshaking
>> with. If so, then its a no-op. If not, it should check, if any
>> snap-bricks for the said snap-volume are hosted in the peer, that it's
>> handshaking with. If no, then again it will be a no-op. Only in a case
>> where the node it is handshaking with, has no missed snap-deletes for
>> the particular snap-id and is hosting a brick for the same, is when it
>> will take precendence over the current node(N5). and the S1VOL files
>> will be overwriten.
>>
>> So the flowchart for any peer having a name conflict during handshake
>> will be.
>>
>> Step 1: Am I hosting any snap-bricks for the conflicted snap. If yes
>> goto step 5
>>
>> Step 2: Is the node I am handshaking with, has any missed deletes for
>> the same snap-id. If yes then goto step 5
>>
>> Step 3: Is the node I am handshaking with, hosting any snap-bricks for
>> the conflicted snap, If no, then goto step 5
>>
>> Step 4: The other node takes precedence, and I should delte my snap-vol
>> files. Move on to the next snap
>>
>> Step 5. It's a no-op. Move on to the next snap.
>>
>> Regards,
>> Avra
>>
>> On 04/10/2014 07:52 PM, Avra Sengupta wrote:
>>> Hi,
>>>
>>> In a cluster of 5 nodes a volumes V1 is present, which hosts 4 bricks
>>> on nodes 1 through 4
>>>
>>>               N1               N2               N3 N4                  N5
>>>            ---------       -----------        --------
>>> ------------          ---------
>>> V1         B1               B2               B3 B4                   X
>>>
>>> Now a Snapshot is taken S1 and snap volfiles will be created on nodes
>>>
>>>               N1               N2               N3 N4                  N5
>>>            ---------       -----------        --------
>>> ------------          ---------
>>> S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL
>>>
>>> At this point N5 goes down, and after that we issue a delete on S1.
>>> The delete will be successful, and will not log any missed entries
>>> because all nodes hosting snap bricks are up and we delete the snap
>>> volfiles in those nodes
>>>
>>>               N1               N2               N3 N4                  N5
>>>            ---------       -----------        --------
>>> ------------          ---------
>>> S1          X                  X                 X X
>>> S1VOL
>>>
>>> At this point we create another snap with the name S1, but it will be
>>> a different snap with a different snap-uuid
>>>
>>>               N1               N2               N3 N4                  N5
>>>            ---------       -----------        --------
>>> ------------          ---------
>>> S1       S1VOL          S1VOL        S1VOL         S1VOL S1VOL(old)
>>>
>>> At this point N5 node comes up, and during handshake it receives S1
>>> snap volume details from other node, which are for a different
>>> snap(have different snap-id).
>>>
>>> The challenge is to make the node aware of the right snap vol, without
>>> resorting to a peer reject. This scenario will occur only with a node,
>>> that is not hosting a snap-brick for the said snapshot because the
>>> missed_snap_list keeps track of the missed creates, deletes and
>>> restores, and will handle if any brick misses any of these ops.
>>>
>>> What I propose is for the node to check if the incoming new Snap ID
>>> features in the missed snap-deletes list for the node it's handshaking
>>> with. If so, then its a no-op. If not, it should check, if any
>>> snap-bricks for the said snap-volume are hosted in the peer, that it's
>>> handshaking with. If no, then again it will be a no-op. Only in a case
>>> where the node it is handshaking with, has no missed snap-deletes for
>>> the particular snap-id and is hosting a brick for the same, is when it
>>> will take precendence over the current node(N5). and the S1VOL files
>>> will be overwriten.
>>>
>>>
>>> So the flowchart for any peer having a name conflict during handshake
>>> will be.
>>>
>>> Step 1: Am I hosting any snap-bricks for the conflicted snap. If yes
>>> goto step 5
>>>
>>> Step 2: Is the node I am handshaking with, has any missed deletes for
>>> the same snap-id. If yes then goto step 5
>>>
>>> Step 3: Is the node I am handshaking with, hosting any snap-bricks for
>>> the conflicted snap, If no, then goto step 5
>>>
>>> Step 4: The other node takes precedence, and I should delte my
>>> snap-vol files. Move on to the next snap
>>>
>>> Step 5. It's a no-op. Move on to the next snap.
>>>
>>> Regards,
>>> Avra
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at nongnu.org
>>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>