[Gluster-users] Recovering out of sync nodes from input/output error

[Alex Florescu]

On Thu, Apr 12, 2012 at 1:45 AM, Robert Hajime Lanning wrote:

> So, now you have a "split-brain" problem.
>
>
> Which directory is the "source of truth"?
>
> did "howareyou" exist on 10.0.2.15 and was deleted during the outage, or
> is it a new file?
> vice versa for "hello"
>
> So, when you look at the directory itself, which state is correct?
>
> Gluster does not have a transaction log for each brick, to sync
> across.
>

What do you mean by "source of truth"?
howareyou did not exist on 10.0.2.15, it was created only on 10.0.2.14 and
ideally it should have been replicated to .15 when connectivity recovered.
Actually, both states are correct. Imagine /a is the document root of a
website deployed on 2 servers. By using DNS round robin, one request was
balanced to 10.0.2.14 and created the file howareyou, and the second
request was balanced to 10.0.2.15 and created the file hello. If gluster
wasn't having connectivity issues, everything would have been fine and the
files would have been replicated among the 2 servers. But gluster was
having connectivity issues at that moment, while the other services
(apache) were not, and when gluster connectivity recovered the split-brain
occurred.

Tar replica1 and untar on replica2.  Then delete everything on replica1.
> Then self-heal should take care of the rest.
>

This does not have any effect. As I cannot work anymore with the mountpoint
/a, I am left with modifying only the local directory, named by me /local.
Error persists.

On 04/11/12 04:00, Alex Florescu wrote:

> The only way to recover this was to delete the offending files. This was
> easy to do on the test environment because there were two files involved
> [...]
>

I was mistaken here. Even by doing this I still get "Input/Output error".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120412/9898bfbf/attachment.html>