[Gluster-users] issues recovering machine in gluster

[Atin Mukherjee]

On 06/15/2016 12:14 PM, Arif Ali wrote:
> 
> On 15 June 2016 at 06:48, Atin Mukherjee <amukherj at redhat.com
> <mailto:amukherj at redhat.com>> wrote:
> 
> 
> 
>     On 06/15/2016 11:06 AM, Gandalf Corvotempesta wrote:
>     > Il 15 giu 2016 07:09, "Atin Mukherjee" <amukherj at redhat.com <mailto:amukherj at redhat.com>
>     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>> ha scritto:
>     >> To get rid of this situation you'd need to stop all the running glusterd
>     >> instances and go into /var/lib/glusterd/peers folder on all the nodes
>     >> and manually correct the UUID file names and their content if required.
>     >
>     > If i understood properly the only way to fix this is by bringing the
>     > whole cluster down? "you'd need to stop all the running glusterd instances"
>     >
>     > I hope you are referring to all instances on the failed node...
> 
>     No, since the configuration are synced across all the nodes, any
>     incorrect data gets replicated through out. So in this case to be on the
>     safer side and validate the correctness all glusterd instances on *all*
>     the nodes should be brought down. Having said that, this doesn't impact
>     I/O as the management path is different than I/O.
> 
> 
> As a sanity, one of the things I did last night, was to reboot the whole
> gluster system, when I had downtime arranged. I thought this is
> something would be asked, as I had seen similar requests on the mailing
> list previously
> 
> Unfortunately though, it didn't fix the problem.

Only reboot is not going to solve the problem. You'd need to correct the
configuration as I explained earlier in this thread. If it doesn't
please send the me the content of /var/lib/glusterd/peers/ &
/var/lib/glusterd/glusterd.info file from all the nodes where glusterd
instances are running. I'll take a look and correct them and send it
back to you.

> 
> Any other suggestions are welcome
>