[Gluster-users] gluster volume heal info split brain command not showing files in split-brain

[Anuradha Talur]

Hi Abhishek,

Apologies for the delay. I was not in office from March 10th till 14th, still catching up with the mails.
I'm going through the logs. Will respond to this once done.

----- Original Message -----
> From: "ABHISHEK PALIWAL" <abhishpaliwal at gmail.com>
> To: "Anuradha Talur" <atalur at redhat.com>
> Cc: gluster-users at gluster.org, gluster-devel at gluster.org
> Sent: Monday, March 14, 2016 10:31:02 AM
> Subject: Re: [Gluster-users] gluster volume heal info split brain command not showing files in split-brain
> 
> Hi Anuradha,
> 
> Please reply on this issue.
> 
> Regards,
> Abhishek
> 
> On Thu, Mar 10, 2016 at 11:22 AM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
> wrote:
> 
> > Hi Anuradha,
> >
> > Please find the glusterfs and glusterd logs directory as an attachment.
> >
> > Regards,
> > Abhishek
> >
> >
> >
> >
> > On Wed, Mar 9, 2016 at 5:54 PM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
> > wrote:
> >
> >> Hi Anuradha,
> >>
> >> Sorry for late reply.
> >>
> >> Please find the requested logs below:
> >>
> >> Remote: 10.32.0.48
> >> Local : 10.32.1.144
> >>
> >> Local:
> >> #gluster volume heal c_glusterfs info split-brain
> >> Brick 10.32.1.144:/opt/lvmdir/c2/brick
> >> Number of entries in split-brain: 0
> >>
> >> Brick 10.32.0.48:/opt/lvmdir/c2/brick
> >> Number of entries in split-brain: 0
> >>
> >> Remote:
> >> #gluster volume heal c_glusterfs info split-brain
> >> Brick 10.32.1.144:/opt/lvmdir/c2/brick
> >> Number of entries in split-brain: 0
> >>
> >> Brick 10.32.0.48:/opt/lvmdir/c2/brick
> >> Number of entries in split-brain: 0
> >>
> >> auto-sync.sh.
> >> Here you can see that i/o error is detected. Below is the required meta
> >> data from both the bricks.
> >>
> >> 1)
> >> stat: cannot stat '/mnt/c//public_html/cello/ior_files/nameroot.ior':
> >> Input/output error
> >> Remote:
> >>
> >> getfattr -d -m . -e hex
> >> opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> >> # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> >> trusted.afr.dirty=0x000000000000000000000000
> >> trusted.bit-rot.version=0x000000000000000256ded2f6000ad80f
> >> trusted.gfid=0x771221a7bb3c4f1aade40ce9e38a95ee
> >>
> >> Local:
> >>
> >> getfattr -d -m . -e hex
> >> opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> >> # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> >> trusted.bit-rot.version=0x000000000000000256ded38f000e3a51
> >> trusted.gfid=0x8ea33f46703c4e2d95c09153c1b858fd
> >>
> >>
> >> 2)
> >> stat: cannot stat '/mnt/c//security/corbasecurity': Input/output error
> >> Remote:
> >>
> >> getfattr -d -m . -e hex opt/lvmdir/c2/brick/security/corbasecurity
> >> # file: opt/lvmdir/c2/brick/security/corbasecurity
> >> trusted.afr.dirty=0x000000000000000000000000
> >> trusted.bit-rot.version=0x000000000000000256ded2f6000ad80f
> >> trusted.gfid=0xd298b7a0c8834f3e99abb39741363013
> >>
> >> Local:
> >>
> >> getfattr -d -m . -e hex opt/lvmdir/c2/brick/security/corbasecurity
> >> # file: opt/lvmdir/c2/brick/security/corbasecurity
> >> trusted.bit-rot.version=0x000000000000000256ded38f000e3a51
> >> trusted.gfid=0x890df0f706184b52803fac3242a2f15b
> >>
> >> I observed that getfattr command output doesn't show all the fields all
> >> the times.
> >>
> >> Here you can check that gluster split-brain command hasn't reported any
> >> split-brains but resulted in IO errors when accessed few files.
> >>
> >> Could you please tell me if "split-brain" command doesn't reported any
> >> entry as output, then is there any way through which we can find out that
> >> the files are in split-brain if we are getting the IO error on those file.
> >>
> >>
> >> Regards,
> >> Abhishek
> >>
> >>
> >>
> >> On Thu, Mar 3, 2016 at 5:32 PM, Anuradha Talur <atalur at redhat.com> wrote:
> >>
> >>>
> >>>
> >>> ----- Original Message -----
> >>> > From: "ABHISHEK PALIWAL" <abhishpaliwal at gmail.com>
> >>> > To: gluster-users at gluster.org, gluster-devel at gluster.org
> >>> > Sent: Thursday, March 3, 2016 12:10:42 PM
> >>> > Subject: [Gluster-users] gluster volume heal info split brain command
> >>> not     showing files in split-brain
> >>> >
> >>> >
> >>> > Hello,
> >>> >
> >>> > In gluster, we use the command "gluster volume heal c_glusterfs info
> >>> > split-brain" to find the files that are in split-brain scenario.
> >>> > We run heal script (developed by Windriver prime team) on the files
> >>> reported
> >>> > by above command to resolve split-brain issue.
> >>> >
> >>> > But we observed that the above command is not showing all files that
> >>> are in
> >>> > split-brain,
> >>> > even though split brain scenario actually exists on the node.
> >>> >
> >>> > Now a days this issue is seen more often and IO errors are reported
> >>> when
> >>> > tried to access these files under split-brain.
> >>> >
> >>> > Can you please check why this gluster command is not showing files
> >>> under
> >>> > split-brain?
> >>> > We can provide you required logs and support to resolve this issue.
> >>> Hi,
> >>>
> >>> Could you paste the output of getfattr -m. -de hex
> >>> <path-to-files-in-split-brain> from all the bricks that the files lie in?
> >>>
> >>> >
> >>> > Please reply on this because I am not getting any reply from the
> >>> community.
> >>> >
> >>> > --
> >>> >
> >>> > Regards
> >>> > Abhishek Paliwal
> >>> >
> >>> > _______________________________________________
> >>> > Gluster-users mailing list
> >>> > Gluster-users at gluster.org
> >>> > http://www.gluster.org/mailman/listinfo/gluster-users
> >>>
> >>> --
> >>> Thanks,
> >>> Anuradha.
> >>>
> >>
> >
> 

-- 
Thanks,
Anuradha.