[Gluster-devel] Brick is Offline

ABHISHEK PALIWAL abhishpaliwal at gmail.com
Fri Apr 1 10:53:18 UTC 2016


Please check the cmd-history file in attached logs of the Board A in which
we are removing-brick at 11:56:48 because at this time after waiting for 1
minute brick was not online.

So, We need to identify near to this time why brick is not online.

Because If brick will be online we will not remove-brick and if we remove
brick, volume become distributed which will also cause problem in our case.

Regards,
Abhishek

On Fri, Apr 1, 2016 at 4:08 PM, Atin Mukherjee <amukherj at redhat.com> wrote:

>
>
> On 04/01/2016 03:57 PM, ABHISHEK PALIWAL wrote:
> >
> >
> > On Fri, Apr 1, 2016 at 3:39 PM, Atin Mukherjee <amukherj at redhat.com
> > <mailto:amukherj at redhat.com>> wrote:
> >
> >
> >
> >     On 04/01/2016 03:06 PM, ABHISHEK PALIWAL wrote:
> >     >
> >     > On Fri, Apr 1, 2016 at 2:59 PM, Atin Mukherjee <
> amukherj at redhat.com <mailto:amukherj at redhat.com>
> >     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>> wrote:
> >     >
> >     >
> >     >
> >     >     On 04/01/2016 02:55 PM, ABHISHEK PALIWAL wrote:
> >     >     > Hi Atin,
> >     >     >
> >     >     > Thanks for reply.
> >     >     >
> >     >     > Could you please help me to identify the error log in the
> respective
> >     >     > brick log file. I tried but not able to identified where the
> problem is
> >     >     > occuring.
> >     >     >
> >     >     > I am attaching the brick log file which is not coming online
> even after
> >     >     > waiting for 1 minute.
> >     >     What time did you reboot B? Could you also attach glusterd log
> file
> >     >     (complete log) for board B?
> >     >     >
> >     >
> >     > it is hard to say at what time we rebooted the B board because we
> are
> >     > continuously rebooting the board B.
> >     >
> >     > Here I am attaching the glusterd and glsuterfs log for board B.
> >     I can see the last restart of glusterd was at 13:11:27 and as per the
> >     brick log there was no restart. This doesn't look like a reboot of
> the
> >     board as in that case brick process should have also died and
> restarted.
> >     Brick log indicates that the process is still running and there is no
> >     interruption.
> >
> > In brick log file at 13:11:27 we have the logs showing reboot of board
> >
> > [2016-03-31 13:11:27.408512] I [MSGID: 115036]
> > [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting
> > connection from
> > 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
> > [2016-03-31 13:11:27.408603] I [MSGID: 101055]
> > [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down
> > connection
> > 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
> FYI..this log indicates client disconnection.
> >
> >     >
> >     >     > Regards,
> >     >     > Abhishek
> >     >     >
> >     >     > On Fri, Apr 1, 2016 at 1:04 PM, Atin Mukherjee <
> amukherj at redhat.com <mailto:amukherj at redhat.com>
> >     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>
> >     >     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>
> >     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>>> wrote:
> >     >     >
> >     >     >
> >     >     >
> >     >     >     On 04/01/2016 12:10 PM, ABHISHEK PALIWAL wrote:
> >     >     >     > Hi,
> >     >     >     >
> >     >     >     >
> >     >     >     > I have the setup of two boards A and B with two bricks
> in
> >     >     replica mode.
> >     >     >     >
> >     >     >     > There is one test scenario
> >     >     >     >
> >     >     >     > 1. A acts as an active board and having the glusterfs
> >     mount
> >     >     point on it.
> >     >     >     > 2. B acts as Passive board.
> >     >     >     > 3. We are repetitively rebooting the B board (In this
> time
> >     >     period peer
> >     >     >     > status on A board will be "peer in cluster
> (Disconnected)"
> >     >     and brick is
> >     >     >     > not present in "gluster volume status") and when Board
> B
> >     >     comes up,
> >     >     >     > starts the gluster daemon.
> >     >     >     > 4. if Gluster daemon starts successfully it will make
> >     "peer in
> >     >     >     > cluster(Connected)"
> >     >     >     > 5. At the same with the immediate effect "gluster
> volume
> >     >     status" command
> >     >     >     > should show the brick is available in online.
> >     >     >     >
> >     >     >     >
> >     >     >     > But in my case sometime step 5 takes immediate
> reflection
> >     >     sometime 10-15
> >     >     >     > second and sometime doesn't show brick is online even
> >     after
> >     >     the 1minute.
> >     >     >     >
> >     >     >     > Could you please confirm why this type of unpredictable
> >     >     behavior is
> >     >     >     > occuring. It should be reflect with immediate effect in
> >     >     "gluster volume
> >     >     >     > status" command.
> >     >     >     When glusterd restarts bricks processes are brought up
> >     >     asynchronously
> >     >     >     and hence you may not see the brick processes reflecting
> in
> >     >     gluster
> >     >     >     volume status output immediately after restart. 5-10
> >     seconds is an
> >     >     >     accepted time frame. However if it doesn't come back
> online
> >     >     post that
> >     >     >     then probably brick fails to start in that case.
> >     >     >
> >     >     >     Please check the respective brick log file and if you
> >     can find
> >     >     any error
> >     >     >     logs in it.
> >     >     >     >
> >     >     >     >
> >     >     >     > --
> >     >     >     > Regards
> >     >     >     > Abhishek Paliwal
> >     >     >     >
> >     >     >     >
> >     >     >     > _______________________________________________
> >     >     >     > Gluster-devel mailing list
> >     >     >     > Gluster-devel at gluster.org
> >     <mailto:Gluster-devel at gluster.org> <mailto:Gluster-devel at gluster.org
> >     <mailto:Gluster-devel at gluster.org>>
> >     >     <mailto:Gluster-devel at gluster.org
> >     <mailto:Gluster-devel at gluster.org> <mailto:Gluster-devel at gluster.org
> >     <mailto:Gluster-devel at gluster.org>>>
> >     >     >     > http://www.gluster.org/mailman/listinfo/gluster-devel
> >     >     >     >
> >     >     >
> >     >     >
> >     >     >
> >     >     >
> >     >     > --
> >     >     >
> >     >     >
> >     >     >
> >     >     >
> >     >     > Regards
> >     >     > Abhishek Paliwal
> >     >
> >     >
> >
> >
> >
> >
> > --
> >
> >
> >
> >
> > Regards
> > Abhishek Paliwal
>



-- 




Regards
Abhishek Paliwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160401/839d54fd/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Board A logs.tar.gz
Type: application/x-gzip
Size: 45433 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160401/839d54fd/attachment-0001.gz>


More information about the Gluster-devel mailing list