[Gluster-users] [Gluster-devel] Brick is Offline

ABHISHEK PALIWAL abhishpaliwal at gmail.com
Mon Apr 4 10:24:50 UTC 2016


Hi,

If brick fails to become online then what is the solution to make it online?

Regards,
Abhishek

On Fri, Apr 1, 2016 at 4:23 PM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com>
wrote:

> Please check the cmd-history file in attached logs of the Board A in which
> we are removing-brick at 11:56:48 because at this time after waiting for 1
> minute brick was not online.
>
> So, We need to identify near to this time why brick is not online.
>
> Because If brick will be online we will not remove-brick and if we remove
> brick, volume become distributed which will also cause problem in our case.
>
> Regards,
> Abhishek
>
> On Fri, Apr 1, 2016 at 4:08 PM, Atin Mukherjee <amukherj at redhat.com>
> wrote:
>
>>
>>
>> On 04/01/2016 03:57 PM, ABHISHEK PALIWAL wrote:
>> >
>> >
>> > On Fri, Apr 1, 2016 at 3:39 PM, Atin Mukherjee <amukherj at redhat.com
>> > <mailto:amukherj at redhat.com>> wrote:
>> >
>> >
>> >
>> >     On 04/01/2016 03:06 PM, ABHISHEK PALIWAL wrote:
>> >     >
>> >     > On Fri, Apr 1, 2016 at 2:59 PM, Atin Mukherjee <
>> amukherj at redhat.com <mailto:amukherj at redhat.com>
>> >     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>> wrote:
>> >     >
>> >     >
>> >     >
>> >     >     On 04/01/2016 02:55 PM, ABHISHEK PALIWAL wrote:
>> >     >     > Hi Atin,
>> >     >     >
>> >     >     > Thanks for reply.
>> >     >     >
>> >     >     > Could you please help me to identify the error log in the
>> respective
>> >     >     > brick log file. I tried but not able to identified where
>> the problem is
>> >     >     > occuring.
>> >     >     >
>> >     >     > I am attaching the brick log file which is not coming
>> online even after
>> >     >     > waiting for 1 minute.
>> >     >     What time did you reboot B? Could you also attach glusterd
>> log file
>> >     >     (complete log) for board B?
>> >     >     >
>> >     >
>> >     > it is hard to say at what time we rebooted the B board because we
>> are
>> >     > continuously rebooting the board B.
>> >     >
>> >     > Here I am attaching the glusterd and glsuterfs log for board B.
>> >     I can see the last restart of glusterd was at 13:11:27 and as per
>> the
>> >     brick log there was no restart. This doesn't look like a reboot of
>> the
>> >     board as in that case brick process should have also died and
>> restarted.
>> >     Brick log indicates that the process is still running and there is
>> no
>> >     interruption.
>> >
>> > In brick log file at 13:11:27 we have the logs showing reboot of board
>> >
>> > [2016-03-31 13:11:27.408512] I [MSGID: 115036]
>> > [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting
>> > connection from
>> > 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
>> > [2016-03-31 13:11:27.408603] I [MSGID: 101055]
>> > [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down
>> > connection
>> > 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
>> FYI..this log indicates client disconnection.
>> >
>> >     >
>> >     >     > Regards,
>> >     >     > Abhishek
>> >     >     >
>> >     >     > On Fri, Apr 1, 2016 at 1:04 PM, Atin Mukherjee <
>> amukherj at redhat.com <mailto:amukherj at redhat.com>
>> >     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>
>> >     >     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>
>> >     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>>> wrote:
>> >     >     >
>> >     >     >
>> >     >     >
>> >     >     >     On 04/01/2016 12:10 PM, ABHISHEK PALIWAL wrote:
>> >     >     >     > Hi,
>> >     >     >     >
>> >     >     >     >
>> >     >     >     > I have the setup of two boards A and B with two
>> bricks in
>> >     >     replica mode.
>> >     >     >     >
>> >     >     >     > There is one test scenario
>> >     >     >     >
>> >     >     >     > 1. A acts as an active board and having the glusterfs
>> >     mount
>> >     >     point on it.
>> >     >     >     > 2. B acts as Passive board.
>> >     >     >     > 3. We are repetitively rebooting the B board (In this
>> time
>> >     >     period peer
>> >     >     >     > status on A board will be "peer in cluster
>> (Disconnected)"
>> >     >     and brick is
>> >     >     >     > not present in "gluster volume status") and when
>> Board B
>> >     >     comes up,
>> >     >     >     > starts the gluster daemon.
>> >     >     >     > 4. if Gluster daemon starts successfully it will make
>> >     "peer in
>> >     >     >     > cluster(Connected)"
>> >     >     >     > 5. At the same with the immediate effect "gluster
>> volume
>> >     >     status" command
>> >     >     >     > should show the brick is available in online.
>> >     >     >     >
>> >     >     >     >
>> >     >     >     > But in my case sometime step 5 takes immediate
>> reflection
>> >     >     sometime 10-15
>> >     >     >     > second and sometime doesn't show brick is online even
>> >     after
>> >     >     the 1minute.
>> >     >     >     >
>> >     >     >     > Could you please confirm why this type of
>> unpredictable
>> >     >     behavior is
>> >     >     >     > occuring. It should be reflect with immediate effect
>> in
>> >     >     "gluster volume
>> >     >     >     > status" command.
>> >     >     >     When glusterd restarts bricks processes are brought up
>> >     >     asynchronously
>> >     >     >     and hence you may not see the brick processes
>> reflecting in
>> >     >     gluster
>> >     >     >     volume status output immediately after restart. 5-10
>> >     seconds is an
>> >     >     >     accepted time frame. However if it doesn't come back
>> online
>> >     >     post that
>> >     >     >     then probably brick fails to start in that case.
>> >     >     >
>> >     >     >     Please check the respective brick log file and if you
>> >     can find
>> >     >     any error
>> >     >     >     logs in it.
>> >     >     >     >
>> >     >     >     >
>> >     >     >     > --
>> >     >     >     > Regards
>> >     >     >     > Abhishek Paliwal
>> >     >     >     >
>> >     >     >     >
>> >     >     >     > _______________________________________________
>> >     >     >     > Gluster-devel mailing list
>> >     >     >     > Gluster-devel at gluster.org
>> >     <mailto:Gluster-devel at gluster.org> <mailto:
>> Gluster-devel at gluster.org
>> >     <mailto:Gluster-devel at gluster.org>>
>> >     >     <mailto:Gluster-devel at gluster.org
>> >     <mailto:Gluster-devel at gluster.org> <mailto:
>> Gluster-devel at gluster.org
>> >     <mailto:Gluster-devel at gluster.org>>>
>> >     >     >     > http://www.gluster.org/mailman/listinfo/gluster-devel
>> >     >     >     >
>> >     >     >
>> >     >     >
>> >     >     >
>> >     >     >
>> >     >     > --
>> >     >     >
>> >     >     >
>> >     >     >
>> >     >     >
>> >     >     > Regards
>> >     >     > Abhishek Paliwal
>> >     >
>> >     >
>> >
>> >
>> >
>> >
>> > --
>> >
>> >
>> >
>> >
>> > Regards
>> > Abhishek Paliwal
>>
>
>
>
> --
>
>
>
>
> Regards
> Abhishek Paliwal
>



-- 




Regards
Abhishek Paliwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160404/c8c32040/attachment.html>


More information about the Gluster-users mailing list