[Gluster-devel] Brick is Offline

[Atin Mukherjee]

On 04/01/2016 03:57 PM, ABHISHEK PALIWAL wrote:
> 
> 
> On Fri, Apr 1, 2016 at 3:39 PM, Atin Mukherjee <amukherj at redhat.com
> <mailto:amukherj at redhat.com>> wrote:
> 
> 
> 
>     On 04/01/2016 03:06 PM, ABHISHEK PALIWAL wrote:
>     >
>     > On Fri, Apr 1, 2016 at 2:59 PM, Atin Mukherjee <amukherj at redhat.com <mailto:amukherj at redhat.com>
>     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>> wrote:
>     >
>     >
>     >
>     >     On 04/01/2016 02:55 PM, ABHISHEK PALIWAL wrote:
>     >     > Hi Atin,
>     >     >
>     >     > Thanks for reply.
>     >     >
>     >     > Could you please help me to identify the error log in the respective
>     >     > brick log file. I tried but not able to identified where the problem is
>     >     > occuring.
>     >     >
>     >     > I am attaching the brick log file which is not coming online even after
>     >     > waiting for 1 minute.
>     >     What time did you reboot B? Could you also attach glusterd log file
>     >     (complete log) for board B?
>     >     >
>     >
>     > it is hard to say at what time we rebooted the B board because we are
>     > continuously rebooting the board B.
>     >
>     > Here I am attaching the glusterd and glsuterfs log for board B.
>     I can see the last restart of glusterd was at 13:11:27 and as per the
>     brick log there was no restart. This doesn't look like a reboot of the
>     board as in that case brick process should have also died and restarted.
>     Brick log indicates that the process is still running and there is no
>     interruption.
> 
> In brick log file at 13:11:27 we have the logs showing reboot of board
> 
> [2016-03-31 13:11:27.408512] I [MSGID: 115036]
> [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting
> connection from
> 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
> [2016-03-31 13:11:27.408603] I [MSGID: 101055]
> [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down
> connection
> 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0
FYI..this log indicates client disconnection.
> 
>     >
>     >     > Regards,
>     >     > Abhishek
>     >     >
>     >     > On Fri, Apr 1, 2016 at 1:04 PM, Atin Mukherjee <amukherj at redhat.com <mailto:amukherj at redhat.com>
>     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>
>     >     > <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>
>     <mailto:amukherj at redhat.com <mailto:amukherj at redhat.com>>>> wrote:
>     >     >
>     >     >
>     >     >
>     >     >     On 04/01/2016 12:10 PM, ABHISHEK PALIWAL wrote:
>     >     >     > Hi,
>     >     >     >
>     >     >     >
>     >     >     > I have the setup of two boards A and B with two bricks in
>     >     replica mode.
>     >     >     >
>     >     >     > There is one test scenario
>     >     >     >
>     >     >     > 1. A acts as an active board and having the glusterfs
>     mount
>     >     point on it.
>     >     >     > 2. B acts as Passive board.
>     >     >     > 3. We are repetitively rebooting the B board (In this time
>     >     period peer
>     >     >     > status on A board will be "peer in cluster (Disconnected)"
>     >     and brick is
>     >     >     > not present in "gluster volume status") and when Board B
>     >     comes up,
>     >     >     > starts the gluster daemon.
>     >     >     > 4. if Gluster daemon starts successfully it will make
>     "peer in
>     >     >     > cluster(Connected)"
>     >     >     > 5. At the same with the immediate effect "gluster volume
>     >     status" command
>     >     >     > should show the brick is available in online.
>     >     >     >
>     >     >     >
>     >     >     > But in my case sometime step 5 takes immediate reflection
>     >     sometime 10-15
>     >     >     > second and sometime doesn't show brick is online even
>     after
>     >     the 1minute.
>     >     >     >
>     >     >     > Could you please confirm why this type of unpredictable
>     >     behavior is
>     >     >     > occuring. It should be reflect with immediate effect in
>     >     "gluster volume
>     >     >     > status" command.
>     >     >     When glusterd restarts bricks processes are brought up
>     >     asynchronously
>     >     >     and hence you may not see the brick processes reflecting in
>     >     gluster
>     >     >     volume status output immediately after restart. 5-10
>     seconds is an
>     >     >     accepted time frame. However if it doesn't come back online
>     >     post that
>     >     >     then probably brick fails to start in that case.
>     >     >
>     >     >     Please check the respective brick log file and if you
>     can find
>     >     any error
>     >     >     logs in it.
>     >     >     >
>     >     >     >
>     >     >     > --
>     >     >     > Regards
>     >     >     > Abhishek Paliwal
>     >     >     >
>     >     >     >
>     >     >     > _______________________________________________
>     >     >     > Gluster-devel mailing list
>     >     >     > Gluster-devel at gluster.org
>     <mailto:Gluster-devel at gluster.org> <mailto:Gluster-devel at gluster.org
>     <mailto:Gluster-devel at gluster.org>>
>     >     <mailto:Gluster-devel at gluster.org
>     <mailto:Gluster-devel at gluster.org> <mailto:Gluster-devel at gluster.org
>     <mailto:Gluster-devel at gluster.org>>>
>     >     >     > http://www.gluster.org/mailman/listinfo/gluster-devel
>     >     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > Regards
>     >     > Abhishek Paliwal
>     >
>     >
> 
> 
> 
> 
> -- 
> 
> 
> 
> 
> Regards
> Abhishek Paliwal