<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 18, 2017 at 2:01 PM, Niels de Vos <span dir="ltr"><<a href="mailto:ndevos@redhat.com" target="_blank">ndevos@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Aug 18, 2017 at 12:22:33PM +0530, Atin Mukherjee wrote:<br>
> You're hitting a race here. By the time glusterd tries to resolve the<br>
> address of one of the remote bricks of a particular volume, the n/w<br>
> interface is not up by that time. We have fixed this issue in mainline and<br>
> 3.12 branch through the following commit:<br>
<br>
</span>We still maintain 3.10 for at least 6 months. It probably makes sense to<br>
backport this? I would not bother with 3.8 though, the last update for<br>
this version has already been shipped.<br></blockquote><div><br></div><div>Agreed. Gaurav is backporting the fix in 3.10 now. <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Thanks,<br>
Niels<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
><br>
> commit 1477fa442a733d7b1a5ea74884cac8<wbr>f29fbe7e6a<br>
> Author: Gaurav Yadav <<a href="mailto:gyadav@redhat.com">gyadav@redhat.com</a>><br>
> Date: Tue Jul 18 16:23:18 2017 +0530<br>
><br>
> glusterd : glusterd fails to start when peer's network interface is<br>
> down<br>
><br>
> Problem:<br>
> glusterd fails to start on nodes where glusterd tries to come up even<br>
> before network is up.<br>
><br>
> Fix:<br>
> On startup glusterd tries to resolve brick path which is based on<br>
> hostname/ip, but in the above scenario when network interface is not<br>
> up, glusterd is not able to resolve the brick path using ip_address or<br>
> hostname With this fix glusterd will use UUID to resolve brick path.<br>
><br>
> Change-Id: Icfa7b2652417135530479d0aa4e2a<wbr>82b0476f710<br>
> BUG: 1472267<br>
> Signed-off-by: Gaurav Yadav <<a href="mailto:gyadav@redhat.com">gyadav@redhat.com</a>><br>
> Reviewed-on: <a href="https://review.gluster.org/17813" rel="noreferrer" target="_blank">https://review.gluster.org/<wbr>17813</a><br>
> Smoke: Gluster Build System <<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>><br>
> Reviewed-by: Prashanth Pai <<a href="mailto:ppai@redhat.com">ppai@redhat.com</a>><br>
> CentOS-regression: Gluster Build System <<a href="mailto:jenkins@build.gluster.org">jenkins@build.gluster.org</a>><br>
> Reviewed-by: Atin Mukherjee <<a href="mailto:amukherj@redhat.com">amukherj@redhat.com</a>><br>
><br>
><br>
><br>
> Note : 3.12 release is planned by end of this month.<br>
><br>
> ~Atin<br>
><br>
> On Thu, Aug 17, 2017 at 2:45 PM, ismael mondiu <<a href="mailto:mondiu@hotmail.com">mondiu@hotmail.com</a>> wrote:<br>
><br>
> > Hi Team,<br>
> ><br>
> > I noticed that glusterd is never starting when i reboot my Redhat 7.1<br>
> > server.<br>
> ><br>
> > Service is enabled but don't works.<br>
> ><br>
> > I tested with gluster 3.10.4 & gluster 3.10.5 and the problem still exists.<br>
> ><br>
> ><br>
> > When i started the service manually this works.<br>
> ><br>
> > I'va also tested on Redhat 6.6 server and gluster 3.10.4 and this works<br>
> > fine.<br>
> ><br>
> > The problem seems to be related to Redhat 7.1<br>
> ><br>
> ><br>
> > This is à known issue ? if yes , can you tell me what's is the workaround?<br>
> ><br>
> ><br>
> > Thanks<br>
> ><br>
> ><br>
> > Some logs here<br>
> ><br>
> ><br>
> > [root@~]# systemctl status glusterd<br>
> > ● glusterd.service - GlusterFS, a clustered file-system server<br>
> > Loaded: loaded (/usr/lib/systemd/system/<wbr>glusterd.service; enabled;<br>
> > vendor preset: disabled)<br>
> > Active: failed (Result: exit-code) since Thu 2017-08-17 11:04:00 CEST;<br>
> > 2min 9s ago<br>
> > Process: 851 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid<br>
> > --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=1/FAILURE)<br>
> ><br>
> > Aug 17 11:03:59 dvihcasc0r systemd[1]: Starting GlusterFS, a clustered<br>
> > file-system server...<br>
> > Aug 17 11:04:00 dvihcasc0r systemd[1]: glusterd.service: control process<br>
> > exited, code=exited status=1<br>
> > Aug 17 11:04:00 dvihcasc0r systemd[1]: Failed to start GlusterFS, a<br>
> > clustered file-system server.<br>
> > Aug 17 11:04:00 dvihcasc0r systemd[1]: Unit glusterd.service entered<br>
> > failed state.<br>
> > Aug 17 11:04:00 dvihcasc0r systemd[1]: glusterd.service failed.<br>
> ><br>
> ><br>
> > ******************************<wbr>******************************<br>
> > ****************************<br>
> ><br>
> > /var/log/glusterfs/glusterd.<wbr>log<br>
> ><br>
> > ******************************<wbr>******************************<br>
> > ******************************<wbr>**<br>
> ><br>
> ><br>
> > 2017-08-17 09:04:00.202529] I [MSGID: 106478] [glusterd.c:1449:init]<br>
> > 0-management: Maximum allowed open file descriptors set to 65536<br>
> > [2017-08-17 09:04:00.202573] I [MSGID: 106479] [glusterd.c:1496:init]<br>
> > 0-management: Using /var/lib/glusterd as working directory<br>
> > [2017-08-17 09:04:00.365134] E [rpc-transport.c:283:rpc_<wbr>transport_load]<br>
> > 0-rpc-transport: /usr/lib64/glusterfs/3.10.5/<wbr>rpc-transport/rdma.so:<br>
> > cannot open shared object file: No such file or directory<br>
> > [2017-08-17 09:04:00.365161] W [rpc-transport.c:287:rpc_<wbr>transport_load]<br>
> > 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not<br>
> > valid or not found on this machine<br>
> > [2017-08-17 09:04:00.365195] W [rpcsvc.c:1661:rpcsvc_create_<wbr>listener]<br>
> > 0-rpc-service: cannot create listener, initing the transport failed<br>
> > [2017-08-17 09:04:00.365206] E [MSGID: 106243] [glusterd.c:1720:init]<br>
> > 0-management: creation of 1 listeners failed, continuing with succeeded<br>
> > transport<br>
> > [2017-08-17 09:04:00.464314] I [MSGID: 106228] [glusterd.c:500:glusterd_<wbr>check_gsync_present]<br>
> > 0-glusterd: geo-replication module not installed in the system [No such<br>
> > file or directory]<br>
> > [2017-08-17 09:04:00.510412] I [MSGID: 106513] [glusterd-store.c:2197:<wbr>glusterd_restore_op_version]<br>
> > 0-glusterd: retrieved op-version: 31004<br>
> > [2017-08-17 09:04:00.711413] I [MSGID: 106194] [glusterd-store.c:3776:<br>
> > glusterd_store_retrieve_<wbr>missed_snaps_list] 0-management: No missed snaps<br>
> > list.<br>
> > [2017-08-17 09:04:00.756731] E [MSGID: 106187] [glusterd-store.c:4559:<wbr>glusterd_resolve_all_bricks]<br>
> > 0-glusterd: resolve brick failed in restore<br>
> > [2017-08-17 09:04:00.756787] E [MSGID: 101019] [xlator.c:503:xlator_init]<br>
> > 0-management: Initialization of volume 'management' failed, review your<br>
> > volfile again<br>
> > [2017-08-17 09:04:00.756802] E [MSGID: 101066]<br>
> > [graph.c:325:glusterfs_graph_<wbr>init] 0-management: initializing translator<br>
> > failed<br>
> > [2017-08-17 09:04:00.756816] E [MSGID: 101176]<br>
> > [graph.c:681:glusterfs_graph_<wbr>activate] 0-graph: init failed<br>
> > [2017-08-17 09:04:00.766584] W [glusterfsd.c:1332:cleanup_<wbr>and_exit]<br>
> > (-->/usr/sbin/glusterd(<wbr>glusterfs_volumes_init+0xfd) [0x7f9bdef4cabd]<br>
> > -->/usr/sbin/glusterd(<wbr>glusterfs_process_volfp+0x1b1) [0x7f9bdef4c961]<br>
> > -->/usr/sbin/glusterd(cleanup_<wbr>and_exit+0x6b) [0x7f9bdef4be4b] ) 0-:<br>
> > received signum (1), shutting down<br>
> ><br>
> > ******************************<wbr>******************************<br>
> > ******************************<br>
> ><br>
> > [root@~]# uptime<br>
> > 11:13:55 up 10 min, 1 user, load average: 0.00, 0.02, 0.04<br>
> ><br>
> ><br>
> > ******************************<wbr>******************************<br>
> > ******************************<br>
> ><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > Gluster-users mailing list<br>
> > <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> > <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
> ><br>
<br>
> ______________________________<wbr>_________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
<br>
</div></div></blockquote></div><br></div></div>