[Gluster-users] [External] Re: glusterfs 3.13 repo unavailable and downgrade to 3.12.9 fails
Davide Obbi
davide.obbi at booking.com
Tue May 15 13:16:13 UTC 2018
i have a 12 nodes cluster with 2 nodes downgraded and currently down. The
procedure of taking down 3.13 nodes means a cluster downtime?
if it matters /var/lib/glusterd/glusterd.info is empty on the 3.12.9 nodes.
On Tue, May 15, 2018 at 2:47 PM, Kaleb S. KEITHLEY <kkeithle at redhat.com>
wrote:
> On 05/15/2018 08:08 AM, Davide Obbi wrote:
> > Thanks Kaleb,
> >
> > any chance i can make the node working after the downgrade?
> > thanks
>
> Without knowing what doesn't work, I'll go out on a limb and guess that
> it's an op-version problem.
>
> Shut down your 3.13 nodes, change their op-version to one of the valid
> 3.12 op-versions (e.g. 31203) and restart. Then the 3.12 nodes should
> work with 3.13, and/or the formerly 3.13 nodes should work with 3.12.
>
> JoeJulian in #gluster may have other ideas.
>
>
> >
> > On Tue, May 15, 2018 at 2:02 PM, Kaleb S. KEITHLEY <kkeithle at redhat.com
> > <mailto:kkeithle at redhat.com>> wrote:
> >
> >
> > You can still get them from
> > https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/
> > <https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/>
> >
> > (I don't know how much longer they'll be there. I suggest you copy
> them
> > if you think you're going to need them in the future.)
> >
> >
> >
> > n 05/15/2018 04:58 AM, Davide Obbi wrote:
> > > hi,
> > >
> > > i noticed that this repo for glusterfs 3.13 does not exists
> anymore at:
> > >
> > > http://mirror.centos.org/centos/7/storage/x86_64/
> > <http://mirror.centos.org/centos/7/storage/x86_64/>
> > >
> > > i knew was not going to be long term supported however the
> downgrade to
> > > 3.12 breaks the server node i believe the issue is with:
> > > *[2018-05-15 08:54:39.981101] E [MSGID: 101019]
> > > [xlator.c:503:xlator_init] 0-management: Initialization of volume
> > > 'management' failed, review your volfile
> again
> > > [2018-05-15 08:54:39.981113] E [MSGID: 101066]
> > > [graph.c:327:glusterfs_graph_init] 0-management: initializing
> translator
> > > failed
>
> > > [2018-05-15 08:54:39.981121] E [MSGID: 101176]
> > > [graph.c:698:glusterfs_graph_activate] 0-graph: init
> > > failed
>
> > > *
> > >
> > > Any help appreciated, thanks
> > >
> > > Details:
> > > *Installed Packages*
> > >
> > glusterfs.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-api.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-cli.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-client-xlators.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-events.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-fuse.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-libs.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > glusterfs-server.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > python2-gluster.x86_64
>
> > >
> > 3.12.9-1.el7
>
> > > @gluster-3.12
> > >
> > > *# journalctl -u
> > >
> > glusterd *
>
>
> > >
> > > -- Logs begin at Wed 2018-03-21 15:06:46 CET, end at Tue 2018-05-15
> > > 10:48:01 CEST. --
> > > Mar 21 15:06:58 glusternode-1003 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > Mar 21 15:07:01 glusternode-1 systemd[1]: Started GlusterFS, a
> > clustered
> > > file-system
> > >
> > server.
>
> > >
> > > Mar 21 15:25:07 glusternode-1 systemd[1]: Stopping GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > Mar 21 15:25:07 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > Mar 21 15:25:09 glusternode-1 systemd[1]: Started GlusterFS, a
> > clustered
> > > file-system
> > >
> > server.
>
> > >
> > > May 10 12:29:11 glusternode-1 systemd[1]: Stopping GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 10 12:29:11 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 10 12:29:13 glusternode-1 systemd[1]: Started GlusterFS, a
> > clustered
> > > file-system
> > >
> > server.
>
> > >
> > > May 15 10:21:01 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service: control
> > > process exited, code=exited
> > > status=1
>
> > > May 15 10:21:01 glusternode-1 systemd[1]: Failed to start
> GlusterFS, a
> > > clustered file-system
> > > server.
>
> > > May 15 10:21:01 glusternode-1 systemd[1]: Unit glusterd.service
> > entered
> > > failed
> > >
> > state.
>
> > >
> > > May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service failed.
> > > May 15 10:22:14 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service: control
> > > process exited, code=exited
> > > status=1
>
> > > May 15 10:22:14 glusternode-1 systemd[1]: Failed to start
> GlusterFS, a
> > > clustered file-system
> > > server.
>
> > > May 15 10:22:14 glusternode-1 systemd[1]: Unit glusterd.service
> > entered
> > > failed
> > >
> > state.
>
> > >
> > > May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service failed.
> > > May 15 10:22:49 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service: control
> > > process exited, code=exited
> > > status=1
>
> > > May 15 10:22:49 glusternode-1 systemd[1]: Failed to start
> GlusterFS, a
> > > clustered file-system
> > > server.
>
> > > May 15 10:22:49 glusternode-1 systemd[1]: Unit glusterd.service
> > entered
> > > failed
> > >
> > state.
>
> > >
> > > May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service failed.
> > > May 15 10:23:36 glusternode-1 systemd[1]: Starting GlusterFS, a
> > > clustered file-system
> > >
> > server...
>
> > >
> > > May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service: control
> > > process exited, code=exited
> > > status=1
>
> > > May 15 10:23:36 glusternode-1 systemd[1]: Failed to start
> GlusterFS, a
> > > clustered file-system
> > > server.
>
> > > May 15 10:23:36 glusternode-1 systemd[1]: Unit glusterd.service
> > entered
> > > failed
> > >
> > state.
>
> > >
> > > May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service failed.
> > > [2018-05-15 08:54:38.354520] I [MSGID: 100030]
> > [glusterfsd.c:2511:main]
> > > 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version
> > 3.12.9
> > > (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level
> INFO)
> > > [2018-05-15 08:54:38.359267] I [MSGID: 106478]
> [glusterd.c:1423:init]
> > > 0-management: Maximum allowed open file descriptors set to
> > > 65536
> > > [2018-05-15 08:54:38.359305] I [MSGID: 106479]
> [glusterd.c:1481:init]
> > > 0-management: Using /var/lib/glusterd as working
> > > directory
>
> > > [2018-05-15 08:54:38.359313] I [MSGID: 106479]
> [glusterd.c:1486:init]
> > > 0-management: Using /var/run/gluster as pid file working
> > > directory
> > > [2018-05-15 08:54:38.363363] E
> > [rpc-transport.c:283:rpc_transport_load]
> > > 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/
> rpc-transport/rdma.so:
> > > cannot open shared object file: No such file or directory
> > > [2018-05-15 08:54:38.363381] W
> > [rpc-transport.c:287:rpc_transport_load]
> > > 0-rpc-transport: volume 'rdma.management': transport-type 'rdma'
> > is not
> > > valid or not found on this machine
> > > [2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_
> listener]
> > > 0-rpc-service: cannot create listener, initing the transport
> > > failed
> > > [2018-05-15 08:54:38.363398] E [MSGID: 106243]
> [glusterd.c:1769:init]
> > > 0-management: creation of 1 listeners failed, continuing with
> > succeeded
> > > transport
> > > [2018-05-15 08:54:39.981028] E [MSGID: 106022]
> > > [glusterd-store.c:2234:glusterd_restore_op_version] 0-management:
> > wrong
> > > op-version (31300) retrieved [Invalid
> argument]
> > > [2018-05-15 08:54:39.981071] E [MSGID: 106244]
> [glusterd.c:1928:init]
> > > 0-management: Failed to restore
> > >
> > op_version
>
> > >
> > > [2018-05-15 08:54:39.981101] E [MSGID: 101019]
> > > [xlator.c:503:xlator_init] 0-management: Initialization of volume
> > > 'management' failed, review your volfile
> > again
> > > [2018-05-15 08:54:39.981113] E [MSGID: 101066]
> > > [graph.c:327:glusterfs_graph_init] 0-management: initializing
> > translator
> > > failed
>
> > > [2018-05-15 08:54:39.981121] E [MSGID: 101176]
> > > [graph.c:698:glusterfs_graph_activate] 0-graph: init
> > >
> > failed
>
> > >
> > > [2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_
> and_exit]
> > > (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd)
> [0x559572dbcf5d]
> > > -->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
> > > [0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
> > > [0x559572dbc37b] ) 0-: received signum (-1), shutting
> > > down
>
> > > */var/log/glusterfs/glusterd.log*
> > > [2018-05-15 08:54:38.354520] I [MSGID: 100030]
> > [glusterfsd.c:2511:main]
> > > 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version
> > 3.12.9
> > > (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level
> INFO)
> > > [2018-05-15 08:54:38.359267] I [MSGID: 106478]
> [glusterd.c:1423:init]
> > > 0-management: Maximum allowed open file descriptors set to
> > > 65536
> > > [2018-05-15 08:54:38.359305] I [MSGID: 106479]
> [glusterd.c:1481:init]
> > > 0-management: Using /var/lib/glusterd as working
> > > directory
>
> > > [2018-05-15 08:54:38.359313] I [MSGID: 106479]
> [glusterd.c:1486:init]
> > > 0-management: Using /var/run/gluster as pid file working
> > > directory
> > > [2018-05-15 08:54:38.363363] E
> > [rpc-transport.c:283:rpc_transport_load]
> > > 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/
> rpc-transport/rdma.so:
> > > cannot open shared object file: No such file or directory
> > > [2018-05-15 08:54:38.363381] W
> > [rpc-transport.c:287:rpc_transport_load]
> > > 0-rpc-transport: volume 'rdma.management': transport-type 'rdma'
> > is not
> > > valid or not found on this machine
> > > [2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_
> listener]
> > > 0-rpc-service: cannot create listener, initing the transport
> > > failed
> > > [2018-05-15 08:54:38.363398] E [MSGID: 106243]
> [glusterd.c:1769:init]
> > > 0-management: creation of 1 listeners failed, continuing with
> > succeeded
> > > transport
> > > [2018-05-15 08:54:39.981028] E [MSGID: 106022]
> > > [glusterd-store.c:2234:glusterd_restore_op_version] 0-management:
> > wrong
> > > op-version (31300) retrieved [Invalid
> argument]
> > > [2018-05-15 08:54:39.981071] E [MSGID: 106244]
> [glusterd.c:1928:init]
> > > 0-management: Failed to restore
> > >
> > op_version
>
> > >
> > > *[2018-05-15 08:54:39.981101] E [MSGID: 101019]
> > > [xlator.c:503:xlator_init] 0-management: Initialization of volume
> > > 'management' failed, review your volfile
> again
> > > [2018-05-15 08:54:39.981113] E [MSGID: 101066]
> > > [graph.c:327:glusterfs_graph_init] 0-management: initializing
> translator
> > > failed
>
> > > [2018-05-15 08:54:39.981121] E [MSGID: 101176]
> > > [graph.c:698:glusterfs_graph_activate] 0-graph: init
> > > failed
>
> > > *
> > > [2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_
> and_exit]
> > > (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd)
> [0x559572dbcf5d]
> > > -->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
> > > [0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
> > > [0x559572dbc37b] ) 0-: received signum (-1), shutting
> > > down
>
> > >
> > >
> > > Kind regards
> > > Davide
> > >
> > >
> > > _______________________________________________
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> > > http://lists.gluster.org/mailman/listinfo/gluster-users
> > <http://lists.gluster.org/mailman/listinfo/gluster-users>
> > >
> >
> >
> >
> >
> >
> > --
> > Davide Obbi
> > System Administrator
> >
> > Booking.com B.V.
> > Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
> > Direct +31207031558
> > Booking.com <http://www.booking.com/>
> > The world's #1 accommodation site
> > 43 languages, 198+ offices worldwide, 120,000+ global destinations,
> > 1,550,000+ room nights booked every day
> > No booking fees, best price always guaranteed
> > Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
>
>
>
--
Davide Obbi
System Administrator
Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207031558
[image: Booking.com] <http://www.booking.com/>
The world's #1 accommodation site
43 languages, 198+ offices worldwide, 120,000+ global destinations,
1,550,000+ room nights booked every day
No booking fees, best price always guaranteed
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180515/b2c32b2a/attachment.html>
More information about the Gluster-users
mailing list