[Gluster-users] Upgrade to 4.1.2 geo-replication does not work
Kotresh Hiremath Ravishankar
khiremat at redhat.com
Thu Aug 30 05:21:02 UTC 2018
Did you fix the library link on node "noi-poc-gluster " as well?
If not please fix it. Please share the geo-rep log this node if it's
as different issue.
-Kotresh HR
On Thu, Aug 30, 2018 at 12:17 AM, Krishna Verma <kverma at cadence.com> wrote:
> Hi Kotresh,
>
>
>
> Thank you so much for you input. Geo-replication is now showing “Active”
> atleast for 1 master node. But its still at faulty state for the 2nd
> master server.
>
>
>
> Below is the detail.
>
>
>
> [root at gluster-poc-noida glusterfs]# gluster volume geo-replication
> glusterep gluster-poc-sj::glusterep status
>
>
>
> MASTER NODE MASTER VOL MASTER BRICK SLAVE USER
> SLAVE SLAVE NODE STATUS CRAWL STATUS
> LAST_SYNCED
>
> ------------------------------------------------------------
> ------------------------------------------------------------
> --------------------------------------------
>
> gluster-poc-noida glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep gluster-poc-sj Active Changelog Crawl
> 2018-08-29 23:56:06
>
> noi-poc-gluster glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep N/A Faulty N/A
> N/A
>
>
>
>
>
> [root at gluster-poc-noida glusterfs]# gluster volume status
>
> Status of volume: glusterep
>
> Gluster process TCP Port RDMA Port Online
> Pid
>
> ------------------------------------------------------------
> ------------------
>
> Brick gluster-poc-noida:/data/gluster/gv0 49152 0 Y
> 22463
>
> Brick noi-poc-gluster:/data/gluster/gv0 49152 0 Y
> 19471
>
> Self-heal Daemon on localhost N/A N/A Y
> 32087
>
> Self-heal Daemon on noi-poc-gluster N/A N/A Y
> 6272
>
>
>
> Task Status of Volume glusterep
>
> ------------------------------------------------------------
> ------------------
>
> There are no active volume tasks
>
>
>
>
>
>
>
> [root at gluster-poc-noida glusterfs]# gluster volume info
>
>
>
> Volume Name: glusterep
>
> Type: Replicate
>
> Volume ID: 4a71bc94-14ce-4b2c-abc4-e6a9a9765161
>
> Status: Started
>
> Snapshot Count: 0
>
> Number of Bricks: 1 x 2 = 2
>
> Transport-type: tcp
>
> Bricks:
>
> Brick1: gluster-poc-noida:/data/gluster/gv0
>
> Brick2: noi-poc-gluster:/data/gluster/gv0
>
> Options Reconfigured:
>
> transport.address-family: inet
>
> nfs.disable: on
>
> performance.client-io-threads: off
>
> geo-replication.indexing: on
>
> geo-replication.ignore-pid-check: on
>
> changelog.changelog: on
>
> [root at gluster-poc-noida glusterfs]#
>
>
>
> Could you please help me in that also please?
>
>
>
> It would be really a great help from your side.
>
>
>
> /Krishna
>
> *From:* Kotresh Hiremath Ravishankar <khiremat at redhat.com>
> *Sent:* Wednesday, August 29, 2018 10:47 AM
>
> *To:* Krishna Verma <kverma at cadence.com>
> *Cc:* Sunny Kumar <sunkumar at redhat.com>; Gluster Users <
> gluster-users at gluster.org>
> *Subject:* Re: [Gluster-users] Upgrade to 4.1.2 geo-replication does not
> work
>
>
>
> EXTERNAL MAIL
>
> Answer inline
>
>
>
> On Tue, Aug 28, 2018 at 4:28 PM, Krishna Verma <kverma at cadence.com> wrote:
>
> Hi Kotresh,
>
>
>
> I created the links before. Below is the detail.
>
>
>
> [root at gluster-poc-noida ~]# ls -l /usr/lib64 | grep libgfch
>
> lrwxrwxrwx 1 root root 30 Aug 28 14:59 libgfchangelog.so ->
> /usr/lib64/libgfchangelog.so.1
>
>
>
> The link created is pointing to wrong library. Please fix this
>
>
>
> #cd /usr/lib64
>
> #rm libgfchangelog.so
>
> #ln -s "libgfchangelog.so.0.0.1" libgfchangelog.so
>
>
>
> lrwxrwxrwx 1 root root 23 Aug 23 23:35 libgfchangelog.so.0 ->
> libgfchangelog.so.0.0.1
>
> -rwxr-xr-x 1 root root 63384 Jul 24 19:11 libgfchangelog.so.0.0.1
>
> [root at gluster-poc-noida ~]# locate libgfchangelog.so
>
> /usr/lib64/libgfchangelog.so.0
>
> /usr/lib64/libgfchangelog.so.0.0.1
>
> [root at gluster-poc-noida ~]#
>
>
>
> Is it looks good what we exactly need or di I need to create any more link
> or How to get “libgfchangelog.so” file if missing.
>
>
>
> /Krishna
>
>
>
> *From:* Kotresh Hiremath Ravishankar <khiremat at redhat.com>
> *Sent:* Tuesday, August 28, 2018 4:22 PM
> *To:* Krishna Verma <kverma at cadence.com>
> *Cc:* Sunny Kumar <sunkumar at redhat.com>; Gluster Users <
> gluster-users at gluster.org>
>
>
> *Subject:* Re: [Gluster-users] Upgrade to 4.1.2 geo-replication does not
> work
>
>
>
> EXTERNAL MAIL
>
> Hi Krishna,
>
> As per the output shared, I don't see the file "libgfchangelog.so" which
> is what is required.
>
> I only see "libgfchangelog.so.0". Please confirm "libgfchangelog.so" is
> present in "/usr/lib64/".
>
> If not create a symlink similar to "libgfchangelog.so.0"
>
>
>
> It should be something like below.
>
>
>
> #ls -l /usr/lib64 | grep libgfch
> -rwxr-xr-x. 1 root root 1078 Aug 28 05:56 libgfchangelog.la
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__libgfchangelog.la&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=77GIqpHy9HY8RQd6lKzSJ-Z1PCuIhZJ3I3IvIuDX-xo&s=kIFnrBaSFV_DdqZezd6PXcDnD8Iy_gVN69ETZYtykEE&e=>
> lrwxrwxrwx. 1 root root 23 Aug 28 05:56 libgfchangelog.so ->
> libgfchangelog.so.0.0.1
> lrwxrwxrwx. 1 root root 23 Aug 28 05:56 libgfchangelog.so.0 ->
> libgfchangelog.so.0.0.1
> -rwxr-xr-x. 1 root root 336888 Aug 28 05:56 libgfchangelog.so.0.0.1
>
>
>
> On Tue, Aug 28, 2018 at 4:04 PM, Krishna Verma <kverma at cadence.com> wrote:
>
> Hi Kotresh,
>
>
>
> Thanks for the response, I did that also but nothing changed.
>
>
>
> [root at gluster-poc-noida ~]# ldconfig /usr/lib64
>
> [root at gluster-poc-noida ~]# ldconfig -p | grep libgfchangelog
>
> libgfchangelog.so.0 (libc6,x86-64) =>
> /usr/lib64/libgfchangelog.so.0
>
> [root at gluster-poc-noida ~]#
>
>
>
> [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep stop
>
> Stopping geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
>
> [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep start
>
> Starting geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
>
>
>
> [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep status
>
>
>
> MASTER NODE MASTER VOL MASTER BRICK SLAVE USER
> SLAVE SLAVE NODE STATUS CRAWL STATUS
> LAST_SYNCED
>
> ------------------------------------------------------------
> ------------------------------------------------------------
> -----------------------------
>
> gluster-poc-noida glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep N/A Faulty N/A N/A
>
> noi-poc-gluster glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep N/A Faulty
> N/A N/A
>
> [root at gluster-poc-noida ~]#
>
>
>
> /Krishna
>
>
>
> *From:* Kotresh Hiremath Ravishankar <khiremat at redhat.com>
> *Sent:* Tuesday, August 28, 2018 4:00 PM
> *To:* Sunny Kumar <sunkumar at redhat.com>
> *Cc:* Krishna Verma <kverma at cadence.com>; Gluster Users <
> gluster-users at gluster.org>
>
>
> *Subject:* Re: [Gluster-users] Upgrade to 4.1.2 geo-replication does not
> work
>
>
>
> EXTERNAL MAIL
>
> Hi Krishna,
>
> Since your libraries are in /usr/lib64, you should be doing
>
> #ldconfig /usr/lib64
>
> Confirm that below command lists the library
>
> #ldconfig -p | grep libgfchangelog
>
>
>
>
>
> On Tue, Aug 28, 2018 at 3:52 PM, Sunny Kumar <sunkumar at redhat.com> wrote:
>
> can you do ldconfig /usr/local/lib and share the output of ldconfig -p
> /usr/local/lib | grep libgf
>
> On Tue, Aug 28, 2018 at 3:45 PM Krishna Verma <kverma at cadence.com> wrote:
> >
> > Hi Sunny,
> >
> > I did the mentioned changes given in patch and restart the session for
> geo-replication. But again same errors in the logs.
> >
> > I have attaching the config files and logs here.
> >
> >
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep stop
> > Stopping geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep delete
> > Deleting geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep create push-pem force
> > Creating geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep start
> > geo-replication start failed for glusterep gluster-poc-sj::glusterep
> > geo-replication command failed
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep start
> > geo-replication start failed for glusterep gluster-poc-sj::glusterep
> > geo-replication command failed
> > [root at gluster-poc-noida ~]# vim /usr/libexec/glusterfs/python/
> syncdaemon/repce.py
> > [root at gluster-poc-noida ~]# systemctl restart glusterd
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep start
> > Starting geo-replication session between glusterep &
> gluster-poc-sj::glusterep has been successful
> > [root at gluster-poc-noida ~]# gluster volume geo-replication glusterep
> gluster-poc-sj::glusterep status
> >
> > MASTER NODE MASTER VOL MASTER BRICK SLAVE USER
> SLAVE SLAVE NODE STATUS CRAWL STATUS
> LAST_SYNCED
> > ------------------------------------------------------------
> ------------------------------------------------------------
> -----------------------------
> > gluster-poc-noida glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep N/A Faulty N/A N/A
> > noi-poc-gluster glusterep /data/gluster/gv0 root
> gluster-poc-sj::glusterep N/A Faulty N/A N/A
> > [root at gluster-poc-noida ~]#
> >
> >
> > /Krishna.
> >
> > -----Original Message-----
> > From: Sunny Kumar <sunkumar at redhat.com>
> > Sent: Tuesday, August 28, 2018 3:17 PM
> > To: Krishna Verma <kverma at cadence.com>
> > Cc: gluster-users at gluster.org
> > Subject: Re: [Gluster-users] Upgrade to 4.1.2 geo-replication does not
> work
> >
> > EXTERNAL MAIL
> >
> >
> > With same log message ?
> >
> > Can you please verify that
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.
> gluster.org_-23_c_glusterfs_-2B_20207_&d=DwIBaQ&c=
> aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_
> 6ZItAWQ2jH8rVLG6tiXhoLFE&m=F0ExtFUfa_YCktOGvy82x3IAxvi2GrbPR72jZ8beuYk&s=
> fGtkmezHJj5YoLN3dUeVUCcYFnREHyOSk36mRjbTTEQ&e= patch is present if not
> can you please apply that.
> > and try with symlinking ln -s /usr/lib64/libgfchangelog.so.0
> /usr/lib64/libgfchangelog.so.
> >
> > Please share the log also.
> >
> > Regards,
> > Sunny
> > On Tue, Aug 28, 2018 at 3:02 PM Krishna Verma <kverma at cadence.com>
> wrote:
> > >
> > > Hi Sunny,
> > >
> > > Thanks for your response, I tried both, but still I am getting the
> same error.
> > >
> > >
> > > [root at noi-poc-gluster ~]# ldconfig /usr/lib [root at noi-poc-gluster ~]#
> > >
> > > [root at noi-poc-gluster ~]# ln -s /usr/lib64/libgfchangelog.so.1
> > > /usr/lib64/libgfchangelog.so [root at noi-poc-gluster ~]# ls -l
> > > /usr/lib64/libgfchangelog.so lrwxrwxrwx. 1 root root 30 Aug 28 14:59
> > > /usr/lib64/libgfchangelog.so -> /usr/lib64/libgfchangelog.so.1
> > >
> > > /Krishna
> > >
> > > -----Original Message-----
> > > From: Sunny Kumar <sunkumar at redhat.com>
> > > Sent: Tuesday, August 28, 2018 2:55 PM
> > > To: Krishna Verma <kverma at cadence.com>
> > > Cc: gluster-users at gluster.org
> > > Subject: Re: [Gluster-users] Upgrade to 4.1.2 geo-replication does not
> > > work
> > >
> > > EXTERNAL MAIL
> > >
> > >
> > > Hi Krish,
> > >
> > > You can run -
> > > #ldconfig /usr/lib
> > >
> > > If that still does not solves your problem you can do manual symlink
> > > like - ln -s /usr/lib64/libgfchangelog.so.1
> > > /usr/lib64/libgfchangelog.so
> > >
> > > Thanks,
> > > Sunny Kumar
> > > On Tue, Aug 28, 2018 at 1:47 PM Krishna Verma <kverma at cadence.com>
> wrote:
> > > >
> > > > Hi
> > > >
> > > >
> > > >
> > > > I am getting below error in gsyncd.log
> > > >
> > > >
> > > >
> > > > OSError: libgfchangelog.so: cannot open shared object file: No such
> > > > file or directory
> > > >
> > > > [2018-08-28 07:19:41.446785] E [repce(worker
> /data/gluster/gv0):197:__call__] RepceClient: call failed
> call=26469:139794524604224:1535440781.44 method=init
> error=OSError
> > > >
> > > > [2018-08-28 07:19:41.447041] E [syncdutils(worker
> /data/gluster/gv0):330:log_raise_exception] <top>: FAIL:
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line
> > > > 311, in main
> > > >
> > > > func(args)
> > > >
> > > > File "/usr/libexec/glusterfs/python/syncdaemon/subcmds.py", line
> > > > 72, in subcmd_worker
> > > >
> > > > local.service_loop(remote)
> > > >
> > > > File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line
> > > > 1236, in service_loop
> > > >
> > > > changelog_agent.init()
> > > >
> > > > File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line
> > > > 216, in __call__
> > > >
> > > > return self.ins(self.meth, *a)
> > > >
> > > > File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line
> > > > 198, in __call__
> > > >
> > > > raise res
> > > >
> > > > OSError: libgfchangelog.so: cannot open shared object file: No such
> > > > file or directory
> > > >
> > > > [2018-08-28 07:19:41.457555] I [repce(agent
> /data/gluster/gv0):80:service_loop] RepceServer: terminating on reaching
> EOF.
> > > >
> > > > [2018-08-28 07:19:42.440184] I [monitor(monitor):272:monitor]
> Monitor:
> > > > worker died in startup phase brick=/data/gluster/gv0
> > > >
> > > >
> > > >
> > > > Below is my file location:
> > > >
> > > >
> > > >
> > > > /usr/lib64/libgfchangelog.so.0
> > > >
> > > > /usr/lib64/libgfchangelog.so.0.0.1
> > > >
> > > >
> > > >
> > > > What I can do to fix it ?
> > > >
> > > >
> > > >
> > > > /Krish
> > > >
> > > > _______________________________________________
> > > > Gluster-users mailing list
> > > > Gluster-users at gluster.org
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.gluster.o
> > > > rg
> > > > _mailman_listinfo_gluster-2Dusers&d=DwIBaQ&c=aUq983L2pue2FqKFoP6PGHM
> > > > JQ
> > > > yoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=_
> > > > u6
> > > > vGRjlVsype7Z8hXDgCONilqVe4sIWkXNqqz2n3IQ&s=i0EUwtUHurhJHyw9UPpepCdLB
> > > > 70
> > > > 1mkxoNZWYvU7XXug&e=
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
>
> https://lists.gluster.org/mailman/listinfo/gluster-users
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.gluster.org_mailman_listinfo_gluster-2Dusers&d=DwMFaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=0E5nRoxLsT2ZXgCpJM_6ZItAWQ2jH8rVLG6tiXhoLFE&m=BuUVeXUwqxdQDoP-IWDI_JPFCQGrRFdzV9g0enLn8kM&s=X0eVYgZ1emUQnEwRTg0AGbAWyoIwIyrE_gzmuolgiPE&e=>
>
>
>
>
> --
>
> Thanks and Regards,
>
> Kotresh H R
>
>
>
>
> --
>
> Thanks and Regards,
>
> Kotresh H R
>
>
>
>
> --
>
> Thanks and Regards,
>
> Kotresh H R
>
--
Thanks and Regards,
Kotresh H R
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180830/f3860727/attachment.html>
More information about the Gluster-users
mailing list