[Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request
Erik Jacobson
erik.jacobson at hpe.com
Wed Apr 15 17:45:57 UTC 2020
> The new split-brain issue is much harder to reproduce, but after several
(correcting to say new seg fault issue, the split brain is gone!!)
> intense runs, it usually hits once.
>
> We switched to pure gluster74 plus your patch so we're apples to apples
> now.
>
> I'm going to see if Scott can help debug it.
>
> Here is the back trace info from the core dump:
>
> -rw-r----- 1 root root 1.9G Apr 15 12:40 core.glusterfs.0.52467a7e67964553aa9971eb2bb0148c.61084.1586972324000000
> -rw-r----- 1 root root 221M Apr 15 12:40 core.glusterfs.0.52467a7e67964553aa9971eb2bb0148c.61084.1586972324000000.lz4
> drwxrwxrwt 9 root root 20K Apr 15 12:40 .
> [root at leader3 tmp]#
> [root at leader3 tmp]#
> [root at leader3 tmp]# gdb core.glusterfs.0.52467a7e67964553aa9971eb2bb0148c.61084.1586972324000000
> GNU gdb (GDB) Red Hat Enterprise Linux 8.2-5.el8
> Copyright (C) 2018 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
>
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> [New LWP 61102]
> [New LWP 61085]
> [New LWP 61087]
> [New LWP 61117]
> [New LWP 61086]
> [New LWP 61108]
> [New LWP 61089]
> [New LWP 61090]
> [New LWP 61121]
> [New LWP 61088]
> [New LWP 61091]
> [New LWP 61093]
> [New LWP 61095]
> [New LWP 61092]
> [New LWP 61094]
> [New LWP 61098]
> [New LWP 61096]
> [New LWP 61097]
> [New LWP 61084]
> [New LWP 61100]
> [New LWP 61103]
> [New LWP 61104]
> [New LWP 61099]
> [New LWP 61105]
> [New LWP 61101]
> [New LWP 61106]
> [New LWP 61109]
> [New LWP 61107]
> [New LWP 61112]
> [New LWP 61119]
> [New LWP 61110]
> [New LWP 61111]
> [New LWP 61118]
> [New LWP 61123]
> [New LWP 61122]
> [New LWP 61113]
> [New LWP 61114]
> [New LWP 61120]
> [New LWP 61116]
> [New LWP 61115]
>
> warning: core file may not match specified executable file.
> Reading symbols from /usr/sbin/glusterfsd...Reading symbols from /usr/lib/debug/usr/sbin/glusterfsd-7.4-1.el8722.0800.200415T1052.a.rhel8hpeerikj.x86_64.debug...done.
> done.
>
> warning: Ignoring non-absolute filename: <linux-vdso.so.1>
> Missing separate debuginfo for linux-vdso.so.1
> Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/06/44254f9cbaa826db070a796046026adba58266
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
>
> warning: Loadable section ".note.gnu.property" outside of ELF segments
> Core was generated by `/usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/run/gluster/n'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 0x00007fe63bb5d7bb in FRAME_DESTROY (frame=0x7fe5ac096288)
> at ../../../../libglusterfs/src/glusterfs/stack.h:193
> 193 FRAME_DESTROY(frame);
> [Current thread is 1 (Thread 0x7fe617fff700 (LWP 61102))]
> Missing separate debuginfos, use: dnf debuginfo-install glibc-2.28-42.el8.x86_64 keyutils-libs-1.5.10-6.el8.x86_64 krb5-libs-1.16.1-22.el8.x86_64 libacl-2.2.53-1.el8.x86_64 libattr-2.4.48-3.el8.x86_64 libcom_err-1.44.3-2.el8.x86_64 libgcc-8.2.1-3.5.el8.x86_64 libselinux-2.8-6.el8.x86_64 libtirpc-1.1.4-3.el8.x86_64 libuuid-2.32.1-8.el8.x86_64 openssl-libs-1.1.1-8.el8.x86_64 pcre2-10.32-1.el8.x86_64 zlib-1.2.11-10.el8.x86_64
> (gdb) bt
> #0 0x00007fe63bb5d7bb in FRAME_DESTROY (frame=0x7fe5ac096288)
> at ../../../../libglusterfs/src/glusterfs/stack.h:193
> #1 STACK_DESTROY (stack=0x7fe5ac6d65f8)
> at ../../../../libglusterfs/src/glusterfs/stack.h:193
> #2 rda_fill_fd_cbk (frame=0x7fe5acf18eb8, cookie=<optimized out>,
> this=0x7fe63c0162b0, op_ret=3, op_errno=0, entries=<optimized out>,
> xdata=0x0) at readdir-ahead.c:623
> #3 0x00007fe63bd6c3aa in afr_readdir_cbk (frame=<optimized out>,
> cookie=<optimized out>, this=<optimized out>, op_ret=<optimized out>,
> op_errno=<optimized out>, subvol_entries=<optimized out>, xdata=0x0)
> at afr-dir-read.c:234
> #4 0x00007fe6400a1e07 in client4_0_readdirp_cbk (req=<optimized out>,
> iov=<optimized out>, count=<optimized out>, myframe=0x7fe5ace0eda8)
> at client-rpc-fops_v2.c:2338
> #5 0x00007fe6479ca115 in rpc_clnt_handle_reply (
> clnt=clnt at entry=0x7fe63c0663f0, pollin=pollin at entry=0x7fe60c1737a0)
> at rpc-clnt.c:764
> #6 0x00007fe6479ca4b3 in rpc_clnt_notify (trans=0x7fe63c066780,
> mydata=0x7fe63c066420, event=<optimized out>, data=0x7fe60c1737a0)
> at rpc-clnt.c:931
> #7 0x00007fe6479c707b in rpc_transport_notify (
> this=this at entry=0x7fe63c066780,
> event=event at entry=RPC_TRANSPORT_MSG_RECEIVED,
> data=data at entry=0x7fe60c1737a0) at rpc-transport.c:545
> #8 0x00007fe640da893c in socket_event_poll_in_async (xl=<optimized out>,
> async=0x7fe60c1738c8) at socket.c:2601
> #9 0x00007fe640db03dc in gf_async (
> cbk=0x7fe640da8910 <socket_event_poll_in_async>, xl=<optimized out>,
> async=0x7fe60c1738c8) at ../../../../libglusterfs/src/glusterfs/async.h:189
> #10 socket_event_poll_in (notify_handled=true, this=0x7fe63c066780)
> at socket.c:2642
> #11 socket_event_handler (fd=fd at entry=19, idx=idx at entry=10, gen=gen at entry=1,
> data=data at entry=0x7fe63c066780, poll_in=<optimized out>,
> poll_out=<optimized out>, poll_err=0, event_thread_died=0 '\000')
> at socket.c:3040
> #12 0x00007fe647c84a5b in event_dispatch_epoll_handler (event=0x7fe617ffe014,
> event_pool=0x563f5a98c750) at event-epoll.c:650
> #13 event_dispatch_epoll_worker (data=0x7fe63c063b60) at event-epoll.c:763
> #14 0x00007fe6467a72de in start_thread () from /lib64/libpthread.so.0
> #15 0x00007fe645fffa63 in clone () from /lib64/libc.so.6
>
>
>
> On Wed, Apr 15, 2020 at 10:39:34AM -0500, Erik Jacobson wrote:
> > After several successful runs of the test case, we thought we were
> > solved. Indeed, split-brain is gone.
> >
> > But we're triggering a seg fault now, even in a less loaded case.
> >
> > We're going to switch to gluster74, which was your intention, and report
> > back.
> >
> > On Wed, Apr 15, 2020 at 10:33:01AM -0500, Erik Jacobson wrote:
> > > > Attached the wrong patch by mistake in my previous mail. Sending the correct
> > > > one now.
> > >
> > > Early results loook GREAT !!
> > >
> > > We'll keep beating on it. We applied it to glsuter72 as that is what we
> > > have to ship with. It applied fine with some line moves.
> > >
> > > If you would like us to also run a test with gluster74 so that you can
> > > say that's tested, we can run that test. I can do a special build.
> > >
> > > THANK YOU!!
> > >
> > > >
> > > >
> > > > -Ravi
> > > >
> > > >
> > > > On 15/04/20 2:05 pm, Ravishankar N wrote:
> > > >
> > > >
> > > > On 10/04/20 2:06 am, Erik Jacobson wrote:
> > > >
> > > > Once again thanks for sticking with us. Here is a reply from Scott
> > > > Titus. If you have something for us to try, we'd love it. The code had
> > > > your patch applied when gdb was run:
> > > >
> > > >
> > > > Here is the addr2line output for those addresses. Very interesting
> > > > command, of
> > > > which I was not aware.
> > > >
> > > > [root at leader3 ~]# addr2line -f -e/usr/lib64/glusterfs/7.2/xlator/
> > > > cluster/
> > > > afr.so 0x6f735
> > > > afr_lookup_metadata_heal_check
> > > > afr-common.c:2803
> > > > [root at leader3 ~]# addr2line -f -e/usr/lib64/glusterfs/7.2/xlator/
> > > > cluster/
> > > > afr.so 0x6f0b9
> > > > afr_lookup_done
> > > > afr-common.c:2455
> > > > [root at leader3 ~]# addr2line -f -e/usr/lib64/glusterfs/7.2/xlator/
> > > > cluster/
> > > > afr.so 0x5c701
> > > > afr_inode_event_gen_reset
> > > > afr-common.c:755
> > > >
> > > >
> > > > Right, so afr_lookup_done() is resetting the event gen to zero. This looks
> > > > like a race between lookup and inode refresh code paths. We made some
> > > > changes to the event generation logic in AFR. Can you apply the attached
> > > > patch and see if it fixes the split-brain issue? It should apply cleanly on
> > > > glusterfs-7.4.
> > > >
> > > > Thanks,
> > > > Ravi
> > > >
> > > >
> > > > ________
> > > >
> > > >
> > > >
> > > > Community Meeting Calendar:
> > > >
> > > > Schedule -
> > > > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> > > > Bridge: https://bluejeans.com/441850968
> > > >
> > > > Gluster-users mailing list
> > > > Gluster-users at gluster.org
> > > > https://lists.gluster.org/mailman/listinfo/gluster-users
> > > >
> > >
> > > > >From 11601e709a97ce7c40078866bf5d24b486f39454 Mon Sep 17 00:00:00 2001
> > > > From: Ravishankar N <ravishankar at redhat.com>
> > > > Date: Wed, 15 Apr 2020 13:53:26 +0530
> > > > Subject: [PATCH] afr: event gen changes
> > > >
> > > > The general idea of the changes is to prevent resetting event generation
> > > > to zero in the inode ctx, since event gen is something that should
> > > > follow 'causal order'.
> > > >
> > > > Change #1:
> > > > For a read txn, in inode refresh cbk, if event_generation is
> > > > found zero, we are failing the read fop. This is not needed
> > > > because change in event gen is only a marker for the next inode refresh to
> > > > happen and should not be taken into account by the current read txn.
> > > >
> > > > Change #2:
> > > > The event gen being zero above can happen if there is a racing lookup,
> > > > which resets even get (in afr_lookup_done) if there are non zero afr
> > > > xattrs. The resetting is done only to trigger an inode refresh and a
> > > > possible client side heal on the next lookup. That can be acheived by
> > > > setting the need_refresh flag in the inode ctx. So replaced all
> > > > occurences of resetting even gen to zero with a call to
> > > > afr_inode_need_refresh_set().
> > > >
> > > > Change #3:
> > > > In both lookup and discover path, we are doing an inode refresh which is
> > > > not required since all 3 essentially do the same thing- update the inode
> > > > ctx with the good/bad copies from the brick replies. Inode refresh also
> > > > triggers background heals, but I think it is okay to do it when we call
> > > > refresh during the read and write txns and not in the lookup path.
> > > >
> > > > Change-Id: Id0600dd34b144b4ae7a3bf3c397551adf7e402f1
> > > > Signed-off-by: Ravishankar N <ravishankar at redhat.com>
> > > > ---
> > > > ...ismatch-resolution-with-fav-child-policy.t | 8 +-
> > > > xlators/cluster/afr/src/afr-common.c | 92 ++++---------------
> > > > xlators/cluster/afr/src/afr-dir-write.c | 6 +-
> > > > xlators/cluster/afr/src/afr.h | 5 +-
> > > > 4 files changed, 29 insertions(+), 82 deletions(-)
> > > >
> > > > diff --git a/tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t b/tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t
> > > > index f4aa351e4..12af0c854 100644
> > > > --- a/tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t
> > > > +++ b/tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t
> > > > @@ -168,8 +168,8 @@ TEST [ "$gfid_1" != "$gfid_2" ]
> > > > #We know that second brick has the bigger size file
> > > > BIGGER_FILE_MD5=$(md5sum $B0/${V0}1/f3 | cut -d\ -f1)
> > > >
> > > > -TEST ls $M0/f3
> > > > -TEST cat $M0/f3
> > > > +TEST ls $M0 #Trigger entry heal via readdir inode refresh
> > > > +TEST cat $M0/f3 #Trigger data heal via readv inode refresh
> > > > EXPECT_WITHIN $HEAL_TIMEOUT "^0$" get_pending_heal_count $V0
> > > >
> > > > #gfid split-brain should be resolved
> > > > @@ -215,8 +215,8 @@ TEST $CLI volume start $V0 force
> > > > EXPECT_WITHIN $PROCESS_UP_TIMEOUT "1" brick_up_status $V0 $H0 $B0/${V0}2
> > > > EXPECT_WITHIN $CHILD_UP_TIMEOUT "1" afr_child_up_status $V0 2
> > > >
> > > > -TEST ls $M0/f4
> > > > -TEST cat $M0/f4
> > > > +TEST ls $M0 #Trigger entry heal via readdir inode refresh
> > > > +TEST cat $M0/f4 #Trigger data heal via readv inode refresh
> > > > EXPECT_WITHIN $HEAL_TIMEOUT "^0$" get_pending_heal_count $V0
> > > >
> > > > #gfid split-brain should be resolved
> > > > diff --git a/xlators/cluster/afr/src/afr-common.c b/xlators/cluster/afr/src/afr-common.c
> > > > index 61f21795e..319665a14 100644
> > > > --- a/xlators/cluster/afr/src/afr-common.c
> > > > +++ b/xlators/cluster/afr/src/afr-common.c
> > > > @@ -282,7 +282,7 @@ __afr_set_in_flight_sb_status(xlator_t *this, afr_local_t *local,
> > > > metadatamap |= (1 << index);
> > > > }
> > > > if (metadatamap_old != metadatamap) {
> > > > - event = 0;
> > > > + __afr_inode_need_refresh_set(inode, this);
> > > > }
> > > > break;
> > > >
> > > > @@ -295,7 +295,7 @@ __afr_set_in_flight_sb_status(xlator_t *this, afr_local_t *local,
> > > > datamap |= (1 << index);
> > > > }
> > > > if (datamap_old != datamap)
> > > > - event = 0;
> > > > + __afr_inode_need_refresh_set(inode, this);
> > > > break;
> > > >
> > > > default:
> > > > @@ -458,34 +458,6 @@ out:
> > > > return ret;
> > > > }
> > > >
> > > > -int
> > > > -__afr_inode_event_gen_reset_small(inode_t *inode, xlator_t *this)
> > > > -{
> > > > - int ret = -1;
> > > > - uint16_t datamap = 0;
> > > > - uint16_t metadatamap = 0;
> > > > - uint32_t event = 0;
> > > > - uint64_t val = 0;
> > > > - afr_inode_ctx_t *ctx = NULL;
> > > > -
> > > > - ret = __afr_inode_ctx_get(this, inode, &ctx);
> > > > - if (ret)
> > > > - return ret;
> > > > -
> > > > - val = ctx->read_subvol;
> > > > -
> > > > - metadatamap = (val & 0x000000000000ffff) >> 0;
> > > > - datamap = (val & 0x00000000ffff0000) >> 16;
> > > > - event = 0;
> > > > -
> > > > - val = ((uint64_t)metadatamap) | (((uint64_t)datamap) << 16) |
> > > > - (((uint64_t)event) << 32);
> > > > -
> > > > - ctx->read_subvol = val;
> > > > -
> > > > - return ret;
> > > > -}
> > > > -
> > > > int
> > > > __afr_inode_read_subvol_get(inode_t *inode, xlator_t *this, unsigned char *data,
> > > > unsigned char *metadata, int *event_p)
> > > > @@ -556,22 +528,6 @@ out:
> > > > return ret;
> > > > }
> > > >
> > > > -int
> > > > -__afr_inode_event_gen_reset(inode_t *inode, xlator_t *this)
> > > > -{
> > > > - afr_private_t *priv = NULL;
> > > > - int ret = -1;
> > > > -
> > > > - priv = this->private;
> > > > -
> > > > - if (priv->child_count <= 16)
> > > > - ret = __afr_inode_event_gen_reset_small(inode, this);
> > > > - else
> > > > - ret = -1;
> > > > -
> > > > - return ret;
> > > > -}
> > > > -
> > > > int
> > > > afr_inode_read_subvol_get(inode_t *inode, xlator_t *this, unsigned char *data,
> > > > unsigned char *metadata, int *event_p)
> > > > @@ -721,30 +677,22 @@ out:
> > > > return need_refresh;
> > > > }
> > > >
> > > > -static int
> > > > -afr_inode_need_refresh_set(inode_t *inode, xlator_t *this)
> > > > +int
> > > > +__afr_inode_need_refresh_set(inode_t *inode, xlator_t *this)
> > > > {
> > > > int ret = -1;
> > > > afr_inode_ctx_t *ctx = NULL;
> > > >
> > > > - GF_VALIDATE_OR_GOTO(this->name, inode, out);
> > > > -
> > > > - LOCK(&inode->lock);
> > > > - {
> > > > - ret = __afr_inode_ctx_get(this, inode, &ctx);
> > > > - if (ret)
> > > > - goto unlock;
> > > > -
> > > > + ret = __afr_inode_ctx_get(this, inode, &ctx);
> > > > + if (ret == 0) {
> > > > ctx->need_refresh = _gf_true;
> > > > }
> > > > -unlock:
> > > > - UNLOCK(&inode->lock);
> > > > -out:
> > > > +
> > > > return ret;
> > > > }
> > > >
> > > > int
> > > > -afr_inode_event_gen_reset(inode_t *inode, xlator_t *this)
> > > > +afr_inode_need_refresh_set(inode_t *inode, xlator_t *this)
> > > > {
> > > > int ret = -1;
> > > >
> > > > @@ -754,7 +702,7 @@ afr_inode_event_gen_reset(inode_t *inode, xlator_t *this)
> > > > "Resetting event gen for %s", uuid_utoa(inode->gfid));
> > > > LOCK(&inode->lock);
> > > > {
> > > > - ret = __afr_inode_event_gen_reset(inode, this);
> > > > + ret = __afr_inode_need_refresh_set(inode, this);
> > > > }
> > > > UNLOCK(&inode->lock);
> > > > out:
> > > > @@ -1187,7 +1135,7 @@ afr_txn_refresh_done(call_frame_t *frame, xlator_t *this, int err)
> > > > ret = afr_inode_get_readable(frame, inode, this, local->readable,
> > > > &event_generation, local->transaction.type);
> > > >
> > > > - if (ret == -EIO || (local->is_read_txn && !event_generation)) {
> > > > + if (ret == -EIO) {
> > > > /* No readable subvolume even after refresh ==> splitbrain.*/
> > > > if (!priv->fav_child_policy) {
> > > > err = EIO;
> > > > @@ -2451,7 +2399,7 @@ afr_lookup_done(call_frame_t *frame, xlator_t *this)
> > > > if (read_subvol == -1)
> > > > goto cant_interpret;
> > > > if (ret) {
> > > > - afr_inode_event_gen_reset(local->inode, this);
> > > > + afr_inode_need_refresh_set(local->inode, this);
> > > > dict_del_sizen(local->replies[read_subvol].xdata, GF_CONTENT_KEY);
> > > > }
> > > > } else {
> > > > @@ -3007,6 +2955,7 @@ afr_discover_unwind(call_frame_t *frame, xlator_t *this)
> > > > afr_private_t *priv = NULL;
> > > > afr_local_t *local = NULL;
> > > > int read_subvol = -1;
> > > > + int ret = 0;
> > > > unsigned char *data_readable = NULL;
> > > > unsigned char *success_replies = NULL;
> > > >
> > > > @@ -3028,7 +2977,10 @@ afr_discover_unwind(call_frame_t *frame, xlator_t *this)
> > > > if (!afr_has_quorum(success_replies, this, frame))
> > > > goto unwind;
> > > >
> > > > - afr_replies_interpret(frame, this, local->inode, NULL);
> > > > + ret = afr_replies_interpret(frame, this, local->inode, NULL);
> > > > + if (ret) {
> > > > + afr_inode_need_refresh_set(local->inode, this);
> > > > + }
> > > >
> > > > read_subvol = afr_read_subvol_decide(local->inode, this, NULL,
> > > > data_readable);
> > > > @@ -3284,11 +3236,7 @@ afr_discover(call_frame_t *frame, xlator_t *this, loc_t *loc, dict_t *xattr_req)
> > > > afr_read_subvol_get(loc->inode, this, NULL, NULL, &event,
> > > > AFR_DATA_TRANSACTION, NULL);
> > > >
> > > > - if (afr_is_inode_refresh_reqd(loc->inode, this, event,
> > > > - local->event_generation))
> > > > - afr_inode_refresh(frame, this, loc->inode, NULL, afr_discover_do);
> > > > - else
> > > > - afr_discover_do(frame, this, 0);
> > > > + afr_discover_do(frame, this, 0);
> > > >
> > > > return 0;
> > > > out:
> > > > @@ -3429,11 +3377,7 @@ afr_lookup(call_frame_t *frame, xlator_t *this, loc_t *loc, dict_t *xattr_req)
> > > > afr_read_subvol_get(loc->parent, this, NULL, NULL, &event,
> > > > AFR_DATA_TRANSACTION, NULL);
> > > >
> > > > - if (afr_is_inode_refresh_reqd(loc->inode, this, event,
> > > > - local->event_generation))
> > > > - afr_inode_refresh(frame, this, loc->parent, NULL, afr_lookup_do);
> > > > - else
> > > > - afr_lookup_do(frame, this, 0);
> > > > + afr_lookup_do(frame, this, 0);
> > > >
> > > > return 0;
> > > > out:
> > > > diff --git a/xlators/cluster/afr/src/afr-dir-write.c b/xlators/cluster/afr/src/afr-dir-write.c
> > > > index 82a72fddd..333085b14 100644
> > > > --- a/xlators/cluster/afr/src/afr-dir-write.c
> > > > +++ b/xlators/cluster/afr/src/afr-dir-write.c
> > > > @@ -119,11 +119,11 @@ __afr_dir_write_finalize(call_frame_t *frame, xlator_t *this)
> > > > continue;
> > > > if (local->replies[i].op_ret < 0) {
> > > > if (local->inode)
> > > > - afr_inode_event_gen_reset(local->inode, this);
> > > > + afr_inode_need_refresh_set(local->inode, this);
> > > > if (local->parent)
> > > > - afr_inode_event_gen_reset(local->parent, this);
> > > > + afr_inode_need_refresh_set(local->parent, this);
> > > > if (local->parent2)
> > > > - afr_inode_event_gen_reset(local->parent2, this);
> > > > + afr_inode_need_refresh_set(local->parent2, this);
> > > > continue;
> > > > }
> > > >
> > > > diff --git a/xlators/cluster/afr/src/afr.h b/xlators/cluster/afr/src/afr.h
> > > > index a3f2942b3..ed6d777c1 100644
> > > > --- a/xlators/cluster/afr/src/afr.h
> > > > +++ b/xlators/cluster/afr/src/afr.h
> > > > @@ -958,7 +958,10 @@ afr_inode_read_subvol_set(inode_t *inode, xlator_t *this,
> > > > int event_generation);
> > > >
> > > > int
> > > > -afr_inode_event_gen_reset(inode_t *inode, xlator_t *this);
> > > > +__afr_inode_need_refresh_set(inode_t *inode, xlator_t *this);
> > > > +
> > > > +int
> > > > +afr_inode_need_refresh_set(inode_t *inode, xlator_t *this);
> > > >
> > > > int
> > > > afr_read_subvol_select_by_policy(inode_t *inode, xlator_t *this,
> > > > --
> > > > 2.25.2
> > > >
> > >
> > >
More information about the Gluster-users
mailing list