[Gluster-users] Upgrade 5.5 -> 5.6: network traffic bug fixed?

[Hu Bert]

Hi Poornima,

thx for your efforts. I made a couple of tests and the results are the
same, so the options are not related. Anyway, i'm not able to
reproduce the problem on my testing system, although the volume
options are the same.

About 1.5 hours ago i set performance.quick-read to on again and
watched: load/iowait went up (not bad at the moment, little traffic),
but network traffic went up - from <20 MBit/s up to 160 MBit/s. After
deactivating quick-read traffic dropped to < 20 MBit/s again.

munin graph: https://abload.de/img/network-client4s0kle.png

The 2nd peak is from the last test.


Thx,
Hubert

Am Di., 16. Apr. 2019 um 09:43 Uhr schrieb Hu Bert <revirii at googlemail.com>:
>
> In my first test on my testing setup the traffic was on a normal
> level, so i thought i was "safe". But on my live system the network
> traffic was a multiple of the traffic one would expect.
> performance.quick-read was enabled in both, the only difference in the
> volume options between live and testing are:
>
> performance.read-ahead: testing on, live off
> performance.io-cache: testing on, live off
>
> I ran another test on my testing setup, deactivated both and copied 9
> GB of data. Now the traffic went up as well, from before ~9-10 MBit/s
> up to 100 MBit/s with both options off. Does performance.quick-read
> require one of those options set to 'on'?
>
> I'll start another test shortly, and activate on of those 2 options,
> maybe there's a connection between those 3 options?
>
>
> Best Regards,
> Hubert
>
> Am Di., 16. Apr. 2019 um 08:57 Uhr schrieb Poornima Gurusiddaiah
> <pgurusid at redhat.com>:
> >
> > Thank you for reporting this. I had done testing on my local setup and the issue was resolved even with quick-read enabled. Let me test it again.
> >
> > Regards,
> > Poornima
> >
> > On Mon, Apr 15, 2019 at 12:25 PM Hu Bert <revirii at googlemail.com> wrote:
> >>
> >> fyi: after setting performance.quick-read to off network traffic
> >> dropped to normal levels, client load/iowait back to normal as well.
> >>
> >> client: https://abload.de/img/network-client-afterihjqi.png
> >> server: https://abload.de/img/network-server-afterwdkrl.png
> >>
> >> Am Mo., 15. Apr. 2019 um 08:33 Uhr schrieb Hu Bert <revirii at googlemail.com>:
> >> >
> >> > Good Morning,
> >> >
> >> > today i updated my replica 3 setup (debian stretch) from version 5.5
> >> > to 5.6, as i thought the network traffic bug (#1673058) was fixed and
> >> > i could re-activate 'performance.quick-read' again. See release notes:
> >> >
> >> > https://review.gluster.org/#/c/glusterfs/+/22538/
> >> > http://git.gluster.org/cgit/glusterfs.git/commit/?id=34a2347780c2429284f57232f3aabb78547a9795
> >> >
> >> > Upgrade went fine, and then i was watching iowait and network traffic.
> >> > It seems that the network traffic went up after upgrade and
> >> > reactivation of performance.quick-read. Here are some graphs:
> >> >
> >> > network client1: https://abload.de/img/network-clientfwj1m.png
> >> > network client2: https://abload.de/img/network-client2trkow.png
> >> > network server: https://abload.de/img/network-serverv3jjr.png
> >> >
> >> > gluster volume info: https://pastebin.com/ZMuJYXRZ
> >> >
> >> > Just wondering if the network traffic bug really got fixed or if this
> >> > is a new problem. I'll wait a couple of minutes and then deactivate
> >> > performance.quick-read again, just to see if network traffic goes down
> >> > to normal levels.
> >> >
> >> >
> >> > Best regards,
> >> > Hubert
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> https://lists.gluster.org/mailman/listinfo/gluster-users