[Gluster-users] double traffic usage since upgrade?

[Liam Slusser]

On Fri, Aug 14, 2009 at 10:22 AM, Anand Avati <avati at gluster.com> wrote:

> > I've been running 2.0.3 with two backend bricks and a frontend client of
> > mod_gluster/apache 2.2.11+worker for a few weeks now without much issue.
> >  Last night i upgraded to 2.0.6 only to find out that mod_gluster has
> been
> > removed and is recommending to use the booster library - which is fine
> but i
> > didnt have time to test it last night so i just mounted the whole
> filesystem
> > with a fuse mount and figured id test the booster config later and then
> > swap.  I did try running the 2.0.3 mod_gluster module with the 2.0.6
> bricks
> > but apache kept segfaulting (every 10 seconds) and then would spawn
> another
> > process which would reconnect and keep going.  I figured it was dropping
> a
> > client request every few seconds which is why i went with the fuse mount
> > until i could test the booster library.
>
> That would not work, swapping binaries across versions.


It actually did work - but like i said it was segfaulting every few seconds.
 Still our traffic looked normal...


> > Well, before with mod_gluster, we would be pushing around 200mbit of web
> > traffic and it would evenly distribute that 200mbit between our two
> bricks -
> > so server1 would be pushing 100mbit and server2 would be pushing another
> > 100mbit.  Basically both inbound from the backend bricks and outbound
> from
> > apache was basically identical.  Except of course if one of the backend
> > glusterd processes died for whatever reason the other remaining brick
> would
> > take the whole load and its traffic would double as you would expect.
> >  Perfect, all was happy.
> >
> > Now using gluster 2.0.6 and fuse both server bricks are pushing the full
> > 200mbit of traffic - so i basically have 400mbit of incoming traffic from
> > the gluster bricks but the same 200mbit of web traffic.  I can deal, but
> i
> > only have a shared gigabit link between my client server and backend
> bricks
> > and im already eating up basically 50% of that pipe.  It is also putting
> a
> > much larger load on both bricks since i have basically doubled the disk
> IO
> > time and traffic.  Is this a feature? Bug?
>
> If I understand correct, 2.0.3 mod_glusterfs = 1x, 2.0.6 fuse = 2x?
> Can you describe the files being served? (average file size and number
> of files)
>
> Avati
>

Correct - i was running 2.0.3 with mod_gluster and now 2.0.6 fuse.  Same
apache+worker, same incoming request traffic, same 200mbit outbound traffic,
just how i'm seeing double the traffic on each brick (from 100mbit to
200mbit).  I am serving ~3 meg AAC and MP3 files audio files - around 5-8
requests per second and around 200mbit/second of traffic.

thanks,
liam