[Gluster-users] double traffic usage since upgrade?

[Shehjar Tikoo]

Mark Mielke wrote:
> Possibly relevant here -
> 
> At work, we have used a tool which does something similar to
> booster to accelerate an extremely slow remote file system. It
> works the same way with LD_PRELOAD, however, it also requires GLIBC
> to be compiled with --disable-hidden-plt. Reviewing the Internet
> for similar solutions, will find PlasticFS which also has the same
> requirement.
> 
> Recent versions of GLIBC call open() internally without following
> the regular the regular PLT name resolution model. This increases 
> performance as the PLT indirect lookup model has an expense
> associated. For example, GLIBC fopen() calls open() directly rather
> than going through the PLT. So, overriding open() does not
> intercept calls to fopen()?
> 
> Is this something the booster developers are aware of? Have they
> found a way around this, or is it possible that booster is only
> boosting *some* types of access, and other types of access are
> still "falling through" to FUSE?
> 
> I've asked the developer who wrote out library what he thought of 
> glusterfs/booster not requiring GLIBC with --disable-hidden-plt,
> and he thinks glusterfs/booster cannot be working (or cannot be
> intercepting all calls and some calls are leaking through to FUSE).
> Comments?
> 
> If some calls were leaking through, this might have the "double
> traffic" effect, since FUSE would have its own cache separate from
> booster?
> 

I dont know what a PLT is but I'll attempt to provide some clarity here.

It is true, that booster does not support or boost all system calls.
We do not require that glibc be built with --disable-hidden-plt
for those calls which we do support.
For a start, we've aimed at getting apache and unfs3 to work with 
booster. The functional support for both in booster is complete in
2.0.6 release.

For a list of system calls supported by booster, please see:
http://www.gluster.org/docs/index.php/BoosterConfiguration

There can be applications which need un-boosted syscalls also to be
usable over GlusterFS. For such a scenario we have two ways booster
can be used. Both approaches are described at the page linked above
but in short, you're right in thinking that when the un-supported
syscalls are also needed to go over FUSE, we are, as you said, leaking
or redirecting calls over the FUSE mount point.

That page is a bit long so feel free to ask any questions here.

Thanks
-Shehjar


> Cheers, mark
> 
> 
> 
> On 08/14/2009 01:22 PM, Anand Avati wrote:
>>> I've been running 2.0.3 with two backend bricks and a frontend
>>> client of mod_gluster/apache 2.2.11+worker for a few weeks now
>>> without much issue. Last night i upgraded to 2.0.6 only to find
>>> out that mod_gluster has been removed and is recommending to
>>> use the booster library - which is fine but i didnt have time
>>> to test it last night so i just mounted the whole filesystem 
>>> with a fuse mount and figured id test the booster config later
>>> and then swap.  I did try running the 2.0.3 mod_gluster module
>>> with the 2.0.6 bricks but apache kept segfaulting (every 10
>>> seconds) and then would spawn another process which would
>>> reconnect and keep going.  I figured it was dropping a client
>>> request every few seconds which is why i went with the fuse
>>> mount until i could test the booster library.
>>> 
>> 
>> That would not work, swapping binaries across versions.
>> 
>> 
>>> Well, before with mod_gluster, we would be pushing around
>>> 200mbit of web traffic and it would evenly distribute that
>>> 200mbit between our two bricks - so server1 would be pushing
>>> 100mbit and server2 would be pushing another 100mbit.
>>> Basically both inbound from the backend bricks and outbound 
>>> from apache was basically identical.  Except of course if one
>>> of the backend glusterd processes died for whatever reason the
>>> other remaining brick would take the whole load and its traffic
>>> would double as you would expect. Perfect, all was happy.
>>> 
>>> Now using gluster 2.0.6 and fuse both server bricks are pushing
>>> the full 200mbit of traffic - so i basically have 400mbit of
>>> incoming traffic from the gluster bricks but the same 200mbit
>>> of web traffic.  I can deal, but i only have a shared gigabit
>>> link between my client server and backend bricks and im already
>>> eating up basically 50% of that pipe.  It is also putting a 
>>> much larger load on both bricks since i have basically doubled
>>> the disk IO time and traffic.  Is this a feature? Bug?
>>> 
>> 
>> If I understand correct, 2.0.3 mod_glusterfs = 1x, 2.0.6 fuse =
>> 2x? Can you describe the files being served? (average file size
>> and number of files)
>> 
>> Avati _______________________________________________ 
>> Gluster-users mailing list Gluster-users at gluster.org 
>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>> 
>> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
> _______________________________________________ Gluster-users
> mailing list Gluster-users at gluster.org 
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users