[Gluster-users] poor performance

[Jaco Kroon]

Hi,

The problem is files generated by wordpress, and uploads etc ... so 
copying them to frontend hosts whilst making perfect sense assumes I 
have control over the code to not write to the local front-end, else we 
could have relied on something like lsync.

As it stands, performance is acceptable with nl-cache enabled, but the 
fact that we get those ENOENT errors are highly problematic.


Kind Regards,
Jaco Kroon


n 2022/12/14 14:04, Péter Károly JUHÁSZ wrote:

> When we used glusterfs for websites, we copied the web dir from 
> gluster to local on frontend boots, then served it from there.
>
> Jaco Kroon <jaco at uls.co.za> 于 2022年12月14日周三 12:49写道：
>
>     Hi All,
>
>     We've got a glusterfs cluster that houses some php web sites.
>
>     This is generally considered a bad idea and we can see why.
>
>     With performance.nl-cache on it actually turns out to be very
>     reasonable, however, with this turned of performance is roughly 5x
>     worse.  meaning a request that would take sub 500ms now takes 2500ms.
>     In other cases we see far, far worse cases, eg, with nl-cache takes
>     ~1500ms, without takes ~30s (20x worse).
>
>     So why not use nl-cache?  Well, it results in readdir reporting files
>     which then fails to open with ENOENT.  The cache also never clears
>     even
>     though the configuration says nl-cache entries should only be
>     cached for
>     60s.  Even for "ls -lah" in affected folders you'll notice ???? mark
>     entries for attributes on files.  If this recovers in a reasonable
>     time
>     (say, a few seconds, sure).
>
>     # gluster volume info
>     Type: Replicate
>     Volume ID: cbe08331-8b83-41ac-b56d-88ef30c0f5c7
>     Status: Started
>     Snapshot Count: 0
>     Number of Bricks: 1 x 2 = 2
>     Transport-type: tcp
>     Options Reconfigured:
>     performance.nl-cache: on
>     cluster.readdir-optimize: on
>     config.client-threads: 2
>     config.brick-threads: 4
>     config.global-threading: on
>     performance.iot-pass-through: on
>     storage.fips-mode-rchecksum: on
>     cluster.granular-entry-heal: enable
>     cluster.data-self-heal-algorithm: full
>     cluster.locking-scheme: granular
>     client.event-threads: 2
>     server.event-threads: 2
>     transport.address-family: inet
>     nfs.disable: on
>     cluster.metadata-self-heal: off
>     cluster.entry-self-heal: off
>     cluster.data-self-heal: off
>     cluster.self-heal-daemon: on
>     server.allow-insecure: on
>     features.ctime: off
>     performance.io-cache: on
>     performance.cache-invalidation: on
>     features.cache-invalidation: on
>     performance.qr-cache-timeout: 600
>     features.cache-invalidation-timeout: 600
>     performance.io-cache-size: 128MB
>     performance.cache-size: 128MB
>
>     Are there any other recommendations short of abandon all hope of
>     redundancy and to revert to a single-server setup (for the web
>     code at
>     least).  Currently the cost of the redundancy seems to outweigh
>     the benefit.
>
>     Glusterfs version 10.2.  With patch for --inode-table-size, mounts
>     happen with:
>
>     /usr/sbin/glusterfs --acl --reader-thread-count=2 --lru-limit=524288
>     --inode-table-size=524288 --invalidate-limit=16 --background-qlen=32
>     --fuse-mountopts=nodev,nosuid,noexec,noatime --process-name fuse
>     --volfile-server=127.0.0.1 --volfile-id=gv_home
>     --fuse-mountopts=nodev,nosuid,noexec,noatime /home
>
>     Kind Regards,
>     Jaco
>
>     ________
>
>
>
>     Community Meeting Calendar:
>
>     Schedule -
>     Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>     Bridge: https://meet.google.com/cpu-eiue-hvk
>     Gluster-users mailing list
>     Gluster-users at gluster.org
>     https://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20221214/ac58a5c9/attachment.html>