[Gluster-users] Slow write times to gluster disk
Pat Haley
phaley at mit.edu
Mon Jun 25 21:51:43 UTC 2018
Hi Raghavendra,
Setting the performance.write-behind off had a small improvement on the
write speed (~3%),
We were unable to turn on "group metadata-cache". When we try get
errors like
# gluster volume set data-volume group metadata-cache
'/var/lib/glusterd/groups/metadata-cache' file format not valid.
Was metadata-cache available for gluster 3.7.11? We ask because the
release notes for 3.11 mentions “Feature for metadata-caching/small file
performance is production ready.”
(https://gluster.readthedocs.io/en/latest/release-notes/3.11.0/).
Do any of these results suggest anything? If not, what further tests
would be useful?
Thanks
Pat
On 06/22/2018 07:51 AM, Raghavendra Gowdappa wrote:
>
>
> On Thu, Jun 21, 2018 at 8:41 PM, Pat Haley <phaley at mit.edu
> <mailto:phaley at mit.edu>> wrote:
>
>
> Hi Raghavendra,
>
> Thanks for the suggestions. Our technician will be in on Monday.
> We'll test then and let you know the results.
>
> One question I have, is the "group metadata-cache" option supposed
> to directly impact the performance or is it to help collect data?
> If the latter, where will the data be located?
>
>
> It impacts performance.
>
>
> Thanks again.
>
> Pat
>
>
>
> On 06/21/2018 01:01 AM, Raghavendra Gowdappa wrote:
>>
>>
>> On Thu, Jun 21, 2018 at 10:24 AM, Raghavendra Gowdappa
>> <rgowdapp at redhat.com <mailto:rgowdapp at redhat.com>> wrote:
>>
>> For the case of writes to glusterfs mount,
>>
>> I saw in earlier conversations that there are too many
>> lookups, but small number of writes. Since writes cached in
>> write-behind would invalidate metadata cache, lookups won't
>> be absorbed by md-cache. I am wondering what would results
>> look like if we turn off performance.write-behind.
>>
>> @Pat,
>>
>> Can you set,
>>
>> # gluster volume set <volname> performance.write-behind off
>>
>>
>> Please turn on "group metadata-cache" for write tests too.
>>
>>
>> and redo the tests writing to glusterfs mount? Let us know
>> about the results you see.
>>
>> regards,
>> Raghavendra
>>
>> On Thu, Jun 21, 2018 at 8:33 AM, Raghavendra Gowdappa
>> <rgowdapp at redhat.com <mailto:rgowdapp at redhat.com>> wrote:
>>
>>
>>
>> On Thu, Jun 21, 2018 at 8:32 AM, Raghavendra Gowdappa
>> <rgowdapp at redhat.com <mailto:rgowdapp at redhat.com>> wrote:
>>
>> For the case of reading from Glusterfs mount,
>> read-ahead should help. However, we've known issues
>> with read-ahead[1][2]. To work around these, can you
>> try with,
>>
>> 1. Turn off performance.open-behind
>> #gluster volume set <volname> performance.open-behind off
>>
>> 2. enable group meta metadata-cache
>> # gluster volume set <volname> group metadata-cache
>>
>>
>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1084508
>> <https://bugzilla.redhat.com/show_bug.cgi?id=1084508>
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1214489
>> <https://bugzilla.redhat.com/show_bug.cgi?id=1214489>
>>
>>
>>
>>
>> On Thu, Jun 21, 2018 at 5:00 AM, Pat Haley
>> <phaley at mit.edu <mailto:phaley at mit.edu>> wrote:
>>
>>
>> Hi,
>>
>> We were recently revisiting our problems with the
>> slowness of gluster writes
>> (http://lists.gluster.org/pipermail/gluster-users/2017-April/030529.html
>> <http://lists.gluster.org/pipermail/gluster-users/2017-April/030529.html>).
>> Specifically we were testing the suggestions in a
>> recent post
>> (http://lists.gluster.org/pipermail/gluster-users/2018-March/033699.html
>> <http://lists.gluster.org/pipermail/gluster-users/2018-March/033699.html>).
>> The first two suggestions (specifying a
>> negative-timeout in the mount settings or adding
>> rpc-auth-allow-insecure to glusterd.vol) did not
>> improve our performance, while setting
>> "disperse.eager-lock off" provided a tiny (5%)
>> speed-up.
>>
>> Some of the various tests we have tried earlier
>> can be seen in the links below. Do any of the
>> above observations suggest what we could try next
>> to either improve the speed or debug the issue?
>> Thanks
>>
>> http://lists.gluster.org/pipermail/gluster-users/2017-June/031565.html
>> <http://lists.gluster.org/pipermail/gluster-users/2017-June/031565.html>
>> http://lists.gluster.org/pipermail/gluster-users/2017-May/030937.html
>> <http://lists.gluster.org/pipermail/gluster-users/2017-May/030937.html>
>>
>> Pat
>>
>> --
>>
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> Pat Haley Email: phaley at mit.edu
>> <mailto:phaley at mit.edu>
>> Center for Ocean Engineering Phone: (617) 253-6824
>> Dept. of Mechanical Engineering Fax: (617)
>> 253-8125
>> MIT, Room 5-213 http://web.mit.edu/phaley/www/
>> 77 Massachusetts Avenue
>> Cambridge, MA 02139-4301
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> <mailto:Gluster-users at gluster.org>
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>> <http://lists.gluster.org/mailman/listinfo/gluster-users>
>>
>>
>>
>>
>>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley Email:phaley at mit.edu <mailto:phaley at mit.edu>
> Center for Ocean Engineering Phone: (617) 253-6824
> Dept. of Mechanical Engineering Fax: (617) 253-8125
> MIT, Room 5-213http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA 02139-4301
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://lists.gluster.org/mailman/listinfo/gluster-users
> <http://lists.gluster.org/mailman/listinfo/gluster-users>
>
>
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: phaley at mit.edu
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA 02139-4301
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180625/5293a301/attachment.html>
More information about the Gluster-users
mailing list