[Gluster-users] Performance drop when upgrading from 3.8 to 6.5

[RAFI KC]

On 11/4/19 2:41 PM, Riccardo Murri wrote:
> Hello Amar,
>
>> Can you please check the profile info [1]  ? That may give some hints.
> I am attaching the output of `sudo gluster volume profile info` as a text file
> to preserve formatting.  This covers the time from Friday night to
> Monday morning;
> during this time the cluster has been the target of a an `rsync` command copying
> a large directory tree (14TB; it's taking more than 2 weeks now...).
>
> My take from cursorily reading the profile:
>
> - metadata operations (opendir, create, rename) seem to have very high
> latency (up to 25 seconds for a rename!)


We have identified a performance drop with ctime and rename operations. 
This is because ctime stores some metadata information as an extended 
attributes which sometimes exceeds the default inode size. In such 
scenarios the additional xattrs won't fit into the default size. This 
will result in additional blocks to be used which will effect the latency.

Is it possible for you to repeat the test by disabling ctime or 
increasing the inode size to a higher value say 1024?


> - large block size (>128kB) are rare; most blocks seems to be 8kB,
> 16kB, or 128kB.
>
> I personally do not see any obvious culprit or optimization
> opportunities; is there something I'm missing out that catches the eye
> of someone more experienced?
>
> Thanks,
> R
>
> ________
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/118564314
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/118564314
>
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20191105/3aa1166b/attachment.html>