<div dir="ltr">With these 2 changes, we are getting a good improvement in file creation and <div>slight improvement in the "ls-l" operation.<div><br></div><div>We are still working to improve the same.<br><br>To validate the same we have executed below script from 6 different clients on 24x3 distributed<br>replicate environment after enabling performance related option <br><br>mkdir /gluster-mount/`hostname`<br>date;<br>for i in {1..100}<br>do<br>echo "directory $i is created" `date`<br>mkdir /gluster-mount/`hostname`/dir$i<br>tar -xvf /root/kernel_src/linux-5.4-rc8.tar.xz -C /gluster-mount/`hostname`/dir$i >/dev/null<br>done<br><br>With no Patch<br>tar was taking almost 36-37 hours<br><br>With Patch<br>tar is taking almost 26 hours<br><br>We were getting a similar kind of improvement in smallfile tool also.<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 25, 2020 at 1:29 PM Mohit Agrawal <<a href="mailto:moagrawa@redhat.com">moagrawa@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,<br></div>We observed performance is mainly hurt while .glusterfs is having huge data.As we know before executing a fop in POSIX xlator it builds an internal path based on GFID.To validate the path it call's (l)stat system call and while .glusterfs is heavily loaded kernel takes time to lookup inode and due to that performance drops<br>To improve the same we tried two things with this patch(<a href="https://review.gluster.org/#/c/glusterfs/+/23783/" target="_blank">https://review.gluster.org/#/c/glusterfs/+/23783/</a>)<br><br>1) To keep the first level entry always in a cache so that inode lookup will be faster we have to keep open first level fds(00 to ff total 256) per brick at the time of starting a brick process. Even in case of cache cleanup kernel will not evict first level fds from the cache and performance will improve<br><br>2) We tried using "at" based call(lstatat,fstatat,readlinat etc) instead of accessing complete path access relative path, these call's were also helpful to improve performance.<br><div><br></div><div>Regards,</div><div>Mohit Agrawal</div><div><br></div><div><pre style="white-space:pre-wrap;color:rgb(0,0,0)"><br></pre></div></div>
</blockquote></div>