<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">For _every_ file copied samba performs readdir() to get all entries of the destination folder. Then the list is searched for filename (to prevent name collisions as SMB shares are not case sensitive). More files in folder, more time it takes to perform readdir(). It is a lot worse for Gluster because single folder contents distributed among many servers and Gluster has to join many directory listings (requested via network) to form one and return it to caller.</div><div class=""><br class=""></div><div class="">Rsync does not perform readdir(), it just checks file existence with stat() IIRC. And as modern Gluster versions has default setting to check for file only at its destination (when volume is balanced) - the check performs relatively fast.</div><div class=""><br class=""></div><div class="">You can hack samba to prevent such checks if your goal is to get files copied not so slow (as you sure the files you are copying are not exists at destination). But try to perform 'ls -l' on _not_ cached folder with thousands of files - it will take tens of seconds. This is time your users will waste browsing shares.</div><br class=""><div><blockquote type="cite" class=""><div class="">8 февр. 2017 г., в 13:17, Gary Lloyd <<a href="mailto:g.lloyd@keele.ac.uk" class="">g.lloyd@keele.ac.uk</a>> написал(а):</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Thanks for the reply<div class=""><br class=""></div><div class="">I've just done a bit more testing. If I use rsync from a gluster client to copy the same files to the mount point it only takes a couple of minutes.</div><div class="">For some reason it's very slow on samba though (version 4.4.4).</div><div class=""><br class=""></div><div class="">I have tried various samba tweaks / settings and have yet to get acceptable write speed on small files.</div><div class=""><br class=""></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><i class="">Gary Lloyd</i><div class=""><span style="color:rgb(136,136,136)" class="">______________________________</span><span style="color:rgb(136,136,136)" class="">__________________</span><br class=""><div class=""><span style="color:rgb(136,136,136)" class="">I.T. Systems:Keele University</span><br style="color:rgb(136,136,136)" class=""><span style="color:rgb(136,136,136)" class="">Finance & IT Directorate</span></div><div class=""><span style="color:rgb(136,136,136)" class="">Keele:Staffs:IC1 Building:ST5 5NB:UK</span><br style="color:rgb(136,136,136)" class=""><a href="tel:%2B44%201782%20733073" value="+441782733073" style="color:rgb(17,85,204)" target="_blank" class="">+44 1782 733063</a><br style="color:rgb(136,136,136)" class=""><span style="color:rgb(136,136,136)" class="">______________________________</span><span style="color:rgb(136,136,136)" class="">__________________</span></div></div></div></div></div></div></div>
<br class=""><div class="gmail_quote">On 8 February 2017 at 10:05, Дмитрий Глушенок <span dir="ltr" class=""><<a href="mailto:glush@jet.msk.su" target="_blank" class="">glush@jet.msk.su</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class="">Hi,</div><div class=""><br class=""></div><div class="">There is a number of tweaks/hacks to make it better, but IMHO overall performance with small files is still unacceptable for such folders with thousands of entries.</div><div class=""><br class=""></div><div class="">If your shares are not too large to be placed on single filesystem and you still want to use Gluster - it is possible to run VM on top of Gluster. Inside that VM you can create ZFS/NTFS to be shared.</div><br class=""><div class=""><blockquote type="cite" class=""><div class="">8 февр. 2017 г., в 12:10, Gary Lloyd <<a href="mailto:g.lloyd@keele.ac.uk" target="_blank" class="">g.lloyd@keele.ac.uk</a>> написал(а):</div><br class="m_6782863723204035709Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hi</div><div class=""><br class=""></div><div class="">I am currently testing gluster 3.9 replicated/distrbuted on centos 7.3 with samba/ctdb.</div><div class="">I have been able to get it all up and running, but writing small files is really slow. </div><div class=""><br class=""></div><div class="">If I copy large files from gluster backed samba I get almost wire speed (We only have 1Gb at the moment). I get around half that speed if I copy large files to the gluster backed samba system, which I am guessing is due to it being replicated (This is acceptable).<br class=""></div><div class=""><br class=""></div><div class="">Small file write performance seems really poor for us though:</div><div class="">As an example I have an eclipse IDE workspace folder that is 6MB in size that has around 6000 files in it. A lot of these files are <1k in size.</div><div class=""><br class=""></div><div class="">If I copy this up to gluster backed samba it takes almost one hour to get there.</div><div class="">With our basic samba deployment it only takes about 5 minutes.</div><div class=""><br class=""></div><div class="">Both systems reside on the same disks/SAN.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">I was hoping that we would be able to move away from using a proprietary SAN to house our network shares and use gluster instead.</div><div class=""><br class=""></div><div class="">Does anyone have any suggestions of anything I could tweak to make it better ?</div><div class=""><br class=""></div><div class="">Many Thanks</div><div class=""><br class=""></div><br clear="all" class=""><div class=""><div class="m_6782863723204035709gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><i class="">Gary Lloyd</i><div class=""><span style="color:rgb(136,136,136)" class="">______________________________</span><span style="color:rgb(136,136,136)" class=""><wbr class="">__________________</span><br class=""><div class=""><span style="color:rgb(136,136,136)" class="">I.T. Systems:Keele University</span><br style="color:rgb(136,136,136)" class=""><span style="color:rgb(136,136,136)" class="">Finance & IT Directorate</span></div><div class=""><span style="color:rgb(136,136,136)" class="">Keele:Staffs:IC1 Building:ST5 5NB:UK</span><br style="color:rgb(136,136,136)" class=""><span style="color:rgb(136,136,136)" class="">______________________________</span><span style="color:rgb(136,136,136)" class=""><wbr class="">__________________</span></div></div></div></div></div></div></div>
</div>
______________________________<wbr class="">_________________<br class="">Gluster-users mailing list<br class=""><a href="mailto:Gluster-users@gluster.org" target="_blank" class="">Gluster-users@gluster.org</a><br class=""><a href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank" class="">http://lists.gluster.org/<wbr class="">mailman/listinfo/gluster-users</a></div></blockquote></div><br class=""><div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div class="">--</div><div class=""><div style="word-wrap:break-word" class=""><div style="word-wrap:break-word" class=""><div style="word-wrap:break-word" class=""><div class="">Dmitry Glushenok</div><div class="">Jet Infosystems</div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">--</div><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">Dmitry Glushenok</div><div class="">Jet Infosystems</div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br class=""></body></html>