[Gluster-users] Slow read performance

Ling Ho ling at slac.stanford.edu
Fri Mar 1 00:53:48 UTC 2013


Tom,

What type of disks do you have? If they are raid 5 or 6, have you try 
setting the read-ahead size to 8192 or 16384 (blockdev --setra 8192 
/dev/<sd?>   ?

...
ling

On 02/28/2013 04:23 PM, Thomas Wakefield wrote:
> Did anyone else have any ideas on performance tuning for reads?
>
> On Feb 27, 2013, at 9:29 PM, Thomas Wakefield <twake at iges.org 
> <mailto:twake at iges.org>> wrote:
>
>> Bryan-
>>
>> Yes I can write at 700-800MBytes/sec, but i can only read at *70-80 
>> MBytes/sec*.  I would be very happy if I could get it to read at the 
>> same speed it can write at.  And the 70-80 is sequential, not random 
>> for reads, same exact test commands on the disk server are in the 
>> 2+GB/s range, so I know the disk server can do it.
>>
>> -Tom
>>
>>
>> On Feb 27, 2013, at 7:41 PM, Bryan Whitehead <driver at megahappy.net 
>> <mailto:driver at megahappy.net>> wrote:
>>
>>> Are your figures 700-800M*Byte*/sec? Because that is probably as 
>>> fast as your 10G nic cards are able to do. You can test that by 
>>> trying to push a large amount of data over nc or ftp.
>>>
>>> Might want to try Infiniband. 40G cards are pretty routine.
>>>
>>>
>>> On Wed, Feb 27, 2013 at 3:45 PM, Thomas Wakefield <twake at iges.org 
>>> <mailto:twake at iges.org>> wrote:
>>>
>>>     I also get the same performance running iozone for large file
>>>     sizes, iozone -u 1 -r 512k -s 2G -I -F.
>>>
>>>     Large file IO is what I need the system to do.  I am just
>>>     shocked at the huge difference between local IO and gluster
>>>     client IO.  I know there should be some difference, but 10x is
>>>     unacceptable.
>>>
>>>     -Tom
>>>
>>>
>>>
>>>     On Feb 27, 2013, at 5:31 PM, Bryan Whitehead
>>>     <driver at megahappy.net <mailto:driver at megahappy.net>> wrote:
>>>
>>>>     Every time you open/close a file or a directory you will have
>>>>     to wait for locks which take time. This is totally expected.
>>>>
>>>>     Why don't you share what you want to do? iozone benchmarks look
>>>>     like crap but serving qcow2 files to qemu works fantastic for
>>>>     me. What are you doing? Make a benchmark that does that. If you
>>>>     are going to have many files with a wide variety of sizes
>>>>     glusterfs/fuse might not be what you are looking for.
>>>>
>>>>
>>>>     On Wed, Feb 27, 2013 at 12:56 PM, Thomas Wakefield
>>>>     <twake at cola.iges.org <mailto:twake at cola.iges.org>> wrote:
>>>>
>>>>         I have tested everything, small and large files.  I have
>>>>         used file sizes ranging from 128k up to multiple GB files.
>>>>          All the reads are bad.
>>>>
>>>>
>>>>
>>>>         Here is a fairly exhaustive iozone auto test:
>>>>
>>>>                                                                    
>>>>         random  random    bkwd   record   stride
>>>>                       KB  reclen   write rewrite    read    reread
>>>>            read   write    read  rewrite     read   fwrite frewrite
>>>>           fread  freread
>>>>                       64       4   40222   63492    26868    30060
>>>>            1620   71037    1572    70570    31294    77096    72475
>>>>           14736    13928
>>>>                       64       8   99207  116366    13591    13513
>>>>            3214   97690    3155   109978    28920   152018   158480
>>>>           18936    17625
>>>>                       64      16  230257  253766    25156    28713
>>>>           10867  223732    8873   244297    54796   303383   312204
>>>>           15062    13545
>>>>                       64      32  255943  234481  5735102  7100397
>>>>           11897  318502   13681   347801    24214   695778   528618
>>>>           25838    28094
>>>>                       64      64  214096  681644  6421025  7100397
>>>>           27453  292156   28117   621657    27338   376062   512471
>>>>           28569    32534
>>>>                      128       4   74329   75468    26428    41089
>>>>            1131   72857    1118    66976     1597    73778    78343
>>>>           13351    13026
>>>>                      128       8  100862  135170    24966    16734
>>>>            2617  118966    2560   120406    39156   125121   146613
>>>>           16177    16180
>>>>                      128      16  115114  253983    28212    17854
>>>>            5307  246180    5431   229843    47335   255920   271173
>>>>           27256    24445
>>>>                      128      32  256042  391360    39848    64258
>>>>           11329  290230    9905   429563    38176   490380   463696
>>>>           20917    19219
>>>>                      128      64  248573  592699  4557257  6812590
>>>>           19583  452366   29263   603357    42967   814915   692017
>>>>           76327    37604
>>>>                      128     128  921183  526444  5603747  5379161
>>>>           45614  390222   65441   826202    41384   662962  1040839
>>>>           78526    39023
>>>>                      256       4   76212   77337    40295    32125
>>>>            1289   71866    1261    64645     1436    57309    53048
>>>>           23073    29550
>>>>                      256       8  126922  141976    26237    25130
>>>>            2566  128058    2565   138981     2985   125060   133603
>>>>           22840    24955
>>>>                      256      16  242883  263636    41850    24371
>>>>            4902  250009    5290   248792    89353   243821   247303
>>>>           26965    26199
>>>>                      256      32  409074  439732    40101    39335
>>>>           11953  436870   11209   430218    83743   409542   479390
>>>>           30821    27750
>>>>                      256      64  259935  571502    64840    71847
>>>>           22537  617161   23383   392047    91852   672010   802614
>>>>           41673    53111
>>>>                      256     128  847597  812329   185517    83198
>>>>           49383  708831   44668   794889    74267  1180188  1662639
>>>>           54303    41018
>>>>                      256     256  481324  709299  5217259  5320671
>>>>           44668  719277   40954   808050    41302   790209   771473
>>>>           62224    35754
>>>>                      512       4   77667   75226    35102    29696
>>>>            1337   66262    1451    67680     1413    69265    69142
>>>>           42084    27897
>>>>                      512       8  134311  144341    30144    24646
>>>>            2102  134143    2209   134699     2296   108110   128616
>>>>           25104    29123
>>>>                      512      16  200085  248787    30235    25697
>>>>            4196  247240    4179   256116     4768   250003   226436
>>>>           32351    28455
>>>>                      512      32  330341  439805    26440    39284
>>>>            8744  457611    8006   424168   125953   425935   448813
>>>>           27660    26951
>>>>                      512      64  483906  733729    48747    41121
>>>>           16032  555938   17424   587256   187343   366977   735740
>>>>           41700    41548
>>>>                      512     128  836636  907717    69359    94921
>>>>           42443  761031   36828   964378   123165   651383   695697
>>>>           58368    44459
>>>>                      512     256  520879  860437   145534   135523
>>>>           40267  847532   31585   663252    69696  1270846  1492545
>>>>           48822    48092
>>>>                      512     512  782951  973118  3099691  2942541
>>>>           42328  871966   46218   911184    49791   953248  1036527
>>>>           52723    48347
>>>>                     1024       4   76218   69362    36431    28711
>>>>            1137   66171    1174    68938     1125    70566    70845
>>>>           34942    28914
>>>>                     1024       8  126045  140524    37836    15664
>>>>            2698  126000    2557   125566     2567   110858   127255
>>>>           26764    27945
>>>>                     1024      16  243398  261429    40238    23263
>>>>            3987  246400    3882   260746     4093   236652   236874
>>>>           31429    25076
>>>>                     1024      32  383109  422076    41731    41605
>>>>            8277  473441    7775   415261     8588   394765   407306
>>>>           40089    28537
>>>>                     1024      64  590145  619156    39623    53267
>>>>           15051  722717   14624   753000   257294   597784   620946
>>>>           38619    44073
>>>>                     1024     128 1077836 1124099    56192    64916
>>>>           36851 1102176   37198  1082454   281548   829175   792604
>>>>           47975    51913
>>>>                     1024     256  941918 1074331    72783    81450
>>>>           26778 1099636   32395  1060013   183218  1024121   995171
>>>>           44371    45448
>>>>                     1024     512  697483 1130312   100324   114682
>>>>           48215 1041758   41480  1058967    90156   994020  1563622
>>>>           56328    46370
>>>>                     1024    1024  931702 1087111  4609294  4199201
>>>>           44191  949834   45594   970656    56674   933525  1075676
>>>>           44876    46115
>>>>                     2048       4   71438   67066    58319    38913
>>>>            1147   44147    1043    42916      967    66416    67205
>>>>           45953    96750
>>>>                     2048       8  141926  134567    61101    55445
>>>>            2596   77528    2564    80402     4258   124211   120747
>>>>           53888   100337
>>>>                     2048      16  254344  255585    71550    74500
>>>>            5410  139365    5201   141484     5171   205521   213113
>>>>           67048    57304
>>>>                     2048      32  397833  411261    56676    80027
>>>>           10440  260034   10126   230238    10814   391665   383379
>>>>           79333    60877
>>>>                     2048      64  595167  687205    64262    87327
>>>>           20772  456430   19960   477064    23190   540220   563096
>>>>           86812    92565
>>>>                     2048     128  833585  933403   121926   118621
>>>>           37700  690020   37575   733254   567449   712337   734006
>>>>           92011   104934
>>>>                     2048     256  799003  949499   143688   125659
>>>>           40871  892757   37977   880494   458281   836263   901375
>>>>          131332   110237
>>>>                     2048     512  979936 1040724   120896   138013
>>>>           54381  859783   48721   780491   279203  1068824  1087085
>>>>           97886    98078
>>>>                     2048    1024  901754  987938    53352    53043
>>>>           72727 1054522   68269   992275   181253  1309480  1524983
>>>>          121600    95585
>>>>                     2048    2048  831890 1021540  4257067  3302797
>>>>           75672  984203   80181   826209    94278   966920  1027159
>>>>          111832   105921
>>>>                     4096       4   66195   67316    62171    74785
>>>>            1328   28963    1329    26397     1223    71470    69317
>>>>           55903    84915
>>>>                     4096       8  122221  120057    90537    60958
>>>>            2598   47312    2468    59783     2640   128674   127872
>>>>           41285    40422
>>>>                     4096      16  238321  239251    29336    32121
>>>>            4153   89262    3986    96930     4608   229970   237108
>>>>           55039    56983
>>>>                     4096      32  417110  421356    30974    50000
>>>>            8382  156676    7886   153841     7900   359585   367288
>>>>           26611    25952
>>>>                     4096      64  648008  668066    32193    29389
>>>>           14830  273265   14822   282211    19653   581898   620798
>>>>           51281    50218
>>>>                     4096     128  779422  848564    55594    60253
>>>>           37108  451296   35908   491361    37567   738163   728059
>>>>           67681    66440
>>>>                     4096     256  865623  886986    71368    63947
>>>>           44255  645961   42689   719491   736707   819696   837641
>>>>           57059    60347
>>>>                     4096     512  852099  889650    68870    73891
>>>>           31185  845224   30259   830153   392334   910442   961983
>>>>           60083    55558
>>>>                     4096    1024  710357  867810    29377    29522
>>>>           49954  846640   43665   926298   213677   986226  1115445
>>>>           55130    59205
>>>>                     4096    2048  826479  908420    43191    42075
>>>>           59684  904022   58601   855664   115105  1418322  1524415
>>>>           60548    66066
>>>>                     4096    4096  793351  855111  3232454  3673419
>>>>           66018  861413   48833   847852    45914   852268   842075
>>>>           42980    48374
>>>>                     8192       4   67340   69421    42198    31740
>>>>             994   23251    1166    16813      837    73827    73126
>>>>           25169    29610
>>>>                     8192       8  137150  125622    29131    36439
>>>>            2051   44342    1988    48930     2315   134183   135367
>>>>           31080    33573
>>>>                     8192      16  237366  220826    24810    26584
>>>>            3576   88004    3769    78717     4289   233751   235355
>>>>           23302    28742
>>>>                     8192      32  457447  454404    31594    27750
>>>>            8141  142022    7846   143984     9322   353147   396188
>>>>           34203    33265
>>>>                     8192      64  670645  655259    28630    23255
>>>>           16669  237476   16965   244968    15607   590365   575320
>>>>           49998    43305
>>>>                     8192     128  658676  760982    44197    47802
>>>>           28693  379523   26614   378328    27184   720997   702038
>>>>           51707    49733
>>>>                     8192     256  643370  698683    56233    63165
>>>>           28846  543952   27745   576739    44014   701007   725534
>>>>           59611    58985
>>>>                     8192     512  696884  776793    67258    52705
>>>>           18711  698854   21004   694124   621695   784812   773331
>>>>           43101    47659
>>>>                     8192    1024  729664  810451   15470    15875  
>>>>         31318  801490   38123   812944   301222   804323   832765  
>>>>         54308    53376
>>>>                     8192    2048  749217   68757    21914    22667
>>>>           48971  783309   48132   782738   172848   907408   929324
>>>>           51156    50565
>>>>                     8192    4096  707677  763960    32063    31928
>>>>           47809  751692   49560   786339    93445  1046761  1297876
>>>>           48037    51680
>>>>                     8192    8192  623817  746288  2815955  3137358
>>>>           48722  741633   35428   753787    49626   803683   823800
>>>>           48977    52895
>>>>                    16384       4   72372   73651    34471    30788
>>>>             960   23610     903    22316      891    71445    71138
>>>>           56451    55129
>>>>                    16384       8  137920  141704    50830    33857
>>>>            1935   41934    2275    35588     3608   130757   137801
>>>>           51621    48525
>>>>                    16384      16  245369  242460    41808    29770
>>>>            3605   75682    4355    75315     4767   241100   239693
>>>>           53263    30785
>>>>                    16384      32  448877  433956    31846    35010
>>>>            7973  118181    8819   112703     8177   381734   391651
>>>>           57749    63417
>>>>                    16384      64  710831  700712    66792    68864
>>>>           20176  209806   19034   207852    21255   589503   601379
>>>>          104567   105162
>>>>                    16384     128  836901  860867   104226   100373
>>>>           40899  358865   40946   360562    39415   675968   691538
>>>>           96086   105695
>>>>                    16384     256  798081  828146   107103   120433
>>>>           39084  595325   39050   593110    56925   763466   797859
>>>>          109645   113414
>>>>                    16384     512  810851  843931   113564   106202
>>>>           35111  714831   46244   745947    53636   802902   760172
>>>>          110492   100879
>>>>                    16384    1024  726399  820219    22106    22987
>>>>           53087  749053   54781   777705  1075341   772686   809723
>>>>          100349    96619
>>>>                    16384    2048  807772  856458    23920    23617
>>>>           66320  829576   72105   740848   656379   864539   835446
>>>>           93499   101714
>>>>                    16384    4096  797470  840596    27270    28132
>>>>           88784  834938   83428   836809   342898   953274   969699
>>>>           96499   100657
>>>>                    16384    8192  760063  812942    41022    40014
>>>>          102020  812135  104063   820346   229870  1252409  1398306
>>>>          100959   103793
>>>>                    16384   16384  743107  825054  3074035  3551062
>>>>           97740  826175  103597   803926    94681   820474   829106
>>>>           97168    92018
>>>>                    32768      64  713528  714585    71310    74296
>>>>           21135  194177   21551   202998    21110   571030   595232
>>>>           98598   103373
>>>>                    32768     128  766698  820615   103028   102269
>>>>           39013  328409   40325   346403    41084   684604   702513
>>>>          103887   100689
>>>>                    32768     256  766591  819138   107557   113878
>>>>           42088  574878   41778   557989    55045   749922   754434
>>>>          100964   104866
>>>>                    32768     512  774365  846915   104255   106094
>>>>           35280  728957   25829   734316    26990   750614   800955
>>>>           55013    63750
>>>>                    32768    1024  759113  806241    22033    22622
>>>>           37322  761870   45003   766950    54519   755666   741339
>>>>           70018    66255
>>>>                    32768    2048  797566  860344    22893    23859
>>>>           49973  778872   53509   779539   796674   817078   797585
>>>>           56750    57047
>>>>                    32768    4096  796693  798985    24267    25222
>>>>           54410  781810   53053   802845   421421   832858   855811
>>>>           56748    60507
>>>>                    32768    8192  741543  795144    29254    29150
>>>>           58678  830011   55545   833914   221323   914182   952916
>>>>           56120    60014
>>>>                    32768   16384  754641  819797    44183    45718
>>>>           67293  785047   65078   807273   132136  1240414  1267713
>>>>           56585    56167
>>>>                    65536      64  663333  656588    32211    31925
>>>>           17324  202043   15477   215981    15496   592126   597188
>>>>           86431    93969
>>>>                    65536     128  716218  791603    92035    95614
>>>>           40330  330127   39295   346542    39469   697385   714453
>>>>          105094    98327
>>>>                    65536     256  769283  798701   110285   110085
>>>>           38404  535730   39939   585023    52402   741206   749899
>>>>           98671   100106
>>>>                    65536     512  728151  814516   112550   108873
>>>>           32284  690164   38911   708244    50047   778669   765303
>>>>           98333    92728
>>>>                    65536    1024  773504  827108    21542    20741
>>>>           45901  775473   57282   807232    69359   770793   766736
>>>>          101465   100905
>>>>                    65536    2048  767661  853446    21077    21260
>>>>           59825  762711   71985   802488    87290   772402   775685
>>>>           96832   103189
>>>>                    65536    4096  767669  828165    20681    21480
>>>>           79798  829757   91755   838367  1379411   791248   783023
>>>>          106433   112769
>>>>                    65536    8192  739291  784946    23100    22454
>>>>           85945  785134   87081   828489   642088   816781   798275
>>>>           93307    95318
>>>>                    65536   16384  736401  791669    26345    26384
>>>>           93082  799356   91419   751766   385721   911919   884688
>>>>          109668    96374
>>>>                   131072      64  658361  661211    79790    81016
>>>>           19460  182084   19325   195428    19553   551809   554426
>>>>           96581    81979
>>>>                   131072     128  759806  838994    51063    56976
>>>>           33624  326699   36329   360593    35679   710602   716538
>>>>           81725    51819
>>>>                   131072     256  787375  843763    54664    65031
>>>>           33847  557817   35093   581552    45163   750234   758452
>>>>           82704    83472
>>>>                   131072     512  777460  818049    87831    87110
>>>>           27994  705621   31860   723071    38483   779489   793068
>>>>           88281    86068
>>>>                   131072    1024  737158  782780    19005    18897
>>>>           36614  735240   38900   812074    52149   746598   762694
>>>>           74376    82534
>>>>                   131072    2048  777386  817330    24169    24693
>>>>           55171  769662   56082   788848    55937   757716   763147
>>>>           73508    82351
>>>>                   131072    4096  751757  792201    25283    25173
>>>>           48455  813945   56557   837418    59578   749106   757939
>>>>           67846    70964
>>>>                   131072    8192  743004  796673    20369    20327
>>>>           88990  780925   95956   777211   951528   762755   739851
>>>>           70850    69943
>>>>                   131072   16384  766023  823611    22333    22142
>>>>          108337  796852  110848   777704   718954   809861   802822
>>>>          115896    82903
>>>>                   262144      64  655769  695392    45973    45668
>>>>           19083  209417   18768   207272    19133   601329   599267
>>>>          107705    87261
>>>>                   262144     128  750052  798999    72711    74713
>>>>           37329  339480   36854   365587    37187   710898   714372
>>>>          108400    78806
>>>>                   262144     256  788355  815575    80897    75141
>>>>           37080  549762   37521   574393    51731   780429   751131
>>>>           97395    73842
>>>>                   262144     512  714994  736649    80224    80195
>>>>           32986  708196   36048   628508    44043   767699   781863
>>>>          123267    83558
>>>>                   262144    1024  793541  800826    20200    20428
>>>>           45599  752027   48232   798229    57011   715643   763691
>>>>           76741    73211
>>>>                   262144    2048  699395  782935    20227    20519
>>>>           65785  779219   80486   803403    93381   650054   712171
>>>>          115611   108457
>>>>                   262144    4096  629531  766261    20544    21563
>>>>           83107  819917   93008   844103    90625   741879   754931
>>>>           94114    94475
>>>>                   262144    8192  731695  804112    21158    21589
>>>>           82987  787869   84858   799483    95127   731822   737397
>>>>           95950    95160
>>>>                   262144   16384  703376  802552    21940    22371
>>>>           87748  777024   90349   779626  1018679   765278   619969
>>>>           96051    92734
>>>>                   524288      64  649937  710342    73778    70893
>>>>           19734  207179   19367   199709    19493   585910   581340
>>>>           91902    91443
>>>>                   524288     128  739270  805153    91507    90395
>>>>           38547  319049   37988   358516    38123   712602   711814
>>>>           92463    90851
>>>>                   524288     256  751279  802496    89481    91022
>>>>           38869  544097   38244   569142    56130   756367   752466
>>>>           92677    92875
>>>>                   524288     512  758393  819788    58085    73437
>>>>           29030  700120   28767   739236    33925   770177   795879
>>>>           58963    72689
>>>>                   524288    1024  778385  818368    18822    18639
>>>>           38774  798827   50898   799492    55679   801849   800917
>>>>           92778    82686
>>>>                   524288    2048  780594  834612    19936    19994
>>>>           63147  769555   63007   802512    66322   772900   771314
>>>>           86628    93485
>>>>                   524288    4096  769806  812570    20165    20137
>>>>           79896  783615   71217    80835    68550    80166    73733
>>>>          122559   116844
>>>>                   524288    8192  797504  887823   123824   124513
>>>>           94520  844880   86441   710319    65077   636713   819868
>>>>           87765    80121
>>>>                   524288   16384  788468  858216    92580   117354
>>>>          109583  850179   63966   856965   121151   789252   805040
>>>>          123810   119088
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>         On Feb 27, 2013, at 2:46 PM, Bryan Whitehead
>>>>         <driver at megahappy.net <mailto:driver at megahappy.net>> wrote:
>>>>
>>>>>         How are you doing the read/write tests on the
>>>>>         fuse/glusterfs mountpoint? Many small files will be slow
>>>>>         because all the time is spent coordinating locks.
>>>>>
>>>>>
>>>>>         On Wed, Feb 27, 2013 at 9:31 AM, Thomas Wakefield
>>>>>         <twake at cola.iges.org <mailto:twake at cola.iges.org>> wrote:
>>>>>
>>>>>             Help please-
>>>>>
>>>>>
>>>>>             I am running 3.3.1 on Centos using a 10GB network.  I
>>>>>             get reasonable write speeds, although I think they
>>>>>             could be faster.  But my read speeds are REALLY slow.
>>>>>
>>>>>             Executive summary:
>>>>>
>>>>>             On gluster client-
>>>>>             Writes average about 700-800MB/s
>>>>>             Reads average about 70-80MB/s
>>>>>
>>>>>             On server-
>>>>>             Writes average about 1-1.5GB/s
>>>>>             Reads average about 2-3GB/s
>>>>>
>>>>>             Any thoughts?
>>>>>
>>>>>
>>>>>
>>>>>             Here are some additional details:
>>>>>
>>>>>             Nothing interesting in any of the log files,
>>>>>             everything is very quite.
>>>>>             All servers had no other load, and all clients are
>>>>>             performing the same way.
>>>>>
>>>>>
>>>>>             Volume Name: shared
>>>>>             Type: Distribute
>>>>>             Volume ID: de11cc19-0085-41c3-881e-995cca244620
>>>>>             Status: Started
>>>>>             Number of Bricks: 26
>>>>>             Transport-type: tcp
>>>>>             Bricks:
>>>>>             Brick1: fs-disk2:/storage/disk2a
>>>>>             Brick2: fs-disk2:/storage/disk2b
>>>>>             Brick3: fs-disk2:/storage/disk2d
>>>>>             Brick4: fs-disk2:/storage/disk2e
>>>>>             Brick5: fs-disk2:/storage/disk2f
>>>>>             Brick6: fs-disk2:/storage/disk2g
>>>>>             Brick7: fs-disk2:/storage/disk2h
>>>>>             Brick8: fs-disk2:/storage/disk2i
>>>>>             Brick9: fs-disk2:/storage/disk2j
>>>>>             Brick10: fs-disk2:/storage/disk2k
>>>>>             Brick11: fs-disk2:/storage/disk2l
>>>>>             Brick12: fs-disk2:/storage/disk2m
>>>>>             Brick13: fs-disk2:/storage/disk2n
>>>>>             Brick14: fs-disk2:/storage/disk2o
>>>>>             Brick15: fs-disk2:/storage/disk2p
>>>>>             Brick16: fs-disk2:/storage/disk2q
>>>>>             Brick17: fs-disk2:/storage/disk2r
>>>>>             Brick18: fs-disk2:/storage/disk2s
>>>>>             Brick19: fs-disk2:/storage/disk2t
>>>>>             Brick20: fs-disk2:/storage/disk2u
>>>>>             Brick21: fs-disk2:/storage/disk2v
>>>>>             Brick22: fs-disk2:/storage/disk2w
>>>>>             Brick23: fs-disk2:/storage/disk2x
>>>>>             Brick24: fs-disk3:/storage/disk3a
>>>>>             Brick25: fs-disk3:/storage/disk3b
>>>>>             Brick26: fs-disk3:/storage/disk3c
>>>>>             Options Reconfigured:
>>>>>             performance.write-behind: on
>>>>>             performance.read-ahead: on
>>>>>             performance.io-cache: on
>>>>>             performance.stat-prefetch: on
>>>>>             performance.quick-read: on
>>>>>             cluster.min-free-disk: 500GB
>>>>>             nfs.disable: off
>>>>>
>>>>>
>>>>>             sysctl.conf settings for 10GBe
>>>>>             # increase TCP max buffer size settable using setsockopt()
>>>>>             net.core.rmem_max = 67108864
>>>>>             net.core.wmem_max = 67108864
>>>>>             # increase Linux autotuning TCP buffer limit
>>>>>             net.ipv4.tcp_rmem = 4096 87380 67108864
>>>>>             net.ipv4.tcp_wmem = 4096 65536 67108864
>>>>>             # increase the length of the processor input queue
>>>>>             net.core.netdev_max_backlog = 250000
>>>>>             # recommended default congestion control is htcp
>>>>>             net.ipv4.tcp_congestion_control=htcp
>>>>>             # recommended for hosts with jumbo frames enabled
>>>>>             net.ipv4.tcp_mtu_probing=1
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>             Thomas W.
>>>>>             Sr.  Systems Administrator COLA/IGES
>>>>>             twake at cola.iges.org <mailto:twake at cola.iges.org>
>>>>>             Affiliate Computer Scientist GMU
>>>>>
>>>>>
>>>>>             _______________________________________________
>>>>>             Gluster-users mailing list
>>>>>             Gluster-users at gluster.org
>>>>>             <mailto:Gluster-users at gluster.org>
>>>>>             http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130228/3f65adff/attachment.html>


More information about the Gluster-users mailing list