<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 18, 2019 at 11:23 PM Lindolfo Meira <<a href="mailto:meira@cesup.ufrgs.br">meira@cesup.ufrgs.br</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">We're running some benchmarks on a striped glusterfs volume.<br>
<br></blockquote><div><br></div><div>Hi Lindolfo,</div><div><br></div><div>We are not supporting Stripe anymore, and planning to remove it from build too by glusterfs-6.0 (ie, next release). See if you can use 'Shard' for the usecase.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
We have 6 identical servers acting as bricks. Measured link speed between <br>
these servers is 3.36GB/s. Link speed between clients of the parallel file <br>
system and its servers is also 3.36GB/s. So we're expecting this system to <br>
have a write performance of around 20.16GB/s (6 times 3.36GB/s) minus some <br>
write overhead.<br>
<br>
If we write to the system from a single client, we manage to write at <br>
around 3.36GB/s. That's okay, because we're limited by the max throughput <br>
of that client's network adapter. But when we account for that and write <br>
from 6 or more clients, we can never get past 11GB/s. Is that right? Is <br>
this really the overhead to be expected? We'd appreciate any inputs.<br>
<br></blockquote><div> </div><div>Lame question: Are we getting more than 11GB/s from disks ?</div><div><br></div><div>Please collect `gluster volume profile gfs0 info`, that can give more information.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Output of gluster volume info:<br>
<br>
Volume Name: gfs0<br>
Type: Stripe<br>
Volume ID: 2ca3dd45-6209-43ff-a164-7f2694097c64<br>
Status: Started<br>
Snapshot Count: 0<br>
Number of Bricks: 1 x 6 = 6<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: pfs01-ib:/mnt/data<br>
Brick2: pfs02-ib:/mnt/data<br>
Brick3: pfs03-ib:/mnt/data<br>
Brick4: pfs04-ib:/mnt/data<br>
Brick5: pfs05-ib:/mnt/data<br>
Brick6: pfs06-ib:/mnt/data<br>
Options Reconfigured:<br>
cluster.stripe-block-size: 128KB<br>
performance.cache-size: 32MB<br>
performance.write-behind-window-size: 1MB<br>
performance.strict-write-ordering: off<br>
performance.strict-o-direct: off<br>
performance.stat-prefetch: off<br>
server.event-threads: 4<br>
client.event-threads: 2<br>
performance.io-thread-count: 16<br>
transport.address-family: inet<br>
nfs.disable: on<br>
cluster.localtime-logging: enable<br>
<br>
<br></blockquote><div><br></div><div> </div></div></div>