<div dir="ltr"><div dir="ltr"><div>I would need following data:</div><div><br></div><div>* client and brick volume profile - <a href="https://glusterdocs.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/">https://glusterdocs.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/</a></div><div>* cmdline of exact test you were running</div><div><br></div><div>regards,<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 10, 2019 at 9:02 PM Jeff Forbes <<a href="mailto:jeff.forbes@mail.nacon.com">jeff.forbes@mail.nacon.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I have two CentOS-6 servers running version 3.12.14 of gluster-server.<br>
Each server as one brick and they are configured to replicate between<br>
the two bricks.<br>
<br>
I also have two CentOS-6 servers running version 3.12.2-18 of<br>
glusterfs.<br>
<br>
These servers use a separate VLAN. Each server has two bonded 1 Gbps<br>
NICs to communicate the gluster traffic. File transfer speeds between<br>
these servers using rsync approaches 100 MBps.<br>
<br>
The client servers mount the gluster volume using this fstab entry:<br>
192.168.40.30:gv0 /store glusterfs defaults, attribute-timeout=600,entry-timeout=600,negative-timeout=600,fopen-keep-cache,use-readdirp=no,log-level=WARNING 1 2<br>
<br>
Reading data from the servers to the clients is similar to the rsync<br>
speed. The problem is that writing from the clients to the mounted<br>
gluster volume is less than 8 MB/s and fluctuates from less than 500<br>
kB/s to 8 MB/s, as measured by the pv command. Using rsync, the speed<br>
fluctuates between 2 and 5 MBps.<br>
<br>
When the bonded nics on one of the gluster servers is shut down, the<br>
write speed to the remaining online brick is now similar to the read<br>
speed<br>
<br>
I can only assume that there is something wrong in my configuration,<br>
since a greater than 10-fold decrease in write speed when the bricks<br>
are replicating makes for an unusable system.<br>
<br>
<br>
Does anyone have any ideas what the problem may be?<br>
<br>
<br>
Server volume configuration:<br>
> sudo gluster volume info<br>
<br>
Volume Name: gv0<br>
Type: Replicate<br>
Volume ID: d96bbb99-f264-4655-95ff-f9f05ca9ff55<br>
Status: Started<br>
Snapshot Count: 0<br>
Number of Bricks: 1 x 2 = 2<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: 192.168.40.20:/export/scsi/brick<br>
Brick2: 192.168.40.30:/export/scsi/brick<br>
Options Reconfigured:<br>
performance.cache-size: 1GB<br>
performance.readdir-ahead: on<br>
features.cache-invalidation: on<br>
features.cache-invalidation-timeout: 600<br>
performance.stat-prefetch: on<br>
performance.cache-samba-metadata: on<br>
performance.cache-invalidation: on<br>
performance.md-cache-timeout: 600<br>
network.inode-lru-limit: 250000<br>
performance.cache-refresh-timeout: 60<br>
performance.read-ahead: disable<br>
performance.parallel-readdir: on<br>
performance.write-behind-window-size: 4MB<br>
performance.io-thread-count: 64<br>
performance.client-io-threads: on<br>
performance.quick-read: on<br>
performance.flush-behind: on<br>
performance.write-behind: on<br>
nfs.disable: on<br>
client.event-threads: 3<br>
server.event-threads: 3<br>
server.allow-insecure: on<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div>