[Gluster-users] Replica add brick slow replication

[Steve Dainard]

Gluster 3.6.6 / CentOS 7.1 / dual Intel E5-2630v3 / 128GB RAM /
Mellanox 10G Ethernet

I just added a 3rd replica to a 2 replica volume and I'm noticing the
network throughput is very slow replicating to the new node,
~30-60MB/s. I'm on 10gig with SSD bricks and typically get 300+MB/s
for normal file transfers.

Looking at this thread:
http://www.gluster.org/pipermail/gluster-users/2014-January/015554.html
it seems this isn't an isolated issue and there is mention of
re-factoring the replication code.

I see also see glusterfsd cpu usage averaging ~200% and peaks of
4-500% occasionally on node1/node2, but node3 is averaging ~50%. This
cpu overhead exists even when network throughput drops to <1MB/sec.

Any insight into why this process is so slow and why CPU usage is so high?

Lastly, this storage is for an Ovirt storage domain and while the
add-brick operation is running all the VM's hang. But I can write to a
fuse mount point so the volume isn't RO.

Thanks,
Steve