[Gluster-users] Gluster performance with VM's

[Alexey Zakurin]

Hi, community

Please, help me with my trouble.

I have 2 Gluster nodes, with 2 bricks on each.
Configuration:
Node1 brick1 replicated on Node0 brick0
Node0 brick1 replicated on Node1 brick0

Volume Name: gm0
Type: Distributed-Replicate
Volume ID: 5e55f511-8a50-46e4-aa2f-5d4f73c859cf
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: gl1:/mnt/brick1/gm0
Brick2: gl0:/mnt/brick0/gm0
Brick3: gl0:/mnt/brick1/gm0
Brick4: gl1:/mnt/brick0/gm0
Options Reconfigured:
cluster.rebal-throttle: aggressive
performance.cache-refresh-timeout: 4
performance.cache-max-file-size: 10MB
performance.client-io-threads: on
diagnostics.client-log-level: WARNING
diagnostics.brick-log-level: WARNING
performance.write-behind-window-size: 4MB
features.scrub: Active
features.bitrot: on
cluster.readdir-optimize: on
server.event-threads: 16
client.event-threads: 16
cluster.lookup-optimize: on
server.allow-insecure: on
performance.read-ahead: disable
performance.readdir-ahead: off
performance.io-thread-count: 64
performance.cache-size: 2GB
diagnostics.count-fop-hits: on
diagnostics.latency-measurement: on
nfs.disable: on
transport.address-family: inet
cluster.self-heal-daemon: enable
cluster.server-quorum-ratio: 51%

Each brick is software RAID10, that contains 6 disks.

20Gb round-robin bonding between servers and clients - one network.

On storage, I have VM's images.
VM's run on 3 clients, Xen Hypervisor.

One of VM's is the FTP-server, that contains large numbers of archives.

Problem.
When I try to upload on FTP-server large file (50-60Gb), other VM's was 
throttled too much. Sometimes, FS on this VM's can be automatically 
re-mounted in ro-mode.
Network monitor saying, that speed ~40MB/sec. Disk monitor displays the 
same.

I try to start VM with FTP on other server (0) - problem still persist.
Mounting other Gluster node - problem still persist.

Please, help to solve this problem.

-- 
С уважением, Закурин Алексей Евгеньевич.
Telegram: @Zakurin
Tel: +7 968 455 88 48