[Gluster-users] slow write perf for disperse volume

[Ingard Mevåg]

I can confirm mounting the disperse volume locally on one of the three
servers i got 211 MB/s with dd if=/dev/zero of=./local.dd.test bs=1M
count=10000.

Its not very good concidering 10gig network, but at least 20x better than
10-12MB/s

2017-04-24 13:53 GMT+02:00 Pranith Kumar Karampuri <pkarampu at redhat.com>:

> +Ashish
>
> Ashish,
>        Could you help Ingard? Do let me know what you find.
>
> On Mon, Apr 24, 2017 at 4:50 PM, Ingard Mevåg <ingard at jotta.no> wrote:
>
>> Hi. I can't see a fuse thread at all. Please see attached screenshot of
>> top process with threads. Keep in mind this is from inside the container.
>>
>> 2017-04-24 12:17 GMT+02:00 Pranith Kumar Karampuri <pkarampu at redhat.com>:
>>
>>> We were able to saturate hardware with EC as well. Could you check 'top'
>>> in threaded mode to see if fuse thread is saturated when you run dd?
>>>
>>> On Mon, Apr 24, 2017 at 3:27 PM, Ingard Mevåg <ingard at jotta.no> wrote:
>>>
>>>> Hi
>>>> I've been playing with disperse volumes the past week, and so far i can
>>>> not get more than 12MB/s when i do a write test. I've tried a distributed
>>>> volume on the same bricks and gotten close to gigabit speeds. iperf
>>>> confirms gigabit speeds to all three servers in the storage pool.
>>>>
>>>> The three storage servers have 10gig nics (connected to the same
>>>> switch). The client is for a now a docker container in a 2nd DC (latency
>>>> roughly 8-9 ms).
>>>>
>>>> dpkg -l|grep -i gluster
>>>> ii  glusterfs-client               3.10.1-ubuntu1~xenial1
>>>>  amd64        clustered file-system (client package)
>>>> ii  glusterfs-common               3.10.1-ubuntu1~xenial1
>>>>  amd64        GlusterFS common libraries and translator modules
>>>> ii  glusterfs-server               3.10.1-ubuntu1~xenial1
>>>>  amd64        clustered file-system (server package)
>>>>
>>>> $ gluster volume info
>>>>
>>>> Volume Name: DFS-ARCHIVE-001
>>>> Type: Disperse
>>>> Volume ID: 1497bc85-cb47-4123-8f91-a07f55c11dcc
>>>> Status: Started
>>>> Snapshot Count: 0
>>>> Number of Bricks: 1 x (4 + 2) = 6
>>>> Transport-type: tcp
>>>> Bricks:
>>>> Brick1: dna-001:/mnt/data01/brick
>>>> Brick2: dna-001:/mnt/data02/brick
>>>> Brick3: dna-002:/mnt/data01/brick
>>>> Brick4: dna-002:/mnt/data02/brick
>>>> Brick5: dna-003:/mnt/data01/brick
>>>> Brick6: dna-003:/mnt/data02/brick
>>>> Options Reconfigured:
>>>> transport.address-family: inet
>>>> nfs.disable: on
>>>>
>>>> Anyone know the reason for the slow speeds on disperse vs distribute?
>>>>
>>>> kind regards
>>>> ingard
>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>
>>>
>>>
>>>
>>> --
>>> Pranith
>>>
>>
>>
>>
>> --
>> Ingard Mevåg
>> Driftssjef
>> Jottacloud
>>
>> Mobil: +47 450 22 834 <+47%20450%2022%20834>
>> E-post: ingard at jottacloud.com
>> Webside: www.jottacloud.com
>>
>
>
>
> --
> Pranith
>



-- 
Ingard Mevåg
Driftssjef
Jottacloud

Mobil: +47 450 22 834
E-post: ingard at jottacloud.com
Webside: www.jottacloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170424/b4965840/attachment.html>