[Gluster-users] Slow write times to gluster disk

[Pranith Kumar Karampuri]

On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N <ravishankar at redhat.com>
wrote:

> Hi Pat,
>
> I'm assuming you are using gluster native (fuse mount). If it helps, you
> could try mounting it via gluster NFS (gnfs) and then see if there is an
> improvement in speed. Fuse mounts are slower than gnfs mounts but you get
> the benefit of avoiding a single point of failure. Unlike fuse mounts, if
> the gluster node containing the gnfs server goes down, all mounts done
> using that node will fail). For fuse mounts, you could try tweaking the
> write-behind xlator settings to see if it helps. See the
> performance.write-behind and performance.write-behind-window-size options
> in `gluster volume set help`. Of course, even for gnfs mounts, you can
> achieve fail-over by using CTDB.
>

Ravi,
      Do you have any data that suggests fuse mounts are slower than gNFS
servers?

Pat,
      I see that I am late to the thread, but do you happen to have
"profile info" of the workload?

You can follow
https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Monitoring%20Workload/
to get the information.


>
> Thanks,
> Ravi
>
>
> On 04/08/2017 12:07 AM, Pat Haley wrote:
>
>
> Hi,
>
> We noticed a dramatic slowness when writing to a gluster disk when
> compared to writing to an NFS disk. Specifically when using dd (data
> duplicator) to write a 4.3 GB file of zeros:
>
>    - on NFS disk (/home): 9.5 Gb/s
>    - on gluster disk (/gdata): 508 Mb/s
>
> The gluser disk is 2 bricks joined together, no replication or anything
> else. The hardware is (literally) the same:
>
>    - one server with 70 hard disks  and a hardware RAID card.
>    - 4 disks in a RAID-6 group (the NFS disk)
>    - 32 disks in a RAID-6 group (the max allowed by the card, /mnt/brick1)
>    - 32 disks in another RAID-6 group (/mnt/brick2)
>    - 2 hot spare
>
> Some additional information and more tests results (after changing the log
> level):
>
> glusterfs 3.7.11 built on Apr 27 2016 14:09:22
> CentOS release 6.8 (Final)
> RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS-3 3108
> [Invader] (rev 02)
>
>
>
> *Create the file to /gdata (gluster)*
> [root at mseas-data2 gdata]# dd if=/dev/zero of=/gdata/zero1 bs=1M count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes (1.0 GB) copied, 1.91876 s, *546 MB/s*
>
> *Create the file to /home (ext4)*
> [root at mseas-data2 gdata]# dd if=/dev/zero of=/home/zero1 bs=1M count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes (1.0 GB) copied, 0.686021 s, *1.5 GB/s - *3 times as fast
>
>
>
> * Copy from /gdata to /gdata (gluster to gluster) *[root at mseas-data2
> gdata]# dd if=/gdata/zero1 of=/gdata/zero2
> 2048000+0 records in
> 2048000+0 records out
> 1048576000 bytes (1.0 GB) copied, 101.052 s, *10.4 MB/s* - realllyyy
> slooowww
>
>
> *Copy from /gdata to /gdata* *2nd time (gluster to gluster)*
> [root at mseas-data2 gdata]# dd if=/gdata/zero1 of=/gdata/zero2
> 2048000+0 records in
> 2048000+0 records out
> 1048576000 bytes (1.0 GB) copied, 92.4904 s, *11.3 MB/s* - realllyyy
> slooowww again
>
>
>
> *Copy from /home to /home (ext4 to ext4)*
> [root at mseas-data2 gdata]# dd if=/home/zero1 of=/home/zero2
> 2048000+0 records in
> 2048000+0 records out
> 1048576000 bytes (1.0 GB) copied, 3.53263 s, *297 MB/s *30 times as fast
>
>
> *Copy from /home to /home (ext4 to ext4)*
> [root at mseas-data2 gdata]# dd if=/home/zero1 of=/home/zero3
> 2048000+0 records in
> 2048000+0 records out
> 1048576000 bytes (1.0 GB) copied, 4.1737 s, *251 MB/s* - 30 times as fast
>
>
> As a test, can we copy data directly to the xfs mountpoint (/mnt/brick1)
> and bypass gluster?
>
>
> Any help you could give us would be appreciated.
>
> Thanks
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
> _______________________________________________
> Gluster-users mailing listGluster-users at gluster.orghttp://lists.gluster.org/mailman/listinfo/gluster-users
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170414/dd3f9839/attachment.html>