[Gluster-users] Gluster on tcp ip compared to infiniband

[Amar S. Tumballi]

Hi Robert, John,
 Thanks for sharing these numbers.

One question for John, are you using performance translators ? we expect to
get higher numbers over IB.

Regards,
Amar


2008/7/21 Robert Dildy <Robert at dildy.net>:

>
> John Marshall wrote:
>
> Jean Spirat wrote:
>
>
>  Hi,
>
>  I have seen the benchmarks on the gluster website and i wondered if
> there is a big difference between infiniband and tcp/ip performances.
> Anyone can share some stats on this i do not know the infiniband
> technology at all.
>
>
>
>  Hi,
>
> Some preliminary numbers I have for a 4x SDR IB network, single
> disk (RAID 5, 4+1, SAS, 300G 15K disks), single user:
>
> transport        read    write   notes
> ---------        ----    -----   -----
> local            ~254     ~255   for a 100G file, not compensating for
> caching
>                                  take them for what their worth :)
> nfs              158     76      not gluster, of course!
> tcp (ipoib)      171     118     no optimizations
> sdp              211     108     no optimizations
> sdp              210     128     iot 4, 32MB
> verbs            218     150     no optimizations
> verbs            220     150     recv/send size 1048675, recv/send-count 16
>
>
> John
>
> _______________________________________________
> Gluster-users mailing listGluster-users at gluster.orghttp://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>  The tuning is just overall so the block sizes can make a huge difference.
> The numbers just give a good feeling for speeds.
>
> We have 8 servers that are also clients of each other with DDR IB each with
> 4x750 SATA II in RAID 5 with 16GB RAM.
>
> rates are in MBs.  A = aggregat    S = per server
>
> Transport            Read            Write            Read notes
> ----------           ------            ------           ------
> IBverbs               5993 (A)      5111 (A)     cache
> IBverbs                 749 (S)        638  (S)     cache
> IBverbs               3616 (A)      2598 (A)     exceeding cache, but mixed
> IBverbs                 452 (S)        325 (S)      exceeding cache, bu
> tmixed
> IBverbs                 958 (A)        747 (A)     no cache
> IBverbs                 120 (S)           93 (S)     no cache
> NFS (GigE)           874 (A)       739 (A)     cache
> NFS (GigE)           109 (S)         92  (S)     cache
>
> Hope it helps.
> -robert
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>


-- 
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Super Storage!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080721/4db71112/attachment.html>