[Gluster-users] Gluster on tcp ip compared to infiniband
John.Marshall at ec.gc.ca
Mon Jul 21 16:48:12 UTC 2008
Amar S. Tumballi wrote:
> Hi Robert, John,
> Thanks for sharing these numbers.
> One question for John, are you using performance translators ? we
> expect to get higher numbers over IB.
No. My first goal was purposely to compare without any extras/optimizations.
Of course, with them, I will expect better numbers as you mention :)
> 2008/7/21 Robert Dildy <Robert at dildy.net <mailto:Robert at dildy.net>>:
> John Marshall wrote:
>> Jean Spirat wrote:
>>> I have seen the benchmarks on the gluster website and i wondered if
>>> there is a big difference between infiniband and tcp/ip performances.
>>> Anyone can share some stats on this i do not know the infiniband
>>> technology at all.
>> Some preliminary numbers I have for a 4x SDR IB network, single
>> disk (RAID 5, 4+1, SAS, 300G 15K disks), single user:
>> transport read write notes
>> --------- ---- ----- -----
>> local ~254 ~255 for a 100G file, not compensating for
>> take them for what their worth :)
>> nfs 158 76 not gluster, of course!
>> tcp (ipoib) 171 118 no optimizations
>> sdp 211 108 no optimizations
>> sdp 210 128 iot 4, 32MB
>> verbs 218 150 no optimizations
>> verbs 220 150 recv/send size 1048675, recv/send-count 16
>> Gluster-users mailing list
>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> The tuning is just overall so the block sizes can make a huge
> difference. The numbers just give a good feeling for speeds.
> We have 8 servers that are also clients of each other with DDR IB
> each with 4x750 SATA II in RAID 5 with 16GB RAM.
> rates are in MBs. A = aggregat S = per server
> Transport Read Write Read notes
> ---------- ------ ------ ------
> IBverbs 5993 (A) 5111 (A) cache
> IBverbs 749 (S) 638 (S) cache
> IBverbs 3616 (A) 2598 (A) exceeding cache,
> but mixed
> IBverbs 452 (S) 325 (S) exceeding
> cache, bu tmixed
> IBverbs 958 (A) 747 (A) no cache
> IBverbs 120 (S) 93 (S) no cache
> NFS (GigE) 874 (A) 739 (A) cache
> NFS (GigE) 109 (S) 92 (S) cache
> Hope it helps.
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> Amar Tumballi
> Gluster/GlusterFS Hacker
> [bulde on #gluster/irc.gnu.org <http://irc.gnu.org>]
> http://www.zresearch.com - Commoditizing Super Storage!
More information about the Gluster-users