[Gluster-users] infiniband speed

[Joe Landman]

On 01/11/2011 10:44 AM, Nathan Stratton wrote:
> On Mon, 10 Jan 2011, Joe Landman wrote:
>
>> Try this:
>>
>> # (on bravo)
>> dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
>>
>> This will write a 20GB file to the same partition. We need to see how
>> fast that write is (outside of cache)
>>
>> Do the same test on the other machine.
>>
>> Infiniband won't make slow disks fast. So unless you are already
>> getting a gigabyte or two per second (we do on our units), adding
>> Infiniband won't get you any significant additional performance.
>>
>> Put another way, you are bound by the slowest aspect of your storage
>> system.
>>
>> What are your underlying disks? Are these individual drives? RAID?
>> What sort of RAID if it is a RAID? How is it organized?
>>
>> We tune our systems pretty hard, so we start with 2+GB/s for TB sized
>> files before we ever touch the next stack up. Each additional stack
>> you traverse takes performance away (you lose it in stack
>> inefficiency). So you should start out with as fast a system as you
>> possibly can, so you don't lose too much performance.
>
> What hardware gives you 2+GB/s? I am using the 16 port 3Ware 9750 cards
> with fast drives and I dont get close to that.
>

Recent results here

http://scalability.org/?p=2912
http://scalability.org/?p=2552

and older results

http://scalability.org/?p=2356

Not a commercial.  Follow the links in the .sig for more info.

We've been a Gluster reseller/service provider for a while now (though 
for some reason we've not been on their site, and have asked them to 
work on that with us).

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615