[Gluster-users] infiniband speed

Joe Landman landman at scalableinformatics.com
Tue Jan 11 15:50:56 UTC 2011

On 01/11/2011 10:44 AM, Nathan Stratton wrote:
> On Mon, 10 Jan 2011, Joe Landman wrote:
>> Try this:
>> # (on bravo)
>> dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
>> This will write a 20GB file to the same partition. We need to see how
>> fast that write is (outside of cache)
>> Do the same test on the other machine.
>> Infiniband won't make slow disks fast. So unless you are already
>> getting a gigabyte or two per second (we do on our units), adding
>> Infiniband won't get you any significant additional performance.
>> Put another way, you are bound by the slowest aspect of your storage
>> system.
>> What are your underlying disks? Are these individual drives? RAID?
>> What sort of RAID if it is a RAID? How is it organized?
>> We tune our systems pretty hard, so we start with 2+GB/s for TB sized
>> files before we ever touch the next stack up. Each additional stack
>> you traverse takes performance away (you lose it in stack
>> inefficiency). So you should start out with as fast a system as you
>> possibly can, so you don't lose too much performance.
> What hardware gives you 2+GB/s? I am using the 16 port 3Ware 9750 cards
> with fast drives and I dont get close to that.

Recent results here


and older results


Not a commercial.  Follow the links in the .sig for more info.

We've been a Gluster reseller/service provider for a while now (though 
for some reason we've not been on their site, and have asked them to 
work on that with us).

Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

More information about the Gluster-users mailing list