[Gluster-users] infiniband speed

Nathan Stratton nathan at robotics.net
Tue Jan 11 15:44:58 UTC 2011

On Mon, 10 Jan 2011, Joe Landman wrote:

> Try this:
> 	# (on bravo)
> 	dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
> This will write a 20GB file to the same partition.  We need to see how fast 
> that write is (outside of cache)
> Do the same test on the other machine.
> Infiniband won't make slow disks fast.  So unless you are already getting a 
> gigabyte or two per second (we do on our units), adding Infiniband won't get 
> you any significant additional performance.
> Put another way, you are bound by the slowest aspect of your storage system.
> What are your underlying disks?  Are these individual drives?  RAID? What 
> sort of RAID if it is a RAID?  How is it organized?
> We tune our systems pretty hard, so we start with 2+GB/s for TB sized files 
> before we ever touch the next stack up.  Each additional stack you traverse 
> takes performance away (you lose it in stack inefficiency).  So you should 
> start out with as fast a system as you possibly can, so you don't lose too 
> much performance.

What hardware gives you 2+GB/s? I am using the 16 port 3Ware 9750 cards 
with fast drives and I dont get close to that.


More information about the Gluster-users mailing list