[Gluster-users] infiniband speed

[Nathan Stratton]

On Mon, 10 Jan 2011, Joe Landman wrote:

> Try this:
>
> 	# (on bravo)
> 	dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
>
> This will write a 20GB file to the same partition.  We need to see how fast 
> that write is (outside of cache)
>
> Do the same test on the other machine.
>
> Infiniband won't make slow disks fast.  So unless you are already getting a 
> gigabyte or two per second (we do on our units), adding Infiniband won't get 
> you any significant additional performance.
>
> Put another way, you are bound by the slowest aspect of your storage system.
>
> What are your underlying disks?  Are these individual drives?  RAID? What 
> sort of RAID if it is a RAID?  How is it organized?
>
> We tune our systems pretty hard, so we start with 2+GB/s for TB sized files 
> before we ever touch the next stack up.  Each additional stack you traverse 
> takes performance away (you lose it in stack inefficiency).  So you should 
> start out with as fast a system as you possibly can, so you don't lose too 
> much performance.

What hardware gives you 2+GB/s? I am using the 16 port 3Ware 9750 cards 
with fast drives and I dont get close to that.

-Nathan