[Gluster-users] infiniband speed

[Christopher Hawkins]

Thanks Joe, you nailed it. These are just test machines and in each case, just a single 10k scsi drive. That is the throughput bottleneck... I was not able to get more than 70MB/s sustained.

Chris

----- "Joe Landman" <landman at scalableinformatics.com> wrote:

> On 01/10/2011 07:43 PM, Christopher Hawkins wrote:
> > I am testing Infiniband for the first time. It seems that I should
> be
> > able to get a lot more speed than I am with some pretty basic
> tests.
> > Maybe someone running Infiniband can confirm that what I am seeing
> is
> > way out of line, and/or help diagnose?
> 
> What is the underlying disk speed?
> 
> Try this:
> 
> 	# (on bravo)
> 	dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
> 
> This will write a 20GB file to the same partition.  We need to see how
> 
> fast that write is (outside of cache)
> 
> Do the same test on the other machine.
> 
> Infiniband won't make slow disks fast.  So unless you are already 
> getting a gigabyte or two per second (we do on our units), adding 
> Infiniband won't get you any significant additional performance.
> 
> Put another way, you are bound by the slowest aspect of your storage
> system.
> 
> What are your underlying disks?  Are these individual drives?  RAID? 
> What sort of RAID if it is a RAID?  How is it organized?
> 
> We tune our systems pretty hard, so we start with 2+GB/s for TB sized
> 
> files before we ever touch the next stack up.  Each additional stack
> you 
> traverse takes performance away (you lose it in stack inefficiency). 
> So 
> you should start out with as fast a system as you possibly can, so you
> 
> don't lose too much performance.
> 
> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics, Inc.
> email: landman at scalableinformatics.com
> web  : http://scalableinformatics.com
>         http://scalableinformatics.com/sicluster
> phone: +1 734 786 8423 x121
> fax  : +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users