[Gluster-users] infiniband speed

Christopher Hawkins chawkins at bplinux.com
Tue Jan 11 06:07:16 UTC 2011


Out of curiosity, what is a typical RAID / spindle count / rpm configuration for you that yields 2 GB/s?

----- "Christopher Hawkins" <chawkins at bplinux.com> wrote:

> Thanks Joe, you nailed it. These are just test machines and in each
> case, just a single 10k scsi drive. That is the throughput
> bottleneck... I was not able to get more than 70MB/s sustained.
> 
> Chris
> 
> ----- "Joe Landman" <landman at scalableinformatics.com> wrote:
> 
> > On 01/10/2011 07:43 PM, Christopher Hawkins wrote:
> > > I am testing Infiniband for the first time. It seems that I
> should
> > be
> > > able to get a lot more speed than I am with some pretty basic
> > tests.
> > > Maybe someone running Infiniband can confirm that what I am
> seeing
> > is
> > > way out of line, and/or help diagnose?
> > 
> > What is the underlying disk speed?
> > 
> > Try this:
> > 
> > 	# (on bravo)
> > 	dd if=/dev/zero of=/cluster/shadow/big.file bs=1M count=20k
> > 
> > This will write a 20GB file to the same partition.  We need to see
> how
> > 
> > fast that write is (outside of cache)
> > 
> > Do the same test on the other machine.
> > 
> > Infiniband won't make slow disks fast.  So unless you are already 
> > getting a gigabyte or two per second (we do on our units), adding 
> > Infiniband won't get you any significant additional performance.
> > 
> > Put another way, you are bound by the slowest aspect of your
> storage
> > system.
> > 
> > What are your underlying disks?  Are these individual drives?  RAID?
> 
> > What sort of RAID if it is a RAID?  How is it organized?
> > 
> > We tune our systems pretty hard, so we start with 2+GB/s for TB
> sized
> > 
> > files before we ever touch the next stack up.  Each additional
> stack
> > you 
> > traverse takes performance away (you lose it in stack inefficiency).
> 
> > So 
> > you should start out with as fast a system as you possibly can, so
> you
> > 
> > don't lose too much performance.
> > 
> > -- 
> > Joseph Landman, Ph.D
> > Founder and CEO
> > Scalable Informatics, Inc.
> > email: landman at scalableinformatics.com
> > web  : http://scalableinformatics.com
> >         http://scalableinformatics.com/sicluster
> > phone: +1 734 786 8423 x121
> > fax  : +1 866 888 3112
> > cell : +1 734 612 4615
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users



More information about the Gluster-users mailing list