[Gluster-devel] Any feedback on performance when migrating from 1gige to 10gbps infiniband?

[Gordan Bobic]

Brandon Lamb wrote:
> Hello,
> 
> We have started looking into some used inifiniband hardware to up our
> network from the typical gigabit ethernet to the 10gigabit infiniband
> stuff. Im curious to hear from anyone that has maybe done this or has
> been able to see how much of a performance gain there is. My primary
> interest would be in a replicated setup (5 or so clients and 2 data
> servers). Our use would be web files and/or mail possibly, both i
> guess are small files.

Have you done testing with 1Gb networking? Do your storage boxes have 
enough CPU resource to handle a 10x increase in bandwidth? What about 
disk I/O? You'll need some pretty fat storage nodes to justify going to 
10Gb.

> We are also however looking into building xen servers, and looking at
> designs for do we stick images on local disks or try nas/san
> solutions, try glusterfs for this maybe?

Considering that virtualized disk I/O performance varies between 10% and 
40% of bare metal, you may be lucky and find that using a fuse based fs 
doesn't actually affect the performance too badly on top of the 
virtualization overheads.

Good luck. Please report your findings. :)

Gordan