[Gluster-users] Substitute for SMP?

[Joe Landman]

On 05/27/2011 07:12 AM, Jon Tegner wrote:
> A general question, suppose I have a parallel application, using mpi,
> where really fast access to the file system is critical.
>
> Would it be stupid to consider a ram disk based setup? Say a 36 port QDR

Ram disks won't work directly, due to lack of locking in tmpfs.  You 
could create a tmpfs, then create a file that fills this up, then a 
loopback device pointing to that file, then build a file system atop 
that, and mount it.  And then mount gluster atop that.

Needless to say, all these layers significantly decrease performance and 
introduce inefficiencies.

> infiniband with half of the ports connected to computational nodes and
> the other half to gluster nodes?

There may be other options, but the options are not going to be 
cheap/inexpensive.  How fast, and by fast do you mean bandwidth and/or 
latency (e.g. streaming bandwidth or random IOPs)?  What does your IO 
profile look like?

You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for 
single readers/writers to single files.  For MPI jobs with single 
readers/writers, this is good.  For very large IO jobs where you need 
10's of GB/s, you probably need a more specific design to your problem.

Regards,

Joe


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615