[Gluster-users] Substitute for SMP?
landman at scalableinformatics.com
Fri May 27 14:31:32 UTC 2011
On 05/27/2011 07:12 AM, Jon Tegner wrote:
> A general question, suppose I have a parallel application, using mpi,
> where really fast access to the file system is critical.
> Would it be stupid to consider a ram disk based setup? Say a 36 port QDR
Ram disks won't work directly, due to lack of locking in tmpfs. You
could create a tmpfs, then create a file that fills this up, then a
loopback device pointing to that file, then build a file system atop
that, and mount it. And then mount gluster atop that.
Needless to say, all these layers significantly decrease performance and
> infiniband with half of the ports connected to computational nodes and
> the other half to gluster nodes?
There may be other options, but the options are not going to be
cheap/inexpensive. How fast, and by fast do you mean bandwidth and/or
latency (e.g. streaming bandwidth or random IOPs)? What does your IO
profile look like?
You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for
single readers/writers to single files. For MPI jobs with single
readers/writers, this is good. For very large IO jobs where you need
10's of GB/s, you probably need a more specific design to your problem.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Gluster-users