[Gluster-users] Substitute for SMP?
tegner at renget.se
Fri May 27 15:13:29 UTC 2011
On 05/27/2011 04:31 PM, Joe Landman wrote:
> On 05/27/2011 07:12 AM, Jon Tegner wrote:
>> A general question, suppose I have a parallel application, using mpi,
>> where really fast access to the file system is critical.
>> Would it be stupid to consider a ram disk based setup? Say a 36 port QDR
> Ram disks won't work directly, due to lack of locking in tmpfs. You
> could create a tmpfs, then create a file that fills this up, then a
> loopback device pointing to that file, then build a file system atop
> that, and mount it. And then mount gluster atop that.
> Needless to say, all these layers significantly decrease performance
> and introduce inefficiencies.
>> infiniband with half of the ports connected to computational nodes and
>> the other half to gluster nodes?
> There may be other options, but the options are not going to be
> cheap/inexpensive. How fast, and by fast do you mean bandwidth and/or
> latency (e.g. streaming bandwidth or random IOPs)? What does your IO
> profile look like?
> You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for
> single readers/writers to single files. For MPI jobs with single
> readers/writers, this is good. For very large IO jobs where you need
> 10's of GB/s, you probably need a more specific design to your problem.
Would you say that the inefficiencies related to the ram disk would
remove all advantages of using ram instead of hard drives (or could it
still be worth a try)?
As for speed, I would think that latency would be the most critical -
but I don't really know, it its the code of a colleague of mine (I was
trying to come up with a replacement of his SMP-machine which is getting
More information about the Gluster-users