[Gluster-users] Substitute for SMP?

[Jon Tegner]

On 05/27/2011 04:31 PM, Joe Landman wrote:
> On 05/27/2011 07:12 AM, Jon Tegner wrote:
>> A general question, suppose I have a parallel application, using mpi,
>> where really fast access to the file system is critical.
>>
>> Would it be stupid to consider a ram disk based setup? Say a 36 port QDR
>
> Ram disks won't work directly, due to lack of locking in tmpfs.  You 
> could create a tmpfs, then create a file that fills this up, then a 
> loopback device pointing to that file, then build a file system atop 
> that, and mount it.  And then mount gluster atop that.
>
> Needless to say, all these layers significantly decrease performance 
> and introduce inefficiencies.
>
>> infiniband with half of the ports connected to computational nodes and
>> the other half to gluster nodes?
>
> There may be other options, but the options are not going to be 
> cheap/inexpensive.  How fast, and by fast do you mean bandwidth and/or 
> latency (e.g. streaming bandwidth or random IOPs)?  What does your IO 
> profile look like?
>
> You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for 
> single readers/writers to single files.  For MPI jobs with single 
> readers/writers, this is good.  For very large IO jobs where you need 
> 10's of GB/s, you probably need a more specific design to your problem.
>
> Regards,
>
> Joe
>
>
Thanks!

Would you say that the inefficiencies related to the ram disk would 
remove all advantages of using ram instead of hard drives (or could it 
still be worth a try)?

As for speed, I would think that latency would be the most critical - 
but I don't really know, it its the code of a colleague of mine (I was 
trying to come up with a replacement of his SMP-machine which is getting 
old).

Regards,

/jon