[Gluster-users] glusterfs on HPC cluster

[Raghavendra Talur]

On Tue, Mar 1, 2016 at 3:47 PM, Fedele Stabile <fedele.stabile at fis.unical.it
> wrote:

> Hello to all,
> my question of the day is: can I use GlusterFS on a 32-node cluster
> HPC?
> Let me explain:
> - each node in the cluster has one extra disk that is a brick of a
> glusterFS cluster-wide mounted
> - interconnection network is 40GB/s Infiniband
> - users run MPI jobs
>
> Everything works perfectly except that a user complains that his MPI
> job after gluster implementation is slower than 50%
>
> I made a linpack test that gives optimal results if runs on 16 nodes,
> but if runs on 32 nodes
>  it happens that one of the gluster nodes disconnects and the linpack
> don't finish.
>
> Can anyone help me?
>

Hi Fedele,

We are interested to solve this, we have received similar description from
other users too.
Note that I am talking only about number of clients and not about MPI. MPI
workload should run fine on Gluster.

Some questions:
1. What version of Gluster?
2. What OS?
3. Is the volume set to RDMA mode or are you using IPoIB?
4. Do all the clients work on the same file from the volume?

Thanks,
Raghavendra Talur




>
> Fedele
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160301/1a131e28/attachment.html>