[Gluster-users] gluster fuse disk state problem (reproducible )

[Marc Geerlings]

I think I have the same problems as:

http://gluster.org/pipermail/gluster-users/2011-June/007980.html

and

http://gluster.org/pipermail/gluster-users/2011-May/007697.html

Recently I installed Glusterfs 3.2.0 on four workstations which share a
total of 28TB of disk-space between them for batch processing of FMRI
and DTI data. Had the same setup between three workstations and 9TB with
Glusterfs 3.0.4 for over a year, but the hard-disk pool was to small.
The three "old" workstations from the old cluster and two more will also
access the 28TB pool.

OS: Scientific Linux 6
Glusterfs 3.2.0 and 3.2.1 compiled from source rpm

I've been testing the new setup for over a week now, but it seemed
unstable, Glusterfs clients would stop responding and tasks hung without
being able to kill -9. Only a reboot would stop the tasks.
a kill of the Glusterfs client and a mount afterwards would make the
gluster storage accessible again.

One of the tasks is converting a medical image format called nifti to
another format called analyze. We use a application included in a
package called mrtrix (2.9.0) for this. The application is mrconvert.

-A conversion of a large nifti file (256MB) to a analyze file will hang
application and the Gluster fuse client
- A small one (10MB) will work. 
- If I convert it reading from the Gluster storage to a local disk
everything works fine. 
- Reading from a local disk to the Gluster storage will hang the
application and gluster fuse client
- disabling the quick-read does nothing.

Errors are the same as a in the second post above, I can post them
Monday. I did a strace on a functioning and a none functioning convert
and the application hangs on the last close like:

munmap .............
close(4

The conversion was completed.
I tried 3.2.0, 3.1.4 and 3.2.1 and all of them have the same problem.
I went back to 3.0.8 and this is functioning.

-- 
Marc Geerlings 

                     Application Manager Research 
                        Department of Radiology

                  Maastricht University Medical Center
                        P.O. Box: 5800, 6202 AZ 
                      Maastricht, The Netherlands

                            P. Debyelaan 25
                          6229 HX, Maastricht
                             Room: 0.C1.037

                        Tel. + 31 43 – 387 49 50
                        Fax. + 31 43 – 387 69 09

                       email: m.geerlings at mumc.nl
                  m.geerlings at maastrichtuniversity.nl