[Gluster-users] 3.1.2 with "No such file" and "Invalid argument" errors

[Raghavendra G]

Hi Steve,

Are the back-end file systems working correctly? I am seeing lots of errors in server log files while accessing back-end filesystem.

gluster-01-brick.log.1:[2011-01-26 03:43:07.353445] E [posix.c:2193:posix_open] post-posix: open on /gluster/01/bri
ck/home/lev/deltah/aadimers/serd/converge/0..75000/serd_phi-psi_hist.4deg.0..75000_map.cmd: Read-only file system
gluster-01-brick.log.1:[2011-01-26 03:43:07.353857] E [posix.c:678:posix_setattr] post-posix: setattr (utimes) on /
gluster/01/brick/home/lev/deltah/aadimers/serd/converge/0..75000/serd_phi-psi_hist.4deg.0..75000_map.cmd failed: Re
ad-only file system
gluster-01-brick.log.1:[2011-01-26 03:43:07.354827] E [posix.c:2318:posix_readv] post-posix: read failed on fd=0x7f
28e50dc1c8: Input/output error
gluster-01-brick.log.1:[2011-01-26 03:43:07.357396] E [posix.c:2193:posix_open] post-posix: open on /gluster/01/bri
ck/home/lev/deltah/aadimers/serd/converge/0..75000/serd_phi-psi_hist.4deg.0..75000_map.ps: Read-only file system
gluster-01-brick.log.1:[2011-01-26 03:43:07.357794] E [posix.c:678:posix_setattr] post-posix: setattr (utimes) on /
gluster/01/brick/home/lev/deltah/aadimers/serd/converge/0..75000/serd_phi-psi_hist.4deg.0..75000_map.ps failed: Rea
d-only file system
gluster-01-brick.log.1:[2011-01-26 03:43:07.358865] E [posix.c:2318:posix_readv] post-posix: read failed on fd=0x7f
28e50dc1c8: Input/output error
gluster-01-brick.log.1:[2011-01-26 03:43:07.359264] E [posix.c:2318:posix_readv] post-posix: read failed on fd=0x7f
28e50dc1c8: Input/output error
gluster-01-brick.log.1:[2011-01-26 03:43:07.359548] E [posix.c:2318:posix_readv] post-posix: read failed on fd=0x7f
28e50dc1c8: Input/output error
gluster-01-brick.log.1:[2011-01-26 03:43:07.367163] E [posix.c:2318:posix_readv] post-posix: read failed on fd=0x7f

I am seeing other errors, which indicate that the backend is read-only filesystem. Due to this distribute and replicate are not able to store the metadata (using xattrs), which in turn is resulting in lots of split-brains and layout NULL errors. Can you please check the backend file system?

regards,
----- Original Message -----
> From: "Steve Wilson" <stevew at purdue.edu>
> To: "Lakshmipathi" <lakshmipathi at gluster.com>
> Cc: "Raghavendra G" <raghavendra at gluster.com>
> Sent: Thursday, February 3, 2011 7:21:36 PM
> Subject: Re: [Gluster-users] 3.1.2 with "No such file" and "Invalid argument" errors
> Hi,
> 
> Thanks for looking into this. Any ideas so far? Or anything you'd like
> me to try?
> 
> Here's some other perhaps relevant information:
> * all bricks are formatted ext4 and mounted with the noatime option
> in addition to default options
> * servers and clients are running Ubuntu 10.04
> * I did try mounting the GlusterFS volume with direct-io-mode
> disabled but that didn't fix the problem
> 
> Thanks!
> 
> Steve
> 
> On 02/01/2011 07:35 AM, Lakshmipathi wrote:
> > Hi,
> > Could you please sent us client and server log files?
> >
> >
> 
> --
> Steven M. Wilson, Systems and Network Manager
> Markey Center for Structural Biology
> Purdue University
> (765) 496-1946