[Gluster-users] Random "No such file or directory" error in gluster client logs - FIXED

[Lee Simpson]

Forgot to mention that my current working theory is that is may be linked to the" Virtio Disk" module in KVM. Switched the gluster server guest over to IDE mode and waiting to see if any errors pop up in the logs.

On Friday 19 Mar 2010 16:13:47 Lee Simpson wrote:
> Hello,
> 
> Thanks for your email. Today ive noticed that this problem is cropping up again. :(  All be it at a much much reduced rate with ext3. 
> 
> I also seem to be getting some STAT() type errors with ext3 which i never saw with ext4. Again if i ssh into the effected client and cat the file in question, it works and the problem for that file disappears.
> 
> Any help you can provide would be most welcome. 
> 
> More info follows;
> 
> The setup is 2 Physical servers split into multiple KVM guests. Gluster clients and servers are all on guests. The guests are all using the "Virtio Disk" module for their disks which on the dom0 is pointed at a LVM block device.
> 
> Kernel =  2.6.31-20-server #58-Ubuntu
> 
> OS = Ubuntu 9.10
> 
> Current Configs = https://gist.github.com/5f0e6a950712d7f24a64 (Ive tried reducing those configs to have no performance volumes)
> 
> Errors in gluster client logs = https://gist.github.com/6e2c6dff75fd3bc19fa9
> 
> 
> Is there any further info i can provide to help your diagnosis?
> 
> 
> 
> - Lee
> 
> 
> 
> 
> 
> 
> 
> On Friday 19 Mar 2010 09:07:59 Harshavardhana wrote:
> > On 03/18/2010 07:18 PM, Lee Simpson wrote:
> > > Hello,
> > >
> > > Just thought id share the experience i had with a gluster client error and the solution i found after much searching and chatting with some IRC guys.
> > >
> > > Im running a simple 2 server with multiple clients using cluster/replicate. Randomly newly created files produced the following error in the gluster client logs when accessed;
> > >
> > > "W [fuse-bridge.c:858:fuse_fd_cbk] glusterfs-fuse: 59480: OPEN() /data/randomfile-here =>  -1 (No such file or directory)"
> > >
> > > These files are created by apache or other scripts (such as awstats on a cron).  Apache is then unable to read the file, and the above message appears in the gluster logs everytime you try. If i SSH into the apache server and cat the file it displays fine and then apache starts reading it fine.
> > >
> > > I upgraded the client and server to 3.03 and tried reducing my configs to the bare min without any performance volumes..  But the problem persisted...
> > >
> > >
> > > SOLUTION
> > >
> > > The solution was quite simple. It turned out that it was because the server's data drive was formatted in ext4. Switched it to ext3 and the problems went away!
> > >
> > >
> > > Hope that helps someone else who finds this.
> > >
> > >
> > > - Lee
> > >
> > >
> > >    
> > 
> > Hi Lee,
> > 
> >      can you provide you kernel version under use?. Your volume files?.  
> > Let us figure out if this is related to "ext4"
> > 
> > Regards
> > 
> > 
> 
> 

-- 
Lee Simpson : Bsc(hons) Software Engineering
Software Development : Server Specification : Manufacture : Configuration

9 Partridge Rd,
St Albans
Herts
AL3 6HH

Email : Lee at LeeSimpson.me.uk
www : http://www.leenix.co.uk

Tel : 01727 855 124
Mob : 07961 348 790





[------------ Disclaimer ------------]
This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender by replying to this e-mail.

This email has been scanned for viruses