[Gluster-devel] another mailstore glusterfs issue

[Jeff Humes]

I upgraded my dovecot imap server so that I could use the 
"disable_mmap=yes" option because I was seeing errors trying to write to 
.index and some other mmap'ed files in a glusterfs mounted mailstore.
Those errors are not showing up anymore, but I am now seeing some other 
(harder to troubleshoot) errors:
 
the mail client will get an email and click on it, and it will show no 
content in the message.  I look at the imap server log and see the 
following error:
dovecot: Nov 18 20:16:56 Error: IMAP(jana at bofus.org 
<mailto:jana at bofus.org>): Maildir: Symlink destination doesn't exist: 
/opt/GFS/postfix/vmail/jana at bofus 
<mailto:jana at bofus>.org/cur/1195435820.V13Iaad099.asknmapr01:2,S
 
So I do an "ls -l" on that file:
# ls -l /opt/GFS/postfix/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S
ls: /opt/GFS/postfix/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S: No 
such file or directory
-rw-------  1 vmail vmail 9247 Nov 18 19:30 
/opt/GFS/postfix/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S
 
first it says "No such file or directory", then it lists it out.
If I try to touch the file, it fails with no such file.  If I try to 
remove the file it fails in the same manner.
 
Here is my client mountpoint:
glusterfs on /opt/GFS/postfix type fuse 
(rw,nosuid,nodev,allow_other,default_permissions,max_read=1048576)
 
 
So now I login to the glusterfs server:
I can do an successful "ls -l" on that same file above:
# ls -l /glusterfs/knworksmail/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S
-rw-------  1 15002 15002 9247 Nov 18 19:30 
/glusterfs/knworksmail/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S
 
Here on the server I can move the file around:
mv /glusterfs/knworksmail/vmail/jana at bofus.org 
<mailto:jana at bofus.org>/cur/1195435820.V13Iaad099.asknmapr01:2,S /var/tmp
- works -
 
Now if I move the file back into place, the mail client can then see it 
like it should have been able to in the first place:
mv /var/tmp/1195435820.V13Iaad099.asknmapr01:2,S 
/glusterfs/knworksmail/vmail/jana at bofus.org <mailto:jana at bofus.org>/cur/
- client now sees it and the content -
 
This happens seemingly randomly to mail.  this did not happen before the 
upgrade, and does not occur if I move the mailstore to local ext3 disk.
 
Any Ideas?  do I need to provide more information?  
 
Thanks!
 
-Jeff Humes