[Gluster-users] booster and apache 2.2.14 permission errors

[Liam Slusser]

I'm having a strange booster+apache issue.  I am unable to get apache
to download any of the files through booster.  I get a 403 (Forbidden)
on any file.  If I enabled directory indexes i can get directory
listings but still a 403 on any file.  I can view/list just files just
fine by using LD_PRELOAD=...glusterfs-booster.so with ls or cat
/pub/data/path/to/myfile.  So its just apache that im having issues
with.  If i mount the file system (to /pub) with fuse and start httpd
without booster it works fine so im pretty sure i have all the
permissions correctly.

Ideas?

thanks,
liam

# wget -S http://x.x.x.x/data/test/test.mp3
--2009-11-05 18:35:07--  http://x.x.x.x/data/test/test.mp3
Connecting to x.x.x.x:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 403 Forbidden
  Date: Fri, 06 Nov 2009 02:35:07 GMT
  Server: Apache/2.2.14 (Unix)
  Content-Length: 228
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=iso-8859-1
2009-11-05 18:35:07 ERROR 403: Forbidden.


# wget -S http://x.x.x.x/data/test/
--2009-11-05 18:36:13--  http://x.x.x.x/data/test/
Connecting to x.x.x.x:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Fri, 06 Nov 2009 02:36:13 GMT
  Server: Apache/2.2.14 (Unix)
  Content-Length: 919
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html;charset=ISO-8859-1
Length: 919 [text/html]
Saving to: `index.html'

100%[=============================================================================>]
919         --.-K/s   in 0s

2009-11-05 18:36:13 (87.6 MB/s) - `index.html' saved [919/919]

(inside the index.html will be an apache pretty output of the files in
/data/test)

my booster-pub.log output:

[2009-11-05 18:41:50] D [libglusterfsclient.c:2908:glusterfs_open]
libglusterfsclient: path /pub/data/test/test.mp3
[2009-11-05 18:41:50] D
[libglusterfsclient.c:1517:_libgf_vmp_search_entry]
libglusterfsclient: VMP Search: path /pub/data/test/test.mp3, type:
LongestPrefix
[2009-11-05 18:41:50] D
[libglusterfsclient.c:1604:libgf_vmp_search_entry] libglusterfsclient:
VMP Entry found: path :/pub/data/test/test.mp3 vmp: /pub/
[2009-11-05 18:41:50] D
[libglusterfsclient.c:851:libgf_resolve_path_light]
libglusterfsclient: Path: /data/test/test.mp3, Resolved Path:
/data/test/test.mp3
[2009-11-05 18:41:50] D
[libglusterfsclient-dentry.c:389:libgf_client_path_lookup]
libglusterfsclient: resolved path(/data/test/test.mp3) to
1118653312/1118655564
[2009-11-05 18:41:50] D [libglusterfsclient.c:2659:libgf_client_open]
libglusterfsclient: open: path /data/test/test.mp3, status: 0, errno
117


My httpd.conf is very simple:

Alias /data/ "/pub/data"
<Directory "/pub/data/">
    Options All
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

booster.fstab:

/home/gluster/apps/glusterfs-2.0.7/etc/glusterfs/glusterfs.vol-pub.booster
/pub/ glusterfs
subvolume=cache,logfile=/home/gluster/apps/glusterfs-2.0.7/var/log/glusterfs/booster-pub.log,loglevel=DEBUG,attr_timeout=0

glusterfs.vol-pub.booster:

/home/gluster/apps/glusterfs-2.volume brick1a
  type protocol/client
  option transport-type tcp
  option remote-host x.x.x.x
  option remote-subvolume brick1a
end-volume

volume brick2a
  type protocol/client
  option transport-type tcp
  option remote-host x.x.x.x
 option remote-subvolume brick2a
end-volume

volume replicate
  type cluster/replicate
  subvolumes brick1a brick2a
end-volume

volume iothreads
  type performance/io-threads
  option thread-count 32
  subvolumes replicate
end-volume

volume readahead
  type performance/read-ahead
  option page-count 16       # cache per file  = (page-count x page-size)
  option force-atime-update off
  subvolumes iothreads
end-volume

volume cache
  type performance/io-cache
  option cache-size 512MB
  subvolumes readahead
end-volume