[Gluster-users] booster with apache permission denied

[Liam Slusser]

I believe i posted this here before but never got any replies.  I'm in
the middle of upgrading to Gluster 2.0.9 and would like to move away
from having to use fuse to serve up files out of apache so im working
again on getting boosting working correctly.

Everything appears to load and work fine but i always get permission
denied, 403, in my apache logs.  Works fine under fuse.  I'm running
Apache under the user nobody which does have read access to all the
files.

I'm not sure where to start debugging this?  Ideas?

thanks,
liam

I can run things from the command line...

[root at server]# LD_PRELOAD="/home/gluster/apps/glusterfs-2.0.9/lib/glusterfs/glusterfs-booster.so"
file /pub/data/tnsc/test/test.mp3
/pub/data/tnsc/test/test.mp3: MPEG ADTS, layer III, v1, 128 kBits,
44.1 kHz, JntStereo

[root at server]# LD_PRELOAD="/home/gluster/apps/glusterfs-2.0.9/lib/glusterfs/glusterfs-booster.so"
ls -agl /pub/data/tnsc/test/test.mp3
ls: /pub/data/tnsc/test/test.mp3: Invalid argument
-rw-r--r-- 1 user 3692251 Aug 27  2007 /pub/data/tnsc/test/test.mp3

[root at server]# wget -S http://x.x.x.x/tnsc/test/test.mp3
--2010-01-08 13:34:55--  http://x.x.x.x/tnsc/test/test.mp3
Connecting to x.x.x.x:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 403 Forbidden
  Date: Fri, 08 Jan 2010 21:34:55 GMT
  Server: Apache/2.2.14 (Unix)
  Content-Length: 228
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html; charset=iso-8859-1
2010-01-08 13:34:55 ERROR 403: Forbidden.

Apache access log:

x.x.x.x - - [08/Jan/2010:12:50:46 -0800] "GET /tnsc/test/test.mp3
HTTP/1.0" 403 228 "-" "Wget/1.11.4 Red Hat modified"

Apache error log:

[Fri Jan 08 12:50:46 2010] [error] [client x..x.x.x] (22)Invalid
argument: file permissions deny server access:
/pub/data/tnsc/test/test.mp3

Gluster error log:

[2010-01-08 12:50:46] D [libglusterfsclient.c:4866:glusterfs_stat]
libglusterfsclient: path /pub/data/tnsc/test/test.mp3
[2010-01-08 12:50:46] D
[libglusterfsclient.c:1541:_libgf_vmp_search_entry]
libglusterfsclient: VMP Search: path /pub/data/tnsc/test/test.mp3,
type: LongestPrefix
[2010-01-08 12:50:46] D
[libglusterfsclient.c:1628:libgf_vmp_search_entry] libglusterfsclient:
VMP Entry found: path :/pub/data/tnsc/test/test.mp3 vmp: /pub/
[2010-01-08 12:50:46] D [libglusterfsclient.c:4788:__glusterfs_stat]
libglusterfsclient: path /data/tnsc/test/test.mp3, op: 2
[2010-01-08 12:50:46] D
[libglusterfsclient.c:869:libgf_resolve_path_light]
libglusterfsclient: Path: /data/tnsc/test/test.mp3, Resolved Path:
/data/tnsc/test/test.mp3
[2010-01-08 12:50:46] D
[libglusterfsclient-dentry.c:389:libgf_client_path_lookup]
libglusterfsclient: resolved path(/data/tnsc/test/test.mp3) to
1118653312/1118655564
[2010-01-08 12:50:46] D [libglusterfsclient.c:4710:libgf_client_stat]
libglusterfsclient: Cache will be used
[2010-01-08 12:50:46] D [libglusterfsclient.c:3001:glusterfs_open]
libglusterfsclient: path /pub/data/tnsc/test/test.mp3
[2010-01-08 12:50:46] D
[libglusterfsclient.c:1541:_libgf_vmp_search_entry]
libglusterfsclient: VMP Search: path /pub/data/tnsc/test/test.mp3,
type: LongestPrefix
[2010-01-08 12:50:46] D
[libglusterfsclient.c:1628:libgf_vmp_search_entry] libglusterfsclient:
VMP Entry found: path :/pub/data/tnsc/test/test.mp3 vmp: /pub/
[2010-01-08 12:50:46] D
[libglusterfsclient.c:869:libgf_resolve_path_light]
libglusterfsclient: Path: /data/tnsc/test/test.mp3, Resolved Path:
/data/tnsc/test/test.mp3
[2010-01-08 12:50:46] D
[libglusterfsclient-dentry.c:389:libgf_client_path_lookup]
libglusterfsclient: resolved path(/data/tnsc/test/test.mp3) to
1118653312/1118655564
[2010-01-08 12:50:46] D [libglusterfsclient.c:2752:libgf_client_open]
libglusterfsclient: open: path /data/tnsc/test/test.mp3, status: 0,
errno 117

Booster.fstab:

/home/gluster/apps/glusterfs-2.0.9/etc/glusterfs/glusterfs.vol-pub.booster
/pub/ glusterfs
subvolume=cache,logfile=/home/gluster/apps/glusterfs-2.0.9/var/log/glusterfs/booster-pub.log,loglevel=DEBUG,attr_timeout=0

glusterfs.vol-pub.booster:

volume brick1a
  type protocol/client
  option transport-type tcp
  option remote-host x.x.x.30
  option remote-subvolume brick1a
end-volume

volume brick2a
  type protocol/client
  option transport-type tcp
  option remote-host x.x.x.35
 option remote-subvolume brick2a
end-volume

volume replicate
  type cluster/replicate
  subvolumes brick1a brick2a
end-volume

volume iothreads
  type performance/io-threads
  option thread-count 32
  subvolumes replicate
end-volume

volume readahead
  type performance/read-ahead
  option page-count 16       # cache per file  = (page-count x page-size)
  option force-atime-update off
  subvolumes iothreads
end-volume

volume cache
  type performance/io-cache
  option cache-size 512MB
  subvolumes readahead
end-volume

httpd.conf:

Alias /tnsc/ "/pub/data/tnsc/"
<Directory "/pub/data/tnsc/">
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>