[Gluster-users] gluster opens thousands of files on one peer
mangoo at wpkg.org
Sun Jul 10 16:33:14 UTC 2011
On 09.07.2011 09:41, Anand Avati wrote:
> On Wed, Jul 6, 2011 at 6:58 AM, Tomasz Chmielewski <mangoo at wpkg.org
> <mailto:mangoo at wpkg.org>> wrote:
> One of the peers is behaving very slow - load is constantly around
> 6-15, and generally all connected clients work very slow.
> As soon as I shut that peer down, clients are relatively fast.
> What I noticed, that on one gluster peer, I only get around 100
> files opened by the gluster process:
> # lsof -n|grep -c gluster
> On the slow peer, it's hundreds, tens of thousands, and growing
> # while true; do lsof -n|grep -c gluster; sleep 10; done
> If I restart glusterd on that peer, it starts to grow again.
> On both peers, I have a big number of such entries:
> [2011-07-06 03:22:00.20873] W
> [server-resolve.c:556:server___resolve] 0-gluster-data-server: pure
> path resolution for /some/path/.../file (LOOKUP)
> Where the path is a symbolic link to the gluster mount.
> Can you describe this a little more? What do you mean by those paths are
> symbolic links to the gluster mount? They are supposed to be files
> within the gluster mount. Do you have a gluster mount on that peer as
> well? If so, does unmounting have any effect?
I mean, glusterfs is mounted in /home/glusterfs, with real data in /home/gluster-data.
/some/ is a symbolic link to /home/www; where we have some more symlinks, this time to somewhere inside the gluster mount point, /home/glusterfs.
For example, with a path like:
/some -> /home/www
2) another symlink
/home/www/storage_cache -> /home/glusterfs/storage_cache
So the real path without symlinks is this one:
> This is strange. There is almost no reason how you can be seeing this
> error unless the extended attributes were modified outside gluster. Can
> you get a dump of those attributes on those files with 'getfattr -d -m .
> -e hex /some/path/../file' and post it here?
# getfattr -d -m . -e hex /home/gluster-data/www/storage_cache/tatuaze/cache/photos/tatuaz/4662/tatuaze-na-przedramieniu-5663_5.jpg
getfattr: Removing leading '/' from absolute path names
# file: home/gluster-data/www/storage_cache/tatuaze/cache/photos/tatuaz/4662/tatuaze-na-przedramieniu-5663_5.jpg
Note that the issue is somehow gone; I've made "ls -lR" on both backends,
More information about the Gluster-users