[Gluster-users] Self healing fails with permission errors

[Francois van der Ven]

Hi,

First my situation. I have two fileservers in replication mode. CentOS with
glusterfs 3.6.3. For a while now they've ran just fine.

I wanted to replace a hard drive of one server for a smaller one. So what I
did on the server was:
1) Stop glusterd service
2) Copy all from old drive to new drive
3) Stop server
4) Replace old drive with new (smaller drive)
5) Start server

The old drive had /dev/sdf and I checked that the new one had the same.
Glusterfs didn't start correctly so I followed this:

https://joejulian.name/blog/replacing-a-brick-on-glusterfs-340/

This made glusterfs start again. However, I got errors about permissions
and thus I decided to just wipe the new drive and use the gluster
self-healing. Now the self-healing doesn't work. I get the following errors:

[2015-05-15 13:31:19.090962] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 58: MKDIR /web (00000000-0000-0000-0000-000000000001/web)
==> (Permission denied)
[2015-05-15 13:31:19.095314] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 67: MKDIR /web (00000000-0000-0000-0000-000000000001/web)
==> (Permission denied)
[2015-05-15 13:31:19.145125] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (6f2e72e3-de39-4dcc-8b0f-d45994a1c49f) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.145168] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (6f2e72e3-de39-4dcc-8b0f-d45994a1c49f) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.145221] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (6f2e72e3-de39-4dcc-8b0f-d45994a1c49f) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.145272] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (6f2e72e3-de39-4dcc-8b0f-d45994a1c49f) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.150942] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (f739372b-4a60-44b2-8bc0-29030025cbf2) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.151004] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (f739372b-4a60-44b2-8bc0-29030025cbf2) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.151061] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (f739372b-4a60-44b2-8bc0-29030025cbf2) is not
found. anonymous fd creation failed
[2015-05-15 13:31:19.151099] W [server-resolve.c:437:resolve_anonfd_simple]
0-server: inode for the gfid (f739372b-4a60-44b2-8bc0-29030025cbf2) is not
found. anonymous fd creation failed
[2015-05-15 13:31:25.030158] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 201: MKDIR /web (00000000-0000-0000-0000-000000000001/web)
==> (Permission denied)
[2015-05-15 13:31:28.378031] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 285: MKDIR /web (00000000-0000-0000-0000-000000000001/web)
==> (Permission denied)
[2015-05-15 13:31:30.041158] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 369: MKDIR /web (00000000-0000-0000-0000-000000000001/web)
==> (Permission denied)
[2015-05-15 13:31:30.940040] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 40: MKDIR /mail (00000000-0000-0000-0000-000000000001/mail)
==> (Permission denied)
[2015-05-15 13:31:30.944764] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 49: MKDIR /mail (00000000-0000-0000-0000-000000000001/mail)
==> (Permission denied)
[2015-05-15 13:31:30.948941] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 58: MKDIR /mail (00000000-0000-0000-0000-000000000001/mail)
==> (Permission denied)
[2015-05-15 13:31:30.953006] I [server-rpc-fops.c:475:server_mkdir_cbk]
0-data-server: 67: MKDIR /mail (00000000-0000-0000-0000-000000000001/mail)
==> (Permission denied)

Apparantly gluster cannot create the directories. This is strange because
some other directories have been healed fine..

Could anyone help with this? I'm a bit worried because there is only one
file server running now.

Kind regards,

Francois van der Ven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150515/dd2a3e42/attachment.html>