[Gluster-users] Gluster problems permission denied LOOKUP () /etc/samba/private/msg.sock

Sun Oct 7 00:21:31 UTC 2018

I have followed the guide and reconfigured everything as per the
Admnistration guide.

The error messages about (Permissions denied) and the path
/etc/samba/private/msg.sock have stopped. However, I continue to have
problems with saving files to the samba shares.

The particular issue is very reproducible when using Revit files
(Autodesk Revit with .rvt extension). I have also found another
problem now downloading simple files like NVIDIA drivers from the
nvidia website and saving them to one of the samba shares using Google
Chrome. What happens is that the machine apparently locks up in trying
to save the file. Firefox does not seem to lock up. The only work
around for now to fix downloads using google chrome and the revit file
writes, is to stop using the vfs gluster objects in all the samba
shares. I simply mount the gluster volume via fuse mount and set the
path to the mounted folder to export from samba.

/etc/samba/smb.conf -> http://termbin.com/e0rp
/etc/ctdb/ctdbd/conf -> http://termbin.com/y0hq
gluster v status -> http://termbin.com/pafu

Logs for the projects share when vfs gluster plugin was enabled ->
http://termbin.com/o7a5

This is likely a different thread, should I start a new one for that issue?

Diego
On Fri, Oct 5, 2018 at 10:20 AM Diego Remolina <dijuremo at gmail.com> wrote:
>
> Hi,
>
> Thanks for the reply!
>
> This was setup a few years ago and was working OK, even when falling back to this server. We had not failed over to this server recently after the latest samba upgrades, so Not sure if maybe the new samba and ctdb packages had a change that is creating the issue.
>
> samba-libs-4.7.1-9.el7_5.x86_64
> samba-client-libs-4.7.1-9.el7_5.x86_64
> samba-common-tools-4.7.1-9.el7_5.x86_64
> samba-common-4.7.1-9.el7_5.noarch
> samba-common-libs-4.7.1-9.el7_5.x86_64
> samba-vfs-glusterfs-4.7.1-9.el7_5.x86_64
> samba-4.7.1-9.el7_5.x86_64
>
> It may not be the right way to do it, so I am going to investigate your suggestion and find out if it works for us. I do need your help with answers to some questions below.
>
> A bit of an explanation on the current setup. Both servers, ysmha01 and ysmha02 are joined against AD using sssd. We are not using winbindd at all.
>
> For each server, we created a machine account in AD, and we also created a computer account for the "Shared" host name. So we have these 3 computer objects in AD
> ysmha01 10.0.0.6
> ysmha02 10.0.0.7
> ysmserver 10.0.0.1 (this ip is handled by ctdb)
>
> We are not controlling smb with ctdb (doing it manually).
>
> Both ysmha01 and ysmha02 were tied to AD using: realm join domain -v unattended
>
> Then we modified the sssd.conf file as follows:
>
> http://termbin.com/wulh
>
> And restarted sssd and everything works fine getting users and groups.
>
> We populate uidNumbers and gidNumbers for all users and groups in AD, so the permissions work.
>
> Then we configured samba to join the domain using the ysmserver machine account and only password (not keytab). So in order to keep the samba information available to both servers, we used the configuration:
>
> private dir = /export/etc/samba/private
>
> Since this is an un-conventional setup, could you explain the process of using both sssd and joining the machine to the AD domain? I am not quite sure I understand how to do that after having used SSSD first. In occasions where I set ysmha01 and ysmha02 as the netbios name for smb.conf and then ran net ads join after realm join, it simply updated the keytab and then sssd would not work anymore. This is why we ended up using the setup above. If you could point to a good process including smb.conf and how to join the machines to the domain, that would be appreciated.
>
> This is the current config for samba. For the Projects share I had to disable vfs gluster because I had issues with one specific type of files, but it would be really nice if I can clean up all of this and get it to work properly using vfs gluster for all shares.
>
> http://termbin.com/2f64
>
> After replacing the motherboard on ysmha02 and bringing it back up last night, things seem to be working fine so far, but I still see the gluster error messages and I want to fix this and run it properly as it should:
>
> [2018-10-05 13:41:21.279685] I [MSGID: 139001] [posix-acl.c:269:posix_acl_log_permit_denied] 0-posix-acl-autoload: cli
> ent: -, gfid: 5b5bed22-ace0-410d-8623-4f1a31069b81, req(uid:1058,gid:513,perm:1,ngrps:3), ctx(uid:0,gid:0,in-groups:0,
> perm:700,updated-fop:LOOKUP, acl:-) [Permission denied]
> [2018-10-05 13:41:21.279758] W [fuse-bridge.c:490:fuse_entry_cbk] 0-glusterfs-fuse: 10521075: LOOKUP() /etc/samba/priv
> ate/msg.sock/6945 => -1 (Permission denied)
> [2018-10-05 13:41:21.279827] W [fuse-bridge.c:490:fuse_entry_cbk] 0-glusterfs-fuse: 10521076: LOOKUP() /etc/samba/priv
> ate/msg.sock/6945 => -1 (Permission denied)
>
> The link you sent is broken, but I think it should be:
>
> https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/administration_guide/#sect-SMB_CTDB
>
> Thanks
>
> Diego
>
>
> On Thu, Oct 4, 2018, 09:16 Poornima Gurusiddaiah <pgurusid at redhat.com> wrote:
>>
>>
>>
>> On Tue, Oct 2, 2018 at 5:26 PM Diego Remolina <dijuremo at gmail.com> wrote:
>>>
>>> Dear all,
>>>
>>> I have a two node setup running on Centos and gluster version
>>> glusterfs-3.10.12-1.el7.x86_64
>>>
>>> One of my nodes died (motherboard issue). Since I had to continue
>>> being up, I modified the quorum to below 50% to make sure I could
>>> still run on one server.
>>>
>>> The server runs ovirt and 2 VMs on top of a volume called vmstorage. I
>>> also had a third node in the peer list, but never configured it as an
>>> arbiter, so it just comes up in gluster v status. The server also run
>>> a file server with samba to serve files to windows machines.
>>>
>>> The issue is that since starting the server on it's own as the samba
>>> server, I am seeing permission denied errors for the "export" volume
>>> in /var/log/glusterfs/export.log
>>>
>>> The errors look like this and repeat over and over:
>>>
>>> [2018-10-02 11:46:56.327925] I [MSGID: 139001]
>>> [posix-acl.c:269:posix_acl_log_permit_denied] 0-posix-acl-autoload:
>>> client: -, gfid: 5b5bed22-ace0-410d-8623-4f1a31069b81,
>>> req(uid:1051,gid:513,perm:1,ngrps:2),
>>> ctx(uid:0,gid:0,in-groups:0,perm:700,updated-fop:LOOKUP, acl:-)
>>> [Permission denied]
>>> [2018-10-02 11:46:56.328004] W [fuse-bridge.c:490:fuse_entry_cbk]
>>> 0-glusterfs-fuse: 20599112: LOOKUP() /etc/samba/private/msg.sock/15149
>>> => -1 (Permission denied)
>>> [2018-10-02 11:46:56.328185] W [fuse-bridge.c:490:fuse_entry_cbk]
>>> 0-glusterfs-fuse: 20599113: LOOKUP() /etc/samba/private/msg.sock/15149
>>> => -1 (Permission denied)
>>> [2018-10-02 11:47:53.766562] W [fuse-bridge.c:490:fuse_entry_cbk]
>>> 0-glusterfs-fuse: 20600590: LOOKUP() /etc/samba/private/msg.sock/15149
>>> => -1 (Permission denied)
>>>
>>> The gluster volume export is mounted on /export, samba and ctdb are
>>> instructed to use /export/etc/samba/private and /export/lock which is
>>> on the gluster file system for the clustered tdb, etc. However, I keep
>>> getting the log messages that fuse seems to try to access a folder
>>> that does not exist, /etc/samba/private/msg.sock
>>
>>
>> This is an unconventional setup, the suggested way of clustering samba is as mentioned in [1]. Sharing tdbs using gluster volume can lead to more issues. Has the setup ever worked? Was this setup suggested somewhere?
>>
>> [1] https://access.qa.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/administration_guide/#sect-SMB_CTDB
>>
>>>
>>> Why is this, how can I fix it?
>>>
>>> [root at ysmha01 export]# gluster v status export
>>> Status of volume: export
>>> Gluster process                             TCP Port  RDMA Port  Online  Pid
>>> ------------------------------------------------------------------------------
>>> Brick 10.0.1.6:/bricks/hdds/brick           49153     0          Y       3516
>>> Self-heal Daemon on localhost               N/A       N/A        Y       3710
>>> Self-heal Daemon on 10.0.1.5                N/A       N/A        Y       4380
>>>
>>> Task Status of Volume export
>>> ------------------------------------------------------------------------------
>>> There are no active volume tasks
>>>
>>> These are all the volume options currently set:
>>>
>>> http://termbin.com/1xm5
>>>
>>> Diego
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> https://lists.gluster.org/mailman/listinfo/gluster-users