[Gluster-users] ovirt read error

[David Gossage]

On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <
f.rothenstein at bodden-kliniken.de> wrote:

> Hi all,
>
> I am stuck with a strange problem using ovirt/gluster on ZFSonLinux.
>
> I detached a data-volume form my Cluster an reimported it to change
> from POSIX-type/glusterfs to GlusterFS-Storage. When I try to attach it
> to the Cluster I always get a Sanlock error. I checked the logs and what I
> found were some "operation not permitted"
> errors in the brick logs:
>
> [2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix-
> helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to get
> anonymous real_path:
> /tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395-ac62-
> b278670c333a _fd = -1 [No such file or directory]
> [2016-07-08 09:19:24.542103] W [MSGID: 113006]
> [posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from
> fd=0x7f15f0003b48 [Operation not permitted]
> [2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc-
> fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV -2
> (c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not permitted)
> [Operation not permitted]
>
> Anyone ever expierienced such problem?
>
> In fact I am able to create/change/delete files in the brick path
> itself and the mount points (NFS/Fuse).
> Ovirt itself seems to test the volumes with /usr/bin/dd, but uses
> iflag=direct. Using this parameter I'm also unable to create any file -
> is there any gluster-option that changed recently? Or is it ZFS, that
> introduced something that blocks now?
>

What version of Gluster are you on?

My errors aren't quite the same as yours, but I have a similar setup

Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux

Updating from 3.7.11 -> 12 or 13 this last weekend I had these issues pop
up for me which while not exactly the same did result in [Operation not
permitted]

[2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init]
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel
7.22
[2016-07-09 15:27:49.555466] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
operation failed [Operation not permitted]
[2016-07-09 15:27:49.556574] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
operation failed [Operation not permitted]
[2016-07-09 15:27:49.556659] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-3f1cf8e3534d
fd=0x7f5224002f68 (Operation not permitted)
[2016-07-09 15:27:59.612477] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
operation failed [Operation not permitted]
[2016-07-09 15:27:59.613700] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
operation failed [Operation not permitted]
[2016-07-09 15:27:59.613781] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-3f1cf8e3534d
fd=0x7f5224002f68 (Operation not permitted)

Another gluster user suggested that in 3.7.12/13 aio support was disabled
and I may need to change disk caching method.  So far I have just stayed at
3.7.11 until I have time to get a better test setup.


> Thanks in advance, Frank
>
>
>
>
>
>
> ______________________________________________________________________________
> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> Sandhufe 2
> 18311 Ribnitz-Damgarten
>
> Telefon: 03821-700-0
> Fax:       03821-700-240
>
> E-Mail: info at bodden-kliniken.de   Internet: http://www.bodden-kliniken.de
>
> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.:
> 079/133/40188
> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko
> Milski
>
> Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten
> Adressaten bestimmt. Wenn Sie nicht der vorge-
> sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten
> Sie bitte, dass jede Form der Veröf-
> fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail
> unzulässig ist. Wir bitten Sie, sofort den
> Absender zu informieren und die E-Mail zu löschen.
>
>
>              Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160712/2d662270/attachment.html>