[Gluster-users] ovirt read error
Krutika Dhananjay
kdhananj at redhat.com
Tue Jul 12 10:20:48 UTC 2016
Frank,
Could you share your volume configuration (`gluster volume info <VOLNAME>`)?
-Krutika
On Tue, Jul 12, 2016 at 3:44 PM, David Gossage <dgossage at carouselchecks.com>
wrote:
>
>
> On Tue, Jul 12, 2016 at 5:08 AM, Frank Rothenstein <
> f.rothenstein at bodden-kliniken.de> wrote:
>
>> Hi David,
>>
>> As I'm on gluster I use
>>
>> glusterfs.x86_64 3.7.12-2.el7 @centos-gluster37
>>
>> I already tried to revert back to 3.7.11 but somehow this didn't help.
>>
>> Do you have more info on aio/caching?
>>
>
> Not at the moment I really haven't had time to dig into it this week yet.
> I'll quote Lindsay Mathieson 's response and maybe she can chime in if she
> sees this chain.
>
>
> --------------------------------------------------------------
> Hi David, I did a quick test this morning with Proxmox and 3.7.13 and was
> able to get it working with the fuse mount *and* libgfapi.
>
>
> One caveat - you *have* to enable qemu caching, either write-back or
> write-through. 3.7.12 & 13 seem to now disable aio support, and qemu
> requires that when caching is turned off.
>
>
> There are setting for aio in gluster that I haven't played with yet.
> -----------------------------------------------------------------
>
> Did you revert client and server's to 3.7.11? It could be also that we
> had 2 completely different issues. The only similarity being the recent
> upgrades and Operation not permitted messages.
>
>
>> FrankAm Dienstag, den 12.07.2016, 04:25 -0500 schrieb David Gossage:
>> >
>> > On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <f.rothenstein at bod
>> > den-kliniken.de> wrote:
>> > > Hi all,
>> > >
>> > > I am stuck with a strange problem using ovirt/gluster on
>> > > ZFSonLinux.
>> > >
>> > > I detached a data-volume form my Cluster an reimported it to change
>> > > from POSIX-type/glusterfs to GlusterFS-Storage. When I try to
>> > > attach it
>> > > to the Cluster I always get a Sanlock error. I checked the logs and
>> > > what I found were some "operation not permitted"
>> > > errors in the brick logs:
>> > >
>> > > [2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix-
>> > > helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to get
>> > > anonymous real_path:
>> > > /tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395-ac62-
>> > > b278670c333a _fd = -1 [No such file or directory]
>> > > [2016-07-08 09:19:24.542103] W [MSGID: 113006]
>> > > [posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from
>> > > fd=0x7f15f0003b48 [Operation not permitted]
>> > > [2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc-
>> > > fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV -2
>> > > (c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not
>> > > permitted)
>> > > [Operation not permitted]
>> > >
>> > > Anyone ever expierienced such problem?
>> > >
>> > > In fact I am able to create/change/delete files in the brick path
>> > > itself and the mount points (NFS/Fuse).
>> > > Ovirt itself seems to test the volumes with /usr/bin/dd, but uses
>> > > iflag=direct. Using this parameter I'm also unable to create any
>> > > file -
>> > > is there any gluster-option that changed recently? Or is it ZFS,
>> > > that
>> > > introduced something that blocks now?
>> >
>> > What version of Gluster are you on?
>> >
>> > My errors aren't quite the same as yours, but I have a similar setup
>> >
>> > Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux
>> >
>> > Updating from 3.7.11 -> 12 or 13 this last weekend I had these issues
>> > pop up for me which while not exactly the same did result
>> > in [Operation not permitted]
>> >
>> > [2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init] 0-
>> > glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22
>> > kernel 7.22
>> > [2016-07-09 15:27:49.555466] W [MSGID: 114031] [client-rpc-
>> > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
>> > operation failed [Operation not permitted]
>> > [2016-07-09 15:27:49.556574] W [MSGID: 114031] [client-rpc-
>> > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
>> > operation failed [Operation not permitted]
>> > [2016-07-09 15:27:49.556659] W [fuse-bridge.c:2227:fuse_readv_cbk] 0-
>> > glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-
>> > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)
>> > [2016-07-09 15:27:59.612477] W [MSGID: 114031] [client-rpc-
>> > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
>> > operation failed [Operation not permitted]
>> > [2016-07-09 15:27:59.613700] W [MSGID: 114031] [client-rpc-
>> > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
>> > operation failed [Operation not permitted]
>> > [2016-07-09 15:27:59.613781] W [fuse-bridge.c:2227:fuse_readv_cbk] 0-
>> > glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-
>> > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)
>> >
>> > Another gluster user suggested that in 3.7.12/13 aio support was
>> > disabled and I may need to change disk caching method. So far I have
>> > just stayed at 3.7.11 until I have time to get a better test setup.
>> >
>> > > Thanks in advance, Frank
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > ___________________________________________________________________
>> > > ___________
>> > > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
>> > > Sandhufe 2
>> > > 18311 Ribnitz-Damgarten
>> > >
>> > > Telefon: 03821-700-0
>> > > Fax: 03821-700-240
>> > >
>> > > E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-klini
>> > > ken.de
>> > >
>> > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-
>> > > Nr.: 079/133/40188
>> > > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr.
>> > > Falko Milski
>> > >
>> > > Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten
>> > > Adressaten bestimmt. Wenn Sie nicht der vorge-
>> > > sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten,
>> > > beachten Sie bitte, dass jede Form der Veröf-
>> > > fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-
>> > > Mail unzulässig ist. Wir bitten Sie, sofort den
>> > > Absender zu informieren und die E-Mail zu löschen.
>> > >
>> > >
>> > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
>> > > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
>> > >
>> > > _______________________________________________
>> > > Gluster-users mailing list
>> > > Gluster-users at gluster.org
>> > > http://www.gluster.org/mailman/listinfo/gluster-users
>> >
>>
>>
>>
>>
>>
>>
>> ______________________________________________________________________________
>> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
>> Sandhufe 2
>> 18311 Ribnitz-Damgarten
>>
>> Telefon: 03821-700-0
>> Fax: 03821-700-240
>>
>> E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-kliniken.de
>>
>> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.:
>> 079/133/40188
>> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko
>> Milski
>>
>> Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten
>> Adressaten bestimmt. Wenn Sie nicht der vorge-
>> sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten,
>> beachten Sie bitte, dass jede Form der Veröf-
>> fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail
>> unzulässig ist. Wir bitten Sie, sofort den
>> Absender zu informieren und die E-Mail zu löschen.
>>
>>
>> Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
>> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
>>
>>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160712/4d41143b/attachment.html>
More information about the Gluster-users
mailing list