[Gluster-users] ovirt read error
Frank Rothenstein
f.rothenstein at bodden-kliniken.de
Tue Jul 12 10:38:49 UTC 2016
Yes, no problem.
This is an export domain in ovirt:
gluster v info export
Volume Name: export
Type: Replicate
Volume ID: 0b50341b-9c9a-4868-9c53-7f0ecdb44162
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: glusternode1:/tank2/export/brick1
Brick2: glusternode2:/tank2/export/brick1
Options Reconfigured:
performance.write-behind: on
performance.cache-size: 1GB
server.event-threads: 8
client.event-threads: 8
client.bind-insecure: on
cluster.eager-lock: enable
performance.stat-prefetch: off
performance.io-cache: off
performance.read-ahead: off
performance.quick-read: off
server.allow-insecure: on
network.remote-dio: enable
cluster.quorum-type: auto
storage.owner-gid: 36
storage.owner-uid: 36
auth.allow: *
user.cifs: enable
nfs.disable: ON
cluster.server-quorum-type: serverFirst problem in ovirt is to init the sanlock...
Am Dienstag, den 12.07.2016, 15:50 +0530 schrieb Krutika Dhananjay:
> Frank,
>
> Could you share your volume configuration (`gluster volume info
> <VOLNAME>`)?
>
> -Krutika
>
> On Tue, Jul 12, 2016 at 3:44 PM, David Gossage <dgossage at carouselchec
> ks.com> wrote:
> >
> > On Tue, Jul 12, 2016 at 5:08 AM, Frank Rothenstein <f.rothenstein at b
> > odden-kliniken.de> wrote:
> > > Hi David,
> > >
> > > As I'm on gluster I use
> > >
> > > glusterfs.x86_64 3.7.12-2.el7 @centos-gluster37
> > >
> > > I already tried to revert back to 3.7.11 but somehow this didn't
> > > help.
> > >
> > > Do you have more info on aio/caching?
> >
> > Not at the moment I really haven't had time to dig into it this
> > week yet. I'll quote Lindsay Mathieson 's response and maybe she
> > can chime in if she sees this chain.
> >
> >
> > --------------------------------------------------------------
> > Hi David, I did a quick test this morning with Proxmox and 3.7.13
> > and was able to get it working with the fuse mount *and* libgfapi.
> >
> >
> > One caveat - you *have* to enable qemu caching, either write-back
> > or write-through. 3.7.12 & 13 seem to now disable aio support, and
> > qemu requires that when caching is turned off.
> >
> >
> > There are setting for aio in gluster that I haven't played with
> > yet.
> > -----------------------------------------------------------------
> >
> > Did you revert client and server's to 3.7.11? It could be also
> > that we had 2 completely different issues. The only similarity
> > being the recent upgrades and Operation not permitted messages.
> >
> > > FrankAm Dienstag, den 12.07.2016, 04:25 -0500 schrieb David
> > > Gossage:
> > > >
> > > > On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <f.rothenste
> > > in at bod
> > > > den-kliniken.de> wrote:
> > > > > Hi all,
> > > > >
> > > > > I am stuck with a strange problem using ovirt/gluster on
> > > > > ZFSonLinux.
> > > > >
> > > > > I detached a data-volume form my Cluster an reimported it to
> > > change
> > > > > from POSIX-type/glusterfs to GlusterFS-Storage. When I try to
> > > > > attach it
> > > > > to the Cluster I always get a Sanlock error. I checked the
> > > logs and
> > > > > what I found were some "operation not permitted"
> > > > > errors in the brick logs:
> > > > >
> > > > > [2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix-
> > > > > helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to
> > > get
> > > > > anonymous real_path:
> > > > > /tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395-
> > > ac62-
> > > > > b278670c333a _fd = -1 [No such file or directory]
> > > > > [2016-07-08 09:19:24.542103] W [MSGID: 113006]
> > > > > [posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from
> > > > > fd=0x7f15f0003b48 [Operation not permitted]
> > > > > [2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc-
> > > > > fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV
> > > -2
> > > > > (c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not
> > > > > permitted)
> > > > > [Operation not permitted]
> > > > >
> > > > > Anyone ever expierienced such problem?
> > > > >
> > > > > In fact I am able to create/change/delete files in the brick
> > > path
> > > > > itself and the mount points (NFS/Fuse).
> > > > > Ovirt itself seems to test the volumes with /usr/bin/dd, but
> > > uses
> > > > > iflag=direct. Using this parameter I'm also unable to create
> > > any
> > > > > file -
> > > > > is there any gluster-option that changed recently? Or is it
> > > ZFS,
> > > > > that
> > > > > introduced something that blocks now?
> > > >
> > > > What version of Gluster are you on?
> > > >
> > > > My errors aren't quite the same as yours, but I have a similar
> > > setup
> > > >
> > > > Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux
> > > >
> > > > Updating from 3.7.11 -> 12 or 13 this last weekend I had these
> > > issues
> > > > pop up for me which while not exactly the same did result
> > > > in [Operation not permitted]
> > > >
> > > > [2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init]
> > > 0-
> > > > glusterfs-fuse: FUSE inited with protocol versions: glusterfs
> > > 7.22
> > > > kernel 7.22
> > > > [2016-07-09 15:27:49.555466] W [MSGID: 114031] [client-rpc-
> > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> > > > operation failed [Operation not permitted]
> > > > [2016-07-09 15:27:49.556574] W [MSGID: 114031] [client-rpc-
> > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> > > > operation failed [Operation not permitted]
> > > > [2016-07-09 15:27:49.556659] W [fuse-
> > > bridge.c:2227:fuse_readv_cbk] 0-
> > > > glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-
> > > > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)
> > > > [2016-07-09 15:27:59.612477] W [MSGID: 114031] [client-rpc-
> > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> > > > operation failed [Operation not permitted]
> > > > [2016-07-09 15:27:59.613700] W [MSGID: 114031] [client-rpc-
> > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> > > > operation failed [Operation not permitted]
> > > > [2016-07-09 15:27:59.613781] W [fuse-
> > > bridge.c:2227:fuse_readv_cbk] 0-
> > > > glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-
> > > > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted)
> > > >
> > > > Another gluster user suggested that in 3.7.12/13 aio support
> > > was
> > > > disabled and I may need to change disk caching method. So far
> > > I have
> > > > just stayed at 3.7.11 until I have time to get a better test
> > > setup.
> > > >
> > > > > Thanks in advance, Frank
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > _________________________________________________________________
> > > __
> > > > > ___________
> > > > > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> > > > > Sandhufe 2
> > > > > 18311 Ribnitz-Damgarten
> > > > >
> > > > > Telefon: 03821-700-0
> > > > > Fax: 03821-700-240
> > > > >
> > > > > E-Mail: info at bodden-kliniken.de Internet: http://www.bodden
> > > -klini
> > > > > ken.de
> > > > >
> > > > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919,
> > > Steuer-
> > > > > Nr.: 079/133/40188
> > > > > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer:
> > > Dr.
> > > > > Falko Milski
> > > > >
> > > > > Der Inhalt dieser E-Mail ist ausschließlich für den
> > > bezeichneten
> > > > > Adressaten bestimmt. Wenn Sie nicht der vorge-
> > > > > sehene Adressat dieser E-Mail oder dessen Vertreter sein
> > > sollten,
> > > > > beachten Sie bitte, dass jede Form der Veröf-
> > > > > fentlichung, Vervielfältigung oder Weitergabe des Inhalts
> > > dieser E-
> > > > > Mail unzulässig ist. Wir bitten Sie, sofort den
> > > > > Absender zu informieren und die E-Mail zu löschen.
> > > > >
> > > > >
> > > > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> > > > > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus
> > > ***
> > > > >
> > > > > _______________________________________________
> > > > > Gluster-users mailing list
> > > > > Gluster-users at gluster.org
> > > > > http://www.gluster.org/mailman/listinfo/gluster-users
> > > >
> > >
> > >
> > >
> > >
> > >
> > > _________________________________________________________________
> > > _____________
> > > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> > > Sandhufe 2
> > > 18311 Ribnitz-Damgarten
> > >
> > > Telefon: 03821-700-0
> > > Fax: 03821-700-240
> > >
> > > E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-kli
> > > niken.de
> > >
> > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919,
> > > Steuer-Nr.: 079/133/40188
> > > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr.
> > > Falko Milski
> > >
> > > Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten
> > > Adressaten bestimmt. Wenn Sie nicht der vorge-
> > > sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten,
> > > beachten Sie bitte, dass jede Form der Veröf-
> > > fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser
> > > E-Mail unzulässig ist. Wir bitten Sie, sofort den
> > > Absender zu informieren und die E-Mail zu löschen.
> > >
> > >
> > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> > > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
> > >
> > >
> >
> >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
______________________________________________________________________________
BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
Sandhufe 2
18311 Ribnitz-Damgarten
Telefon: 03821-700-0
Fax: 03821-700-240
E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-kliniken.de
Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 079/133/40188
Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski
Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorge-
sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröf-
fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail unzulässig ist. Wir bitten Sie, sofort den
Absender zu informieren und die E-Mail zu löschen.
Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
*** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
More information about the Gluster-users
mailing list