[Gluster-users] [Gluster-devel] 3.7.13 & proxmox/qemu

Krutika Dhananjay kdhananj at redhat.com
Wed Aug 3 04:26:50 UTC 2016


Glad the fixes worked for you. Thanks for that update!

-Krutika

On Tue, Aug 2, 2016 at 7:31 PM, David Gossage <dgossage at carouselchecks.com>
wrote:

> So far both dd commands that failed previously worked fine on 3.7.14
>
> Once I deleted old content from test volume it mounted to oVirt via
> storage add when previously it would error out.  I am now creating a test
> VM with default disk caching settings (pretty sure oVirt is defaulting to
> none rather than writeback/through).  So far all shards are being created
> properly.
>
> Load is sky rocketing but I have all 3 gluster bricks running off 1 hard
> drive on test box so I would expect horrible io/load issues with that.
>
> Very promising so far.  Thank you developers for your help in working
> through this.
>
> Once I have the VM installed and running will test for a few days and make
> sure it doesn't have any freeze or locking issues then will roll this out
> to working cluster.
>
>
>
> *David Gossage*
> *Carousel Checks Inc. | System Administrator*
> *Office* 708.613.2284
>
> On Wed, Jul 27, 2016 at 8:37 AM, David Gossage <
> dgossage at carouselchecks.com> wrote:
>
>> On Tue, Jul 26, 2016 at 9:38 PM, Krutika Dhananjay <kdhananj at redhat.com>
>> wrote:
>>
>>> Yes please, could you file a bug against glusterfs for this issue?
>>>
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=1360785
>>
>>
>>>
>>>
>>> -Krutika
>>>
>>> On Wed, Jul 27, 2016 at 1:39 AM, David Gossage <
>>> dgossage at carouselchecks.com> wrote:
>>>
>>>> Has a bug report been filed for this issue or should l I create one
>>>> with the logs and results provided so far?
>>>>
>>>> *David Gossage*
>>>> *Carousel Checks Inc. | System Administrator*
>>>> *Office* 708.613.2284
>>>>
>>>> On Fri, Jul 22, 2016 at 12:53 PM, David Gossage <
>>>> dgossage at carouselchecks.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jul 22, 2016 at 9:37 AM, Vijay Bellur <vbellur at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> On Fri, Jul 22, 2016 at 10:03 AM, Samuli Heinonen <
>>>>>> samppah at neutraali.net> wrote:
>>>>>> > Here is a quick way how to test this:
>>>>>> > GlusterFS 3.7.13 volume with default settings with brick on ZFS
>>>>>> dataset. gluster-test1 is server and gluster-test2 is client mounting with
>>>>>> FUSE.
>>>>>> >
>>>>>> > Writing file with oflag=direct is not ok:
>>>>>> > [root at gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct
>>>>>> count=1 bs=1024000
>>>>>> > dd: failed to open ‘file’: Invalid argument
>>>>>> >
>>>>>> > Enable network.remote-dio on Gluster Volume:
>>>>>> > [root at gluster-test1 gluster]# gluster volume set gluster
>>>>>> network.remote-dio enable
>>>>>> > volume set: success
>>>>>> >
>>>>>> > Writing small file with oflag=direct is ok:
>>>>>> > [root at gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct
>>>>>> count=1 bs=1024000
>>>>>> > 1+0 records in
>>>>>> > 1+0 records out
>>>>>> > 1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s
>>>>>> >
>>>>>> > Writing bigger file with oflag=direct is ok:
>>>>>> > [root at gluster-test2 gluster]# dd if=/dev/zero of=file3
>>>>>> oflag=direct count=100 bs=1M
>>>>>> > 100+0 records in
>>>>>> > 100+0 records out
>>>>>> > 104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s
>>>>>> >
>>>>>> > Enable Sharding on Gluster Volume:
>>>>>> > [root at gluster-test1 gluster]# gluster volume set gluster
>>>>>> features.shard enable
>>>>>> > volume set: success
>>>>>> >
>>>>>> > Writing small file  with oflag=direct is ok:
>>>>>> > [root at gluster-test2 gluster]# dd if=/dev/zero of=file3
>>>>>> oflag=direct count=1 bs=1M
>>>>>> > 1+0 records in
>>>>>> > 1+0 records out
>>>>>> > 1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s
>>>>>> >
>>>>>> > Writing bigger file with oflag=direct is not ok:
>>>>>> > [root at gluster-test2 gluster]# dd if=/dev/zero of=file3
>>>>>> oflag=direct count=100 bs=1M
>>>>>> > dd: error writing ‘file3’: Operation not permitted
>>>>>> > dd: closing output file ‘file3’: Operation not permitted
>>>>>> >
>>>>>>
>>>>>>
>>>>>> Thank you for these tests! would it be possible to share the brick and
>>>>>> client logs?
>>>>>>
>>>>>
>>>>> Not sure if his tests are same as my setup but here is what I end up
>>>>> with
>>>>>
>>>>> Volume Name: glustershard
>>>>> Type: Replicate
>>>>> Volume ID: 0cc4efb6-3836-4caa-b24a-b3afb6e407c3
>>>>> Status: Started
>>>>> Number of Bricks: 1 x 3 = 3
>>>>> Transport-type: tcp
>>>>> Bricks:
>>>>> Brick1: 192.168.71.10:/gluster1/shard1/1
>>>>> Brick2: 192.168.71.11:/gluster1/shard2/1
>>>>> Brick3: 192.168.71.12:/gluster1/shard3/1
>>>>> Options Reconfigured:
>>>>> features.shard-block-size: 64MB
>>>>> features.shard: on
>>>>> server.allow-insecure: on
>>>>> storage.owner-uid: 36
>>>>> storage.owner-gid: 36
>>>>> cluster.server-quorum-type: server
>>>>> cluster.quorum-type: auto
>>>>> network.remote-dio: enable
>>>>> cluster.eager-lock: enable
>>>>> performance.stat-prefetch: off
>>>>> performance.io-cache: off
>>>>> performance.quick-read: off
>>>>> cluster.self-heal-window-size: 1024
>>>>> cluster.background-self-heal-count: 16
>>>>> nfs.enable-ino32: off
>>>>> nfs.addr-namelookup: off
>>>>> nfs.disable: on
>>>>> performance.read-ahead: off
>>>>> performance.readdir-ahead: on
>>>>>
>>>>>
>>>>>
>>>>>  dd if=/dev/zero of=/rhev/data-center/mnt/glusterSD/192.168.71.11\:_glustershard/
>>>>> oflag=direct count=100 bs=1M
>>>>> 81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/ __DIRECT_IO_TEST__
>>>>>      .trashcan/
>>>>> [root at ccengine2 ~]# dd if=/dev/zero
>>>>> of=/rhev/data-center/mnt/glusterSD/192.168.71.11\:_glustershard/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test
>>>>> oflag=direct count=100 bs=1M
>>>>> dd: error writing ‘/rhev/data-center/mnt/glusterSD/192.168.71.11:_glustershard/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test’:
>>>>> Operation not permitted
>>>>>
>>>>> creates the 64M file in expected location then the shard is 0
>>>>>
>>>>> # file:
>>>>> gluster1/shard1/1/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test
>>>>>
>>>>> security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
>>>>> trusted.afr.dirty=0x000000000000000000000000
>>>>> trusted.bit-rot.version=0x0200000000000000579231f3000e16e7
>>>>> trusted.gfid=0xec6de302b35f427985639ca3e25d9df0
>>>>> trusted.glusterfs.shard.block-size=0x0000000004000000
>>>>>
>>>>> trusted.glusterfs.shard.file-size=0x0000000004000000000000000000000000000000000000010000000000000000
>>>>>
>>>>>
>>>>> # file: gluster1/shard1/1/.shard/ec6de302-b35f-4279-8563-9ca3e25d9df0.1
>>>>>
>>>>> security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
>>>>> trusted.afr.dirty=0x000000000000000000000000
>>>>> trusted.gfid=0x2bfd3cc8a727489b9a0474241548fe80
>>>>>
>>>>>
>>>>>> Regards,
>>>>>> Vijay
>>>>>> _______________________________________________
>>>>>> Gluster-users mailing list
>>>>>> Gluster-users at gluster.org
>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160803/67ea8c95/attachment.html>


More information about the Gluster-users mailing list