[Gluster-users] Can't write to volume using vim/nano

[Jim Kinney]

I have rdma capability. Will test and report back. I'm still on v 3.12.

On January 24, 2019 12:54:26 AM EST, Amar Tumballi Suryanarayan <atumball at redhat.com> wrote:
>I suspect this is a bug with 'Transport: rdma' part. We have called out
>for
>de-scoping that feature as we are lacking experts in that domain right
>now.
>Recommend you to use IPoIB option, and use tcp/socket transport type
>(which
>is default). That should mostly fix all the issues.
>
>-Amar
>
>On Thu, Jan 24, 2019 at 5:31 AM Jim Kinney <jim.kinney at gmail.com>
>wrote:
>
>> That really sounds like a bug with the sharding. I'm not using
>sharding on
>> my setup and files are writeable (vim) with 2 bytes and no errors
>occur.
>> Perhaps the small size is cached until it's large enough to trigger a
>write
>>
>> On Wed, 2019-01-23 at 21:46 -0200, Lindolfo Meira wrote:
>>
>> Also I noticed that any subsequent write (after the first write with
>340
>>
>> bytes or more), regardless the size, will work as expected.
>>
>>
>>
>> Lindolfo Meira, MSc
>>
>> Diretor Geral, Centro Nacional de Supercomputação
>>
>> Universidade Federal do Rio Grande do Sul
>>
>> +55 (51) 3308-3139
>>
>>
>> On Wed, 23 Jan 2019, Lindolfo Meira wrote:
>>
>>
>> Just checked: when the write is >= 340 bytes, everything works as
>>
>> supposed. If the write is smaller, the error takes place. And when it
>>
>> does, nothing is logged on the server. The client, however, logs the
>>
>> following:
>>
>>
>> [2019-01-23 23:28:54.554664] W [MSGID: 103046]
>>
>> [rdma.c:3502:gf_rdma_decode_header] 0-rpc-transport/rdma: received a
>msg
>>
>> of type RDMA_ERROR
>>
>>
>> [2019-01-23 23:28:54.554728] W [MSGID: 103046]
>>
>> [rdma.c:3939:gf_rdma_process_recv] 0-rpc-transport/rdma: peer
>>
>> (172.24.1.6:49152), couldn't encode or decode the msg properly or
>write
>>
>> chunks were not provided for replies that were bigger than
>>
>> RDMA_INLINE_THRESHOLD (2048)
>>
>>
>> [2019-01-23 23:28:54.554765] W [MSGID: 114031]
>>
>> [client-rpc-fops_v2.c:680:client4_0_writev_cbk] 0-gfs-client-5:
>remote
>>
>> operation failed [Transport endpoint is not connected]
>>
>>
>> [2019-01-23 23:28:54.554850] W [fuse-bridge.c:1436:fuse_err_cbk]
>>
>> 0-glusterfs-fuse: 1723199: FLUSH() ERR => -1 (Transport endpoint is
>not
>>
>> connected)
>>
>>
>>
>>
>> Lindolfo Meira, MSc
>>
>> Diretor Geral, Centro Nacional de Supercomputação
>>
>> Universidade Federal do Rio Grande do Sul
>>
>> +55 (51) 3308-3139
>>
>>
>> On Wed, 23 Jan 2019, Lindolfo Meira wrote:
>>
>>
>> Hi Jim. Thanks for taking the time.
>>
>>
>> Sorry I didn't express myself properly. It's not a simple matter of
>>
>> permissions. Users can write to the volume alright. It's when vim and
>nano
>>
>> are used, or when small file writes are performed (by cat or echo),
>that
>>
>> it doesn't work. The file is updated with the write in the server,
>but it
>>
>> shows up as empty in the client.
>>
>>
>> I guess it has something to do with the size of the write, because I
>ran a
>>
>> test writing to a file one byte at a time, and it never showed up as
>>
>> having any content in the client (although in the server it kept
>growing
>>
>> accordingly).
>>
>>
>> I should point out that I'm using a sharded volume. But when I was
>testing
>>
>> a striped volume, it also happened. Output of "gluster volume info"
>>
>> follows bellow:
>>
>>
>> Volume Name: gfs
>>
>> Type: Distribute
>>
>> Volume ID: b5ef065f-1ba2-481f-8108-e8f6d2d3f036
>>
>> Status: Started
>>
>> Snapshot Count: 0
>>
>> Number of Bricks: 6
>>
>> Transport-type: rdma
>>
>> Bricks:
>>
>> Brick1: pfs01-ib:/mnt/data
>>
>> Brick2: pfs02-ib:/mnt/data
>>
>> Brick3: pfs03-ib:/mnt/data
>>
>> Brick4: pfs04-ib:/mnt/data
>>
>> Brick5: pfs05-ib:/mnt/data
>>
>> Brick6: pfs06-ib:/mnt/data
>>
>> Options Reconfigured:
>>
>> nfs.disable: on
>>
>> features.shard: on
>>
>>
>>
>>
>> Lindolfo Meira, MSc
>>
>> Diretor Geral, Centro Nacional de Supercomputação
>>
>> Universidade Federal do Rio Grande do Sul
>>
>> +55 (51) 3308-3139
>>
>>
>> On Wed, 23 Jan 2019, Jim Kinney wrote:
>>
>>
>> Check permissions on the mount. I have multiple dozens of systems
>>
>> mounting 18 "exports" using fuse and it works for multiple user
>>
>> read/write based on user access permissions to the mount point space.
>>
>> /home is mounted for 150+ users plus another dozen+ lab storage
>spaces.
>>
>> I do manage user access with freeIPA across all systems to keep
>things
>>
>> consistent.
>>
>> On Wed, 2019-01-23 at 19:31 -0200, Lindolfo Meira wrote:
>>
>> Am I missing something here? A mere write operation, using vim or
>>
>> nano, cannot be performed on a gluster volume mounted over fuse! What
>>
>> gives?
>>
>> Lindolfo Meira, MScDiretor Geral, Centro Nacional de
>>
>> SupercomputaçãoUniversidade Federal do Rio Grande do Sul+55 (51)
>>
>> 3308-3139_______________________________________________Gluster-users
>>
>> mailing
>>
>> listGluster-users at gluster.org
>>
>>
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>>
>> --
>>
>> James P. Kinney III
>>
>>
>> Every time you stop a school, you will have to build a jail. What you
>>
>> gain at one end you lose at the other. It's like feeding a dog on his
>>
>> own tail. It won't fatten the dog.
>>
>> - Speech 11/23/1900 Mark Twain
>>
>>
>> http://heretothereideas.blogspot.com/
>>
>>
>>
>> --
>>
>> James P. Kinney III Every time you stop a school, you will have to
>build a
>> jail. What you gain at one end you lose at the other. It's like
>feeding a
>> dog on his own tail. It won't fatten the dog. - Speech 11/23/1900
>Mark
>> Twain http://heretothereideas.blogspot.com/
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>
>
>
>-- 
>Amar Tumballi (amarts)

-- 
Sent from my Android device with K-9 Mail. All tyopes are thumb related and reflect authenticity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190124/802211af/attachment.html>