[Gluster-users] The dht-layout interval is missing

[Susant Palai]

On Fri, May 29, 2020 at 1:28 PM jifeng-call <17607319886 at 163.com> wrote:

> Hi All,
> I have 6 servers that form a glusterfs 2x3 distributed replication volume,
> the details are as follows:
>
> [root at node1 ~]# gluster volume info
> Volume Name: ksvd_vol
> Type: Distributed-Replicate
> Volume ID: c9848daa-b06f-4f82-a2f8-1b425b8e869c
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 2 x 3 = 6
> Transport-type: tcp
> Bricks:
> Brick1: node1:/export/kylin/ksvd
> Brick2: node2:/export/kylin/ksvd
> Brick3: node3:/export/kylin/ksvd
> Brick4: node4:/export/kylin/ksvd
> Brick5: node5:/export/kylin/ksvd
> Brick6: node6:/export/kylin/ksvd
> Options Reconfigured:
> diagnostics.client-log-level: DEBUG
> diagnostics.brick-log-level: DEBUG
> performance.client-io-threads: off
> nfs.disable: on
> transport.address-family: inet
> cluster.quorum-type: auto
>
> Restart 6 servers at a time, the server automatically mounts the ksvd_vol
> volume to the / home / kylin-ksvd directory during the startup process：
>
> [root at node1 ~]# df -h | grep ksvd_vol
> node1:/ksvd_vol            4.5T  362G  4.1T    9% /home/kylin-ksvd
>
> Failed to create a file in the glusterfs mount directory of node3, the log
> shows the error as follows (the files created by other servers are normal)：
>
> Possibly the layout setxattr failed which should not have happened given
it is a replicated volume. You can check log on client side and brick side
for the parent directory for more information.
Post remount, a fresh triggers which heals the layout missing before and
create goes in.


> [2020-05-29 05:55:03.065656] E [MSGID: 109011]
> [dht-common.c:8683:dht_create] 0-ksvd_vol-dht: no subvolume in layout for
> path=/.ksvd.time.405fa8a6-575d-4474-a97f-a107cbf1c673.18512
> [2020-05-29 05:55:03.065719] W [fuse-bridge.c:2122:fuse_create_cbk]
> 0-glusterfs-fuse: 2454790:
> /.ksvd.time.405fa8a6-575d-4474-a97f-a107cbf1c673.18512 => -1 (输入/输出错误)
> [2020-05-29 05:55:03.680303] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> [2020-05-29 05:55:04.687456] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 491618322
> [2020-05-29 05:55:04.688612] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> [2020-05-29 05:55:05.694446] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> [2020-05-29 05:55:05.830555] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 491618322
> [2020-05-29 05:55:06.700423] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> [2020-05-29 05:55:07.706536] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> [2020-05-29 05:55:07.833049] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 491618322
> [2020-05-29 05:55:08.712128] W [MSGID: 109011]
> [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash
> (value) = 1738969696
> The message "W [MSGID: 109011] [dht-layout.c:186:dht_layout_search]
> 0-ksvd_vol-dht: no subvolume for hash (value) = 1738969696" repeated 2
> times between [2020-05-29 05:55:08.712128] and [2020-05-29 05:55:09.718541]
>
> The pathinfo information in the / home / kylin-ksvd directory is displayed
> as follows:
>
> [root at node3 kylin-ksvd]# getfattr -d -m .  -n trusted.glusterfs.pathinfo
> /home/kylin-ksvd/
> getfattr: Removing leading '/' from absolute path names
> # file: home/kylin-ksvd/
> trusted.glusterfs.pathinfo="((<DISTRIBUTE:ksvd_vol-dht>
> (<REPLICATE:ksvd_vol-replicate-0>
> <POSIX(/export/kylin/ksvd):node2:/export/kylin/ksvd/>
> <POSIX(/export/kylin/ksvd):node3:/export/kylin/ksvd/>
> <POSIX(/export/kylin/ksvd):node1:/export/kylin/ksvd/>)
> (<REPLICATE:ksvd_vol-replicate-1>
> <POSIX(/export/kylin/ksvd):node5:/export/kylin/ksvd/>
> <POSIX(/export/kylin/ksvd):node6:/export/kylin/ksvd/>
> <POSIX(/export/kylin/ksvd):node4:/export/kylin/ksvd/>))
> (ksvd_vol-dht-layout (ksvd_vol-replicate-0 0 0) (ksvd_vol-replicate-1
> 3539976838 4294967295)))"
>
> It can be seen from the above information that ksvd_vol-dht-layout is
> missing the interval 0 to 3539976837
>
> After umount, remounting returned to normal ... What is the reason?
>
> Best regards
>
>
>
>
>
>
> ________
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
>
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200529/1535ca4e/attachment.html>