[Gluster-devel] [Gluster-users] Fwd: dht_is_subvol_filled messages on client

David Gossage dgossage at carouselchecks.com
Thu May 5 09:06:35 UTC 2016 

On Thu, May 5, 2016 at 3:28 AM, Serkan Çoban <cobanserkan at gmail.com> wrote:

> Hi,
>
> You can find the output below link:
> https://www.dropbox.com/s/wzrh5yp494ogksc/status_detail.txt?dl=0
>
> Thanks,
> Serkan
>

Maybe not issue, but playing one of these things is not like the other I
notice of all the bricks only one seems to be different at a quick glance

Brick                : Brick 1.1.1.235:/bricks/20
TCP Port             : 49170
RDMA Port            : 0
Online               : Y
Pid                  : 26736
File System          : ext4
Device               : /dev/mapper/vol0-vol_root
Mount Options        : rw,relatime,data=ordered
Inode Size           : 256
Disk Space Free      : 86.1GB
Total Disk Space     : 96.0GB
Inode Count          : 6406144
Free Inodes          : 6381374

Every other brick seems to be 7TB and xfs but this one.




>
> On Thu, May 5, 2016 at 9:33 AM, Xavier Hernandez <xhernandez at datalab.es>
> wrote:
> > Can you post the result of 'gluster volume status v0 detail' ?
> >
> >
> > On 05/05/16 06:49, Serkan Çoban wrote:
> >>
> >> Hi, Can anyone suggest something for this issue? df, du has no issue
> >> for the bricks yet one subvolume not being used by gluster..
> >>
> >> On Wed, May 4, 2016 at 4:40 PM, Serkan Çoban <cobanserkan at gmail.com>
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I changed cluster.min-free-inodes to "0". Remount the volume on
> >>> clients. inode full messages not coming to syslog anymore but I see
> >>> disperse-56 subvolume still not being used.
> >>> Anything I can do to resolve this issue? Maybe I can destroy and
> >>> recreate the volume but I am not sure It will fix this issue...
> >>> Maybe the disperse size 16+4 is too big should I change it to 8+2?
> >>>
> >>> On Tue, May 3, 2016 at 2:36 PM, Serkan Çoban <cobanserkan at gmail.com>
> >>> wrote:
> >>>>
> >>>> I also checked the df output all 20 bricks are same like below:
> >>>> /dev/sdu1 7.3T 34M 7.3T 1% /bricks/20
> >>>>
> >>>> On Tue, May 3, 2016 at 1:40 PM, Raghavendra G <
> raghavendra at gluster.com>
> >>>> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, May 2, 2016 at 11:41 AM, Serkan Çoban <cobanserkan at gmail.com
> >
> >>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>>> 1. What is the out put of du -hs <back-end-export>? Please get this
> >>>>>>> information for each of the brick that are part of disperse.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Sorry. I needed df output of the filesystem containing brick. Not du.
> >>>>> Sorry
> >>>>> about that.
> >>>>>
> >>>>>>
> >>>>>> There are 20 bricks in disperse-56 and the du -hs output is like:
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 1.8M /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>> 80K /bricks/20
> >>>>>>
> >>>>>> I see that gluster is not writing to this disperse set. All other
> >>>>>> disperse sets are filled 13GB but this one is empty. I see directory
> >>>>>> structure created but no files in directories.
> >>>>>> How can I fix the issue? I will try to rebalance but I don't think
> it
> >>>>>> will write to this disperse set...
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Sat, Apr 30, 2016 at 9:22 AM, Raghavendra G
> >>>>>> <raghavendra at gluster.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Apr 29, 2016 at 12:32 AM, Serkan Çoban
> >>>>>>> <cobanserkan at gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Hi, I cannot get an answer from user list, so asking to devel
> list.
> >>>>>>>>
> >>>>>>>> I am getting [dht-diskusage.c:277:dht_is_subvol_filled] 0-v0-dht:
> >>>>>>>> inodes on subvolume 'v0-disperse-56' are at (100.00 %), consider
> >>>>>>>> adding more bricks.
> >>>>>>>>
> >>>>>>>> message on client logs.My cluster is empty there are only a couple
> >>>>>>>> of
> >>>>>>>> GB files for testing. Why this message appear in syslog?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> dht uses disk usage information from backend export.
> >>>>>>>
> >>>>>>> 1. What is the out put of du -hs <back-end-export>? Please get this
> >>>>>>> information for each of the brick that are part of disperse.
> >>>>>>> 2. Once you get du information from each brick, the value seen by
> dht
> >>>>>>> will
> >>>>>>> be based on how cluster/disperse aggregates du info (basically
> statfs
> >>>>>>> fop).
> >>>>>>>
> >>>>>>> The reason for 100% disk usage may be,
> >>>>>>> In case of 1, backend fs might be shared by data other than brick.
> >>>>>>> In case of 2, some issues with aggregation.
> >>>>>>>
> >>>>>>>> Is is safe to
> >>>>>>>> ignore it?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> dht will try not to have data files on the subvol in question
> >>>>>>> (v0-disperse-56). Hence lookup cost will be two hops for files
> >>>>>>> hashing
> >>>>>>> to
> >>>>>>> disperse-56 (note that other fops like read/write/open still have
> the
> >>>>>>> cost
> >>>>>>> of single hop and dont suffer from this penalty). Other than that
> >>>>>>> there
> >>>>>>> is
> >>>>>>> no significant harm unless disperse-56 is really running out of
> >>>>>>> space.
> >>>>>>>
> >>>>>>> regards,
> >>>>>>> Raghavendra
> >>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> Gluster-devel mailing list
> >>>>>>>> Gluster-devel at gluster.org
> >>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Raghavendra G
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Gluster-devel mailing list
> >>>>>> Gluster-devel at gluster.org
> >>>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Raghavendra G
> >>
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-users
> >>
> >
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160505/0c4638d5/attachment.html>

More information about the Gluster-devel mailing list