[Gluster-users] Wrong volume size for distributed dispersed volume on 4.1.5

[Nithya Balachandran]

Hi,



On 16 October 2018 at 18:20, <jring at mail.de> wrote:

> Hi everybody,
>
> I have created a distributed dispersed volume on 4.1.5 under centos7 like
> this a few days ago:
>
> gluster volume create data_vol1 disperse-data 4 redundancy 2 transport tcp
> \
> \
> gf-p-d-01.isec.foobar.com:/bricks/brick1/brick \
> gf-p-d-03.isec.foobar.com:/bricks/brick1/brick \
> gf-p-d-04.isec.foobar.com:/bricks/brick1/brick \
> gf-p-k-01.isec.foobar.com:/bricks/brick1/brick \
> gf-p-k-03.isec.foobar.com:/bricks/brick1/brick \
> gf-p-k-04.isec.foobar.com:/bricks/brick1/brick \
> \
> gf-p-d-01.isec.foobar.com:/bricks/brick2/brick \
> gf-p-d-03.isec.foobar.com:/bricks/brick2/brick \
> gf-p-d-04.isec.foobar.com:/bricks/brick2/brick \
> gf-p-k-01.isec.foobar.com:/bricks/brick2/brick \
> gf-p-k-03.isec.foobar.com:/bricks/brick2/brick \
> gf-p-k-04.isec.foobar.com:/bricks/brick2/brick \
> \
> ... same for brick3 to brick9...
> \
> gf-p-d-01.isec.foobar.com:/bricks/brick10/brick \
> gf-p-d-03.isec.foobar.com:/bricks/brick10/brick \
> gf-p-d-04.isec.foobar.com:/bricks/brick10/brick \
> gf-p-k-01.isec.foobar.com:/bricks/brick10/brick \
> gf-p-k-03.isec.foobar.com:/bricks/brick10/brick \
> gf-p-k-04.isec.foobar.com:/bricks/brick10/brick
>
> This worked nicely and resulted in the following filesystem:
> [root at gf-p-d-01 ~]# df -h /data/
> Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
> gf-p-d-01.isec.foobar.com:/data_vol1 219T 2,2T 217T 2% /data
>
> Each of the bricks resides on its own 6TB disk with 1 big partition
> formated with xfs.
>
> Yesterday a colleague looked at the filesystem and found some space
> missing...
> [root at gf-p-d-01 ~]# df -h /data/
> Filesystem Size Used Avail Use% Mounted on
> gf-p-d-01.isec.foobar.com:/data_vol1 22T 272G 22T 2% /data
>
> Some googling brought the following bug report against 3.4 which looks
> familiar:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1541830
>
> So we did a quick grep shared-brick-count /var/lib/glusterd/vols/data_vol1/*
> on all boxes and found that on 5 out of 6 boxes this was
> shared-brick-count=0 for all bricks on remote boxes and 1 for local bricks.
>
> Is this the expected result or should we have all 1 everywhere (as the
> quick fix script from the case sets it)?
>

No , this is fine. The shared-brick-count only needs to be 1 for the local
bricks. The value for the remote bricks can be 0.


>
> Also on one box (the one where we created the volume from, btw) we have
> shared-brick-count=0 for all remote bricks and 10 for the local bricks.
>

This is a problem. The shared-brick-count should be 1 for the local bricks
here as well.


> Is it possible that the bug from 3.4 still exists in 4.1.5 and should we
> try the filter script which sets shared-brick-count=1 for all bricks?
>
>
Can you try
1. restarting glusterd on all the nodes one after another (not at the same
time)
2. Setting a volume option (say gluster volume set <volname>
cluster.min-free-disk 11%)

and see if it fixes the issue?

Regards,
Nithya



> The volume is not currently in production so now would be the time to play
> around and find the problem...
>
> TIA and regards,
>
> Joachim
>
>
> ------------------------------------------------------------
> -------------------------------------
> FreeMail powered by mail.de - MEHR SICHERHEIT, SERIOSITÄT UND KOMFORT
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20181016/3edfeddd/attachment.html>