[Gluster-users] volume add-brick: failed: Pre Validation failed

Thu Mar 5 09:14:53 UTC 2020

Hi,

Thank you for your message.

Since I had no response within one day, I uninstalled glusterFS on all 
nodes and installed version 7.2

I re-created all volumes using the 4 servers in peer and now I rsync all 
the data from the "old" bricks to the new volumes. This avoids me also 
to do a re-balance.

I know this is not the way you normally should do but at least it works. 
And it is just a backup cluster, so users are not affected.

To respond to your questions:

- Yes, I did peer probe, as said in my first message, the output was 
"Peer in Cluster (Connected)" and the log file was posted.

- No, it is not a heterogeneous cluster. There are 4 identical machines 
with the same OS. Just the glusterFS version was different, since the 2 
new servers were added afterwards.

Anyway, I managed the issue a bit differently and now they are clustered.

Kind regards,
Birgit

On 05/03/20 09:01, Strahil Nikolov wrote:
> On March 5, 2020 7:52:48 AM GMT+02:00, Sanju Rakonde <srakonde at redhat.com> wrote:
>> Not sure whether I understood it correctly, are you using an
>> heterogeneous cluster? If yes, why?
>>
>> Are the new servers are in part of the cluster? You need to peer probe
>> the
>> servers before adding bricks. Copying the configurations of volume will
>> not
>> help.
>>
>> scp -r /var/lib/glusterd/vols/* root at diufnas22:/var/lib/glusterd/vols/
>> scp -r /var/lib/glusterd/vols/* root at diufnas23:/var/lib/glusterd/vols/
>>
>> Please paste the peer status output and glusterd logs from peers around
>> the
>> time when add-brick is performed.
>>
>> On Tue, Mar 3, 2020 at 2:56 PM DUCARROZ Birgit
>> <birgit.ducarroz at unifr.ch>
>> wrote:
>>
>>>
>>> Just some additional info:
>>>
>>> On diufnas26 and diufnas27:
>>> glusterfs 4.1.8
>>>
>>> On diufnas22 and diufnas23:
>>> glusterfs 7.2
>>>
>>>
>>> setfattr -x --> does not help
>>>
>>> On 03/03/20 10:01, DUCARROZ Birgit wrote:
>>>> Hi,
>>>>
>>>> Having 2 servers diufnas26 and diufnas27 with 3 volumes
>>> (Type=Distribute):
>>>> vol-backup
>>>> vol-restore
>>>> vol-veeam
>>>>
>>>> diufnas26
>>>> /bigdisk/brick1
>>>>
>>>> diufnas27
>>>> /bigdisk/brick2
>>>>
>>>>
>>>> I would like to add 2 new servers: diufnas22 and diufnas23.
>>>>
>>>> diufnas22
>>>> /bigdisk/brick3
>>>>
>>>> diufnas23
>>>> /bigdisk/brick4
>>>>
>>>> gluster peer status on all servers shows State: Peer in Cluster
>>> (Connected)
>>>>
>>>> Wanna add the 2 new bricks:
>>>>
>>>> root at diufnas26:/bigdisk/brick1# gluster volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force
>>>> *volume add-brick: failed: Pre Validation failed on diufnas23.
>> Please
>>>> check log file for details.
>>>> Pre Validation failed on diufnas22. Please check log file for
>> details.*
>>>>
>>>>
>>>>
>>>> I tried this, but it does not help:
>>>>    scp -r /var/lib/glusterd/vols/*
>> root at diufnas22:/var/lib/glusterd/vols/
>>>>    scp -r /var/lib/glusterd/vols/*
>> root at diufnas23:/var/lib/glusterd/vols/
>>>>
>>>>
>>>> root at diufnas26:/bigdisk/brick1# tail /var/log/glusterfs/*log -f |
>> grep E
>>>>
>>>> [2020-03-03 08:37:52.544881]  : volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force : FAILED
>> : Pre
>>>> Validation failed on diufnas22. Please check log file for details.
>>>> [2020-03-03 08:39:35.991559]  : volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force : FAILED
>> : Pre
>>>> Validation failed on diufnas23. Please check log file for details.
>>>> [2020-03-03 08:42:40.800060]  : volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force : FAILED
>> : Pre
>>>> Validation failed on diufnas23. Please check log file for details.
>>>> [2020-03-03 08:50:15.699112]  : volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force : FAILED
>> : Pre
>>>> Validation failed on diufnas22. Please check log file for details.
>>>> [2020-03-03 08:54:56.472727]  : volume add-brick vol-backup
>>>> diufnas22:/bigdisk/brick3 diufnas23:/bigdisk/brick4 force : FAILED
>> : Pre
>>>> Validation failed on diufnas23. Please check log file for details.
>>>> [2020-03-03 08:50:15.698286] E [MSGID: 106121]
>>>> [glusterd-mgmt.c:1148:glusterd_mgmt_v3_pre_validate] 0-management:
>> Pre
>>>> Validation failed on peers
>>>> [2020-03-03 08:50:15.698315] E [MSGID: 106121]
>>>> [glusterd-mgmt.c:2242:glusterd_mgmt_v3_initiate_all_phases]
>>>> 0-management: Pre Validation Failed
>>>> [2020-03-03 08:54:56.471628] E [MSGID: 106115]
>>>> [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Pre
>>>> Validation failed on diufnas23. Please check log file for details.
>>>> [2020-03-03 08:54:56.471687] E [MSGID: 106115]
>>>> [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Pre
>>>> Validation failed on diufnas22. Please check log file for details.
>>>> [2020-03-03 08:54:56.471908] E [MSGID: 106121]
>>>> [glusterd-mgmt.c:1148:glusterd_mgmt_v3_pre_validate] 0-management:
>> Pre
>>>> Validation failed on peers
>>>> [2020-03-03 08:54:56.471935] E [MSGID: 106121]
>>>> [glusterd-mgmt.c:2242:glusterd_mgmt_v3_initiate_all_phases]
>>>> 0-management: Pre Validation Failed
>>>>
>>>>
>>>> What am I missing?
>>>> Thank you for any help.
>>>> Regards, Birgit
>>>>
>>>> ________
>>>>
>>>>
>>>>
>>>> Community Meeting Calendar:
>>>>
>>>> Schedule -
>>>> Every Tuesday at 14:30 IST / 09:00 UTC
>>>> Bridge: https://bluejeans.com/441850968
>>>>
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>> ________
>>>
>>>
>>>
>>> Community Meeting Calendar:
>>>
>>> Schedule -
>>> Every Tuesday at 14:30 IST / 09:00 UTC
>>> Bridge: https://bluejeans.com/441850968
>>>
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>
> 
> Mixing major versions is always a recipe for a disaster.
> 
> Consider upgrading the v4.X nodes as soon as possible.
> 
> Best Regards,
> Strahil Nikolov
> 