[Gluster-users] volume process does not start - glusterfs is happy with it?
Strahil Nikolov
hunter86_bg at yahoo.com
Wed Jul 1 18:33:27 UTC 2020
Sometimes the brick comes up slower than glusterd service (which starts the brick processes).
The problem is that if you leave glusterd to depend on both bricks and a brick fails (for example FS problem) then the other brick will not come up too.
After a system crash, the VDO service replay log was taking too much and the glusterd failed (as bricks were not ready yet), so I just created an override like this one:
# /etc/systemd/system/glusterd.service.d/01-dependencies.conf
[Unit]
[root at ovirt1 ~]# cat /etc/systemd/system/glusterd.service.d/01-dependencies.conf
[Unit]
Description=GlusterFS, a clustered file-system server
After=network.target rpcbind.service gluster_bricks-engine.mount gluster_bricks-data.mount gluster_bricks-fast1.mount gluster_bricks-fast2.mount gluster_bricks-fast3.mount gluster_bricks-fast4.mount
Before=network-online.target
I have created systemd mount units, due to VDO , but most probably the local-fs.target will generate the mount units for you from the fstab.
Best Regards,
Strahil Nikolov
На 1 юли 2020 г. 20:57:22 GMT+03:00, "Felix Kölzow" <felix.koelzow at gmx.de> написа:
>Hey,
>
>
>what about the device mapper? Everything was mount properly during
>reboot?
>
>It happens to me if the lvm device mapper got a timeout during the
>reboot
>
>process while mounting the brick itself.
>
>
>Regards,
>
>Felix
>
>On 01/07/2020 16:46, lejeczek wrote:
>>
>> On 30/06/2020 11:31, Barak Sason Rofman wrote:
>>> Greetings,
>>>
>>> I'm not sure if that's directly related to your problem,
>>> but on a general level, AFAIK, replica-2 vols are not
>>> recommended due to split brain possibility:
>>>
>https://docs.gluster.org/en/latest/Administrator%20Guide/Split%20brain%20and%20ways%20to%20deal%20with%20it/
>>>
>>> It's recommended to either use replica-3 or arbiter Arbiter.
>>>
>>> Regards,
>>>
>>> On Tue, Jun 30, 2020 at 1:14 PM lejeczek
>>> <peljasz at yahoo.co.uk <mailto:peljasz at yahoo.co.uk>> wrote:
>>>
>>> Hi everybody.
>>>
>>> I have two peers in the cluster and a 2-replica volume
>>> which seems okey if it was not for one weird bit -
>>> when a peer reboots then on that peer after a reboot I
>>> see:
>>>
>>> $ gluster volume status USERs
>>> Status of volume: USERs
>>> Gluster process TCP Port
>>> RDMA Port Online Pid
>>>
>------------------------------------------------------------------------------
>>> Brick swir.direct:/00.STORAGE/2/0-GLUSTER-U
>>> SERs N/A
>>> N/A N N/A
>>> Brick dzien.direct:/00.STORAGE/2/0-GLUSTER-
>>> USERs 49152
>>> 0 Y 57338
>>> Self-heal Daemon on localhost N/A
>>> N/A Y 4302
>>> Self-heal Daemon on dzien.direct N/A
>>> N/A Y 57359
>>>
>>> Task Status of Volume USERs
>>>
>------------------------------------------------------------------------------
>>> There are no active volume tasks
>>>
>>> I do not suppose it's expected.
>>> On such rebooted node I see:
>>> $ systemctl status -l glusterd
>>> ● glusterd.service - GlusterFS, a clustered
>>> file-system server
>>> Loaded: loaded
>>> (/usr/lib/systemd/system/glusterd.service; enabled;
>>> vendor preset: enabled)
>>> Drop-In: /etc/systemd/system/glusterd.service.d
>>> └─override.conf
>>> Active: active (running) since Mon 2020-06-29
>>> 21:37:36 BST; 13h ago
>>> Docs: man:glusterd(8)
>>> Process: 4071 ExecStart=/usr/sbin/glusterd -p
>>> /var/run/glusterd.pid --log-level $LOG_LEVEL
>>> $GLUSTERD_OPTIONS (code=exited, status>
>>> Main PID: 4086 (glusterd)
>>> Tasks: 20 (limit: 101792)
>>> Memory: 28.9M
>>> CGroup: /system.slice/glusterd.service
>>> ├─4086 /usr/sbin/glusterd -p
>>> /var/run/glusterd.pid --log-level INFO
>>> └─4302 /usr/sbin/glusterfs -s localhost
>>> --volfile-id shd/USERs -p
>>> /var/run/gluster/shd/USERs/USERs-shd.pid -l /var/log/g>
>>>
>>> Jun 29 21:37:36 swir.private.pawel systemd[1]:
>>> Starting GlusterFS, a clustered file-system server...
>>> Jun 29 21:37:36 swir.private.pawel systemd[1]: Started
>>> GlusterFS, a clustered file-system server.
>>>
>>> And I do not see any other apparent problems nor errors.
>>> On that node I manually:
>>> $ systemctl restart glusterd.service
>>> and...
>>>
>>> $ gluster volume status USERs
>>> Status of volume: USERs
>>> Gluster process TCP Port
>>> RDMA Port Online Pid
>>>
>------------------------------------------------------------------------------
>>> Brick swir.direct:/00.STORAGE/2/0-GLUSTER-U
>>> SERs 49152
>>> 0 Y 103225
>>> Brick dzien.direct:/00.STORAGE/2/0-GLUSTER-
>>> USERs 49152
>>> 0 Y 57338
>>> Self-heal Daemon on localhost N/A
>>> N/A Y 103270
>>> Self-heal Daemon on dzien.direct N/A
>>> N/A Y 57359
>>>
>>> Is not a puzzle??? I'm on glusterfs-7.6-1.el8.x86_64
>>> I hope somebody can share some thoughts.
>>> many thanks, L.
>>>
>> That cannot be it!? If the root cause of this problem is
>> 2-replica volume then it would be a massive cock-up! Then
>> 2-volume replica should be banned and forbidden.
>>
>> I hope some can suggest a way to troubleshoot it.
>>
>> ps. we all, I presume all, know problems of 2-replica volumes.
>>
>> many thanks, L.
>>
>>
>>> ________
>>>
>>>
>>>
>>> Community Meeting Calendar:
>>>
>>> Schedule -
>>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>>> Bridge: https://bluejeans.com/441850968
>>>
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> <mailto:Gluster-users at gluster.org>
>>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>>
>>>
>>> --
>>> *Barak Sason Rofman*
>>>
>>> Gluster Storage Development
>>>
>>> Red Hat Israel <https://www.redhat.com/>
>>>
>>> 34 Jerusalem rd. Ra'anana, 43501
>>>
>>> bsasonro at redhat.com <mailto:adi at redhat.com>
>>> T: _+972-9-7692304_
>>> M: _+972-52-4326355_
>>>
>>> @RedHat <https://twitter.com/redhat> Red Hat
>>> <https://www.linkedin.com/company/red-hat> Red Hat
>>> <https://www.facebook.com/redhat.il/>
>>> <https://red.ht/sig>
>>>
>> ________
>>
>>
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>________
>
>
>
>Community Meeting Calendar:
>
>Schedule -
>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>Bridge: https://bluejeans.com/441850968
>
>Gluster-users mailing list
>Gluster-users at gluster.org
>https://lists.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list