[Gluster-devel] GlusterD2 - 4.0.0rc1 (warning: we have a blocker for GD2)

[Kaushal M]

On Thu, Mar 1, 2018 at 12:52 PM, Kaushal M <kshlmster at gmail.com> wrote:
> On Wed, Feb 28, 2018 at 9:50 PM, Kaleb S. KEITHLEY <kkeithle at redhat.com> wrote:
>> On 02/28/2018 10:49 AM, Kaushal M wrote:
>>> We have a GlusterD2-4.0.0rc1 release.
>>>
>>> Aravinda, Prashanth and the rest of the GD2 developers have been
>>> working hard on getting more stuff merged into GD2 before the 4.0
>>> release.
>>>
>>> At the same time I have been working on getting GD2 packaged for Fedora.
>>> I've been able to get all the required dependencies updated and have
>>> submitted to the package maintainer for merging.
>>> I'm now waiting on the maintainer to accept those updates. Once the
>>> updates have been accepted, the GD2 spec can get accepted [2].
>>> I expect this to take at least another week on the whole.
>>>
>>> In the meantime, I've been building all the updated dependencies and
>>> glusterd2-v4.0.0rc1, on the GD2 copr [3].
>>>
>>> I tried to test out the GD2 package with the GlusterFS v4.0.0rc1
>>> release from [4]. And this is where I hit the blocker.
>>>
>>> GD2 does not start with the packaged glusterfs-v4.0.0rc1 bits. I've
>>> opened an issue on the GD2 issue tracker for it [5].
>>> In short, GD2 fails to read options from xlators, as dlopen fails with
>>> a missing symbol error.
>>>
>>> ```
>>> FATA[2018-02-28 15:02:53.345686] Failed to load xlator options
>>>         error="dlopen(/usr/lib64/glusterfs/4.0.0rc1/xlator/protocol/server.so)
>>> failed; dlerror =
>>> /usr/lib64/glusterfs/4.0.0rc1/xlator/protocol/server.so: undefined
>>> symbol: glusterfs_mgmt_pmap_signout" source="[main.go:79:main.main]"
>>
>>
>> see https://review.gluster.org/#/c/19225/
>>
>>
>> glusterfs_mgmt_pmap_signout() is in glusterfsd. When glusterfsd dlopens
>> server.so the run-time linker can resolve the symbol — for now.
>>
>> Tighter run-time linker semantics coming in, e.g. Fedora 28, means this
>> will stop working in the near future even when RTLD_LAZY is passed as a
>> flag. (As I understand the proposed changes.)
>>
>> It should still work, e.g., on Fedora 27 and el7 though.
>>
>> glusterfs_mgmt_pmap_signout() (and glusterfs_autoscale_threads()) really
>> need to be moved to libglusterfs. ASAP. Doing that will resolve this issue.
>
> Thanks for the pointer Kaleb!
>
> But, I'm testing on Fedora 27, where this shouldn't theoretically happen.
> So then, why am I hitting this. Is it something to do with the way the
> packages are built?
> Or is there some runtime ld configuration that has been set up.
>
> In any case, we should push and get the offending functions moved into
> libglusterfs.
> That should solve the problem for us.

I took a shot at this, and it's not as easy simple as it appeared.
I ended up in a recursive linking situation with libglusterfs,
libgfxdr and libgfrpc.
Looks like the solution is to create a libglusterfsd.

>
>>
>> --
>>
>> Kaleb