[Gluster-devel] GlusterD2 - 4.0.0rc1 (warning: we have a blocker for GD2)

[Kaushal M]

On Thu, Mar 1, 2018 at 6:22 PM, Milind Changire <mchangir at redhat.com> wrote:
> Maybe: gcc -Wl,--start-group foo.o bar.o -Wl,--end-group
>
> quote from man ld:
> It is best to use it only when there are unavoidable circular references
> between two or more archives.
>
>
> On Thu, Mar 1, 2018 at 6:18 PM, Kaushal M <kshlmster at gmail.com> wrote:
>>
>> On Thu, Mar 1, 2018 at 6:14 PM, Kaushal M <kshlmster at gmail.com> wrote:
>> > On Thu, Mar 1, 2018 at 12:52 PM, Kaushal M <kshlmster at gmail.com> wrote:
>> >> On Wed, Feb 28, 2018 at 9:50 PM, Kaleb S. KEITHLEY
>> >> <kkeithle at redhat.com> wrote:
>> >>> On 02/28/2018 10:49 AM, Kaushal M wrote:
>> >>>> We have a GlusterD2-4.0.0rc1 release.
>> >>>>
>> >>>> Aravinda, Prashanth and the rest of the GD2 developers have been
>> >>>> working hard on getting more stuff merged into GD2 before the 4.0
>> >>>> release.
>> >>>>
>> >>>> At the same time I have been working on getting GD2 packaged for
>> >>>> Fedora.
>> >>>> I've been able to get all the required dependencies updated and have
>> >>>> submitted to the package maintainer for merging.
>> >>>> I'm now waiting on the maintainer to accept those updates. Once the
>> >>>> updates have been accepted, the GD2 spec can get accepted [2].
>> >>>> I expect this to take at least another week on the whole.
>> >>>>
>> >>>> In the meantime, I've been building all the updated dependencies and
>> >>>> glusterd2-v4.0.0rc1, on the GD2 copr [3].
>> >>>>
>> >>>> I tried to test out the GD2 package with the GlusterFS v4.0.0rc1
>> >>>> release from [4]. And this is where I hit the blocker.
>> >>>>
>> >>>> GD2 does not start with the packaged glusterfs-v4.0.0rc1 bits. I've
>> >>>> opened an issue on the GD2 issue tracker for it [5].
>> >>>> In short, GD2 fails to read options from xlators, as dlopen fails
>> >>>> with
>> >>>> a missing symbol error.
>> >>>>
>> >>>> ```
>> >>>> FATA[2018-02-28 15:02:53.345686] Failed to load xlator options
>> >>>>
>> >>>> error="dlopen(/usr/lib64/glusterfs/4.0.0rc1/xlator/protocol/server.so)
>> >>>> failed; dlerror =
>> >>>> /usr/lib64/glusterfs/4.0.0rc1/xlator/protocol/server.so: undefined
>> >>>> symbol: glusterfs_mgmt_pmap_signout" source="[main.go:79:main.main]"
>> >>>
>> >>>
>> >>> see https://review.gluster.org/#/c/19225/
>> >>>
>> >>>
>> >>> glusterfs_mgmt_pmap_signout() is in glusterfsd. When glusterfsd
>> >>> dlopens
>> >>> server.so the run-time linker can resolve the symbol — for now.
>> >>>
>> >>> Tighter run-time linker semantics coming in, e.g. Fedora 28, means
>> >>> this
>> >>> will stop working in the near future even when RTLD_LAZY is passed as
>> >>> a
>> >>> flag. (As I understand the proposed changes.)
>> >>>
>> >>> It should still work, e.g., on Fedora 27 and el7 though.
>> >>>
>> >>> glusterfs_mgmt_pmap_signout() (and glusterfs_autoscale_threads())
>> >>> really
>> >>> need to be moved to libglusterfs. ASAP. Doing that will resolve this
>> >>> issue.
>> >>
>> >> Thanks for the pointer Kaleb!
>> >>
>> >> But, I'm testing on Fedora 27, where this shouldn't theoretically
>> >> happen.
>> >> So then, why am I hitting this. Is it something to do with the way the
>> >> packages are built?
>> >> Or is there some runtime ld configuration that has been set up.
>> >>
>> >> In any case, we should push and get the offending functions moved into
>> >> libglusterfs.
>> >> That should solve the problem for us.
>> >
>> > I took a shot at this, and it's not as easy simple as it appeared.
>> > I ended up in a recursive linking situation with libglusterfs,
>> > libgfxdr and libgfrpc.
>> > Looks like the solution is to create a libglusterfsd.
>>
>> I see two ways to do this.
>> 1. Make a library out of the whole glusterfsd.
>> Rename `main` to `init`
>> And then create a simple executable which loads this library and calls
>> `init`.
>>
>> Or,
>> 2. Create a very small library with just the pmap_signout and
>> autoscale functions. And use that instead.
>>
>> If anyone else has a better idea about how to do this please let me know.

I was able to create libglusterfsd, with just the pmap_signout nad
autoscale functions.
Turned out to be easy enough to do in the end.
I've pushed a patch for review [1] on master.

I've also opened new bugs to track the fixes for master[2] and
release-4.0[3]. They have been made blockers to the glusterfs-4.0.0
tracker bug [4].

Shyam,
To backport the fix from master to release-4.0, also requires
backporting one more change [5].
Would you be okay with backporting that as well, in a single patch?

[1]: https://review.gluster.org/19657
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1550895
[3]: https://bugzilla.redhat.com/show_bug.cgi?id=1550894
[4]: https://bugzilla.redhat.com/show_bug.cgi?id=1539842
[5]: https://review.gluster.org/19337

>>
>> >
>> >>
>> >>>
>> >>> --
>> >>>
>> >>> Kaleb
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
>
>
> --
> Milind
>