<div dir="ltr">Hi,<div><br><div> In gluster we do support one kind of address family (either IPV4 or IPV6) and it depends on the user</div><div> what address family they want to use.It is a configurable option a user can set the value in volfile . Here</div><div> It seems you are facing an issue as you mentioned " HPE is forcing IPV4 servers in to to an IPV6 address", i </div><div> think you can avoid an error if u pass address-family=inet6(<span style="color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";font-size:14px">xlator-option="transport.address-family=inet6")</span> </div><div> during mount a volume and in that case you should not face an issue.</div><div> For more please refer this <a href="https://github.com/gluster/glusterfs/pull/2666">https://github.com/gluster/glusterfs/pull/2666</a></div><div><br></div><div>Thanks,</div><div>Mohit Agrawal</div><div><br></div><div> </div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 21, 2021 at 2:42 PM Erik Jacobson <<a href="mailto:erik.jacobson@hpe.com">erik.jacobson@hpe.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear devel team -<br>
<br>
I botched the email address here. I type "hpcm-devel" like 30 times a<br>
day so I mistyped that. Sorry about that.<br>
<br>
Any advice appreciated and see attached patch that "gets it going for<br>
us" but obviously not something you could release.<br>
<br>
Erik<br>
<br><br><br>---------- Forwarded message ----------<br>From: Erik Jacobson <<a href="mailto:erik.jacobson@hpe.com" target="_blank">erik.jacobson@hpe.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, <a href="mailto:hpcm-devel@gluster.org" target="_blank">hpcm-devel@gluster.org</a><br>Cc: <br>Bcc: <br>Date: Mon, 20 Sep 2021 16:46:12 -0500<br>Subject: [Gluster-users] gluster forcing IPV6 on our IPV4 servers, glusterd fails (was gluster update question regarding new DNS resolution requirement)<br>I pretended I'm a low-level C programmer with network and filesystem<br>
experience for a few hours.<br>
<br>
I'm not sure what the right solution is but what was happening was the<br>
code was trying to treat our IPV4 hosts as AF_INET6 and the family was<br>
incompatible with our IPV4 IP addresses. Yes, we need to move to IPV6<br>
but we're hoping to do that on our own time (~50 years like everybody<br>
else :)<br>
<br>
I found a chunk of the code that seemed to be force-setting us to<br>
AF_INET6.<br>
<br>
While I'm sure it is not 100% the correct patch, the patch attached and<br>
pasted below is working for me so I'll integrate it with our internal<br>
build to continue testing.<br>
<br>
Please let me know if there is a configuration item I missed or a<br>
different way to do this. I added -devel to this email.<br>
<br>
In the previous thread, you would have seen that we're testing a<br>
hopeful change that will upgrade our deployed customers from gluster<br>
7.9 to gluster 9.3.<br>
<br>
Thank you!! Advice on next steps would be appreciated !!<br>
<br>
<br>
diff -Narup glusterfs-9.3-ORIG/rpc/rpc-transport/socket/src/name.c glusterfs-9.3-NEW/rpc/rpc-transport/socket/src/name.c<br>
--- glusterfs-9.3-ORIG/rpc/rpc-transport/socket/src/name.c 2021-06-29 00:27:44.381408294 -0500<br>
+++ glusterfs-9.3-NEW/rpc/rpc-transport/socket/src/name.c 2021-09-20 16:34:28.969425361 -0500<br>
@@ -252,9 +252,16 @@ af_inet_client_get_remote_sockaddr(rpc_t<br>
/* Need to update transport-address family if address-family is not provided<br>
to command-line arguments<br>
*/<br>
+ /* HPE This is forcing our IPV4 servers in to to an IPV6 address<br>
+ * family that is not compatible with IPV4. For now we will just set it<br>
+ * to AF_INET.<br>
+ */<br>
+ /*<br>
if (inet_pton(AF_INET6, remote_host, &serveraddr)) {<br>
sockaddr->sa_family = AF_INET6;<br>
}<br>
+ */<br>
+ sockaddr->sa_family = AF_INET;<br>
<br>
/* TODO: gf_resolve is a blocking call. kick in some<br>
non blocking dns techniques */<br>
<br>
<br>
On Mon, Sep 20, 2021 at 11:35:35AM -0500, Erik Jacobson wrote:<br>
> I missed the other important log snip:<br>
> <br>
> The message "E [MSGID: 101075] [common-utils.c:520:gf_resolve_ip6] 0-resolver: error in getaddrinfo [{family=10}, {ret=Address family for hostname not supported}]" repeated 620 times between [2021-09-20 15:49:23.720633 +0000] and [2021-09-20 15:50:41.731542 +0000]<br>
> <br>
> So I will dig in to the code some here.<br>
> <br>
> <br>
> On Mon, Sep 20, 2021 at 10:59:30AM -0500, Erik Jacobson wrote:<br>
> > Hello all! I hope you are well.<br>
> > <br>
> > We are starting a new software release cycle and I am trying to find a<br>
> > way to upgrade customers from our build of gluster 7.9 to our build of<br>
> > gluster 9.3<br>
> > <br>
> > When we deploy gluster, we foribly remove all references to any host<br>
> > names and use only IP addresses. This is because, if for any reason a<br>
> > DNS server is unreachable, even if the peer files have IPs and DNS, it<br>
> > causes glusterd to be unable to reach peers properly. We can't really<br>
> > rely on /etc/hosts either because customers take artistic licene with<br>
> > their /etc/hosts files and don't realize that problems that can cause.<br>
> > <br>
> > So our deployed peer files look something like this:<br>
> > <br>
> > uuid=46a4b506-029d-4750-acfb-894501a88977<br>
> > state=3<br>
> > hostname1=172.23.0.16<br>
> > <br>
> > That is, with full intention, we avoid host names.<br>
> > <br>
> > When we upgrade to gluster 9.3, we fall over with these errors and<br>
> > gluster is now partitioned and the updated gluster servers can't reach<br>
> > anybody:<br>
> > <br>
> > [2021-09-20 15:50:41.731543 +0000] E [name.c:265:af_inet_client_get_remote_sockaddr] 0-management: DNS resolution failed on host 172.23.0.16<br>
> > <br>
> > <br>
> > As you can see, we have defined on purpose everything using IPs but in<br>
> > 9.3 it appears this method fails. Are there any suggestions short of<br>
> > putting real host names in peer files?<br>
> > <br>
> > <br>
> > <br>
> > FYI<br>
> > <br>
> > This supercomputer will be using gluster for part of its system<br>
> > management. It is how we deploy the Image Objects (squashfs images)<br>
> > hosted on NFS today and served by gluster leader nodes and also store<br>
> > system logs, console logs, and other data.<br>
> > <br>
> > <a href="https://www.olcf.ornl.gov/frontier/" rel="noreferrer" target="_blank">https://www.olcf.ornl.gov/frontier/</a> <br>
> > <br>
> > <br>
> > Erik<br>
> > ________<br>
> > <br>
> > <br>
> > <br>
> > Community Meeting Calendar:<br>
> > <br>
> > Schedule -<br>
> > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
> > Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a> <br>
> > Gluster-users mailing list<br>
> > <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> > <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a> <br>
> ________<br>
> <br>
> <br>
> <br>
> Community Meeting Calendar:<br>
> <br>
> Schedule -<br>
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
> Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a> <br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a> <br>
________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a> <br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a> <br>
-------<br>
<br>
Community Meeting Calendar:<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://meet.google.com/cpu-eiue-hvk" rel="noreferrer" target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br>
<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-devel" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-devel</a><br>
<br>
</blockquote></div>