[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: glibc's getaddrinfo() sort order

On Tue, Sep 18, 2007 at 03:33:51PM +0100, Ian Jackson wrote:
> Anthony Towns writes ("Re: glibc's getaddrinfo() sort order"):
> > I'm not familiar with how getaddrinfo() has been implemented in the
> > past
> I think this is an important point.  If you're not familiar with the
> history then perhaps I can help explain.
> hostname-to-address lookups have up to recently generally been done
> with gethostbyname.

Right, gethostbyname I am familiar with (along with the corresponding
DNS round-robin behaviour), and changing its behaviour is certainly

> [...]

> So far so good.  (For clarify, it is the above round-robin
> functionality that I am arguing ought to be preserved.)

> [...]

> However, additionally, it was realised that if getaddrinfo can return
> a mixture of IPv4 and v6 addresses it was necessary to specify in what
> order they ought to be returned.
> When RFC3484 was written its authors evidently felt that the best way
> to do this was to define a comparison function over all addresses,
> which would define which address was to be preferred.
> Heedless of the effect on the DNS round-robin functionality I describe
> above, the authors of RFC3484 specified (s6 rule 9) that all addresses
> should be sorted by "proximity" to the host making the choice - where
> "proximity" is defined as the length of the common initial address
> prefix.

So if getaddrinfo() has always behaved in this way, I don't see a great
deal of justification in changing it. The bug log indicated that there
were pre-rfc implementations of getaddrinfo() that behaved more like
gethostbyname() at least wrt round-robin DNS; but I've got no way of
verifying that.

> This may have been a disputed but arguable definition of real network
> proximity for IPv6 in at the time 3484 was written.  But it is clear
> now that it is not such a measure in the real IPv6 internet, and it
> has never been such a measure in the IPv4 internet.

I hadn't seen any indication it was disputed for IPv6 prior to your mail.
The patch in glibc only affected IPv4 addresses, for that matter.

> So RFC3484 s6 rule 9 is just wrong, because the reasons behind it do
> not apply any more if they ever did.

To give an analogy to the lines I'm thinking along: the definition of
tm_year in the tm struct in time.h is wrong, "years since 1900" should be
"years since 0 AD", but the spec says otherwise, so programs simply need
to deal with that historical craziness.

That's not quite the same here, in that the spec does (by my reading)
explicitly allow implementors to not behave in that way, but if you're
coding to the spec you certainly can't rely on DNS round-robin being passed
through an invocation of getaddrinfo().

> However, it's worse than that: rule 9 is trying to change the
> behaviour of existing systems.  If we agree with rule 9 it ought to
> apply just as well to applications using gethostbyname.
> All existing applications using gethostbyname are not in compliance
> with rule 9.  

The RFC specifies the behaviour of getaddrinfo(), not gethostbyname(),
so doesn't affect any apps that solely use gethostbyname(). So no, it
shouldn't be applied to other functions anymore than the definition of
tm_year should mean we count from 1900 in every year related function.

I think we can safely say that Rule 9 isn't useful for IPv4 addresses.
I'm not sure that's true or not for IPv6 addresses -- it certainly seems
an inappropriately hierarchial way of viewing a network that's connected
much more ... fluidly than that, at any rate. But even if Rule 9 is
completely useless and counterproductive, it's still the standard for
that function, which, afaics, we should be meeting.

> What about getaddrinfo ?  Well, there is no reason why a change in API
> (to add additional richness needed for new functionality) should so
> radically change the behaviour.

Agreed in principle, but this is a rule the RFC should've followed;
since they haven't, I'm not convinced we should.

> It is not reasonable for the RFC to attempt to specify that the
> addresses be returned in a predictable ordering when the established
> behaviour, relied on throughout the internet for decades, has been
> that the addresses are _not_ returned in a predictable order.

Again, I agree with that, but the RFC *has* done that.

> > I'd say it's more important that getaddrinfo() on Debian behave "the same"
> > as on other operating systems, than that it behave in the same way as
> > other functions. I can only take the RFC's assertion as to getaddrinfo()'s
> > proper behaviour though; I don't have a more direct idea how getaddrinfo()
> > behaves in previous versions of Debian, other Linux distros, other libcs,
> > Windows, etc.
> This argument is an argument for accepting any crap that comes out of
> glibc upstream.

No, it's an argument for accepting any crap that comes out of the Internet
standards process. :-/

> As I have demonstrated above, the RFC is wrong, inconsistent with
> existing practice, 

It's certainly inconsistent with gethostbyname()'s existing practice; but
I've seen indications that it's inconsistent with how getaddrinfo()'s past
practice too.

If the Internet standard is explicitly different to what we've shipped
in the past (back to when getaddrinfo() was introduced, probably slink
or hamm), and previous documentation hasn't included the info that DNS
round-robin won't work with getaddrinfo() (which I don't know about,
but I suspect it didn't), then that's a good argument for ignoring the
RFC and keeping the sensible behaviour (and contacting the RFC drafters
with better text).

> > What getaddrinfo() should and shouldn't do is defined by the standard,
> > not by what would be most useful. :-/
> The document which defines getaddrinfo is RFC3493 which doesn't refer
> to RFC3484.  It doesn't mention ordering at all.

RFC3484 explicilty encompasses getaddrinfo though:

   2. Context in Which the Algorithms Operate

   As a consequence, we intend that implementations of getaddrinfo()
   will use the destination address selection algorithm specified here
   to sort the list of IPv6 and IPv4 addresses that they return.

> Note that RFC3484 is NOT A STANDARD.  It is a `proposed standard' -
> the earliest possible IETF standards track document status.

Hey, there's no need to shout... (And yes, I had noticed that; but there's
a lot of proposed standards that're worth following to the letter too)

AFAICS round-robin DNS isn't an Internet standard either, for that matter,
which seems odd to me. There's a Microsoft page that references rfc1794
for round-robin load balancing, but it doesn't seem entirely on point, and
is just a memo anyway.

> > FWIW, if the standard should be changed, it seems to me that it'd carry
> > more weight having the Debian tech ctte put that recommendation in than
> > a random DD.
> Absolutely.  We should recommend that RFC3484 rule 9 should be
> deprecated, and should be abolished (definitely for IPv4 and probably
> for IPv6 as well).

There's a thread on the curl dev list about this apparently, including:

] I agree that we need to consider the default action properly, but I have
] access to a lot of machines and I don't know of anyone who ever modified their
] /etc/gai.conf file and I would argue that most admins don't know that this is
] needed or even how to do it, just to get tools like curl to access *remote*
] round-robin DNS sites "properly".
] It would be very interesting to hear what the people putting together
] (lib)curl for distros think about this. 

 -- http://curl.haxx.se/mail/lib-2007-07/0181.html

The first message in that thread refers to a test run back in 2005 to
see how random getaddrinfo() was, which looks like it gave the same sort
of results we're seeing here.

] ---Check 2000 runs of getaddrinfo(bad11.haxx.se)
] --- 4 different IPs were returned
] IP index 0
] 4040404 2000 times (100%)
] IP index 1
] 2020202 2000 times (100%)
] IP index 2
] 1010101 977 times (48%)
] 3030303 1023 times (51%)
] IP index 3
] 1010101 1023 times (51%)
] 3030303 977 times (48%)

   -- http://curl.haxx.se/mail/lib-2005-11/att-0008/bad11-2000
   -- http://curl.haxx.se/mail/lib-2005-11/0008.html

Running the roundrobin.c program from that mail, I get really odd results.

At home (lenny, I get reasonably well distributed results; on
another box (etch, I get well ordered results (,,,, and on another box on the same network
(~etch, I get the same results quoted above. On another
box (woody, I get well distributed results again.


Attachment: signature.asc
Description: Digital signature

Reply to: