Bug #36330 for bind9-public: [Patch] EDNS fail - problems resolving blog.rop.io IN AAAA

I am currently running git rev 06e0d6b plus trivial patches. I have been trying to work out why I get a SERVFAIL resolving and validating blog.rop.io IN AAAA Named seems to go into a loop re-querying for dns2v6.cdns.net/A and getting a truncated response. It does not fall back to TCP. A similar thing happens for rop.io/DNSKEY. I can only reproduce this response with 'dig' if I send a query without EDNS. So the question is, why is named sending queries without EDNS? It seems to be because the authority servers are a bit broken. Early in the resolution process named made a query for blog.rop.io AAAA and got a truncated response with a missing EDNS record and a missing TC flag - see the first query/response pair below At this point it marked the server as not supporting EDNS. Similarly, when named queried for dns2v6.cdns.net/AAAA it got a response without an EDNS packet. This does not seem to be due to truncation, but rather a buggy EDNS implementation which drops the record if the buffer size is 512 or less. See the second query/response pair below. *** 1 ; <<>> DiG 9.11.0pre-alpha <<>> -4 +qr +multiline +norec +dnssec +bufsize=512 blog.rop.io in aaaa @ns1.r4ns.com. ;; global options: +cmd ;; Sending: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9821 ;; flags: ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 512 ;; QUESTION SECTION: ;blog.rop.io. IN AAAA ;; QUERY SIZE: 40 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9821 ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 0 ;; QUESTION SECTION: ;blog.rop.io. IN AAAA ;; AUTHORITY SECTION: rop.io. 3600 IN SOA ns1.r4ns.com. info.egeektronic.com. ( 2014061518 ; serial 1200 ; refresh (20 minutes) 180 ; retry (3 minutes) 604800 ; expire (1 week) 3600 ; minimum (1 hour) ) rop.io. 3600 IN RRSIG SOA 7 2 3600 ( 20140626000000 20140612000000 26739 rop.io. gCmNnHyTtVLbgLDOKuVou9KexzhqBeHdLoqtN9KpGPmu XHNYjk21RaFAi91ly1Z4JaiPSWk4dj+uZjUKtAde63np OdPB0N3HYX/NPaaQ2fXIE9d7qYJAOy8tEaczxQIs5hkL KBor61w4zrpypfI6uzcmqNWZ0mHibmTUumGYzwA= ) m44202ac9ca4jsqum1248sjcmff74004.rop.io. 3600 IN NSEC3 1 1 1 BEEF ( M44202AC9CA4JSQUM1248SJCMFF74005 A NS SOA MX TXT AAAA SSHFP RRSIG DNSKEY NSEC3PARAM ) ;; Query time: 34 msec ;; SERVER: 176.124.112.100#53(176.124.112.100) ;; WHEN: Mon Jun 16 14:01:04 BST 2014 ;; MSG SIZE rcvd: 342 *** 2 ; <<>> DiG 9.11.0pre-alpha <<>> +qr +multiline +ignore +norec +dnssec +bufsize=512 dns2v6.cdns.net in aaaa @194.0.1.1 ;; global options: +cmd ;; Sending: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23456 ;; flags: ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 512 ;; QUESTION SECTION: ;dns2v6.cdns.net. IN AAAA ;; QUERY SIZE: 44 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23456 ;; flags: qr aa; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;dns2v6.cdns.net. IN AAAA ;; ANSWER SECTION: dns2v6.cdns.net. 86400 IN AAAA 2001:678:5::1 dns2v6.cdns.net. 86400 IN RRSIG AAAA 8 3 86400 ( 20140712152242 20140607075037 1616 cdns.net. n0/yzR0wAJZ/6P1QyALIbBenMYs+mYddGV9oSYNoB+UU AS8IfHHpSBLSK+T27r/u8nMacJ26TvBQ3nYb5JcZGfHM i2V6WjKoSs/Fs64Uz8GbiCX5pNUdsbZCN+3KbYFzh4Jn Req223p88Lk2l9+itq8FYLElAV8V9r7p9UNDEB8= ) ;; Query time: 36 msec ;; SERVER: 194.0.1.1#53(194.0.1.1) ;; WHEN: Mon Jun 16 14:14:13 BST 2014 ;; MSG SIZE rcvd: 229 Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ German Bight: Northwest 5 to 7, veering north 4 or 5. Moderate or rough. Fair. Good.

---

I don't think there is anything for us to do here. The authoritative server is breaking too many DNS fundamentals to interoperate. If it set tc=1 it might be worth hacking around the broken EDNS support but to fix this we would also have to figure out tc=1 should have been set or force TCP for these servers. Add to that there is a updated version of the server available I think the best solution is to accept the odd report like this and encourage the authoritative server to upgrade. Note they have introduced a CNAME since the report was first published.

---

Mark Andrews via RT <bind9-bugs@isc.org> wrote: > I don't think there is anything for us to do here. I can see a couple of changes that would help. I think it would make sense to use different EDNS logic when resolving a signed zone. In this situation named should never send a query without EDNS DO. If an auth server sends a non-EDNS response to a query for a signed zone, named should treat it as a broken server not as a pre-EDNS server. This should cause named to try the other servers for the zone, which might work better. At the moment named can fall back to non-EDNS, get an unsigned reply, try to validate it, and give up, rather than trying to get a properly signed response from another server. The other improvement would be to start with a less pessimistic EDNS buffer size. Plausible choices would be the Ethernet MTU minus a bit of slop for VLAN tags and tunnels, or the IPv6 minimum MTU. I haven't looked at the new EDNS logic in detail yet; I used to have a patch which added a third intermediate fallback level, though I never properly investigated whether it improved things. Its downside was that it increased latency for non-EDNS servers. If you start at an intermediate buffer size, then if it workd you can try big buffers to see if fragmented packets work, and if not you can fall back to small buffers and/or no EDNS; this should be a good mix of the new and old behaviours. Regarding the two specific examples in this bug report, the Rage4 people have fixed their custom build of PowerDNS, which is nice. I have not heard back from CommunityDNS. This is a bit more worrying since they provide authoritative service for A LOT of really important zones, including TLDs and dotat.at... ; <<>> DiG 9.11.0pre-alpha <<>> +dnssec +ignore +multiline +norec +bufsize=512 +qr dotat.at @ns3.gratisdns.dk ;; global options: +cmd ;; Sending: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22659 ;; flags: ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 512 ;; QUESTION SECTION: ;dotat.at. IN A ;; QUERY SIZE: 37 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22659 ;; flags: qr aa; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;dotat.at. IN A ;; ANSWER SECTION: dotat.at. 3600 IN A 212.13.197.229 dotat.at. 3600 IN RRSIG A 5 2 3600 ( 20140628093111 20140618091908 56700 dotat.at. OqSbw9PGyPaq35tjm/UxEglUataufjWvKpkb8A5mT4CW FKxQNTfwPwq1aXnSfpzL+5oorIf5pqdDd0le8WCKtcUv rlPh6RsAea08WfsQc226cM0bHVJuU13PVVYBP+Y9PFQ8 aXBP2APJOFWpRpbhu72irU66UpIcdEwnGDV4Weo= ) ;; Query time: 23 msec ;; SERVER: 2001:678:5::6#53(2001:678:5::6) ;; WHEN: Thu Jun 19 11:21:03 BST 2014 ;; MSG SIZE rcvd: 210 Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Humber, Thames: North or northwest 3 or 4 increasing 5 or 6, but 4 at times later. Slight or moderate. Showers. Moderate or good, occasionally poor at first.

---

--- lib/dns/include/dns/resolver.h | 2 -- lib/dns/resolver.c | 79 ++++-------------------------------------- 2 files changed, 6 insertions(+), 75 deletions(-) diff --git a/lib/dns/include/dns/resolver.h b/lib/dns/include/dns/resolver.h index e9aabc2..a8f35e4 100644 --- a/lib/dns/include/dns/resolver.h +++ b/lib/dns/include/dns/resolver.h @@ -93,8 +93,6 @@ typedef struct dns_fetchevent { #define DNS_FETCHOPT_NOEDNS0 0x008 /*%< Do not use EDNS. */ #define DNS_FETCHOPT_FORWARDONLY 0x010 /*%< Only use forwarders. */ #define DNS_FETCHOPT_NOVALIDATE 0x020 /*%< Disable validation. */ -#define DNS_FETCHOPT_EDNS512 0x040 /*%< Advertise a 512 byte - UDP buffer. */ #define DNS_FETCHOPT_WANTNSID 0x080 /*%< Request NSID */ #define DNS_FETCHOPT_PREFETCH 0x100 /*%< Do prefetch */ #define DNS_FETCHOPT_NOCDFLAG 0x200 /*%< Don't set CD flag. */ diff --git a/lib/dns/resolver.c b/lib/dns/resolver.c index e6e5315..884aeb8 100644 --- a/lib/dns/resolver.c +++ b/lib/dns/resolver.c @@ -145,7 +145,7 @@ * This defines the maximum number of timeouts we will permit before we * disable EDNS0 on the query. */ -#define MAX_EDNS0_TIMEOUTS 3 +#define MAX_EDNS0_TIMEOUTS 6 typedef struct fetchctx fetchctx_t; @@ -245,7 +245,6 @@ struct fetchctx { dns_fwdpolicy_t fwdpolicy; isc_sockaddrlist_t bad; ISC_LIST(struct tried) edns; - ISC_LIST(struct tried) edns512; isc_sockaddrlist_t bad_edns; dns_validator_t *validator; ISC_LIST(dns_validator_t) validators; @@ -1709,39 +1708,6 @@ add_triededns(fetchctx_t *fctx, isc_sockaddr_t *address) { ISC_LIST_INITANDAPPEND(fctx->edns, tried, link); } -static struct tried * -triededns512(fetchctx_t *fctx, isc_sockaddr_t *address) { - struct tried *tried; - - for (tried = ISC_LIST_HEAD(fctx->edns512); - tried != NULL; - tried = ISC_LIST_NEXT(tried, link)) { - if (isc_sockaddr_equal(&tried->addr, address)) - return (tried); - } - - return (NULL); -} - -static void -add_triededns512(fetchctx_t *fctx, isc_sockaddr_t *address) { - struct tried *tried; - - tried = triededns512(fctx, address); - if (tried != NULL) { - tried->count++; - return; - } - - tried = isc_mem_get(fctx->mctx, sizeof(*tried)); - if (tried == NULL) - return; - - tried->addr = *address; - tried->count = 1; - ISC_LIST_INITANDAPPEND(fctx->edns512, tried, link); -} - #ifdef ISC_PLATFORM_USESIT static void compute_cc(resquery_t *query, unsigned char *sit, size_t len) { @@ -2011,26 +1977,12 @@ resquery_send(resquery_t *query) { if (fctx->timeout && (query->options & DNS_FETCHOPT_NOEDNS0) == 0) { - isc_sockaddr_t *sockaddr = &query->addrinfo->sockaddr; - struct tried *tried; - - if (fctx->timeouts > (MAX_EDNS0_TIMEOUTS * 2) && + if (fctx->timeouts > MAX_EDNS0_TIMEOUTS && !EDNSOK(query->addrinfo)) { query->options |= DNS_FETCHOPT_NOEDNS0; fctx->reason = "disabling EDNS"; - } else if ((tried = triededns512(fctx, sockaddr)) != NULL && - tried->count >= 2U && !EDNSOK(query->addrinfo)) { - query->options |= DNS_FETCHOPT_NOEDNS0; - fctx->reason = "disabling EDNS"; - } else if ((tried = triededns(fctx, sockaddr)) != NULL) { - if (tried->count == 1U) { - hint = dns_adb_getudpsize(fctx->adb, - query->addrinfo); - } else if (tried->count >= 2U) { - query->options |= DNS_FETCHOPT_EDNS512; - fctx->reason = "reducing the advertised EDNS " - "UDP packet size to 512 octets"; - } + } else { + hint = dns_adb_getudpsize(fctx->adb, query->addrinfo); } } fctx->timeout = ISC_FALSE; @@ -2049,8 +2001,7 @@ resquery_send(resquery_t *query) { unsigned char sit[64]; #endif - if ((flags & FCTX_ADDRINFO_EDNSOK) != 0 && - (query->options & DNS_FETCHOPT_EDNS512) == 0) { + if ((flags & FCTX_ADDRINFO_EDNSOK) != 0) { udpsize = dns_adb_probesize(fctx->adb, query->addrinfo); if (udpsize > res->udpsize) @@ -2064,12 +2015,6 @@ resquery_send(resquery_t *query) { udpsize = 512; /* - * Was the size forced to 512 in the configuration? - */ - if (udpsize == 512U) - query->options |= DNS_FETCHOPT_EDNS512; - - /* * We have talked to this server before. */ if (hint != 0U) @@ -2159,12 +2104,9 @@ resquery_send(resquery_t *query) { */ query->udpsize = udpsize; - if (udpsize > 512U) + if (udpsize > 0) add_triededns(fctx, &query->addrinfo->sockaddr); - if (udpsize == 512U) - add_triededns512(fctx, &query->addrinfo->sockaddr); - /* * Clear CD if EDNS is not in use. */ @@ -3427,13 +3369,6 @@ fctx_destroy(fetchctx_t *fctx) { isc_mem_put(fctx->mctx, tried, sizeof(*tried)); } - for (tried = ISC_LIST_HEAD(fctx->edns512); - tried != NULL; - tried = ISC_LIST_HEAD(fctx->edns512)) { - ISC_LIST_UNLINK(fctx->edns512, tried, link); - isc_mem_put(fctx->mctx, tried, sizeof(*tried)); - } - for (sa = ISC_LIST_HEAD(fctx->bad_edns); sa != NULL; sa = next_sa) { @@ -3831,7 +3766,6 @@ fctx_create(dns_resolver_t *res, dns_name_t *name, dns_rdatatype_t type, fctx->fwdpolicy = dns_fwdpolicy_none; ISC_LIST_INIT(fctx->bad); ISC_LIST_INIT(fctx->edns); - ISC_LIST_INIT(fctx->edns512); ISC_LIST_INIT(fctx->bad_edns); ISC_LIST_INIT(fctx->validators); fctx->validator = NULL; @@ -7452,7 +7386,6 @@ resquery_response(isc_task_t *task, isc_event_t *event) { broken_server = DNS_R_TRUNCATEDTCP; keep_trying = ISC_TRUE; } else if ((query->options & DNS_FETCHOPT_NOEDNS0) == 0 && - (query->options & DNS_FETCHOPT_EDNS512) == 0 && !triededns(fctx, &query->addrinfo->sockaddr)) { resend = ISC_TRUE; } else { -- 2.0.1

---

I have just sent some patches which seem to improve things for me. > I think it would make sense to use different EDNS logic when resolving a > signed zone. In this situation named should never send a query without > EDNS DO. Interestingly there's a NEEDEDNS0 flag which was not actually used. One of my patches deletes it for tidiness, since I was hacking around in that area. My trivial test is: $ dig axfr . | sed -E '/^([0-9a-z-]+)[.][ ].*/!d;s//\1/' | sort -u | while read d; do dig dnskey $d. | grep 'status: SERVFAIL' && echo $d; done When running rev. e58154a6ec0a8a0bde32bb1e39ad2f1fbc3d2ef2 I get: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 9205 ac ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 60366 am ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 17601 college ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 46668 cologne ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 44970 eus ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 44278 feedback ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 65460 foo ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 49232 gal ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 48754 host ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61070 ink ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 26656 koeln ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 15330 lacaixa ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 26155 lu ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 30175 mango ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 3460 museum ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 54959 nrw ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 6631 quebec ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 43315 ruhr ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 51870 scot ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 46699 soy ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 56911 ua ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 5356 xn--80asehdb ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 62827 xn--80aswg ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 36947 xn--l1acc ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 49480 xn--mgbab2bd ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 39194 xn--q9jyb4c With my patch that deletes the EDNS512 logic I get: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 57772 foo ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 49115 soy ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 49205 xn--l1acc ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 33837 xn--q9jyb4c With the change of initial buffer size from 512 to 1232 I get just: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 2320 xn--l1acc which is an operational fuckup not a protocol bug. Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ South-east Iceland: Variable 3 or 4, occasionally southwesterly 5 in north. Slight, occasionally moderate in north. Showers, fog patches. Moderate or good, occasionally very poor.

---

3912. [bug] Address some unrecoverable lookup failures. [RT #36330] I went with a more conservative approach. 9.10.1b2 should be out in a couple of days.

---