spfquery times out checking apparently valid record

[Lee Maguire]

$ /usr/bin/spfquery -debug=1 -ip=208.72.90.125 -<email address hidden>
spf_compile.c:523 Debug: Parsing macro starting at Please%_see%_http://www.openspf.org/Why?id=%{S}&ip=%{C}&receiver=%{R}
spf_compile.c:1210 Debug: Compiling record v=spf1
spf_dns.c:54 Debug: DNS[cache] lookup: uk.playstation.com SPF (99)
spf_dns.c:54 Debug: DNS[resolv] lookup: uk.playstation.com SPF (99)
spf_dns_resolv.c:311 Debug: query failed: err = -1 No address associated with name (4): uk.playstation.com
spf_dns.c:66 Debug: DNS[resolv] found record
spf_dns.c:69 Debug: DOMAIN: uk.playstation.com TYPE: SPF (99)
spf_dns.c:76 Debug: TTL: 0 RR found: 0 herrno: 4 source: resolv
spf_dns.c:66 Debug: DNS[cache] found record
spf_dns.c:69 Debug: DOMAIN: uk.playstation.com TYPE: SPF (99)
spf_dns.c:76 Debug: TTL: 0 RR found: 0 herrno: 4 source: resolv
spf_server.c:370 Debug: get_record(uk.playstation.com): NO_DATA
spf_dns.c:54 Debug: DNS[cache] lookup: uk.playstation.com TXT (16)
spf_dns.c:54 Debug: DNS[resolv] lookup: uk.playstation.com TXT (16)
[.... hangs at this point ...]

The site openspf.org has no problem marking this as approved.

http://www.openspf.org/Why?show-form=1&identity=help%40uk.playstation.com&ip-address=208.72.90.125

Is it failing because the record is too long and it needs to fall back to TCP?

$ host -t TXT uk.playstation.com
;; Truncated, retrying in TCP mode.
uk.playstation.com descriptive text "v=spf1 include:spf.messagelabs.com include:custhelp.com include:rnmk.com mx a:a:cetclx020.online.scee.com a:cetclx021.online.scee.com a:cetcdns002.online.scee.com a:cetcdns003.online.scee.com a:mail.clients.eu.clientlogic.com -all"
uk.playstation.com descriptive text "v=spf2.0/pra include:spf.messagelabs.com include:custhelp.com include:rnmk.com mx a:a:cetclx020.online.scee.com a:cetclx021.online.scee.com a:cetcdns002.online.scee.com a:cetcdns003.online.scee.com a:mail.clients.eu.clientlogic.com -all"

[Scott Kitterman (kitterman)]

I think it's likely related to the TCP fallback/retry. That record is right
on the edge of if it will fit in a UDP packet or not and so it'll probably
work some times and not others. The SPF RFCs recommend against records that
large for this exact reason.

When you checked, was it from the same host you're having trouble with
spfquery on? If not, it might also be because DNS over TCP was blocked for
the host running spfquery.

If you install spf-tools-python, it includes an alternate spfquery
implementation that I know supports TCP fallback, as long as TCP isn't blocked
in a firewall somewhere.

[Lee Maguire (leemaguire)]

The query was performed on the same host - TCP DNS is not blocked. I ran the same queries on multiple machines/networks to the same result.

[Lee Maguire (leemaguire)]

I've tried both libspf and pyspf. Python takes 30 seconds to return a result, which is useless for realtime checking. libspf takes, it turns out, 7 minutes.

$ time /usr/bin/spfquery.pyspf -ip=208.72.90.125 -<email address hidden>
pass
()
('spfquery:', 'domain of uk.playstation.com designates 208.72.90.125 as permitted sender')
('Received-SPF:', 'Pass (spfquery: domain of uk.playstation.com designates 208.72.90.125 as permitted sender) client-ip=208.72.90.125; <email address hidden>"; receiver=spfquery; mechanism="include:custhelp.com"; identity=mailfrom')

real 0m33.216s
user 0m0.080s
sys 0m0.010s

$ time /usr/bin/spfquery.libspf2 -ip=208.72.90.125 -<email address hidden>
pass

spfquery: domain of uk.playstation.com designates 208.72.90.125 as permitted sender
Received-SPF: pass (spfquery: domain of uk.playstation.com designates 208.72.90.125 as permitted sender) client-ip=208.72.90.125; <email address hidden>;

real 7m11.776s
user 0m0.000s
sys 0m0.000s

[Lee Maguire (leemaguire)]

Is it possible that the size of the fully resolved SPF record may also be the problem?

$ host -t TXT uk.playstation.com
;; Truncated, retrying in TCP mode.
uk.playstation.com descriptive text "v=spf1 include:spf.messagelabs.com include:custhelp.com include:rnmk.com mx a:a:cetclx020.online.scee.com a:cetclx021.online.scee.com a:cetcdns002.online.scee.com a:cetcdns003.online.scee.com a:mail.clients.eu.clientlogic.com -all"
uk.playstation.com descriptive text "v=spf2.0/pra include:spf.messagelabs.com include:custhelp.com include:rnmk.com mx a:a:cetclx020.online.scee.com a:cetclx021.online.scee.com a:cetcdns002.online.scee.com a:cetcdns003.online.scee.com a:mail.clients.eu.clientlogic.com -all"
$ host -t TXT spf.messagelabs.com
spf.messagelabs.com descriptive text "v=spf1 include:nets1.spf.messagelabs.com include:nets2.spf.messagelabs.com ~all"
$ host -t TXT nets1.spf.messagelabs.com
nets1.spf.messagelabs.com descriptive text "v=spf1 ip4:85.158.136.0/21 ip4:193.109.254.0/23 ip4:194.106.220.0/23 ip4:195.245.230.0/23 ip4:95.131.104.0/21 ip4:46.226.48.0/21 ip4:196.14.170.0/23"
$ host -t TXT nets2.spf.messagelabs.com
nets2.spf.messagelabs.com descriptive text "v=spf1 ip4:216.82.240.0/20 ip4:67.219.240.0/20 ip4:117.120.16.0/21 ip4:103.9.96.0/22 ip4:62.25.80.152/29 ip4:62.208.159.152/29"
$ host -t TXT custhelp.com
custhelp.com descriptive text "v=spf1 include:spf-a.rnmk.com include:spf-b.rnmk.com include:spf-c.rnmk.com ~all"
$ host -t TXT spf-a.rnmk.com
spf-a.rnmk.com descriptive text "v=spf1 ip4:216.136.162.64/26 ip4:74.117.203.64/26 ip4:74.117.201.64/26 ip4:208.72.90.64/26 ip4:208.72.94.64/26 ip4:205.223.81.64/26 ip4:205.223.83.64/26 ip4:205.223.85.64/26 ip4:205.223.87.64/26 ip4:216.136.168.93 ip4:216.136.229.13 ~all"
$ host -t TXT spf-b.rnmk.com
spf-b.rnmk.com descriptive text "v=spf1 ip4:216.136.229.48 ip4:74.117.200.18 ip4:74.117.200.22 ip4:74.117.202.18 ip4:74.117.202.22 ip4:74.117.202.148 ip4:74.117.202.152 ip4:74.117.203.135 ip4:74.117.203.192 ip4:74.117.203.209 ip4:74.117.205.22 ip4:74.117.205.36 ~all"
$ host -t TXT spf-c.rnmk.com
spf-c.rnmk.com descriptive text "v=spf1 ip4:74.117.206.22 ip4:74.117.207.22 ip4:208.72.90.16 ip4:208.72.90.17 ip4:208.72.90.135 ip4:208.72.91.18 ip4:208.72.91.22 ip4:208.72.92.12 ip4:208.72.92.28 ip4:208.72.92.202 ip4:208.72.92.212 ip4:208.72.93.28 ip4:64.56.194.155 ~all"
$ host -t TXT rnmk.com
rnmk.com descriptive text "v=spf1 include:spf-a.rnmk.com include:spf-b.rnmk.com include:spf-c.rnmk.com include:spf-d.rnmk.com ~all"
$ host -t TXT spf-d.rnmk.com
spf-d.rnmk.com descriptive text "v=spf1 ip4:160.34.16.0/24 ip4:205.223.86.0/24 ip4:129.152.76.0/23 ip4:199.167.173.0/24 ~all"

[Scott Kitterman (kitterman)]

On Monday, June 09, 2014 23:02:24 you wrote:
> uk.playstation.com

The correct result for their record would be permerror due to too many DNS
lookups. spfquery should survive that though.

See http://kitterman.com/spf/validate.html for a validator that checks for
that.

[Scott Kitterman (kitterman)]

I've measured the two implementations against each other. The C
implementation (libspf2) is roughly twice as fast as the python one when DNS
records are all in the local cache. Most of the processing time is spent in
DNS queries for both. Those take as long as they take.

The current pyspf version does have a shorter timeout as recommended by RFC
7208.