What Is The SPF “PermError Too Many DNS Lookups” And How To Fix It

Email security and authentication is a serious matter, as we have learned in this series about email authentication processes, which include SPF, DKIM, DMARC, their comparison, BIMI, and MTA-STS. However, while performing an SPF DNS lookup to check its status, you may occasionally encounter the error “PermError: Too many DNS lookups“.

Sender Policy Framework (SPF) is a DNS TXT record, stored on a DNS server, which stores the names of the domains allowed to send emails to your domain. An SPF record contains the IP address or the server names of remote servers with SPF records, and has the following syntax:

v=spf1 +a +mx ip4:65.181.111.142 ip4:65.181.111.145 include:spf.examplesender.email ~all

Seeing the “PermError” prompt when performing an SPF DNS Lookup means your SPF record is not configured correctly. When the SPF isn’t configured correctly, it means that the SPF alignment is affected, which in turn affects the DMARC authentication. When that happens, the receiver may stop getting the emails intended for them.

In this post, we are going to discuss, in detail, what the error indicates, and what you can do to fix it so that your email authentication mechanisms are configured correctly and effectively.

What the “PermError too many DNS lookups” means

A Domain Name Server (DNS) stores information that translates a domain’s name into its IP address. A DNS also stores other “records”, such as the MX record, SPF record, DMARC record, DKIM record, etc. Every time a server or a device reaches out to fetch information from the DNS (regardless of its type), it is considered a “lookup” – the device is looking up the information from the DNS server.

For example, when fetching the IP address of the domain, the server will perform a DNS lookup and grab its IP. This will be considered as a single DNS lookup. However, as in the case of a mail server, it performs multiple lookups to fetch a domain’s IP information, its MX record, its SPF record, the DMARC record, and other things. Each of these is counted as a DNS lookup.

Now, take the same concept and apply it to an SPF lookup. If you understand SPF records, you would know that it uses “mechanisms” that define the type of value. The common tags you see in an SPF record are usually v, a, mx, ip4, ip6, include, redirect, and all. Some of these tags perform further DNS lookups, while some of them don’t (this has been discussed in detail ahead in this post).

Right now, all you need to understand is that the SPF record performs DNS lookups, and if the number of SPF lookups exceeds greater than 10, it results in the “PermError too many DNS lookups” error.

The SPF DNS Lookup Limit: What and Why

According to the documentation by the Internet Engineering Task Force (IETF) – an organization that defines the rules and standards for email authentication protocols – the SPF lookup limit is limited to only 10. This is what they have to say in their RFC7208 documentation:

SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the “include” mechanism or the “redirect” modifier. If this number is exceeded during a check, a PermError MUST be returned. The “include”, “a”, “mx”, “ptr”, and “exists” mechanisms as well as the “redirect” modifier do count against this limit. The “all”, “ip4”, and “ip6” mechanisms do not require DNS lookups and therefore do not count against this limit.

This explains why the “PermError too many DNS lookups” error occurs. But what is the reason for this 10-SPF-lookup rule?

A maximum of 10 SPF DNS lookup queries are allowed per SPF record. This rule has been implemented to tackle DOS attacks. A Denial of Service (DoS) or a Distributed DoS attack is where a single, or multiple compromised devices are used to generate and send data packets to a single server or endpoint, making it extremely slow, or in some cases, completely incapable of performing its regular functions.

Similarly, if an unlimited number of SPF lookups were allowed, hackers could do the same with the DNS server and take them out with the help of sending emails with huge SPF lookup requests.

You can better understand the purpose of the 10-SPF-lookup-limit with this example:

1. A malicious actor creates an SPF record on a specific domain named “malware.com” with numerous references to another domain, let’s say “itechtics.org.”
2. Using the “malware.com” domain, they send a plethora of emails to mailboxes hosted by various email service providers with SPF implemented.
3. The service provider then queries the DNS for “itechtics.org” on receiving such emails.
4. As it involves numerous email providers, it amplifies the traffic and becomes a DoS attack at “itechtics.org”.

That said, if this could happen, the source of the attack would remain anonymous. You can see how a malevolent actor takes use of the email authentication system this way. Therefore, limiting the amount of DNS lookups helps to reduce the risk of such DOS attacks.

How to check if domain exceeds SPF lookup limit

There are plenty of DNS lookup tools available online that can be used to fetch your domain’s SPF records. You can use our Email Validation Tool that looks up SPF as well as other information for the queried domain.

If your domain exceeds the 10-lookup limit for SPF queries, it will inform you of such by showing a red cross beside the “SPF Record should be resolved within 10 DNS queries” check.

How the SPF lookup counter works

If you find that your domain is exceeding the SPF lookup limit, you can optimize your SPF record to fix the error. However, before you do, you must understand how the SPF lookup counter works. When you understand what counts as a DNS lookup, you will know about the changes required in your SPF record.

As discussed earlier in this post, certain mechanisms inside the SPF record perform additional DNS lookups. The table below details this information:

To explain it further, the “a” mechanism means to validate if the IP address equals any of the DNS A records of the domain. For that purpose, to compare the two, the SPF record needs to perform a DNS lookup for an A or a AAAA record. Therefore, it counts as a DNS lookup query. The same logic applies to the rest of the mechanisms that count as 1 additional DNS lookup.

That said, an “include” mechanism can count as more than one additional DNS lookup. The “include” mechanism defines the SPF records of the declared domain as its own. If the SPF record of the declared domain had further “include” records, each of those would be considered as an additional DNS lookup, and be counted.

This is known as nested include. However, it only goes till this tier. If each of those domains further had “include” mechanisms, they would not be considered, or counted.

This concept could be better understood with this example:

I have a domain "itechtics.com" which has the follownig SPF record:

v=spf1 +a +mx ip4:65.17.88.102 include:domain1.com include:domain2.com ~all

Here, the SPF lookup count would be 4: a, mx, and 2 include. However, this would be true if "domain1.com" and "domain2.com" did not have the "include" mechanism,or next include.

However, if each of those domains had used he "include" mechanism twice, the total number of SPF DNS lookups would have gone up to 8. How? The "domain1.com" lookup would count as 1 lookup, but has 2 additional lookups, reaching 3. The same logic would apply to "domain2.com". Including "a" and "mx", the total tally would go up to 8.

With this understanding, you would now understand how to drop the DNS lookup counts for your SPF record so that both SPF and DMARC policies are aligned.

How to fix “PermError too many DNS lookups”

Reduce the SPF mechanisms

The obvious answer is to reduce the number of mechanisms you are using in your SPF DNS record. This means removing the mechanisms that are counted as a DNS lookup query, which we have discussed above.

This also applies to the redundant domains that exist within an SPF record. If more than one SPf mechanism is pointing toward the same domain, you can simply eliminate it. Note that two (or more) domains in the “include” mechanism might be different on the surface, but the nested “include” domains could be pointing toward the same domains. Therefore, consider those as well.

You can simply eliminate the “include” and the “redirect” mechanisms, which will disclude other domains from the SPF record. However, this can negatively impact your email delivery, since emails can be blocked from reaching the recipient’s inbox if the SPF is not configured correctly, or all domains are not included. To overcome this obstacle, there are several approaches you can take.

SPF Record Flattening

“SPF Record Flatening” refers to the process of replacing the domain value after the “include” mechanism in the SPF record with the actual IP addresses in the SPF record of that domain. Since the mechanisms that define the IP address (ip4 and ip6) do not perform a DNS lookup, they aren’t counted as one.

Here is an example of the concept. Consider the following SPF record for the domain “itechtics.com”:

v=spf1 +a +mx ip4:65.17.88.102 include:domain1.com ~all

Now, let us assume that the domain “domain1.com” has the following SPF record:

v=spf1 ip4:65.17.88.133 ip4:88.17.90.22 ~all

Now, if I were to perform “SPF record flattening” for “itechtics.com”, I would replace “include:domain1.com” with the IP addresses inside the SPF record for “domain1.com”, and the final SPF record for “itechtics.com” would look like this:

v=spf1 +a +mx ip4:65.17.88.102 ip4:65.17.88.133 ip4:88.17.90.22 ~all

With record flattening, I just reduced one count of DNS lookup. However, this method is not always feasible as it has its downside.

Imagine if there were a plethora of nested “include” mechanisms, and you must include the IP addresses for all the domains. The SPF record might start looking something like this:

v=spf1 ip4:216.99.5.67 ip4:216.99.5.68 ip4:202.177.148.100 ip4:203.122.32.250 ip4:202.177.1 ip4:213.199.128.145 ip4:207.46.50.72 ip4:207.46.50.82 ip4:65.55.42.224 ip4:207.46.100.0/24 ip4:207.46.163.0/24 ip4:65.55.169.0/24 ip4:157.56.110.0/23 ip4:157.55.234.0/24 ip4:213.199.154.0/24 ip4:213.199.180.128/26 ip4:52.100.0.0/14 ip4:157.56.112.0/24 ip4:207.46.51.64/26 ip4:64.4.22.64/26 ip4:40.92.0.0/15 -all

This would surely exterminate the “PermError too many DNS lookups” error since the SPF record does not include even a single DNS lookup query. However, this wouldn’t entirely solve your problem. You see, while one problem would be solved, another will rise in its place.

If even a single IP address for a domain is changed, your whole SPF record will become invalid. Moreover, updating such an SPF record would become a nightmare. This method is susceptible to human error, and besides, you will need to monitor your SPF records all the time.

While SPF record flattening is a great way to overcome the DNS counts, it might not always work. At other times, when the total number of IP addresses is limited, it might be your go-to solution.

Avoid using the “ptr” mechanism

The “ptr” mechanism is used to perform a reverse DNS lookup, which means finding the domain name using its IP address. Using it in an SPF record isn’t really required, and adding it unnecessarily would resulting an additional DNS query.

Remove the “a” and “mx” mechanisms

The “a” and “mx” mechanisms in an SPF record are valid entries, but not always necessary.

The ‘a’ mechanism implies that the IP address of the sending domain should be allowed to transmit messages for the domain name in the header of the email. This approach is usually wrong because that IP address is the domain’s web host IP address. While this can sometimes be a legitimate use, it is not normal.

The ‘mx’ mechanism indicates that the IP addresses for the domain’s MX records should be allowed to transmit emails for the domain name found in the header. This logic can be incorrect because the MX hosts are used for ingress traffic, and outbound emails do not go through that same environment. 

Many times this is completely legitimate, but it is incorrect for most large hosted environments such as M365, Google, Proofpoint, etc. If your SPF record already has an allowance for traffic from your email provider, the addition of the “mx” mechanism in an SPF record is incorrect.

Include active domains only

Some domains might have become inactive over the years. Make sure to only include the active domains in the “include” mechanism and delete the rest.

By active domains, I mean the domains that are regularly sending emails on your domain’s behalf.

Conclusion

Although performing DNS lookups is a great way to transfer information from one server/service to the next, it also has its caveats, which hackers can benefit from. Therefore, all servers must exercise the default limit for an SPF DNS lookup, which is 10.

If the number of SPF lookups exceeds 10, then the SPF check would fail, which in turn would fail the DMARC test. In such a case, the decision to let the invalidated email through would then rely on the configured DMARC policy.

That said, there are ways to tackle the “PermError too many DNS lookups” error without compromising the IP addresses in the record, or the domains.