Information Security
| On
January 1, 2018 6:07 pm

Lessons Learned from the US Federal Government’s Ongoing Deployment of SPF and DMARC

By Sean Whalen

SPF and DMARC are standards that describe how the origins of email messages should be verified, to prevent email spoofing. I spent some free time over the past few weeks creating checkdmarc , a Python 3 module and command-line interface that can validate and troubleshoot SPF and DMARC records across multiple domains, with the intent of building it into a web application that will process DMARC reports. The Department of Homeland Security recently launched an initiative to deploy SPF, DMARC, and other best practices on most federal agency domains by issuing BOD 18-01. This created the perfect case study of common challenges and mistakes when deploying SPF and DMARC across very large organizations, and even a few small ones.

2018-01-30 update: I have made many improvements to my script, corrected a few of my own misconceptions about DMARC I had in this post, and switched to updated results from 2018-01-28.

DHS Binding Operational Directive 18-01 (BOD 18-01)

On October 17, 2017 the US Department of Homeland Security (DHS) issued Binding Operational Directive 18-01 (BOD 18-01). This directive requires most federal agencies to deploy cyber hygiene best practices to secure their websites and email services, including SPF, DMARC, STARTTLS, and HTTPS with HTS:

  • “Within 30 days of BOD issuance (November 16th, 2017), submit an “Agency Plan of Action” to and begin implementing the plan.
  • Within 90 days (January 15, 2018) of BOD issuance:
    • Configure all internet-facing mail servers to offer STARTTLS.
    • Configure all second-level domains to have valid SPF/DMARC records, with at minimum a DMARC policy of “p=none” and at least one address defined as a recipient of aggregate and/or failure reports.
  • By 120 days (February 13, 2018) after BOD issuance:
    • Ensure all publicly accessible Federal websites and web services provide service through a secure connection (HTTPS-only, with HSTS).
    • Identify agency second-level domains that can be HSTS preloaded, and provide a list to DHS at
    • Disable SSLv2 and SSLv3 on web and mail servers.
    • Disable 3DES and RC4 ciphers on web and mail servers.
    • Within 15 days of the establishment of a centralized NCCIC reporting location, add DHS as a recipient of DMARC aggregate reports.
  • Within one year (October 16, 2018) of BOD issuance, set a DMARC policy of “reject” for all second-level domains and mail-sending hosts.”

Unfortunately, Intelligence Community and/or National Security systems are not required to comply with BODs, although they would still benefit from applying them.

Enabiling STARTTLS on mail servers allows senders to establish an encrypted connection before sending email. Enabling HTS on a web server tells clients to only use an encrypted (HTTPS) connection whenever they are accessing the site in the future. Disabling obsolete, weak cypher suites prevents easy exploits such as POODLE from working. SPF and DMARC are standards that work in tandem to prevent email spoofing. This blog post will focus on these standards. However, all of the best practices are key parts of defense in depth.

What is email spoofing?

Emails are spoofed when the from field of the message appears to be from someone other than the actual sender. For example, an attacker sending a phishing email can set the from address to match the target’s CEO. In many organizations, the target’s mail client will even helpfully display a known email address as a name and picture from their internal profile, which makes the phishing email look more legitimate.

Spoofing is a powerful, simple tool for attackers, but there are legitimate reasons for spoofing email from addresses. If an organization is using an outside email marketing service, helpdesk, or any other outside service that sends email, there is a need to make the items appear they are coming from their trusted domain, and not the third party, so recipients know who is responsible for the messages.

So how can a recipient mail server differentiate from legitimate and illegitimate spoofing? SPF, DKIM, and DMARC, of course!

What is SPF?

The Sender Policy Framework (SPF) is a standard for listing mail servers that are authorized to send email from your domain. It is defined in RFC 7208. The domain owner publishes a specially formatted TXT record in the DNS zone containing a list of authorized servers.

Details about the exact record format can be found here.

When a receiver mail server receives an email, it checks to see if the domain in the MAIL FROM part of the SMTP envelope has published a SPF record. If a record is found, the IP address of the sending mail server is compared to the list. Note that SPF does not check the from header that the user sees, and the SMTP envelope is not part of the message headers. so it is easy to bypass SPF and still spoof email to the user. The best illustration I have found of this comes from the great guide at that explains exactly how to bypass SPF:

An illustration of email spoofing that bypasses SPF checks
Used under fair use for educational purposes

Also, It is up the receiving mail server to decide what to do with an email if the SPF check fails. SPF does not specify an action for the recipient mail server to take in the event of a validation failure. DMARC fills these gaps, and more.

Common mistakes when deploying SPF and how to avoid them

Not deploying SPF (and DMARC) on domains that do not send email

An attacker can still spoof email to make it look as if it is coming from a domain even if it doesn’t have MX records and/or you don’t use it for email at all; these domains are the easiest to deploy SPF on: just add a TXT record containing "v=spf1 -all"

Including the same servers more than once

There are multiple mechanisms to allow a server, but a server only needs to be allowed by one of them. For example, if you use the ipv4 mechanism to state that a whole netblock is allowed allowed to send email, you don’t need to use an a mechanism to include a sever by hostname.

Using the wrong mechanism

The most common mechanisms in use are ipv4, ipv6, mx, a, and include

  • ipv4: used to specify a single IPv4 address or CIDR  netblock
  • ipv6:  used to specify a single IPv6 address or CIDR  netblock
  • mx: used to specify the servers listed in a domain’s MX resource records, not for allowing individual mail servers by hostname. If a mx mechanism includes a value (e.g., the MX records located at that value are queried; if a value is not included with the mechanism (e.g. mx), the domain’s own records are queried
  • a: used to specify a server by hostname, or a domain’s own A/AAAA records if no hostname is included with it; this is frequently used to allow web servers to send mail, since they are not included in MX records
  • include: used to include SPF mechanisms from another domain. This is frequently used to include lists of ipv4/ipv6 mechanisms from third party services that send email as the domain including cloud email and marketing services, such as Office 365, Google, or Salesforce; multiple include statements can bu used (e.g. include mechanisms do not include the all value; to include mechanisms and the all value (e.g. to have a standard configuration for all of your domains), use the redirect= modifier at the end of the record – this modifier can only be used once per record

Using too many mechanisms that trigger DNS lookups

RFC 7208, section 4.6.4 limits the combined number of a, mx, include, exists,  and redirect mechanisms and modifiers used in a SPF record to 10, including any mechanisms from included records –  the total number of MX records returned per mx mechanism is also cannot exceed a separate limit of 10; this is done to prevent DNS-based denial-of-service (DoS) attacks

To reduce the number of DNS lookup mechanisms in a record:

  • If you have sending server IP addresses that won’t change often, consider using ipv4 or ipv6 mechanisms, which don’t trigger DNS lookups, instead of mx or a.
  • Have third party services send emails from dedicated subdomains; for example, if a billing services vendor currently sends emails as, have them send from, and make that email address an alias if you want to receive replies at; this way gets its own SPF record and limits. Having MX records for these subdomains improves en emails trustworthiness to many gateways, even if you don’t actually have mailboxes for that subdomain.

Using the ptr mechanism

According to RFC 7208, section 5.5, the ptr mechanism should not be used

What is DKIM?

DKIM is a stronger method of validating that an email was sent from an authorized server than SPF.  The sending server signs outgoing mail with a private key, and places the signature in a message header and the receiving server validates the signature using a public key published in DNS, as defined in RFC 6376 – DomainKeys Identified Mail (DKIM) Signatures. Messages may have multiple DKIM signature headers if they have been forwarded through multiple gateways.

Take a look at this example, lightly modified from Wikipedia:

DKIM-Signature: v=1; a=rsa-sha256;; s=brisbane;

     c=relaxed/simple; q=dns/txt; l=1234; t=1117574938; x=1118006938;





A verifier queries the TXT resource record type of

Here, is the author domain to be verified against (in the d tag), brisbane is a selector given in the s tag, and _domainkey is a fixed part of the protocol.

There are no CAs or revocation lists involved in DKIM key management. The selector is a straightforward method to allow signers to add and remove keys whenever they wish. Long lasting signatures for archival purposes are outside DKIM’s scope. Other tags are visible in the example are:

  • v – version
  • a – signing algorithm
  • d – domain
  • s – selector
  • c – canonicalization algorithm(s) for header and body
  • q – default query method
  • l – length of the canonicalized part of the body that has been signed
  • t – signature timestamp
  • x – expire time
  • h – list of signed header fields, repeated for fields that occur multiple times

What is DMARC?

DMARC is the Domain-based Message Authentication, Reporting, and Conformance standard, as described in RFC 7489. It consists of a specially-formatted DNS TXT resource record published by a Domain Owner, that tells mail servers what to do when they encounter a message that appears to be from that domain, but fails SPF and/or DKIM. It also provides methods for mail receivers to notify Domain Owners when mail fails the DMARC check.

DMARC records are TXT resource records of a subdomain of a second-level domain called _dmarc, i.e. Unlike SPF, DMARC options apply to all subdomains, under the base domain (i.e. DMARC records are composed of tags and values that are joined by =, and delimited  by ;. All DMARC records start with v=DMARC1.

The most important and commonly used DMARC tags are p, sp, pct, rua, and ruf.

The p tag sets the DMARC policy, which specifies the policy to be enacted by the Receiver at the request of the Domain Owner on mail that is supposedly from the Domain Owner’s domain fails the DMARC check. The policy applies to the domain and to its subdomains, unless subdomain policy is explicitly described using the sp tag. Possible policy values include:

Policy Description
none The Domain Owner requests no specific action be taken regarding delivery of messages. This is the implicit default policy.
quarantine The Domain Owner wishes to have email that fails the  DMARC mechanism check be treated by Mail Receivers as suspicious. Depending on the capabilities of the Mail Receiver, this can mean “place into spam folder”, “scrutinize with additional intensity”, and/or “flag as suspicious”.
reject The Domain Owner wishes for Mail Receivers to reject email that fails the DMARC mechanism check. Rejection should occur during the SMTP transaction.

The pct tag value is an integer percentage of messages from the Domain Owner’s mail stream to which the DMARC policy is to be applied. However, this must not be applied to the DMARC-generated reports, all of which must be sent and received unhindered. The purpose of the pct tag is to allow Domain Owners to enact a slow rollout of enforcement of the DMARC mechanism. The implicit default value is 100.

I strongly recommend leaving pct set at 100 (ideally by not setting it at all), and using the policies to slow the rollout of DMARC instead. That way, the same policy is applied to all mail, not an arbitrary percentage.

The rua tag specifies comma-separated delivery locations for aggregate reports, which are typically daily emailed reports of compressed XML files that describe how many emails a Receiver observed from a given mail server IP address that failed the DMARC check.

The ruf tag specifies comma-separated delivery locations for forensic reports, which are emails with copies of the original emails that failed the DMARC check attached. These are particularly useful for allowing Domain Owners to examine phishing emails sent third parties as well as their own organization, that were made took look like they came from their domain. However,  many Receivers only send aggregate reports, if they send DMARC reports at all; others may only provide the email headers, and not the full content.

Both rua and ruf tags are expected to have a value of a list of comma-separated DMARC URIs, the only URI type currently supported in the DMARC specification is mailto. Each email address in the list must be prefixed with mailto: in order to be a valid DMARC URI.

Additionally, if an email address in the rua and ruf tag values has a domain with a base domain that is different than the base domain that _dmarc record is a subdomain of, an authorization record must be added on the base domain of the email address. For example, if had a DMARC record like this: TXT "v=DMARC1; p=none;"

because does not match the base domain of receivers will check for the existence of the following record, before sending a report to TXT "v=DMARC1"

Likewise, if was using a third party service like Agari to receive, store, and process DMARC reports: TXT "v=DMARC1; p=none;"

Then the following DNS records need to be added to the DNS zone (which they do when a domain is added to an account): TXT "v=DMARC1" TXT "v=DMARC1"

These records are defined in RFC 7489, section 7.1. and are required when reports are directed external domains, so that someone can’t flood your DMARC inbox with reports for a domain that you do not operate by setting up a domain like,  adding the record: TXT "v=DMARC1; p=none;;"

And sending a ton of emails as that purposefully fail the DMARC check to many different organizations.

Return Path has an great series of blog posts of how to read DMARC reports. Check it out once you start getting reports.

What is DMARC alignment?

Passing a DMARC check requires passing SPF or DKIM checks, but DMARC goes beyond by also requiring that the identifiers for at least one of these checks are aligned, so spoofing is prevented.

Passing The signature in the DKIM header is validated using a public key that is  published as a DNS record of the domain name specified in the signature The mail server’s IP address is listed in the SPF record of the domain in the SMTP mail from envelope header
Alignment The signing domain aligns with the domain in the message’s from header The domain in the SMTP mail from envelope header aligns with the domain in the message’s from header

DMARC without DKIM will break (some of) your email delivery

You must sign outgoing email at your gateway with a unique DKIM key before publishing a DMARC policy oa DMARC quarantine or reject policy. Simply allowing a sending server via SPF in not enough.

Why? Forwarded emails. Not the kinds of forwards that users send from their mail clients or webmail, but the forwards from mailbox rules. These forwards take place during SMTP delivery, and are common when someone changes addresses, or when a company gets acquired.

When forwards happen at the SMTP level, the envelope mail from will not match the header from, so the SPF indicators are not aligned, and the DMARC check will fail, unless the original sending mail server signed the message with DKIM, and the forwarding server(s) did not tamper with it. DKIM signatures are included in forwarded messages, because they are in the message headers.

The state of .gov in early 2018

I made some improvements to my scripts, and reran the tests on 2018-01-28 to see how the federal government had improved, now that the deadline to at least have a p=none DMARC policy had passed.

Unfortunately, due to the bug fixes I’ve made in between tests, the results are not comparable, but these results are more accurate.

The generated files can be downloaded here.

A pie chart of SPF deployment on .gov domains as of 2018-01-28
A pie chart of DMARC deployment on .gov domains as of 2018-01-28
A column chart of DMARC deployment by policy on .gov domains as of 2018-01-28;
Note: “none” is a valid DMARC policy – the “(empty)” chart value represents domains that have missing or invalid DMARC record
Invalid/Missing 429 645
Valid 881 665
DMARC Policy (p value) Count of Domains
none 463
quarantine 10
reject 192
Missing/Invalid DMARC record 645

High profile .gov domains

Rather than trying to examine all 1,310 .gov domains in a single blog post, let’s focus on a much smaller subset of 44 domains from agencies of political, social, and/or bureaucratic significance:

Domain Agency SPF DMARC National Archives and Records Administration Valid p=reject Congressional Budget Office Uses 13/10 DNS lookups Missing record Centers for Disease Control Valid p=none US Census Bureau Valid does not indicate that it accepts DMARC reports about Centers for Medicare and Medicaid Services Valid p=none Department of Commerce Valid does not indicate that it accepts DMARC reports about Consumer Financial Protection Bureau Missing record does not indicate that it accepts DMARC reports about Department of Commerce Valid p=none Department of the Interior Valid p=none Department of Transportation Valid p=none Department of Education Valid p=none Environmental Protection Agency Valid p=none Federal Aviation Administration Valid p=none Federal Communication Commission Valid p=none Food and Drug Administration Valid p=none Federal Deposit Insurance Corporation Valid p=reject Federal Reserve Valid p=reject Federal Emergency Management Agency Valid p=none Federal Emergency Management Agency Valid does not indicate that it accepts DMARC reports about General Services Agency Uses 13/10 DNS lookups pct value is less than 100. This leads to inconsistent and unpredictable policy enforcement. Federal Healthcare Exchange Valid p=reject Department of Health and Human Services The ptr mechanism should not be used – p=none Department of Housing and Urban Development Valid p=none Immigration and Customs Enforcement Missing record Missing record Department of the Interior Valid p=none Internal Revenue Service DNS servers fail to respond to query for p=none Library of Congress Valid Missing Record National Aeronautics and Space Administration Valid p=none Medicaid Valid p=reject Medicare Valid p=none National Archives and Records Administration Valid p=none National Aeronautics and Space Administration Valid p=none National Credit Union Administration Valid p=none National Highway Traffic Safety Administration Valid does not indicate that it accepts DMARC reports about National Park Service Valid p=none Internal Revenue Service Valid p=none Office of Management and Budget Valid Invalid fo tag: Values 0 and 1 are mutually exclusive Securities and Exchange Commission Valid p=none Social Security Administration Valid p=quarentine US Citizenship and Immigration Services Missing record Missing record US Department Department of Agriculture Valid Missing record US Mint Valid p=none US Patent and Trademark Office Valid p=none Department of Veteran’s Affairs Valid p=none

The US Intelligence Community

Some very significant agencies are missing from the list above, because they are members of the Intelligence Community (IC). DHS Binding Operational Directives do not apply to IC and/or National Security systems. However, SPF and DMARC would still be great security controls for them to deploy, as they are much more likely to be targets of spoofed phishing email than other agencies. At the very least, it could provide counterintelligence on phishing attempts sent to Unclassified email systems, because nothing like that has ever happened before. So, let’s check on the deployment of SPF and DMARC in the IC too.

Domain Agency SPF DMARC US Air Force Missing record Missing record US Army Missing record Missing record Central Intelligence Agency Unnecessary SPF mechanisms: Missing record Drug Enforcement Agency Valid p=none Department of Homeland Security Valid p=none Defense Intelligence Agency Valid Missing record Director of National Intelligence Missing record Missing record Department of Defense Missing record Missing record Department of Energy Valid p=none Federal Bureau of Investigation Valid p=reject Department of Homeland Security Valid p=none Department of Justice Valid p=none Shared Department of Defense email system hosted by DISA Valid Unrelated TXT records were discovered. These should be removed, as some receivers may not expect to find unrelated TXT records at

v=spf2.0/pra ~all US Marine Corps Missing record Missing record US Navy Valid Missing record National Geospacial-Intelligence Agency Missing record Missing record National Reconnaissance Office Missing record Missing record National Reconnaissance Office Missing record Missing record Department of State Valid p=none Department of the Treasury Valid p=none US Coast Guard Valid Missing record Department of Justice Valid p=none US Marine Corps Valid Missing record The White House Valid Missing record

Shout out to the FBI for still being the only IC member with a DMARC policy currently set to reject!

IC Support Agencies

These agencies provide services to IC agencies:

Domain Agency SPF DMARC Defense Information Systems Agency Missing record Missing record Defense Logistics Agency Valid Missing record Office of Personnel Management has multiple spf1 TXT records p=none

Defense contractors

Let’s check the major defense contractors for comparison. I also threw in Leidos, a spin-off of Lockheed Martian that provides information security consulting services to a variety of industries, and SpaceX, to see how newer, companies compare to the established giants.

Several of these companies have separate domains for their email and web services. The email domains are protected by SPF and DMARC, but the web domains are not. Although those web domains do not host email services, those domains could still be useful to spoof for phishing.

Domain Company SPF DMARC BAE Systems Missing record Missing record Booz Allen Hamilton (Email domain) Valid p=none Boeing Valid Missing record Booz Allen Hamilton Missing record Missing record General Dynamics Valid Missing record L3 Communications Valid Missing record L3 Technologies Valid Missing record Leidos Valid Missing record Lockheed Martin (Email domain) Valid Missing record Lockheed Martin Missing record Missing record Northrop Grumman (Email domain) Valid Missing record Northrop Grumman Missing record Missing record Palantir Valid Missing record Raytheon Valid p=none SpaceX Valid p=none United Technologies Valid p=none

This post was last modified on February 11, 2019 11:54 pm

Sean Whalen

Sean Whalen is an Information Security Engineer in the healthcare industry, and founder of the InfoSec Speakeasy, specializing in intelligence and malware analysis. Previously, he worked as an intelligence analyst in the defense industry. He has a passion for open source software, and sci-fi.

View Comments

  • Great post! Very interested in seeing update since Jan 15 BOD 18-01 deadline for DMARC records & aggregate reporting. Could you shoot me an email....would like to discuss this further with you. Thanks!

    Cancel reply

    Leave a Comment

  • Informative. I'm trying to setup my own policies and learned from it. Thank you.

    Cancel reply

    Leave a Comment

  • Would you please post an update page to your blog so we can see the results to date. It is almost a year later, and it would be useful to see how the effort to implement DHS Binding Operational Directive 18-01 (BOD 18-01) is going.

    Thank you.

    Cancel reply

    Leave a Comment

  • very helpful post !! but how the directive force US agencies to implement SPF and DMARC( with p= reject) with DKIM instructions, as you mention that "DMARC without DKIM will break (some of) your email delivery", could you explain for us this misunderstanding.
    Thanks a lots

    Cancel reply

    Leave a Comment

  • updated comment:

    very helpful post !! but how the directive force US agencies to implement SPF and DMARC( with p= reject) without DKIM instructions, as you mention that "DMARC without DKIM will break (some of) your email delivery", could you explain for us this misunderstanding.
    Thanks a lots

    Cancel reply

    Leave a Comment

  • sorry i meant "without DKIM instructions"

    Cancel reply

    Leave a Comment

Leave a Comment