Hard Filtering SPF

I've talked before here about SPF. To recap it is an anti-spam measure and is one we've used for a long while. It works like this. A domain, the-hug.org say, publishes an SPF record as one of its DNS records. This SPF record indicates which IP addresses will send email purporting to come from the-hug.org.

So in the case of the-hug.org it looks like this:

which means we send email from our server with one IPv4 address and a block of IPv6 addresses plus we may send email from home.the-hug.net (which is the FTTC coming into our house) and two other servers misc.the-hug.net and map.the-hug.net but from nowhere else, the "-all" clause.

A receiving SMTP server receives mail from us, checks to see if we have an SPF record and, if we have, checks that the sending SMTP server is in the list specified in the SPF record. If it's not it may choose to treat the email as spam.

Now you don't have to publish an SPF record, but having one counts as an uptick in spam scoring so it's worth doing and is recommended by all the big ISPs, especially if you send bulk email (as we do).

I'm bringing this up now because we do check SPF on our incoming mail for both us and our customers whose email we host but until a couple of weeks ago we treated it as a soft fail. So SpamAssassin, which is our core spam filter, would increase the spam score of an email if the sending domain had an SPF record but the sender's IP address wasn't valid according to the record.

That was fine as far as it went but both we and, perhaps more importantly, some of our customers were finding that some spam was getting through and when I investigated a common feature of these emails was that the the sender's IP address wasn't valid according to the sender's domain SPF record.

Did I dare turn on hard bouncing in these circumstances? I did some reading and the general conclusion from other IT professionals was that this was the right thing to do. So we did that and monitored the logs to see what was bounced.

The good news was that quite a lot of spam was bounced. The interesting news was that some valid email was also bounced because the senders' IT department had screwed up the SPF record in some way.

On a couple of occasions early on I did contact the sender's IT department to tell them they had an issue but after being told by Comic Relief IT no less that there was "nothing wrong with their mail" (there was, their SPF record somehow pointing to a 127.0.0.0/8 IP address) I stopped bothering. (Eventually they appeared to concede there was an issue as they've now deleted their SPF record.)

Anyway, the bottom line is that hard filtering on SPF is worthwhile, you just need to take a fairly strong approach to any issues. If a user contacts you to say the mail isn't getting through then say that the problem is on the sender's end but you're happy to talk the sender's IT department through the issue if necessary.