email like it’s 1995
The old joke
There used to be a joke that people who didn’t ‘get it’ would send an email and then call the recipient up to check if it had been received.
Now the joke’s on us
Now, I am that joke. I think we all are. Email has reached a point where it’s just not reliable enough for business use.
Update 17 Dec – clearly I’m not the only one to notice this. Babbage over at The Economist shares my pain.
Spam
The heart of the problem is of course spam
False positives
The real trouble though is that the cure is maybe worse than the disease. Our reaction to spam, and the systems that we use to filter it out are destroying the utility of email.
I can think of numerous examples in the last year or so when I’ve emailed somebody and got no reply, only to hear sometimes weeks later that my message got caught in their ‘junk folder’.
Bacn
One of the problems with spam is how to define it. There’s obvious stuff from the peddlers of dodgy drugs, body enhancements and financial scams – the stuff that none of us really want to see in our inboxes. Then there’s the stuff from marketing droids, where you had to give your email address to register for an event, but never really want to hear from them again. Then there are the mailing lists that you chose to subscribe to, but don’t want to read right now (for which the term ‘Bacn‘ was coined – it’s fine for breakfast, but you don’t want to eat it all day).
Caught in the crossfire
The problem is when legitimate mail, that’s neither spam nor bacn get’s classified as junk. I noticed last week that a perfectly normal email from a colleague had found it’s way into my Postini quarantine. When I fished it out of limbo I had a look at the headers to see if anything odd was happening. I found this:
Received-SPF: error (google.com: error in processing during lookup of [email protected]: DNS timeout) client-ip=74.125.149.185; Authentication-Results: mx.google.com; spf=temperror (google.com: error in processing during lookup of [email protected]: DNS timeout) [email protected]
Oops – looks like it’s time for me to set up SPF.
SPF
Sender Policy Framework (SPF) is a system where a domain can nominate approved senders for its email as a means to cut down spam. Sadly Google Apps doesn’t configure this for us (hardly a surprise as they don’t manage our DNS), and nor do they have any tool to help you configure SPF. Luckily there’s a bunch of stuff out there on the web, which led me to the following settings (for our Postini outbound servers, and Blackberry):
@ TXT v=spf1 ip4:207.126.144.0/20 ip4:64.18.0.0/20 ip4:74.125.148.0/22 ptr:blackberry.com ~all
This seemed to work fine, and I started seeing the good news in email headers from colleagues:
Received-SPF: pass (google.com: domain of [email protected] designates 74.125.149.180 as permitted sender) client-ip=74.125.149.180;
Troubles not over
Sadly SPF doesn’t seem to cut it with Postini, and a few days later another colleague told me that an email I’d sent him had been quarantined (it was a reply to an old thread asking if he’d made any further progress). I could see nothing more that I could do, so I raised a support case with Google/Postini. Here’s their reply in its full horror:
Chris,This is a known issue and our team is working to correct the issue. The Problem is that you internal mail goes out to the internet and back in through Postini. The Postini system is designed to be very critical on spoofed mail, because a huge percentage of spoofed mail is spam. I have no ETA as to when this will be corrected, but we do have a work around.We have a feature called IP lock. Turning this feature on will allow you to specify the IP’s you allow to spoof your domain. Once you’ve turned this on and added the Google Apps IP ranges you can add your domain to the organization level approved senders list and not have to worry about unauthorized spoofed mail making it through to your end users.Here’s a link that provides configuration instructions for the IP Lock feature.http://www.postini.com/admindoc/secur_iplock.html
and here is are our IP ranges…
64.233.160.0/19
Range:
64.233.160.0 to
64.233.191.255
216.239.32.0/19
Range:
216.239.32.0 to
216.239.63.255
66.249.80.0/20
Range:
66.249.64.0 to
66.249.95.255
72.14.192.0/18
Range:
72.14.192.0 to
72.14.255.255
209.85.128.0/17
Range:
209.85.128.0 to
209.85.255.255
66.102.0.0/20
Range:
66.102.0.0 to
66.102.15.255
74.125.0.0/16
Range:
74.125.0.0 to
74.125.255.255
64.18.0.0/20
Range:
64.18.0.0 to 64.18.15.255
207.126.144.0/20
Range:
207.126.144.0 – 207.126.159.255
Regards,
Redacted
Technical Support Engineer III
Wow – how incredibly unhelpful is that?
Postini could have a system that works (and that respects SPF showing that mail has originated from their own servers) – but they don’t.
The support engineer could have configured IP Locks for me – but he didn’t.
The support engineer could have provided me with the script that I needed to run – but he didn’t.
So… off to the documentation, and a bit of trial and error to discover that I needed to run the following batch script in the Postini services console (Orgs and Users > Batch):
# add Postini IPs
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:64.233.160.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:216.239.32.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:66.249.80.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:72.14.192.0/18
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:209.85.128.0/17
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:66.102.0.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:74.125.0.0/16
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:64.18.0.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:207.126.144.0/20
# add BlackBerry IPs
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:206.51.26.0/24
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:193.109.81.0/24
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:204.187.87.0/24
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:216.9.240.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:206.53.144.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:67.223.64.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:93.186.16.0/20
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:68.171.224.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:74.82.64.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:173.247.32.0/19
addallowedip [capitalscf.com] Account Administrators, capitalscf.com:178.239.80.0/20
# show what happened
showallowedips [capitalscf.com] Account Administrators
Note how complex the OrgName is compared to the examples in the documentation.
It’s also completely unclear to me whether IP Lock should be used in combination with having my domain as an ‘Approved Sender’ or not? For the time being I do (I can always tighten things up if a flood of spam ensues).
Fix this Google
I only use Postini for its ability to add footers with a disclaimer, and this whole thing has me wondering if it’s more trouble than it’s worth – after all the anti-spam in regular gmail is pretty good (and it’s a question for another day why that functionality isn’t better integrated into Postini quarantine – still, they’ve only had 3 years – how hard can it be?).
Or will Facebook fix it for us all?
There has been much fuss in the last few weeks about Facebook launching an integrated email/messaging service. I personally can think of few things I’d less like to do than use Facebook more. I don’t see them as a business service, and I’ve yet to hear anything about how they will deal will the spam problem. But many others seem to think what they’re doing is the shape of things to come – so I could be wrong.
[update 22 Nov] Google/Postini have now been in touch to say that I was apparently the victim of ‘a recent incident that we have since resolved’. PIR_11_Nov_16_Spam_Quarantine. This leaves me wondering how often this sort of thing happens and I don’t notice, and why the first support engineer went straight down the IP lock route?
[update 22 Nov #2] Looks like I missed a bunch of BlackBerry IP ranges first time around – one of the inherent problems with using such a fragile approach. The definitive list is here. At this stage I’m quite tempted to turn IP Lock off given that Google have come clean about their incident.
Filed under: could_do_better, grumble, howto, software | 7 Comments
Tags: bacn, batch, blackberry, business edition, DNS, email, fail, false positive, GABE, GAPE, gapps, google, Google Apps, IP Lock, Postini, premier edition, quarantine, spam, SPF, support
Subscribe
Recent Comments
Tim Coote on Ross Anderson RIP iain on Skiing in Espace Killy (Val… Chris Swan on Getting more from a British Ga… Murray Cowell on Getting more from a British Ga… JL on AI MacGuffin 
Pinboard.in bookmarks- Open Source Security (OpenSSF) and OpenJS Foundations Issue Alert for Social Engineering Takeovers of Open Source Projects
- AI isn't useless. But is it worth it?
- Pluralistic: The true post-cyberpunk hero is a noir forensic accountant
- Subaru battery drain due to missing 3G network
- Paying maintainers: the HOWTO
- SARS-CoV-2 airborne infection probability estimated by using indoor carbon dioxide | Environmental Science and Pollution Research
- An unbiased evaluation of environment management and packaging tools
- Computer says no: British judge refuses to cancel divorce resulting from computer mistake
- Philosophy Museum
- Royal Navy offer must improve to attract and retain staff – you can't motivate an empty chair
Chris Swan's Weblog 
Some comments on Hacker News
Much of the problem is that anti-spam systems shouldn’t “swallow” email they don’t like. i.e. they tell the sender that the delivery has gone OK, but quietly bin or file the email into a Junk folder.
My company’s anti-spam system has a default reaction of rejecting a spammy-looking email at delivery time, so that a polite apology is sent back immediately to the sender of a legitimate message, with a phone number. There’s no black hole, just an immediate bounce back.
I suspect the reason commercial solutions don’t do this is that they don’t want to be called out so quickly on their false positives, or to give them a chance to “fix” a general mistake and redeliver later on. The upshot is exactly what you said, that a sender thinks an email has been delivered when it hasn’t, which is a much worse failure case (for me) than the “embarrassment” of bouncing a legitimate sender’s message.
Spammers *have* made email less reliable, but anti-spam services trying to spare their users’ blushes have made it even worse.
One reason why even well-designed antispam systems will swallow email is to avoid “joe-jobbing” the envelope sender with reject notices, given that the envelopes on SPAM are almost always forged.
ugh, SPF. http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/smtp-spf-is-harmful.html
better, and you wouldn’t have this problem: http://en.wikipedia.org/wiki/DomainKeys
My setting up SPF wasn’t a vote in its favour. I did it merely because Postini was clearly indicating that the lack of SPF was hurting.
I took a look at DKIM at the same time, but it seems to be something that needs support in both DNS (which I can control) and the SMTP server (which I can’t do anything about – I need Google to fix that, and judging by the age of some forum posts it doesn’t seem to be a priority).
PS This might be a good time to mention that I find it peculiar that gmail supports verification for email from Paypal and eBay, but doesn’t bother with its own bulletins (which can often end up in the spam folder).
This shouldn’t be any surprise, but it turns out that IP Lock does bad things to MailChimp. One of my colleagues has just wasted far too much time trying to get a test message through (so that we can send out invites for our Xmas drinks event).
IP Lock now off. I could have added the MailChimp IPs I guess, but this whack a mole game is getting old far too quickly.
This guide might be useful if you find stuff regularly going into your Gmail spam folder when it shouldn’t (I found that adding the sender to my address book didn’t sure the issue).