Posts Tagged ‘block list’

When Blacklists Die

Wednesday, November 17th, 2010

Blacklists have been popular targets for complaints and criticism for years. Senders complain they are too stringent and lack transparency. The anti-spam community howls with outrage when they’re not as aggressive as they think they should be.

One blacklist in particular, called the five-ten-sg.com block list, has been a thorn in the side of ESPs since 2001 – but not because lots of ISPs use the list to block mail. In fact, they don’t; the list generates too many false positives, and as my colleague Al Iverson demonstrated a few years ago, you’d get significantly better results by randomly blocking any mail from an IP address in which the number 7 appears.

The list operator is a guy named Carl Byington, and I’ve been reading what he has to say about spam and e-mail for years. He’s a smart, reasonable guy who’s always been honest about the nature of his list. He lists sources of bulk e-mail for a broad range of reasons, and he’s quick to agree with anyone who points out that his listing criteria are not useful for filtering decisions in a high inbound e-mail volume production environment. But it’s his list, and he can do with it what he pleases – and ISPs and other network operators are similarly free to ignore it.

ESPs, on the other hand, have been getting an earful from their customers about Fiveten for a long time.

When a sender runs into deliverability problems, they’ll often turn to web sites that offer to look up an IP address on a bazillion block lists all at once. In altogether too many instances, they discover they’re listed by Carl. They’ll fire off a few angry e-mail messages or phone calls to their poor, harried deliverability guy. It always seems to take a few days to explain why the listing is almost certainly not the root cause of their deliverability issue, and to redirect time and energy back to the real issues.

Not all lists are created equal. Some are more important than others, because ISPs find them more useful, and so they become deployed more widely. In the scheme of things, Fiveten is not that important, and a Fiveten listing has no measurable impact on deliverability.

This weekend, Fiveten went dark. On Friday, any lookup at the site yielded a response reading “blackholes.five-ten-sg.com has been retired.” As of this writing, the domain doesn’t answer at all. Carl hasn’t provided any public explanation for his decision to decommission his list, and he really doesn’t have to. No one has to pay money to use his list, and maintaining a list takes more time, energy and resources than most folks realize. I suspect Carl simply ran out of one or more.

Senders have a love-hate relationship with blacklists; they do a good job of keeping the deluge of pill spam, virus and malware messages at bay, and are an important reason why e-mail remains a viable channel for marketing and commerce. But when senders find themselves at the pointy end of a listing, they often feel as though the listing must be capricious, or even malicious.

The demise of Fiveten demonstrates that, contrary to all the complaints over the years, block lists as a category generally are not capricious. It turns out that market forces are as immutable for block lists as for any business, and block lists operators are just as answerable. Over-aggressive listings are not useful to ISPs, because they tend to generate false positives by blocking wanted mail. When a list isn’t useful anymore, ISPs stop using it, and it goes away. Just like Fiveten did.

Blacklists will continue to exist and operate much as they always have, and I predict that both senders and anti-spammers will continue to complain about them just as loudly. If either side were to stop – well, that’s when I’d start to worry whether blacklists are still doing a good job.

Andrew Barrett is Sr. Director, ISP Relations & Deliverability for Real Magnet.

What You Need to Know About Block Lists Before You’re Blocked

Tuesday, July 20th, 2010

If you’re sending high volumes of e-mail, chances are that you’ll find yourself on a block list sooner or later. The secret to surviving (and correcting) a listing is to be ready before it happens. Here’s what you need to know now, before you find yourself listed.

Stay Cool. No one ever got a listing removed by screaming down a phone line or threatening legal action. Don’t expect (or demand) a good customer service experience from a block list – you are not their customer.

Block Lists Don’t Block Mail. In the initial panic following the discovery of your listing, it’s easy to forget that block lists don’t actually block any mail; it’s your recipients’ mail servers that do all the blocking. The filters used by many ISPs and companies reference data from block list, reputation scoring firms, and especially feedback from their customers to inform their filtering decisions.

Some Block Lists Matter More Than Others. The vast majority of public block lists don’t matter at all. There are plenty of web sites that offer to look up your sending IP on hundreds of lists all at once, but unless you’re listed on one of only about a half-dozen, you probably have nothing to worry about.

So which are the ones worth worrying about? Any of the lists operated by Spamhaus.org, the CBL, URIBL, CloudMark CSI, SpamCop, Barracuda Central, and sometimes SURBL and SORBS. The cast of characters changes a little from time to time, but these are usually the heavy lifters.

Different Lists Do Different Things. A listing on the Spamhaus SBL means something very different from a listing on URIBL, which is entirely different again from a listing on Spamhaus PBL. Only one of these (SBL) is a list of suspected spam sources. The URIBL lists domains that appear in spam. The PBL is a list of IP space from which unauthenticated e-mail is not supposed to be sent. Don’t assume you’ve been listed because someone thinks you’re sending spam; make sure you understand the reason for your listing before you waste time fixing a problem you don’t have.

Many Block Lists are Automated. Some block lists operate with as little human input as possible. The URIBL is a good example. It automatically adds the domains it sees in the links contained in spam, so that users of the list can block mail based on presence of those domains. The good news is that delisting is pretty straightforward – just submit a short request on their web site. But expect the listing to be reinstated automatically if it sees more spam that contains links to the offending domain.

Avoid the Death of A Thousand Cuts. The most dangerous block lists are the private, home-grown lists created and maintained by IT professionals at the companies you’re sending to.  These lists are unpublished, unqueriable, and are controlled by harried mail administrators who don’t have time to check every few weeks to see if it’s okay to delist you..

Imagine a temp firm that specializes in the placement of legal secretaries with medium and small law offices. They may not be sending a lot in overall volume, but if the temp firm is listed by just a few of their target customers, the impact on deliverability will be noticeable. Once they’re on one of these lists, the affect is very localized, but very difficult to reverse. As the number of listings at individual law offices grows, the temp firm may find their target market is all but inaccessible to them via the e-mail channel.

Ironically, one of the benefits to senders of the large, centralized block lists is that it takes just one delisting to get mail unblocked across great swathes of the Internet. It’s a lot easier than contacting every domain you send to, one by one.

Block lists seem a lot less scary once you understand how they’re assembled and used. If you find yourself listed, keep calm, find out why, and gather the data together you need to fix it. At Real Magnet, we have deliverability professionals ready to manage the process for you, and even help prevent a listing in the first place.

Andrew Barrett is Sr. Director, ISP Relations & Deliverability at Real Magnet.