The Spam Battles

Bob Morse

Email is one of the oldest protocols (ways of transferring data) on the Internet. In fact, it evolved from a simple messaging system on the precursor of the Internet called Arpanet in the 1960s. Like that bacteria that hasn't evolved for 2 billion years, Email as remained largely unchanged in all that time except perhaps for our dependence on it. 

The pernicious spread of Spam (officially know as Unsolicited Bulk Emai or UBE) has made electronic mail seem like a necessary evil. Major Email providers like Google have developed some pretty sophisticated systems of filters to protect your inbox. As a provider of Email servers ourselves, we have worked really hard to protect our users from receiving spam and our servers from being vulnerable to sending spam. Our efforts have mulitple layers and each one is fraught with trade offs. Here's a brief, not too technical description of some steps we've taken particularly in keeping spam out of our user's inbox:

Blocklists These are are third party services which monitor Email traffic. When a mail server is detected as having been used to generate a certain volume of spam the server's identifying number (IP or Internet Protocol address) gets added to the services' database of blocked systems. Our mail server checks the IP address of every mail message it receives against the databases of these services. If the IP address is found in one the mail gets blocked before goes through any further processing. Servers that clean up their systems and stop sending spam are generally removed from the Blocklist database after 24 hours, sooner if the systems administrator requests removal. After a good deal of trial and error we have chosen to use a few highly regarded blocklist services and that has proven effective to reduce the amount of spam our servers have to deal with on other levels. Sometimes people using legitimate servers get their email blocked because someone on that same system had an account compromised. We work to clear those as soon as possible.


MailScanner This script is installed on all our servers and tries to detect spam in the email that gets through the Blocklists by using sophisticated algorithms to scan each message and score it. Rather than simply blocking anythingif a message meets a certain, lower threshhold it will add a tag (Spam?) to the subject line. If the message meets a higher threshhold it will add the a different tag (Definitely Spam?) to the subject line. Each hosted domain can determine how it would like to handle mail tagged at either level. They can choose to accept one or both of the types of messages and handle them in some way at the receiving end on then own devices. Or they can have either or both levels deleted before they are delivered. 

MailScanner Front-End (MSFE) is a licensed script that adds a great deal of functionality and control to MailScanner. The main use for MSFE is to monior the flow of incoming mail and watchfor obvious spam that has slipped through. We can then block the spammer manually  by email address, domain or the IP address of the sending server. We've even blocked a list of new Top Levle Domains (TLDs) that have recently been made available through the Internet Corporation for Assigned Names and Numbers (ICAAN). These are domains like .shop, .space, .website, etc. While some of these are useful given the shortage of good .com names, many of the them are used to send spam from all over the world. 

The result of these efforts is that I as a user on my own system, see almost no spam!

The danger in all of these tactics and layers of protrection is that ocassionally legitmate messages are lost or bounced. While this is rare, it's possible to correct if discovered. So far, people seem pretty pleased with the results. We try to give our clients as much control over these factors as possible while still automating the core of spam protection. It will always be a battle with no end to the war.

Now For Something Funny About Spam: