Why SpamAssassin does not work

The SpamAssassin spam filter does not work. it rejects legitimate email and accepts spam. Why does it fail? Here are several reasons and alternative approaches.

Experience

The best spammers have more experience at writing email than your family, friends, colleagues, and most marketing people. Professional spammers subscribe to the same anti spam services as you and practice writing spam that gets through. Your family, friends, colleagues, and marketing people accidentally include links, keywords and other things that trigger the spam rejection in SpamAssassin.

Most marketing people focus on bling, features, benefits, fashions, fads, and forget to test their not-spam© through anti spam software. You get mail from professional marketing people that includes the same junk gimicks used by non professional spammers. Who will get through first? The professional spammers. Who will fail? The marketing people.

Keywords

Some anti spam software learns by the spam you reject. The software matches words across several rejected spam and finds words that repeat. The software rejects future email containing those repeated words. The keywords for rejection are often innocent words used in non spam email.

Look at one example. Visit my.imisfriendraising.com.au/personalPage.aspx?SID=87336. The page is about a six year old girl shaving her head to raise money for a special charity. Can you find anything in the page that would trigger a spam rejection? No. But.... Some adult content refers to women/girls/girl and shave/shaved body parts. If you reject several spam items containing the word shave, your anti spam service might reject all email containing the word shave, including email from your female friends who participate in fund raising by shaving their head.

Anti spam services reject email for a really weird collection of keywords that are in common use. Professional spammers change the keywords slightly and their email gets through. You friends send you jokes containing references to body parts and the jokes are rejected because they look like adult content spam to spam filters. Americans refer to one body part as butt. Professional spammers know that in most cases they can replace butt with but, an acceptable word, but one that reads the other way within the context of their email. Your email filter rejects mail form for friends and lets through the mail from spammers.

Phrases

Phrase rejection is more accurate because you can decide the meaning of some words by their context. Phrase rejection is harder because a simple phrase can be written in several sequences without making the phrase difficult to understand. Professional spammers are good at rearranging text to sound harmless.

Adjoining words

Looking for adjoining works and words in close proximity gives you the opportunity to pick up phrases written in different sequences. Now you know why professional spam has the spicy words spread out over several lines. That spread overcomes proximity tests.

Links

Spam filters look for those email with just a link. Usually it is spam inviting you to download a virus. Now think about another sequence. Fred sends you an email telling you about Fred completing a new Web site. You send the reply Fred you forgot to include a link to the Web site. Fred sends you a reply containing just the link. You never receive the reply because SpamAssassin assassinated the email.

Images

Repeat the Links example with an email containing just an image. SpamAssassin eats the email. Think of all those times a friend sends you photographs of his new kid or her new Harley Iron 883™. The photographs will not go through because they are too big. Your friend then sends you an email containing the text followed be several email with one picture in each email. The single picture email is assassinated.

Mail clones

You register on a mailing list. A professional spammer registers on the same mailing list. They receive the same email you receive. The change their copy slightly and use as the base of their next spam. If you receive the original, you will receive the clone. You might receive the spam clone but not the original because the original contained an accidental combination of things to trigger a SpamAssassin assassination but the spam clone has some of the tricky keywords removed because the professional spammer knows they will cause problems.

Solutions

Turn of SpamAssassin and test these alternatives. You need SpamAssassin off because SpamAssassin will confuse your test results.

Catch all address

You have a Web site with the contact email address set to contact@example.com. Someone sends you an email using webmaster@example.com. Will you receive the email or not? you can set up a catch all address to receive all the incorrectly addressed email. I did that once for a small site and received 14000 spam email per hour.

A better approach is to switch the catch all address off and set up only common email mistakes as aliases to a local address. webmaster@example.com could be an alias for contact@example.com.

The following description assumes you manage your Web site with cPanel. There are a lot of alternatives to Cpanel and you should be able to an equivalent action for your site. Ask your Web site hosting service for help if you do not find the right setting.

Go to Cpanel, mail, Default Address. Under Send all unrouted e-mail for:, select Discard with error to sender (at SMTP time).

Go to Cpanel, mail, User Level Filtering. Select Manage Filters for the account that needs an alias. Select Create a new Filter. Under Actions, select Redirect to email. Fill in the other fields as appropriate. Set Filter name and the field after Redirect to email to the alias, webmaster@example.com in our example. Set the empty field after Rules to the target email, contact@example.com in our example.

SPF

SPF is Sender Policy Framework and is described in www.openspf.org/Introduction. SPF is an open standard already used by 20 percent or more of the Internet.

Microsoft produced something different named Sender ID that confuses the issue because Sender ID is based on SPF but does something different and clashes with SPF. You can set up a fake Sender ID specification to make Microsoft sites not use Sender ID for your email. You can also change your email headers slightly to pass the Sender ID test without damaging the SPF test.

A test of SPF on a site receiving 1000 email per hour, 2 legitimate and 998 spam, reduced email to 4 per hour, 2 legitimate and 2 spam.

Thunderbird

Switch you email client to Thunderbird. use the email filtering in Thunderbird. You can manually mark mail as spam and Thunderbird will learn to mark the email as spam in the future. Of more importance is the fact that you can add email addresses to your contact list and have those addresses automatically accepted even if they look like spam. If you are already using SPF to make sure you accept only email from legitimate sources, the email passed because of the contact list should be legitimate email.

One serious advantage of using Thunderbird to filter your email after you kill most of the spam using SPF is that Thunderbird can move the spam to a special mail box you can review. If mail you want arrives and is marked spam, you can add the mail address to your contact list and always receive future mail from that address. You need this when your contacts change their email addresses and their new email is marked as spam.

Captcha

Captcha is something you can add to your Web site to reduce spam sent through Web site contact forms and comment forms. Captcha stops most automated spam and a lot of human spammers cannot be bothered with sites that use Captcha.

There are alternatives to Captcha including services you pay for but the paid services run into a problem problem with professional spammers who register for the same paid service then test ways to bypass the service.

Conclusion

External anti spam services do not know what mail you want and what mail is spam. Do not let them destroy your mail. The best they can do is mark your mail as possible spam then let you decide. SPF and other approaches kill most of the spam and you can teach Thunderbird to reject the rest.