A few months ago we were invited by the smartertools team to become a beta tester for Smartermail 4.0. The biggest change in Smartermail 4.0 for us is the improved spam fighting techniques. With the earlier versions of Smartermail, RBL checks and bayesian filtering was released, unfortunately spammers were already aware of bayesian filtering and had already found ways to corrupt the effectiveness of bayesian filtering. RBLs are just hit or miss and not really effective for the most part (they tend to throw the baby out with the bath water). One of our comments at that time was “hey look at spamassassin” but unfortunately the bits had been set in stone and it couldn’t be implemented. Appearantly we weren’t the only ones commenting on this because today we have spamassassin integration but it’s integrated far better than I could have expected.
Enter the ninja.. Spamassassin
Spamassassin is a widely used anti-spam tool mainly used in linux. it uses a set of rules that are constantly being updated, revised and added to but also supports using 3rd party resources like razor, DCC, and pyzor which are distributed spam databases if you will.
Smartertools really researched spamassassin and realized that a windows platform would simply not run it as effectively as on unix. So they not only integrated spamassassin but set it up such that you can run a farm of linux based spamassassin servers to filter mail through. Very cool! That there is smart planning!
Daddy don’t want your mail unless you really want to get it to him.
Next is the addition of greylisting. Greylisting is an extremely simple idea. It basically rejects a message on the first attempt and then accepts it on the second attempt. Legitimate mail servers will send a message and if it doesn’t send the first time will re-attempt to send the message again a few minutes later and will continue to re-attempt the message for a set period of time until it finally times out at which point it’s bounced. The thought being is that spammers are hit-and-run mailers. They have so many email addresses to attempt to deliver to that they simply attempt a send, if it doesn’t go through immediately they move on to the next address and abandon the previous one. Now Grady (from Smartertools) said greylisting would probably be the biggest help in the fight on spam and I didn’t believe him. Boy was I wrong. Greylisting by itself has almost completely eliminated spam on our beta test domains. I’d say less than 10% of the spam quantity is making it past greylisting and that’s a high ballpark estimate.
What’s the trade-off for all this?
Well there’s no such thing as a free-lunch. Greylisting does delay your messages for a couple minutes and I personally have found it to not be a problem. However if it does cause a problem for you, Smartermail allows you to opt out of greylisting on your domain if you wish.
Any messages that make it through greylisting are then fed to spamassassin’s rules, dcc, razor, pyzor and ofcourse the RBLs and only then a message is delivered. Now you’re probably wondering won’t all of those post greylisting tests delay my email from being immediate? Well we’re seeing between 1 and 10 seconds per message for processing through spamassassin and based on the accuracy it’s a very acceptable trade-off and this is running on a standard linux VPS account.
Why is all this necessary?
Spam has reached epidemic proportions and has simply grown out of control. 2/3rds of all email I used to receive on my personal domains was spam. Think about that 2/3rds of every email I’d have to wade through was spam. This means only 30% of the time I spent working in outlook was spent doing anything productive. For us as a hosting company spam represents a major part of our support requests each day and as a result costs us a great deal in time, resources and manpower. Not to mention the lost revenue, time and manpower it costs our clients each day. Not to mention the cost in server hardware necessary to deal with the increased message processing (thanks to spam!). Spam is simply out of control and needs to be stopped. By implementing systems like SmarterMail 4 we may not be able to stop spam but we can definitely lessen the impact it has on us and our customers.
