FIGHTING SPAM Faced with a small but scurrilous band armed with formidable tools to vacuum up e-mail addresses and bulk send e-mail without a server, how do netizens fight back? |
||||
by Kris Buytaert, Senior Consultant, Stone IT Belgium |
From the start of the Internet, a large percentage of net usage has been taken up by the sending of e-mail. The importance of e-mail as an Internet application quickly led to its targeting by malicious users. In particular, the first big problem was, and still remains, the use of e-mail to spread all sorts of disruptive virus programs. The key to success for the perpetrator is inattention on the part of users to open every attachment they get from any stranger. Today there are over 570 million electronic mailboxes on the Internet. That's almost 2 mailboxes per user. For many of these users, there is now emerging an even bigger problem than the occasional e-mail virus: The new menace is the daily bombardment of unsolicited commercial e-mail (UCE) dubbed spam. No longer just a means of interpersonal communication, e-mail has become a powerful and cost-effective business tool subject to abuse. Those not bound by the slightest thought of netiquette have developed a clever set of techniques for collecting e-mail addresses at a very low cost. Collectively dubbed spamware, there are a number of software packages that enable the practice of spamming and often have no other apparent purpose. Atomic Harvester III is a web-crawling e-mail address extractor that will crawl through entire web sites and extract every e-mail address listed on the site. Packages like Desktop Server IV will bulk-mail to lists directly without using a full-fledged e-mail server. Such electronic infrastructure opens the potential of sending millions of unsolicited direct-marketing messages at the push of a button. Spam is getting out of hand and, with the exception of the EU, government legislation lags years behind. In France, the Commission Nationale de l'Informatique et des Libertés (CNIL) has developed a working definition of spam: “bulk-mailing, sometimes repeatedly, of unsolicited e-mail messages, usually of a commercial nature, to individuals with whom the mailer has had no previous contact and whose e-mail addresses the mailer collected from the public spaces of the Internet: newsgroups, mailing lists, directories, web sites etc.” Without waiting for US Congress to act, a number of states are tackling the problem of spam on a very basic level. Local statutes prohibit the forging of addresses and the doctoring of message headers and subject lines. Nonetheless, they fall short of the real problem of spam: It is unsolicited. The timid response to this issue is a requirement for opt-out requests to be honored. |
|
A similar debate on opt-in versus opt-out has raged in the European Parliament. The European Commission led by Erkki Liikanen, Finnish parliamentarian and European Commissioner for Enterprise and the Information Society who is also an ardent Open Source proponent, favored an opt-in position on commercial e-mail. In his argument, Liikanen cited the case of Japan where up to 85% of 850 million text messages contain unsolicited junk mail. While there was universal condemnation of spam, Baroness Sarah Ludford, Liberal Democrat from London, along with other EU parliamentarians, did not think that civil liberties were an issue. They argued that sending junk mail was already illegal and the additional constraints of an 'opt-in' e-mail system could have adverse consequences for small firms and charities. Liikanen, however, would not agree to differentiate between different forms of communications such as faxes, text mobile phone messages, and e-mail when it came to opt-in versus opt-out rules. In the end, Liikanen held the day and on May 30th the EU Parliament voted to adopt the EC directive making it illegal to send unsolicited e-mail, text message or other similar advertisements to individuals with whom companies do not have a preexisting business relationship. This formalized adoption of an opt-in policy will make Europe a spam-free zone by the end of 2003. |
There is a very practical side to the argument against spam: It wastes resources and resources are money. How much time do your employees lose while browsing their mailbox searching the valuable e-mails from clients and suppliers in the huge pile of spam? Maybe you are still using a dial-up connection to fetch your e-mail and your phone bill rises because you have to stay online longer than actually needed. The storage you planned on your mail server isn't enough for the needs of your users, and their mailboxes get filled up with spam each day. You pay for your bandwidth. You pay to get e-mail delivered to you. Do you want to pay to have unwanted, irrelevant, marketing e-mail messages clogging your inbox? To focus legislators on these adverse consequences that spam has on users—spamming is far from a victimless crime—there is the Coalition Against Unsolicited Commercial e-mail (CAUCE). This volunteer organization, which is active in driving US legislation against spam and spammers, now has affiliates in Europe, Canada, Australia, and India. But while you are waiting for government action, how can you defend against spammers and their spamware today? To be successful, spammers need to trick people into opening their mail. Therefore, you have to be able to recognize spam before you see it. Fortunately, that’s a lot easier than it sounds. In particular the Internet Engineering Task Force posted recommendations (rfc2505) in early 1999 on how to improve the effectiveness of e-mail servers when filtering spam. Characteristically, most of the time spam is not directed to you. It’s sent either to a long list of people or to an invisible mailing list. Either way, you are most likely on that list not as a recipient, but as an alias as they try to hide the thousands of recipients. What’s more, spammers invariably try to hide from where they are mailing. They will try to forge headers and write from "non-existing" e-mail addresses. If they use bulk-mailing desktop software, their mail will not have a legitimate MX DNS record. As a result, they will often hijack an insecure e-mail server that does not check the validity of messages before sending them. Such an e-mail server is dubbed an open relay. With all of this in mind, the IETF focused on stopping spam at the e-mail server and in particular honed in on preventing open relays. They recommended testing for a fully qualified domain names (FQDN) and not just an IP address number. In addition, they recommended verification testing of the sending server by the receiving server using the SMTP HELO statement. These are precisely the kinds of spam-prevention tests that have been added to Open Source e-mail servers as openBench Labs demonstrated in a recent review of the Postfix-based SuSE eMail Server 3.1. They will fill the subject line with lots of "!", "?", and other tricks. The body of a spam message will also contain some typical telltale elements. The message will often contain language telling you how much money you can earn with their program. Having never heard of the word netiquette, spammers will also use caps to yell at you and get your attention. Often spammers will promise goodies, if you go to a website, fill in a form, or call their hotline. Spammers will often use html-based messages in order to embed hidden links to their websites in order to track you. And even more insidious, most spam will contain exhaustive efforts to convince you that it is not spam. All of those techniques can be detected using spam filters. Most e-mail applications can filter messages based on the subject. So you could decide to delete automatically every e-mail with a subject that starts with "Get rich fast." That could become quite tedious, however, if spammers can fool your e-mail application with a simple change of the subject like “Get very rich very fast!” What you need is some good Open Source anti-spamware. Blacklists are as old as the Internet. Often people don't want to receive mail from a certain domain, blocking mails from recruitment agencies, blocking mails from competitors etc, so they create a list of domains that they block. There are many different blacklists, http://www.mail-abuse.org and http://spam.abuse.net are just a couple of the existing lists. What it comes down to is that if you are a spammer, you will get listed sooner or later. And once you are listed, people will block your domain or servers from sending mail to their servers. The most basic characteristic of spam message is the presence of thousands of thousands of duplicates. As a result, if you can catch multiple versions of the same message, there is a good chance that the message you just received might be spam. Vipul’s Razor is a distributed, collaborative, spam detection and filtering network. Through user contributions, Razor establishes a catalogue of spam in propagation based on message checksums. This catalogue is consulted by e-mail clients to filter out known spam. Detection is done with statistical and randomized signatures that efficiently spot mutating spam content. Another alternative is SpamAssassin. This software uses a blacklist, header checking, and body analysis just as Razor. In fact, SpamAssassin includes Razor as one of its tests. Based on the content of the subject line, the number of recipients, and either the originating mail server or the relay server used, the header of a mail message reveals a lot of information about that message. SpamAssassin scores each of these characteristics, either negative (unlikely to be spam) or positive (likely to be spam) values. Following the header, SpamAssassin then analyzes the body of an e-mail. Based on the total score a mail gets, on all different parts, you can decide whether a mail gets filtered into a separate mailbox or not. |
What happens with a legitimate e-mail that gets wrongfully tagged? First of all, any message that gets tagged shouldn't be automatically deleted. The best practice is to filter all suspect messages into a separate mailbox. This mailbox can be checked later if your correspondents contact you about lost mail. I have been using a combination of these tools for the past year and most of the messages wrongfully tagged as spam could have easily been spam. More often than not, mistakes happen with newsletters. Taking a look at the tests used by SpamAssassin gives good hints on how not to send e-mail messages to your contacts. We used this technique to help improve the weekly Open magazine e-mail. One particularly knotty problem for newsletters is the status of the mailing list server ISP. If your list server partner is blacklisted, you have a problem. |
|
While legislation is lagging on the spam problem, we have to kill the spam problem now. Tools such as SpamAssassin and Vipul’s Razor are important to keep the Internet a place where we can continue to do our job. Checking your spam-box for legitimate mails every so often is far less time-consuming than searching through spam all day long. |