Seminar Report

A Plan For No Spam

Unwanted and irrelevant mass mailings, commonly known as spam are becoming a serious nuisance that if left unchecked may soon be regarded as a Denial of Service Attack against the email infrastructure of the Internet itself. The term spam is derived from a Monty python sketch set in a cafeteria in which the principal protagonists have difficulty making them heard about a group of Vikings singing. "SPAM" in honor of the meat product manufactured by Hormel.


Seminar Report

A Plan For No Spam

• An ideal spam control would have the following properties • Eliminate all unwanted mails • Eliminate no wanted mails • Require no user input on the part of either the sender or the receiver. • Be compatible with all uses of mail. • Be compatible with all email infrastructure configurations. • Be scalable, that is remain effective if 90% of internet users adopt it. • Resist attempts to evade it. • Create no new problems. No perfect spam control solution has been found so far. Filtering approaches are compatible with a broad range of email uses and infrastructure but no filter perfectly identifies even a fraction of unwanted emails without eliminating at least some wanted emails. Further more the more widely a filter is used the greater the incentive becomes for the spam senders to test against it to ensure that their spam gets through.


seminarsonly. The only mechanisms that rule out of hand are those based on vigilante actions that attempt to gain compliance by coercion. We should not however.Seminar Report A Plan For No Spam TOWARDS SOLUTIONS There are many types of spam and it is likely therefore that any realistic program to eliminate spam will have to address the problem in multiple ways. www. we conclude from the fact that a mechanism fails in certain circumstances that it fails completely in all circumstances. In particular we should not reject a mechanism out of hand simply because it fails to deal with a particular type of spam sender .com 3 .For example it is highly unlikely that spam senders engaged in organized crime will respect Opt-Out lists or Legislative approaches.

Acceptable Use policies Perhaps the most important anti-spam measure to date has been the adoption of acceptable use policies that prohibit sending of spam by Internet service providers of all types. even if such statements do not have the endorsement of a legislative body they can quickly acquire the force of law when the worst malefactors are being dealt with. www. In a recent case a university discovered that some of its students had been renting out the use of their machines to spam senders. The short survey and prosecutions by the FTC and others show that the spam senders are in many cases outright criminals. they protect the Internet service provider against abuse by their own users. An appropriate statement by an authoritative body can help a court to decide that a particular form of behavior is unacceptable. Such use policies do not just protect the other users of the Internet.Seminar Report A Plan For No Spam BEST PRACTICES The traditional response of the internet to problem uses administrators of deployed protocols is to specify some form of 'Best Practices'. Spam is an attack on the Internet community.seminarsonly. Many of the problems being caused by irresponsible blacklists could be addressed through common agreement on best practices for blacklists setting out criteria for issues such as notice to the listed parties. seriously degrading the network service for all the 4 . Best practices can also provide backing for legislative approaches. appeals processes and the acceptability of 'collateral damage'. how then can best practices help? One area in which best practices can provide concrete benefit is in ensuring that the vast majority of Internet users who are acting in good faith do not inadvertently make the problem worse by poorly chosen or poorly coordinated mitigation strategies.

The most important of these being obtaining a commitment from the major software client to supporting the protocol to the commonly used email clients. A mechanism is also required that tells the client where the updates will be available from.Seminar Report A Plan For No Spam Pull Vs Push One of the major problems caused by spam is positive identification as spam of legitimate emails.This is unlikely to prove practical in deployment however since any benefits would require changes to all parts of the email infrastructure-clients. Consent is thus implicit in the pull request since the client will not request content unless directed by the user. there is no way for that consent to be communicated to the mail infrastructure that transports the message.seminarsonly. the format etc. This is a particular problem with commercial mailing lists and newsletter messages requested by the recipient which frequently carry advertising and make use of content that is likely to be flagged by content inspection filters as spam. While some have argued that the commercial nature of these emails means that this type of false positive is unimportant it seems odd to consider the loss of messages specifically requested by the user to be less important than the loss of unsolicited personal emails. the expected frequency of those updates. One possible solution to this problem would be to add some form of authentication mechanism that communicated the recipient's consent to the mail infrastructure . Another option would be to distribute this form of content using the pull model of the Web [HTTP] and NNTP [NNTP] rather than the push model of the 5 . no content is sent until the client specifically requests it. originating servers and receiving server. For the pull model to become practical a number of difficulties must be overcome. The core problem with these messages is that although the sender and the recipient know that the user has consented to receive the messages. In the pull model. The Really Simple www.

even if spam senders observe them since the spam victim is required to Opt-Out with each spam sender individually. that has been sold to several thousand spam senders. Opt-Out schemes of this type are unacceptable 6 . The research by the FTC has demonstrated that large proportion of these Opt-Out options are bogus [FTC 3] and it is not possible for the user to connect to the Opt-Out site. The most powerful argument against Opt-Out lists is that it is likely that spam senders would routinely abuse lists of Opted-Out email addresses as the source of email addresses. We know that a significant proportion of spam is sent on behalf of legitimate businesses that may be considered likely to respect such matters. although it is currently being applied in a different way.seminarsonly. the address owner Opt-Outs out once and it is the responsibility of the spam senders to ensure that their lists are clean. For Opt-Out to be practical it must function in the same way that state Do-Not-call telephone lists do. Opt-Out lists are proven to be of value in controlling the volume of unsolicited mail and unsolicited telephone calls from legitimate businesses. This is impossibility if the victims email address has been listed on a CDROM.The Opt-Out list consists of a sequence of message digest value to permit rapid lookup by binary search or similar means.Seminar Report A Plan For No Spam Syndication (RSS) Protocol [RSS] appears to have these properties. Opt-Out and Do-Not-Spam lists Many spam messages carry a 'click here to Opt-Out' option. This objection may be addressed by appropriate use of cryptography in which the entries in the lists are obscured using a one way message digest function such as SHA-1. www. Certainly a mechanism of this type widely deployed would deprive spam senders of the claim that they respect Opt-Out requests.

www.Seminar Report A Plan For No Spam Accountability One of the major difficulties faced with distinguishing legitimate bulk senders from illegitimate ones is the difficulty of determining whether the claims made that the bulk sender observes a particular set of best practices is true or not. Technical mechanisms are required that enable this form of accountability to be achieved. This problem also faces the bulk senders 7 . At present there is no way that a outsourced provider of bulk mail services can tell if the email addresses on a list a customer wants used for a mailing have intact Opted-In as is claimed.

There are many forms of Content inspection. The symptoms of spam are relieved to a considerable degree. If the spam sender knows the criteria applied by the Content inspection technique. each of which has advantages and disadvantages. As a result a large number of spam messages are incomprehensive to the reader. Naive Language Inspection The Internet is an international medium and spam is sent in many languages. but the patient is still under attack from the 8 . then they can construct their messages so that they are not caught. Keyword Inspection alone is simple to implement but tends to have very high rate of false positives.Seminar Report A Plan For No Spam CONTENT INSPECTION Aspirin will not cure a cold but it will relieve the symptoms and make the cold more tolerable. This type of filtering is implemented in many common email clients such as Outlook [MSFT].seminarsonly. One of the principal difficulties with content inspection mechanism is the ease of evasion by the spam senders. While fully understanding the meaning of the mail message is a complex problem that is AI complete. Naive Keyword Inspection Messages are scanned for the presence of words or phrases that occur frequently in spam messages such as HGH or multi-level marketing. that is it requires a solution of the artificial www. Content inspection has a similar effect on spam. Content inspection is a form of spam filtering that uses the content of the messages as the basis for the decision to filter.

in most cases in combination with some form of Bayesian Inference. Another problem with language inspection is that a large software vendors would face a considerable problem from foreign users of their product if they introduced language sensitive filters in their products. This approach provides some resistance to spam sender counter measures since the individual users maintain separate databases for desirable and undesirable messages.seminarsonly. In many cases the character set in which the email is sent may be used as a proxy for language. Keyword Inspection with Statistical Techniques The effectiveness of Keyword Inspection can be substantially improved if combined with Statistical Techniques that access the probability of a message being used as a spam based on the presence of multiple keywords. detecting the language that the email is written in is considerably easier.Seminar Report A Plan For No Spam intelligence problem. Keyword Inspection with User Feedback There has been considerable recent interest in content inspection mechanisms that employ user 9 . although this technique can result in false positives as a Japanese user may send all his messages in a Japanese character set. using the ASCII subset for messages in English. Keyword Inspection with Dynamic Update Another approach to improving the effectiveness of keyword inspection is the combination of Statistical methods with an online source that provides www. Various techniques may be used for this including Bayesian Inference and least square approaches. The drawback to this approach is that it requires user intervention on a per message basis which experience demonstrates limits the effectiveness of the approach severely.

This approach is frequently combined with the Template approach described below. While this technique can be very effective it is also costly to maintain.seminarsonly. www. either by hand or using some form of automated 10 . The templates are constructed using the spam messages sent to 'honey pot' email addresses.Seminar Report A Plan For No Spam regular Updates. Dynamic Template Response / Fuzzy Matching / Checksums Dynamic Template Response uses templates or 'finger prints' of known spam messages to identify spam messages. particularly since the spam sender may employ counter measures taken by the spam senders.

A particularly insidious spam sender trick borrowed from the Klez computer virus is to harvest email addresses from the mailing lists archived on the web and send a large number of emails purporting to come from one member of the list to other members of the lists.Seminar Report A Plan For No Spam AUTHENTICATION AND AUTHORIZATION Practically all spam messages sent today attempt to evade anti-spam measures by use of false header information. some form of authentication scheme is required that provide an unequivocal proof that the sender address is 11 . authenticated sender address. To address this problem. Some contained no sender address at all.seminarsonly. If every email carried an authenticated sender address spam senders would be forced to obtain a new DNS address www. We find this objection to be a weak one since it is clear that Spam senders have a considerable motivation to conceal their identity. This suggests that a robust method of determining the false sender addresses would provide an effective means of eliminating spam. None of the spam messages that were examined in the writing of this paper carried a genuine sender address. A Spam sender might attempt to circumvent an authentication scheme by sending messages with a legitimate. Detecting false sender addresses would be a simple task but for the fact that the SMTP protocol allows the sender to forge a message that purports to come from any sender. The use of forged email address is likely to become very common if the use of anti-spam filters that detect missing or obviously false from addresses become common. Most of the massages contained from addresses that were obviously fake. In some cases the addresses were not even valid. The use of forged email addresses is currently rare but becoming more common.

Message Identifier Matching: A mail service may use the In-Reply-To and References headers to identify messages that are replies to messages that originated at that service.seminarsonly. This causes increase in costs of spam senders & reducing overall profit of the spam sender enterprise. which maps IP addresses to domain names for this 12 . SMTP Routing information: An SMTP message carries information that describes the path it has taken from one mail server to another. This mechanism is not completely reliable since an IP address is a valid source for a particular sender address. Network based If a mail server knows the set of all possible IP addresses from which an email with a particular sender address is known the IP addresses may be used to provide a light weight means of authenticating the email sender. Some mail servers use the reverse DNS. Authentication Authentication techniques are broadly divided into two types. Network based and Crypto graphically based. Construction of the original message identifiers using a secret key and Message www.Seminar Report A Plan For No Spam frequently to conceal their identity. This approach only works if the email is sent via a mail relay that is configured with reverse DNS entries for the domain of the sender address.

The callback loop mechanism is unusual in that is an active authentication mechanism that is applied at the request of the receiver rather than being applied passively to every message by the sender.seminarsonly. Otherwise the recipient is unable to distinguish a message from a user does not use the authentication mechanism from a forged www. Cryptographic S/MIME S/MIME provides end to end authentication of the sender address and message body. The sender need not send their message through any specific email relay.509 digital certificates. Callback loop.Seminar Report A Plan For No Spam Authentication code allows this to be archived without the need to maintain a list of all message identifiers issued by the service. If the confirmation message is received the sender address is considered to be 13 . Challenge/Response: When an email is received a message requesting confirmation is sent to the purported sender address. Authentication Policy For passive authentication mechanism to be useful as a means of detecting forged sender address it is necessary to know whether a purported sender has a policy of using authentication. Although SSL allows the email relay to use any IP address without the need for configuration of a reverse DNS address the sender must send their outgoing mail. Cryptographic SSL An extension to the SMTP protocol allows the use of the Secure Socket Layer via the START TLS operations allows authentication of both the sending and receiving email servers using X.

The Security Policy Advisory Mechanism provides a set of DNS extensions that address this need. Fortunately the security weakness of the DNS do not lend themselves to exploitation on the scale necessary to make exploitation of these weakness a viable means of defeating an authentication based anti-spam 14 .seminarsonly. Fortunately the DNS architecture provides the necessary functionality and may be readily adapted to the purpose without putting an undue load on the DNS. Revocable Credentials Cryptographic authentication schemes such as S/MIME or SSL use X. www.509 digital certificates as credentials. The principal disadvantage of using the DNS as a means of communicating security policies is that the DNS itself is insecure.Seminar Report A Plan For No Spam message purporting to come from a user who always uses an authentication mechanism. The Internet architecture does not include a mechanism designed for the purpose of communicating security policies.

seminarsonly. A false sender address that impersonates another party is a form of identity theft and may also be actionable as defamation. Spam senders also violate law in their methods. Legislators are reluctant to pass any legislation until they are confident that the implications are fully understood. While it is unlikely that the criminal legislation alone would eliminate spam. Such legislation would certainly create a deterrent for both the spam senders and the advertisers seeking their services.Seminar Report A Plan For No Spam LEGISLATION AND LITIGATION The purpose of criminal legislation in a democratic is to deter persons from engaging in prohibited conduct. cases caused by ISPs & individuals who object to the use of their resources to send spam and cases caused by spam senders to prevent ISPs from cutting off their resources when they violate terms of use. Prosecutions and Litigation under Existing Legislation As previously noted a substantial proportion of spam is illegal under the existing laws. Legislators will have to be convinced that any new legislation to address the problem of spam will bring benefits that significantly outweigh both the cost of enforcement and the political cost of committing the scarce resource of legislative time to the problem of spam rather than to other pressing problems. Use of false sender address is an attempt to gain access to a computer system by 15 . The legislative process is very slow & time consuming. Spam Litigation Spam has spawned two types of litigation. Anti-spam Legislation www.

A number of anti-spam measures have been proposed in the US congress that seeks to regulate spam by requiring spam senders to respect 'Opt-Out' requests and prohibiting the use of forged sender addresses. the Direct Mail Association (DMA) that represents centers of junk mail and telemarketers has demonstrated a considerable degree of influence opposing opt-in requirements to protect privacy. The spam senders compete with the DMA members and so the DMA is opposed to spam but will oppose any measure requiring opt-in which might set a precedent that may later be applied to its members. Some of the 'anti-spam' proposals being raised in private are really 'pro-spam' legislation intended to solve problems for senders of spam. The chief objection to the 'Opt-Out' measures is that a recipient of a spam has no way to know the origin of a spam.Seminar Report A Plan For No Spam In May 2002 the European Union issued a directive that directs member nations to introduce Legislation that prohibits sending of unsolicited marketing messages unless the recipients 'Opt-in'. Pro-Spam Legislation Not all the anti-spam legislation being proposed is intended to solve the problem of spam. While email spam senders have negligible support in Congress. Such a list would meet the need for verification without threatening the interest of DMA and its 16 . Most proposals are based on existing legislation concerning junk faxes and telemarketing calls that have already been tested with constitutional challenges. A common tactic used in the US to derail popular legislation is to introduce legislation at the federal level that proposes 'harmonization' of stateless that does www. One possible resolution of this problem would be to require spam senders to respect a one way encrypted opt-out list of the type described earlier.seminarsonly. It is therefore impossible for a spam recipient to know if the Opt-Out requests are being respected or not.

If that definition is sufficiently loosely worded. Another potential tactic is to attempt to force ISPs to accept all mail that does not meet some definition of spam. www. ISPs might become legally obliged to accept any email content. however objectionable provided only that it was sent from a member of a particular 17 .Seminar Report A Plan For No Spam so by pre-empting the existing state laws with weaker or sometimes completely ineffectual federal laws. It is therefore important that any spam legislation be carefully evaluated on its merits and not merely its title.seminarsonly.

Mail clients to cause mail servers to apply filtering at the server Mail clients to disable server based filtering www. Mail servers to communicate the results of server based authentication and filtering procedures to mail clients.seminarsonly. Legislation provides a means of slowing the rate of growth of spam so that the content inspection based approaches maintain their effectiveness long enough for the long-term authentication based solutions to be effective. Legislative approaches can increase the cost of spam senders by forcing them to employ costly counter measures such as moving offshore but are unlikely to eliminate spam together. Authentication based approaches provide a robust means of identifying messages that are not spam but are of limited utility unless widely adapted. This analysis suggests that content inspection. Protocol changes to facilitate spam measures Previous sections have identified the need for mechanisms are that allow:Unsolicited call back request to be identified and 18 .Seminar Report A Plan For No Spam DEPLOYMENT Content inspection based on approaches can be readily deployed but tends to loose their effectiveness as widespread adoption encourages spam senders to employ counter measures. Content inspection provides short-term mitigation of the effects of spam. legislation and authentication are complementary approaches. Authentication provides a robust long-term solution.

com 19 .Seminar Report A Plan For No Spam Support for lightweight authentication mechanisms to be applied at the server level www.seminarsonly.

com 20 .seminarsonly. Mechanisms identifying messages sent through a mailing list and the means by which the subscription can be cancelled. We propose the following set of protocol changes to remedy this solution. Authentication mechanisms We identify a need for the following:- A means of advertising authentication policy through the DNS A lightweight authentication mechanism that can be deployed with minimal overhead Definition of a standard call back mechanism that allows a mail client to automatically detect call back loop requests that were initiated by the client and those that result from a message sent with a forged header Mailing list management Support for mailing list software in SMTP is less than satisfactory.Seminar Report A Plan For No Spam APIs for spam filters The definition of a standard Application Program Interface (API) for filtering modules would greatly simplify deployment and allow development of spam filtering mechanisms to take place independently of email client development. www.

Mechanisms for identifying automatically generated messages that should not be replied to under any circumstances.Seminar Report A Plan For No Spam Mechanism that allows a mail server to determine that an email user solicited a message from a particular mailing list or other bulk sender Authenticated subscription management mechanism Mechanism for communicating the authenticated mailing list subscription mechanisms to email clients. Means of determining that an email client confirms to a consistent means of implementing the above 21 .seminarsonly.

including verification of I P address. AUTHENTICATION PROCESSOR The mail server performs the authentication procedures that are available to it. use of authenticated SSL and S/MIME authentication on the message 22 . www. DENIAL OF SERVICE PROCESSOR When the mail server receives a request to send an email.Seminar Report A Plan For No Spam COMPREHENSIVE EMAIL MANAGEMENT The various mechanisms described in this draft may be brought together to provide a comprehensive email management solution as shown in figure. it first checks to see whether the email is being sent from an I P address that has attempted to perform a denial of service attack on the server by consulting the Denial Of Service blacklist.seminarsonly.

DENIAL OF SERVICE BLACK LIST Denial Of Service attack typically targets a single site or a small group of sites. otherwise the message is quarantined. If the message passes the authentication policy or there is no authentication policy. The blacklist and the white list may contain information collected locally and information from external sources. AUTHORIZATION PROCESSOR The authorization processor checks to see if the sender of the message is on the local white list of approved senders. otherwise the message is passed to the content inspection module. CONTENT INSPECTION PROCESSOR The content inspection module uses the heuristic approaches described earlier such as keyword filtering.Seminar Report A Plan For No Spam AUTHENTICATION POLICY PROCESSOR If the sending domain has an authentication policy the mail server checks to see if the message meets the authentication policy. black listing and Bayesian inference to determine whether the message is likely to be a wanted message or Spam. If so the message is accepted. It is likely therefore that in most cases the source of information for the denial of service blacklist would be information collected at the site itself. www.seminarsonly. either by placing the message in a likely junk folder or by marking it in a manner that can be verified by the users mail client. the message is passed to the authorization processor. header verification. If the message is identified as likely to be good it is passed to the user. The use of authentication techniques may be taken into account when determining the likelihood that the message is Spam. Otherwise the message is 23 .

it may prove adequate to list exceptions to the white lists. This feedback may employ Bayesian inference. www. In cases where robust authentication techniques such as digital certificates are used.Seminar Report A Plan For No Spam SPAM SENDER BLACKLIST It is recommended that spam blacklists be used as one input to the content filtering decision making process than to provide an accept/deny decisions. suppliers and partners. In most cases neither the sender nor the receiver has direct influence over these policies and the annoyance caused to the use of these tactic is almost certain to be greater than that caused by the spam itself. The use of feedback from end users is highly recommended to assist in the maintenance of the black lists and white lists and in the configuration of the content inspection processor.seminarsonly. AUTHORIZED SENDER WHITELIST The authorized sender white lists will typically be based on information configured locally from lists of known customers. least squares minimization 24 . In particular end users should not be deprived of wanted email messages because they are send from a source address that has been listed to create ‘collateral damage’ as a means of establishing leverage to cause an ISP to change policy.

Authentication provides a means of identifying desirable messages that allows the problem of false positives associated with content inspection to be reduced and with widespread deployment offers the possibility of a comprehensive solution.Seminar Report A Plan For No Spam CONCLUSION There are many techniques that address a part of the spam problem. there are many techniques that in combination can provide an effective strategy for addressing the spam problem. It is the lack of authentication and authorization in the email system that allows sit to be abused for any purpose spam is a community problem and it is the internet community as a whole that must find. Spam is a security problem.seminarsonly. No currently known technique provides a complete solution and it is unlikely that address a part of the problem. Protocol improvements provide support for the content inspection and authentication mechanisms and should be pursued aggressively. protected using one-way encryption provide an objective definition of a spam message. Do not send lists. www. it use as a first line of defense is not acceptable. Callback loop authentication is acceptable as a last resort when a message has been identified as likely spam. Legislative initiatives provide a means of increasing the costs of the spam senders and reducing the profitability of their enterprise. Content inspection provides a means of identifying undesirable messages that can be deployed with immediate advantage to the 25 . Authentication policy provides a means of knowing that a message that does not meet the Authentication policy specified by the domain name owner has been forged and should be discarded as spam. implement and deploy solutions. No currently known technique provides a complete solution and it is unlikely that any technique will be found in the future that provides a complete and costless solution. Even so.

userland.txt • Kantor.ISI http://www."Simple Mail Transfer Protocol".Leach and Frystyk. Microsoft Junk E-Mail Filter Read me http://office. Manister.txt • Dave 26 . P. Hypertext Transfer Protocol-HTTP RSS 2. Mogul. and P.RFC 2616 http://www.1. Fielding. Better Bayesian Filtering http://www.seminarsonly. L.html • Microsoft corp.txt • www.RFC 821. Getty’ • Postel. B.html • Paul"Network News Transfer Protocol".seminarsonly. A Plan for Spam http://www.RFC 977 http://www.Berners-Lee.Seminar Report A Plan For No Spam REFERENCES • Paul www.aspx • R. J. 27 .Seminar Report A Plan For No Spam TABLE OF CONTENTS • • • • • • • • • • • INTRODUCTION IDEAL SPAM CONTROL SYSTEM PROPERTIES TOWARDS SOLUTIONS BEST PRACTICES CONTENT INSPECTION AUTHENTICATION AND AUTHORIZATION LEGISLATION AND LITIGATION DEPLOYMENT COMPREHENSIVE EMAIL MANAGEMENT CONCLUSIONS REFERENCES 1 2 3 4 8 11 15 18 21 24 25 www.

commonly known as spam is starting to seriously degrade the usefulness of 28 . www. In this paper we provide a survey of the principal approaches currently being applied to spam control and propose a strategy by which these mechanisms may be combined to provide a comprehensive solution to the spam menace.Seminar Report A Plan For No Spam ABSTRACT The problem of unwanted and irrelevant mass mailings.

Sign up to vote on this title
UsefulNot useful