ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Note: This info is current for SME 7.2 and ASSP 1.3.3.1.

Little testing of this revision has been by me. Please submit error reports, suggestions and clarifications as I won't be able to do much testing of this myself. I would also appreciate suggestions for configuration settings.

Installation
This will install the ASSP SMTP anti-spam proxy on a single SME 7 server.

Using the Server-Manager E-Mail panel, disable virus scanning and spam filtering before proceeding.

Log into your SME server as root. I use SSH from my Linux desktop, or PuTTY if I'm at a Windows box. Install Perl Modules Configuring CPAN is a little more confusing than installing a .rpm, but not much...and it only needs to be done once. You can just hit 'enter' for almost all prompts and it will still work. Be patient where necessary. Choose a mirror near you (more or less).

Initial configuration (run as root):

perl -MCPAN -e 'shell'

then at the cpan> prompt:

1 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

install Email::Valid Net::RBLClient Mail::SPF::Query Mail::SRS File::Scan::ClamAV File::ReadBackwards exit

If you get errors installing File::Scan::ClamAV, do this:

perl -MCPAN -e shell #gets you to the cpan> prompt test File::Scan::ClamAV #this will fail but is necessary to download the module look File::Scan::ClamAVvi clamav.conf #make the changes make install exit

and add 'true' after these two lines like this:

Foreground true ScanArchive true Download and Install ASSP Download ASSP-1.3.3.1 (or later) and spamdb.zip from http://assp.sourceforge.net.

Logged in as root, download and unzip ASSP:

cd ~ wget http://easynews.dl.sourceforge.net/sourceforge/assp/ASSP_1.3.3.1-Install.zip wget http://superb-west.dl.sourceforge.net/sourceforge/assp/spamdb.zip unzip ASSP_1.3.3.1-Install.zip unzip spamdb.zip

Move the resulting sample SPAM database and the ASSP directory to /opt:

2 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

mv spamdb ASSP_1.3.3.1-Install/ASSP/spamdb.sample mv ASSP_1.3.3.1-Install/ASSP /opt

SME Server Configuration

Here we change the port that SME Server uses for SMTP.

Create a custom template directory:

mkdir -p /etc/e-smith/templates-custom/etc/services/

Copy the original template fragment to customize:

cp /etc/e-smith/templates/etc/services/10standard /etc/e-smith/templates-custom/etc/services/10standard

Edit the new fragment:

vim /etc/e-smith/templates-custom/etc/services/10standard

Change the line that says:

smtp 25/tcp

mail

To say:

3 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

smtp 125/tcp

mail

Then rebuild the /etc/services file:

/sbin/e-smith/expand-template /etc/services

Look at the /etc/services file to verify the changes:

cat /etc/services|grep smtp

Actuate the changes:

killall qmail-remote #optional only needed if server is bogged down with SPAM /sbin/e-smith/config setprop smtpd TCPPort 125 /sbin/e-smith/config set ASSP service TCPPort 25 status enabled access public /sbin/e-smith/signal-event remoteaccess-update /sbin/e-smith/signal-event email-update

Your SMTP server should now be listening on port 125. Test it like this:

telnet localhost 125

You should get something like:

4 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 server.your.org mailfront ESMTP

Type 'QUIT' then <enter> to exit.

Configure the SMTP Proxy

Start the SMTP proxy ignoring error messages:

perl assp.pl

Open a web browser on a network-attached PC to http://ip.of.your.server:55555. You can also use a second virtual terminal on your SME Server and the Lynx text-mode browser: 'lynx localhost:55555'. See also Configure for Remote Maintenance below in the Notes section.

Log into the configuration page. Use 'admin' to log in with the initial password of 'nospam4me'.

You may want to click the Expand All option to see or search all available options.

You will want to change: Network Setup - Listen Port (set it to 25) - SMTP Destination (set it to 125) SPAM Control - Spam Error ( postmaster@your.domain is a good choice)

5 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

CC Mail - Copy Spam and Send to this Address (I always create a 'spambucket' mailbox/account and use 'spambucket' here) (this implements a single organization-wide quarantine as opposed to a per-user quarantine) SPAM Lover/No Processing - All Spam-Lover (postmaster is the default and matches the Spam Error setting) - Unprocessed Addresses (enter the e-mail addresses of anyone you want ASSP to ignore here) Whitelisting - Regular Expression to Identify Non-SPAM (optional - see Getting Messages Through the Filter below) - Whitelisted Domains (optional don't put your e-mail domains in here) Relaying - Local Domains (add your domain(s)) - Default Local Host (your primary domain) Validate Local Addresses Recipient validation can save you a LOT of time and trouble. However, these settings can cause ASSP to fail if they are not exactly correct. That being said, you may be better off disabling recipient validation until you have your system all up and running nicely. You can 'tail /opt/ASSP/maillog.txt' to look for errors. - Validate Recipient Addresses to Conform with RFC 822 (enabled) - Do LDAP lookup for valid local addresses (enable LDAP lookups for recipient validation) - Lookup valid Local Addresses from here (may be blank, but see important notes section below) Attachments & Viruses - External Attachment Blocking (set to 1 to block executable attachments) - Port or file socket for ClamAV (/var/clamav/clamd.socket) Bayesian Options - Bayesian Check (set to 1 to enable Bayesian filtering)

6 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

TestModes - Bayesian Test Mode (enable only for testing, normally disabled) E-Mail Interface - Admin Mail Address (admin) Collecting - Spam Collect Addresses (user names of long-gone users may work well here, else blank - Use Subject as Maillog Names (disable to automatically manage spam/notspam collections) LDAP Setup - LDAP - LDAP - LDAP - LDAP Root) - LDAP Host (localhost) Login (blank) Password (blank) Root container (dc=yourcompany,dc=com) (in Server-Manager, Directory Server Filter (mail=EMAILADDRESS)

Server Setup - Run ASSP as a Daemon (don't use this if you want to see console messages - Web Admin Password (change it now!)

Don't forget to click on 'Apply Changes'.

Testing the ASSP Proxy

Stop the ASSP proxy by hitting Control-C then restart it:

perl assp.pl

7 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Check the startup messages for errors. See Troubleshooting below.

You can test the running proxy, from another terminal window:

telnet localhost 25

You should get the same response as before when you tested port 125.

Rebuild the SPAM Database Periodically

Hit control-C to stop the proxy or just open another console window, or, if ASSP is already running as a daemon, just continue. Modify /etc/crontab Create another custom template fragment:

mkdir -p /etc/e-smith/templates-custom/etc/crontab vim /etc/e-smith/templates-custom/etc/crontab/40assp

Add the following two lines (plus a blank line at the end) to the new file and save:

# Perform the nightly ASSP proxy updates at 3:20am 20 3 * * * root /opt/ASSP/nightly.sh >/dev/null 2>&1

Recreate the /etc/crontab file:

/sbin/e-smith/expand-template /etc/crontab

8 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Verify your modifications:

cat /etc/crontab Create the nightly.sh Shell Script Create nightly.sh in the /opt/ASSP directory:

vim /opt/ASSP/nightly.sh

Add the following:

#!/bin/sh # # nightly.sh - Gene Cooper # Please e-mail modifications or updates to gcooper(at)sonoracomm.com. # # Script to update the DNS Blackhole List and rebuild the # SPAM database for the ASSP SMTP proxy

BASE=/opt/ASSP cd $BASE # Rebuild the SPAM database perl rebuildspamdb.pl # Reload the assp.cfg kill -HUP `cat pid`

9 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Make the nightly.sh script executable:

chmod a+x /opt/ASSP/nightly.sh

To test the newly created script:

/opt/ASSP/nightly.sh

and monitor the log.

Once the script runs properly, copy the sample spamdb to the proper location. Note that running the nightly.sh (rebuildspamdb.pl) destroys the sample spam database.

cp /opt/ASSP/spamdb.sample /opt/ASSP/spamdb

Configure ASSP to Start Automatically at Bootup

Edit the /etc/rc.local file.

vi /etc/rc.local

And add the following lines at the bottom:

10 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

# Start the ASSP proxy /usr/bin/perl /opt/ASSP/assp.pl /opt/ASSP

Then reboot the server to test:

signal-event reboot

Configure for Remote Maintenance

Most mail servers are behind a firewall of some sort. We often use NAT devices such as Linux-based SnapGear firewall routers. ( http://www.snapgear.com ) You will want to forward port 55555 on your firewall to your SME mail server to enable remote access to the ASSP Configuration web page.

Troubleshooting
ASSP Log File

tail -50 /opt/ASSP/logs/maillog.txt #last 50 lines tail -f /opt/ASSP/logs/maillog.txt #monitor the log Find and Remove SPAM from the NOTSPAM Collection Change to the 'notspam' directory

cd /opt/ASSP/notspam

Find and remove SPAM

rm -f `grep -lir &quot;assp-spam: yes&quot; *`

11 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Analyze Individual Messages Use a web browser to open the ASSP management console. ( http://server:55555 )

On the left, click on &quot;Mail Analyzer&quot;.

Copy the headers and the message content into the window and click 'Analyze'.

Notes

ASSP Daemon ASSP does NOT have to be stopped to rebuild the SPAM database, update the DNS black hole list or update the virus databases. A HUP signal will reload the assp.cfg. (note the use of backticks):

kill -HUP `cat /opt/ASSP/pid` Recipient Validation - Local Users The LDAP Recipient validation is quite strict...and so useful that you will want tto implement it. However, you need to know how to configure the exceptions.

If you are hosting multiple domains or if any of your users are using a mail alias (pseudonym), such as firstname_lastname@yourdomain.com, you need an entry in the 'Lookup valid Local Addresses from here' field.

If you only have a few aliases in use, enter them here separated by pipe symbols (vertical bars). If you have multiple domains, or lots of aliases in use, enter something like:

file:/home/e-smith/files/samba/netlogon/email_users.txt

12 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

In this example, the admin user can log in on a Windows box to edit the file in the netlogon share. On the Windows workstation, open a command prompt, enter

net use x: \servernamenetlogon

then edit the text file with notepad. Place one entry per line. Getting Messages Through the Filter I recommend using your phone number as a 'secret key' that will allow a message to pass through the SPAM filter. To do this, edit the 'Regular Expression to Identify Non-Spam' (under Whitelisting) and enter something like this (modify for YOUR phone number):

520D{0,3}322D{0,3}9557

Then if you need 'road warriors' or anyone else to get a message through, just have them place the phone number in their signature (or anywhere else in the message). Vacation Messages (Autoresponders) If you ever send a message to a spammer, that spammer becomes whitelisted...a bad thing. In the same vein, an autoresponder (vacation message) automatically replies to any incoming message and this just might be a spammer. Autoresponders can cause your SPAM filtering to become less accurate by whitelisting spammers.

To remedy the threat, I recommend using the 'Expression to Identify Redlisted Mail'. Redlisted messages are filtered, but they do not contribute to the whitelist.

[autoreply]

13 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

You can use any 'code' you want, but most autoresponders already include the above text, so it makes a good place to start. Make sure that all of your users that implement autoresponders to include this text.

Uninstall ASSP
Notes: Perform as root. Dots in commands and the order are important.

Move the custom template fragments created during installation to root's home directory:

cd ~ mv /etc/e-smith/templates-custom/etc/services/10standard . mv /etc/e-smith/templates-custom/etc/crontab/40assp .

Edit the startup script so ASSP doesn't start on boot:

vi /etc/rc.local

Comment out the ASSP startup line by putting a # sign in front of it and save.

Recreate the /etc/crontab and /etc/services files:

/sbin/e-smith/expand-template /etc/crontab /sbin/e-smith/expand-template /etc/services

Check your work with:

14 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

cat /etc/crontab cat /etc/services|grep smtp

Actuate the changes:

/sbin/e-smith/config setprop smtpd TCPPort 25 /sbin/e-smith/signal-event email-update

Stop the ASSP proxy service:

kill `cat /usr/local/assp/pid`

Test the changes:

telnet localhost 25

You should get something like:

Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 server.your.org mailfront ESMTP

Type 'QUIT' then <enter> to exit.

End User Instructions

15 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

General The latest ASSP is now installed and running on your server. All junk mail and virus-infected messages should now be filtered out unless a particular mailbox is specifically marked to not be filtered. Your server messaging performance should improve dramatically.

This new version of ASSP includes antivirus scanning as one of it's capabilities. If you currently are paying for another mail-scanning antivirus package for your server, you may not want to renew it when it next comes up for renewal.

This new version of ASSP also includes Recipient Validation using the LDAP database already on your server. This drastically reduces junk, particularly the junk that builds up in the spambucket, greatly easing the SPAM Administrators' reviewing task. Furthermore, this will eliminate all of the bounce messages normally generated by this junk.

ASSP automatically maintains a 'whitelist' of valid senders whose mail is not to be filtered. Anyone you SEND a message to is automatically whitelisted and mail from that person will never be filtered.

ASSP 'learns' as it goes. It will make more errors at first and fewer errors later. Important: be sure to whitelist all of your company's contacts immediately! See Getting Started below. Documentation The web page and documentation are here: http://assp.sourceforge.net, if you are interested. Administration All messages flagged as junk will be sent to a new mailbox we created called 'spambucket'. Someone (the SPAM Administrator) needs to review this mailbox at least daily in order to: 1. deal with any falsely filtered messages (if any), and 2. delete the accumulated junk so that it doesn't build up

16 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

This review is a fairly quick and easy procedure. As long as someone reviews the spambucket for false-positives, no valid mail will ever be lost.

I suggest using webmail (https://<servername>/webmail) to review the junk. Log in as 'spambucket' with a password of 'spambucket'. You may also configure your regular mail client by: - add a new mail account username 'spambucket', password 'spambucket' - create a new SPAM folder for junk messages usually a subfolder of Inbox - create a message processing rule that directs new messages from the new account into the SPAM folder
Falsely Filtered Mail

If a valid message is found in the spambucket, do this (in two separate steps or the user will not get their mail!): 1. Forward the message to the proper user so he/she gets their mail, then 2. Forward the message to 'asspwhite@your.domain'.

This will whitelist the sender so their mail is never falsely filtered again.
Junk That Gets Through

If any junk messages make it through the filter, there is an easy way to use that message to train the Bayesian filter to be more accurate in the future. Just forward the junk message (preferably as an attachment) to 'asspspam@your.domain'. Management Console The management console for ASSP is at http://<servername>:55555.

PLEASE read everything carefully before making ANY changes. We don't want to:

17 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

- screw up the mail routing, or - poison the SPAM database

BE CAREFUL if you choose to make ANY changes! You have been warned. Important 'Getting Started' Information I strongly suggest you have all users send a special whitelisting message to 'asspwhite@your.domain'. DO THIS NOW.

Have them put all the contacts in their address book as recipients in the CC: or BCC: field. This will automatically 'whitelist' all of their contacts. This will go a long way towards mitigating any 'friction' you might encounter as a result of implementing a SPAM filter.

Anyone you send a message to is also automatically whitelisted.

TELL PEOPLE NEVER REPLY TO OR OTHERWISE SEND MAIL TO A SPAMMER. This may not be obvious to everyone, but it is critical. We don't want to whitelist spammers now, do we?

If any user uses an autoreply or 'vacation message', make sure that when the messages are sent out that they include this exact text: [autoreply] This will keep the autoreply messages from poisoning the SPAM database. If this is not done, your SPAM database may become corrupted and your SPAM detection accuracy will suffer.

The text I selected is commonly included by default in many autoreply messages, so you may not have to change anything. But please make sure.

18 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Tell your 'road warriors' or users that work from home to place your main phone number in their e-mail signature and always use their signature. This will keep their mail from being filtered. This is important. Not only will mail fail to get through, but it will also 'poison' the database and impede accuracy.

I generally program the main phone number as a 'magic code' that will allow any message to pass through the SPAM filter. We can add any other code you want.

If a junk message makes it through to your inbox, you can - just delete it and forget it, or - you can forward it to 'asspspam@your.domain' and then delete it.

Reporting the errors will help ASSP become more accurate over time. E-Mail Interface The ASSP E-Mail Interface allows you to add recipients to your whitelist, report SPAM that made it through the filter or to report messages that were falsely classified as SPAM, messages to this address if a spam message makes it through the filter to your inbox.

asspspam@your.domain

Forward ham messages here if you find them in the spambucket and they were mis-classified as spam.

asspnotspam@your.domain

19 / 20

ASSP Spam Filter on SME 7

Written by Gene Cooper Monday, 07 August 2006 17:44 - Last Updated Sunday, 20 July 2008 08:43

Any e-mail addresses anywhere in messages sent here will be added to the whitelist to never be filtered.

asspwhite@your.domain

Any e-mail addresses anywhere in messages sent here will be removed from the whitelist.

asspnotwhite@your.domain

20 / 20