Emergency Information
 General Information

Spam Flagging Service


Overview

This service helps IMSS users reduce the amount of unsolicited commercial email (spam) landing in their inboxes.

IMSS has software on our mail servers to automatically identify likely spam email. This software does not remove the suspected-as-spam messages, but instead flags each message it believes to be spam with a special header.

Note that this is an automated flagging mechanism. It is quite possible that real emails will be flagged with the special header, just as it is possible that spam mail messages will get through without being flagged.

This service allows users to configure their own desktop mail program (Eudora, Outlook, Netscape, etc.) to do specific things with the suspected spam email.

After October 11, 2003, a new aspect of this service will be available on our upgraded mail system: server-side spam processing controls. Users may configure their account via the utils.its.caltech.edu web interface to filter messages flagged as spam into a special spam folder without downloading the messages to their local computer.

Below, we have configuration guides for both the server-side spam processing controls as well as guides for specific desktop email programs to use this spam-flagging service. If you do not see your preferred email program listed below, please send email to unix-admins@caltech.edu requesting instructions for your particular program.


Configuring your account to use the IMSS Server-Side Spam Processing Controls

1. Go to https://utils.its.caltech.edu/ and log in.

2. Go to "Change email settings".

3. Under "Use of legacy mail server", if your account preference for "Keep your mail on the legacy mail server?" is set to No, you can skip to step 6.

4. If your account preference under "Use of legacy mail server", if your account preference for "Keep your mail on the legacy mail server?" is set to Yes, then you need to read over our Choices for Accessing IMSS Email guide and decide if you are ready to transition from the legacy system to the upgraded mail system. It is dependent on which program or programs you use to access your email. We have instructions for transitioning to the upgraded mail system for pine and mutt users online; for assistance with other email programs, please contact unix-admins@caltech.edu.

5. If you are ready to transition from the legacy mail system to the upgraded mail system, do so according to the instructions for your particular email program.

6. In the "Change email settings" section of utils.its.caltech.edu, under "Spam processing", set the preference "File messages marked as spam into a folder named spam on the mail server?" to "yes".

Note: if you use POP to read mail, you won't be able to see messages in this folder without using IMAP or the IMSS Webmail system.

Note: This setting has no effect if "forwarding" or "use of the legacy mail server"is set to "yes".

7. Set your preferred number of days in the "Keep spam messages in spam folder for [ ] days (enter "0" to keep forever)" field.

8. Submit changes.

9. Log out of https://utils.its.caltech.edu/.


Configuring your desktop email software to use the IMSS Spam Flagging Service

IMSS has automated software on our mail servers, an analysis package called SpamAssassin, to automatically identify likely spam email.

This software does not remove the suspected-as-spam messages, but instead flags each message it believes to be spam with a special header. This special header allows users to configure their own email software to do specific things with the suspected spam email.

The SpamAssasin server software is configurable in various ways, but it is an automated flagging mechanism. It is quite possible that real emails will be flagged with the special header, just as it is possible that junk mail messages will get through without being flagged.

If this page about our new flagging service does not meet your needs, please email unix-admins@caltech.edu with your specific email program and your needs, and we will do our best to suggest additional solutions.

Contents of the rest of this page:


General instructions

The message filter criteria is the presence of the x-header:

X-Spam-Flag: YES

Depending on your mail client software, you may have the choice to configure your filter to move the messages with that flag to a separate folder, to change their priority, to mark the messages as special, to mark them as read, to delete them, or a few other options.

IMSS recommends choosing to filter messages with the X-Spam-Flag: YES header to a separate folder called "spam" or "junkmail". That way, you can periodically check the folder to make sure no real, possibly important, messages have been flagged by mistake. Every so often, check that spam folder to be sure no messages of value were put there in error, and then bulk-delete the contents of that folder.

If you choose to configure your filter to simply delete the messages with the X-Spam-Flag: YES header, your mail client software may delete flagged messages you would not wish to be deleted, without you ever seeing those messages. At this time, IMSS does not have a way to restore such deleted messages. We strongly recommend against configuring your filter to delete messages with the special header!


Advice About Specific Email Clients

Eudora

When you create a filter on a header, if the header isn't in the pull-down list (which this one won't be), you can overtype into the blank where the pull-down menu is and simply type in the header to search (X-Spam-Flag in this case).

  1. Go to Tools/Filter/New and then type "X-Spam-Flag" (without the quotes) into the Header window.
  2. Click Incoming and Manual options under Match (but don't select Outgoing). That way when you've finished creating the filter you can test it by doing a Manual run of the filter.
  3. For the match, select Contains and then type (or paste) YES.
  4. For Action, you can do Transfer To and then make a special mailbox for spam, or you can even just flag it in your inbox by changing its color.
  5. Save before you exit the Filters window.
  6. Test the new filter by doing Select All in your inbox, and then Special/Filter Messages.

Let us know if you have trouble. If you do encounter a problem, we'd like to know the version of Eudora you're using. We've not created filters using every version, and it's possible there will be some variation between the filtering capabilities of various versions.

Outlook

  1. Go to Tools->Rules Wizard...
  2. Click 'New...' (on the top right)
  3. Choose 'Check messages when they arrive'
  4. Click 'Next'.
  5. Check 'With specific words in the message header'.
  6. Click on 'specific words'.
  7. Type in: X-Spam-Flag: YES (one space between the : and the Y)
  8. Click 'Ok'.
  9. Click 'Next'.
  10. Check 'Move it to the specified folder'.
  11. Click on 'specified'.
  12. Highlight an existing folder, or create a new one.
  13. Click 'Ok'.
  14. Click 'Next'.
  15. Click 'Next'. (Again, unless you want to add exceptions.)
  16. Give the rule a name. (The default is what you typed for specific words above.)
  17. Check 'Turn on this rule'. (You may or may not want to check 'Run this rule on my Inbox now'.)
  18. Click 'Finish'.

 

Outlook Express

Outlook Express for Windows:

It doesn't look like Outlook Express for Windows can filter messages based on the contents of their headers, so users of this mail client are unable to take advantage of the special headers using their mail client software at this time.

Outlook Express 5.x on Macintosh:

  1. From the menu bar, choose Tools, then Rules.
  2. Select POP or IMAP (most users are using POP), and then hit for a new rule.
  3. Under the section marked "If", choose "specific header" and then type or paste in the name of the header, which is "X-Spam-Flag".
  4. Under "Contains:" type in YES.
  5. In the section marked "Then", specify an action -- move to a new folder, change its status or color, as you see fit. Note that we do not recommend simply deleting messages found by this rule.
  6. The Enabled box needs to be checked in order for this rule to be active -- it will be by default.

Outlook Express 4.5 on Macintosh:

The instructions are the same as for Outlook Express 5.x for Mac, but the menu item under Tools is called Mail Rules, and there's no choice between POP/IMAP. The rest is the same, except that if you are sending messages found by this rule to a special mail folder, you must already have created the destination folder before you create the rule.

Netscape

Netscape 7 allows you to create a custom filter. You can supply the special x-header information to Netscape 7 by doing the following:

  1. In the Netscape mail window, select Tools->Message Filters.
  2. In the resulting dialog box, select New
  3. In that dialog box:
    • Set the filter name to "Spam"
    • Leave the radio button on "Match any of the following"
    • Set the drop-down that says "Subject" to "Customize". You will then get a dialog box allowing you to define a "New message header". Enter "X-Spam-Flag" (without the quotes), press Add, then OK.
    • Click on the "Subject" drop-down again. "X-Spam-Flag" or a truncated version thereof should be selectable. Select it.
    • Set the drop-down with "contains" to "is"
    • Enter "YES" (in all caps, without quotes) in the type-in box.
    • Under "Perform this action", leave the action as "Move to folder", and use the other drop-down to choose a folder to move to ("Spam" is probably a good choice; use "New folder" to create it if not).
    • Click OK.
  4. Click OK again.

     

Netscape 6.2.1 does not allow you to create custom filters, so users of this version are unable to take advantage of the special headers using their mail client software at this time.

Netscape 4.7.8 allows you to create a custom filter. You can supply the special x-header information to Netscape 4.7.8 by doing the following:

  1. In the pulldown bar at the top of your Netscape 4.78 window, go to "Edit: Message Filters". A new window will open.
  2. Click "New". Click "Advanced". A new window will open.
  3. Enter "X-Spam", click "Add", click "OK". The latest new window will close.
  4. In the pulldown list, select "X-Spam-Flag".
  5. In the "contains" box, enter "X-Spam-Flag: YES".
  6. In the "Perform this action" pulldown list, select "move to folder".
  7. Click "new folder" and create a spam folder. It should then be selected in the pulldown list of your folders.
  8. Click OK.
  9. The next time you check your mail, check to see if any messages were automatically filtered into your spam folder!

Mac OS X Mail

Mac OS X's built-in Mail program can create filters based on custom headers.

  1. In the menu bar click 'Mailbox' then 'New Mailbox' and create the mailbox
    you want the spam to end up in.
  2. In the menu bar click 'Mail' then 'Preferences...'
  3. Click 'Rules' then 'Create Rule'
  4. Add a description of the rule, then click the 'From' Criteria then
    click 'Expert...'
  5. In the Header: field enter 'X-Spam-Flag', click 'Add Header' and 'OK'
  6. Now click 'From' and select 'X-Spam-Flag', select 'Contains' in the
    next box and enter 'YES' in the third Criteria box.
  7. In the Action section check 'Transfer to mailbox' and select the
    desired mailbox. click 'OK'
  8. Adjust the rule priorities if you want and dismiss the Mail Preferences
    dialog box.
  9. The next time you check your mail, check to see if any messages were automatically filtered into your spam mailbox!

Mutt

The mutt philosophy is to make it easy for you do do things after reading your email. It assumes you are using something like procmail if you want to distribute your email to various places before you start reading.

If you use procmail to, for example, distribute your email to /home/yourname/mail/spam and /home/yourname/mail/goodmail you need to add a line like this to your /home/yourname/.muttrc file (or your /home/yourname/mutt/muttrc file):

mailboxes /home/yourname/mail/goodmail /home/yourname/mail/spam

that will tell mutt where to look for new mail. You can also indicate which messages have been marked as possible spam on the index page by adding:

mono index bold "~h \'X-Spam-Flag: YES"

 

Elm

The elm-filter program does not allow you to filter on the special X-header. We recommend our elm users familiarize themselves with procmail, which allows you to filter based on the special header outside of your mail program.

Pine

Start from pine's main menu.
  1. Press S for Setup.
  2. Press R for Rules; then press F for Filters.
  3. Press A for Add.
  4. Initially, the Nickname field should be highlighted. Select it by pressing enter/return, then set the nickname to "Spam Filter".
  5. Press X to add another header. Enter "X-Spam-Flag". After you do this, the line allowing you to set the match pattern for X-Spam-Flag should be highlighted. Press enter/return, then as the text to be added enter YES (all caps).
  6. Check the CURRENT FOLDER CONDITIONS BEGIN HERE section. Make sure that the Specific option is set and the Folder List is "INBOX". If not, make the required changes so that that is the configuration.
  7. Scroll down to the ACTIONS BEGIN HERE section. Make sure the Filter Action option is set to Move. Select the Folder List item in that section, and set it to "spam".
  8. Press E for Exit Setup and press Y to commit the change. You may be asked to create the spam folder; accept that change if this happens.
  9. Press E again for Exit Setup and press Y to commit the change.
At this point your filtering setup is in place; whenever you read mail, suspected spam will wind up in the "spam" folder in the "mail" directory within your home directory.


Using procmail to Filter Your Messages

New Mail System Instructions

The new IMSS mail system allows users to set up complex filtering using a web-based interface. Please read our instructions page carefully before setting up complex filtering.

Legacy Mail System Instructions

procmail allows users to perform filtering tasks on the UNIX Cluster.

To use procmail to filter spam, first, set up .procmailrc and .forward files suitable for using procmail and then, edit the .procmailrc file so it contains the following text right after the line setting LOCKFILE: 

:0
* ^X-Spam-Flag: YES
spam
Make sure there's a blank line before and after the added text.

Suspected spam mails will then start being saved in the spam file in your mail folder directory (as specified by the MAILDIR variable in .procmailrc).

(Note to users already using procmail: all you have to do is add the three-line rule above to your .procmailrc. You can position it where you like to allow other rules to apply to suspected spam first.)


Advanced Client-side Filtering Details

If you want to configure your mail client to perform more advanced filtering, you may.

SpamAssassin modifies email only by adding headers. Suspected non-spam mail will get an "X-Spam-Status" header that starts with "No," added. (The header also contains a brief summary of the filter processing.) Suspected spam mail gets:

- an X-Spam-Status header starting with "Yes,"
- an "X-Spam-Flag: YES" header
- an "X-Spam-Report: Caltech-IMSS-tagging-program-classifies-this-message-as-spam" header
- an "X-Spam-Checker-Version" header with version info on SA

Mail that is not suspected to be spam gets something like this:

X-Spam-Status: No, hits=1.8 required=5.0 tests=NO_MX_FOR_FROM version=2.20

Mail that is suspected to be spam gets all sorts of X-headers you can filter on, for example:

X-Spam-Status: Yes, hits=30.7 required=5.0
tests=CLICK_BELOW,REMOVAL_INSTRUCTIONS,EXCUSE_12,EXCUSE_3,DIRECT_EMAIL,OPT_
IN,SENT_IN_COMPLIANCE,CALL_FREE,EMAIL_MARKETING,SECTION_301,SUBJ_REMOVE,LINES
_OF_YELLING,MAILTO_WITH_SUBJ,MAILTO_WITH_SUBJ_REMOVE,MAILTO_TO_REMOVE,MAILTO_
TO_SPAM_ADDR,BIG_FONT,CLICK_HERE_LINK,ASCII_FORM_ENTRY,MAILTO_LINK,CTYPE_JUST
_HTML version=2.20
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 2.20 (devel $Id: SpamAssassin.pm,v
1.77 2002/04/06 19:28:30 hughescr Exp $)
X-Spam-Prev-Content-Type: text/html; charset="iso-8859-1"
X-Spam-Prev-Content-Transfer-Encoding: quoted-printable
X-Spam-Report: Caltech-IMSS-tagging-program-classifies-this-message-as-spam

 

 
 Email Services