yossman.net - user accounts - email help and configuration information.
document URL: http://www.yossman.net/support/email.html
document last revised: yossarian holmberg/info at yossman.net/2007-02-18.


".forward" file - an introduction to email forwarding.

if you want email arriving at your yossman.net email address to be automatically forwarded to another email address, create a text file called ".forward" in your home directory. the only line in the file should be the email address of where you would like the email to go instead.

a simple example way to create this .forward file from your yossman.net shell account prompt is shown below. you@somewhereelse.com should be changed to whatever email address you want email forwarded to:


yossman.net> cd
yossman.net> echo 'you@somewhereelse.com' > .forward
yossman.net>

note: the above example will erase any previous ".forward" file you may have installed at some point in the past.


procmail - filtering your email.

using the ".forward" file in combination with procmail, a tool for unix that has already been installed for anyone to use, you have the option of sorting email as it arrives for you into different folders, which stay stored in your account's disk space on yossman.net. these filtered emails do not just get delivered to your email INBOX, the default place all your incoming email is stored if you are not already forwarding it to another email address.

note: using the ".forward" file in this manner means your email will not be sent automatically to another email address or system, as in the previous ".forward" example above. instead, mail is directly handled by the system here and sorted into folders that stay residing on yossman.net. while it is possible to tweak the procmail filtering rules listed below to selectively forward emails to another email address, we don't cover that information here yet.

we discuss some simple procmail rules below.


procmail is at once simple and complex to use. while my instructions here take you step-by-step through the initial procmail setup, i strongly recommend you do a "man procmail" from the yossman.net unix shell, which browses through the procmail user reference manual. thanks to procmail.org for some of the information seen here, and for making great email filter software. you can click the link to check with them for more filtering examples and information.

to get started, create a text file in your home directory called ".procmailrc". since you start in your home directory automatically when you log into yossman.net via SSH, you can just use the following command from the unix shell to get into a "text editor" tool for unix called "pico" to create the ".procmailrc" text file if you don't already know how to do so:

yossman.net> pico -w .procmailrc

and then put the following text in it:

#Set on when debugging
VERBOSE=off

#Replace `mail' with your mail directory (Pine uses mail, Elm uses Mail)
MAILDIR=/home/username/mail

#Directory for storing procmail log and rc files
PMDIR=/home/username/.procmail

#logfile
LOGFILE=$PMDIR/log

#sorting rules in this file
INCLUDERC=$PMDIR/rc.maillists

"VERBOSE" should be left as "off" unless you want more detailed information recorded in your procmail "log" file, which shows what procmail is doing when it processes every piece of incoming email.

"MAILDIR" is the directory in your home directory which holds your email. Almost all of you are using pine here, so use "/home/username/mail" where "username" is your account name.

"PMDIR" is the directory where logfiles and your personal configuration files (.rc files) will be kept, usually in "/home/yourlogin/.procmail". you will need to make this directory.

"LOGFILE" is the name of the file where the log of procmail's activities will go (duh).

"INCLUDERC" tells procmail the name of the file where the rules for sorting your email will go, which in my above example, is 'rc.maillists'. this file goes into the directory you put for "PMDIR", which in my example is "/home/username/.procmail/". the format of this file varies.

simple filtering rules for procmail are usually three lines each. an example rule you might put into "rc.maillists" follows:

:0:
* ^TO.*FreeBSD.org
freebsd-mail

leave the top line exactly as it is written, a zero with a colon before and after.

the second line and third lines translate to "if it's originally addressed to anyone@FreeBSD.org, put the email in the folder "freebsd-mail". many mailing lists have a "To:" line that shows the mail going back to them, and not to your email address as most normal email would. if the freebsd-mail folder doesn't already exist, procmail will create it for you and start storing emails that matched there, instead of in your INBOX. you can then check that folder whenever you like to catch up on the latest email(s) from FreeBSD mailing lists.

many users have asked if there is a simple way to sort email based on it's size. after some brief testing it looks like you can use the following additional rule in your rc.maillists file:


:0B:
* >512000
too-big

that rule above should catch any emails coming into your inbox that are over 512KB (half a megabyte) and automatically move it into a folder in your home directory's mail/ directory called 'too-big'. a lot of virus-type emails may fall into this catagory at a smaller size; but you may also have to dig out some legitimate emails where people have attached pictures or other valid items you actually want. you will probably want to increase or decrease the '512000' number as you see fit to find a good balance.


when you have finished creating the ".procmailrc" file, created the ".procmail directory, and have finished generating the intial rc.maillists procmail filter rules file, you should be set to go.

the final step to enable your new procmail setup is to put the following line into your .forward file, usually replacing anything you already have in there:


"|IFS=' ' && exec /usr/local/bin/procmail -f- || exit 75 #username"

exactly as it is written, including the quotes. note: make sure you replace username with your username in the above string.

you may want some sort of reporting on the sorting as well. procmail comes with a program called mailstat which reads through your procmail logs to tell you the latest sorting that occured. i have added a line to my ".login" file in my home directory that says:


mailstat /home/username/.procmail/log

replace username with your username and keep the rest of it the same (assuming you named your procmail directory '.procmail' as in the examples above).

this should give you a report just before giving your shell prompt. you can run that command line at any time to get an up-to-date report; by default mailstat will report changes since last time you asked for a report.

you may also want to watch the procmail program's log file in a raw format without the aid of mailstat, from the yossman.net shell command prompt try:


yossman.net> cd
yossman.net> tail -f .procmail/log

you can hit ctrl-c to stop the display of the log at any time. tail -f allows you to continue watching the procmail log file as it is being written, as procmail processes each email coming into your inbox.



procmail filtering with spamassassin - config notes for catching "spam" (unwanted commercial email).

information and examples of more advanced procmail filtering rules.

thanks to spamassassin.org for some of the information seen here, and for making great email filter software.

to start using spamassassin, the following is an example of some rules to put into the 'rc.maillists' file as we talked about above. you can copy the contents of the following box directly into a newly-created rc.maillists file if you haven't started using procmail yet:

# SpamAssassin sample procmail rc file.
#
# Pipe all email through spamassassin.
#
#
# The first rule ensures that only emails SMALLER than 250 KB
# (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam
# isn't bigger than a few KB, and working with big messages can bring
# SpamAssassin to its knees.
#
# The part about the '.lock' file ensures that only 1 spamassassin
# is loaded at a time, to help control load spikes on yossman.net
# if a temporary flood of email arrives.
#
#
# 1. RUN EMAIL THROUGH SPAMASSASSIN:
#
:0fw: spamassassin.lock
* < 256000
| spamassassin
#
#
#
# 2. IS EMAIL LIKELY SPAM?
#
# All mail tagged as spam (eg. with a score higher than the set threshold)
# is moved to mail folder "spam-probables".
:0:
* ^X-Spam-Status: Yes
spam-probables
#
#
#
# 3. WORKAROUND FOR DISAPPEARING 'F'ROM HEADER:
#
# Work around procmail bug: any output on stderr will cause the "F" in "From"
# to be dropped.  This will re-add it.
:0
* ^^rom[ ]
{
  LOG="*** Dropped F off From_ header! Fixing up. "
  
  :0 fhw
  | sed -e '1s/^/F/'
}

if you are fairly confident that spamassassin's assessment of every email is probably correct, you could go one step farther and add the following to the rc.maillists file after the first rule:

#
# 1.5 DOES EMAIL SCORE REALLY HIGH SPAM PROBABILITY RIGHT AWAY?
#
# Mails with a score of 15 or higher are almost certainly spam (with 0.05%
# false positives according to rules/STATISTICS.txt). Let's put them in a
# different mbox. (This part of the configuration is optional.)
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
spam


there are many help sites around the Net that demonstrate some very powerful ways to essentially 'pre-sort' your email using procmail to dramatically lower the number of minutes or even hours you have to spend looking at email you didn't want in the first place.

spamassassin configuration generator is available from Michael Moncur (thanks mike!) to help facilitate the creation of your spamassassin filtering rules file.

spamassassin creates a directory, in your home directory, called '.spamassassin', when it first runs. in this directory it creates a file called 'user_prefs'. this file, at first, is a copy of the yossman.net system-wide generic 'user_prefs' for spamassassin. it's meant as a starter file for the spamassassin configuration, so it can nail more of your spam right off the bat. by tweaking this 'user_prefs' file to set the 'spam score' based on various details of each email coming into your account, you can take emails you KNOW are always spam and score them much higher on the 'probably spam' meter than the system normally would for you.

since you will automatically get your own copy of this file in the .spamassassin directory in your home directory, you can go edit this file and change the scores as you see fit. changes take effect as soon as you've saved the file again, and will apply to all future emails received. from the yossman.net shell prompt:


yossman.net> pico -w ~/.spamassassin/user_prefs

you will notice at the bottom of this user_prefs file the 'score' parts, one per line. you can get the spamassassin.org list of spamassassin spam score test names (2003-09-16), or you can get a local one-time mirror list of spamassassin score test names here as well (as of spamassassin 2.5.5 yossman/2003-09-15).



additional notes for people using POP3 and IMAP email clients:

if you do not use 'pine' or 'mutt' or a similar unix shell email client, the 'user_prefs' file can be modified to tell spamassassin to re-write the actual SUBJECT line of emails it thinks are spam. this makes it much easier to get POP3 clients to 're-filter' the email once you've downloaded it, into your own local 'spam' or 'spam-probables' folder on your computer. there are two lines in the 'user_prefs' file to uncomment (remove the '#' symbol), which turns this feature on, as by default spamassassin only adds an 'X-Spam-Status' line to the email which is usually ignored by most POP3 email software.

below is the part of the 'user_prefs' file that controls the subject re-writing:

# NOTE: this is DISABLED by default; you have to uncomment the 'rewrite_subject'
# and 'subject_tag' lines to enable this subject-line re-writing method.
# (uncommenting means, remove the '#' symbol from in front of the two lines.)
#
# Change the subject line of suspected spam emails? (0=no, 1=yes)
rewrite_subject         1
#
# Text to prepend to subject if rewrite_subject is set to '1':
subject_tag             *****SPAM*****


you will also have to comment out the following three lines in your 'rc.maillists' file, in your '.procmail/' directory. put the hash symbol (#) in front of the lines to deactivate them:

# :0:
# * ^X-Spam-Status: Yes
# spam-probables

if you do not comment out those three lines, your email that gets tagged as possible spam will be filtered to a mail folder on yossman.net, instead of sitting in your INBOX on yossman.net, waiting to be picked up by your mail software for filtering on your end.

here is a link to the default spamassassin user_prefs configuration file used by yossman.net, if you would like to take a look at the config file before you do any editing (config as of spamassassin v2.5.5).



you can use 'spamassassin' at the yossman.net unix shell prompt (command-line) directly. this is particularly useful if you want to test spamassassin out with some emails that you feed it directly, to see if the rule changes you are making to your user_prefs are working right.

to try this, take any piece of email that you find particularly annoying, and save it as a .txt file, with FULL HEADERS. save it as something like 'surelyspam.txt', for example, in your home directory. then run spamassassin against it like this:


yossman.net> spamassassin -t < surelyspam.txt

what prints out is the original email you fed it (saved in 'surelyspam.txt'), with a couple extra header lines, such as:


[...]
X-Spam-Status: No, hits=4.3 required=5.0
        tests=FROM_NUM_AT_WEBMAIL,ONLINE_PHARMACY,RESENT_TO,SUBJ_FREE_CAP
        version=2.55
X-Spam-Level: ****
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)
[...]

these lines have been added to the original 'surelyspam.txt' by spamassassin, to indicate it's been processed and what tested positive (matched). at the bottom of the email, some more lines have been added:


Content analysis details:   (4.30 points, 5 required)
RESENT_TO          (-0.2 points) Has a Resent-To header
SUBJ_FREE_CAP      (0.7 points)  Subject contains "FREE" in CAPS
FROM_NUM_AT_WEBMAIL (2.9 points)  From address is webmail, but starts with a number
ONLINE_PHARMACY    (0.9 points)  BODY: Online Pharmacy

the above example shows spamassassin is close to marking this email with the default score of '5' that marks an email as spam. in your own copy of 'user_prefs', you would probably increase the score given to 'ONLINE_PHARMACY' so that future emails with that kind of wording would have a higher chance of begin marked (correctly) as spam and be rejected.

i hope this information enables you to start reducing the amount of 'spurious' email you are getting. with the ever-increasing number of unwanted emails and viruses being delivered world-wide, it makes good sense to trash as much as possible not just on principles but for your own safety in the case of viruses (virii).



this document first appeared on the internet in july 1996 as www.canweb.net/email.html for the yoss.canweb.net unix server.
this document moved to www.yossman.net/canweb/email.html in 1998.
this document moved to www.yossman.net/support/email.html in 2003.