isode-dealing-with-spam-november2003.ppt

dealing with spam

1. LINX 43 November 17/18
Dealing with Spam
November 2003
Steve Kille
CEO, Isode

2. LINX 43 November 17/18
Talk Summary
•Isode (a brief commercial)
•Server based spam
removal
–Solutions close to the
recipient that remove spam
–Focus on solutions for
service providers
–Anti-spam techniques
–Benchmarking anti-spam

3. LINX 43 November 17/18
Isode & Its ISP Products
•Software Product Company
–Founded 1992, based in Hampton
•Mail and Directory server products
–ISPs, Telcos, Government and Military customers
•Technical Company
–Most employees write code
–Open standards focus (long IETF history)
•M-Vault
–LDAP Directory server, typically used for storing email, radius, and
service information
•M-Switch Anti-Spam
–SMTP switching anti-spam product
–Can also provide general purpose boundary messaging and anti-
virus

4. LINX 43 November 17/18
Recipient Side Spam Solutions
•A quick run through major techniques for
dealing with Spam
•Goal is to help in understanding choice of
options in various products

5. LINX 43 November 17/18
Non-server Anti-Spam
•Legal & Regulatory
–Important long term, but currently slow and ineffective
•Spammer side solutions
–Detecting spammers and removing them from ISPs is
important
•Client Filtering
–Can be effective, but too much effort for most users
•Challenge/Response
–Only accept email from recognized senders
–Effective, but too awkward and disruptive for most users

6. LINX 43 November 17/18
Monitoring real spam in real time
•Monitoring net for spam is basis of some other controls
•Can be effective if done on large scale
–Small scale monitoring does not give enough
coverage and reacts too slowly to new spam
–Monitoring needs to scale as spam levels grow
•Large scale monitoring can use:
–Big commercial operation (e.g., Brightmail, Postini)
–Distributed collaborative approach (e.g., Vipul’s razor)
–More focussed effort (e.g., Spamhaus, MAPS)

7. LINX 43 November 17/18
Controlling spam by origination
•Relies on monitoring
•Inference: “because Spam came from a given IP
address, other traffic from this address will also be
Spam”
•Information on IP addresses distributed as “Real Time
Black-Hole” Lists” (RBLs)
http://www.sdsc.edu/~jeff/spam/Blacklists_Compared.html
•Can give high false positive rate (blocking real
messages), as spammers often use addresses (e.g.,
open relays) that also have real traffic
•Not sufficiently effective to be a complete solution

8. LINX 43 November 17/18
Controlling spam by content
•Relies on monitoring (more than RBLs)
•Takes “signature” of messages being sent out
•The “signature” can be used to match other
messages
•Spammers attack this by “snowflaking”, to make
each message different
–Some signature techniques are resistant to some
forms of snowflaking
•May be effective, but long term risk and high cost

9. LINX 43 November 17/18
Content Filtering
•Matching specific words
in Subject line or
message body (e.g.,
Viagra)
•Not very effective (either
low hit rate, or very high
false positives)
•Content filtering can be
useful in other
applications (e.g., filtering
out all messages with
offensive words)

10. LINX 43 November 17/18
“Looks Like Spam”
•Examining messages to see if they look like spam
•Can be very effective, but…
–Some real messages look like spam, so get matched
–Usually these are lower interest messages
•Many products (including Isode) do this
•Lots of techniques to do this type of matching, typically
integrated with Bayesian logic
–Products can describe these techniques in much detail
–Techniques evolve, in “war with spammers”
•Isode currently views that this is the best long term
approach

11. LINX 43 November 17/18
Changing the email infrastructure
•It is easy for anyone to send email
•If it was harder to connect to the email
infrastructure, it would be harder to send
spam
•Various proposals are being made to
change the email infrastructure to achieve
this (e.g., IETF, some vendors)
•Could be effective, but will not be easy

12. LINX 43 November 17/18
Greylisting
•Technique discussed recently in IETF Anti Spam
Research Group
–http://projects.puremagic.com/greylisting/
–Free Sendmail prototype
–Isode Release 10.2 believed to be first commercial
implementation
•Works by recording, sender, recipient, and source IP
–Allow through known tuples
–Temporary fail everything else, and then add as a known tuple
–Forces retry
•Removes 80-90% of spam before it gets to server
–Most spam is sent by scripts
–Especially effective against harvest attacks

13. LINX 43 November 17/18
Greylisting –not perfect
•Drawback is that small percentage (0.1%?) of
real traffic also gets hit
–Buggy email servers
–Opt in marketing sometimes uses script
distribution
•Careful implementation is important
–Whitelisting
•It appears viable and is very effective
–Removing 80-90% of spam is good

14. LINX 43 November 17/18
Controls needed (applies to all techniques)
•Simple filtering and applying anti-spam check is
insufficient
•System controllable by ISP and/or Customer
•Choice of action (bounce; discard; mark; reroute;
quarantine)
•Multiple levels (and independent action control for each
level)
•White lists and Black lists (system and per user)
•Choice of anti-spam checks to apply
•Reporting

15. LINX 43 November 17/18
Key Product Characteristics
•Performance
–Easy to measure
–Important for service providers
•How well does it remove spam?
–Amount of spam removed
–Number of false positives
–Harder to measure
•Control and Management

16. LINX 43 November 17/18
Network World Fusion Tests
•Marketing material from most anti-spam vendors is
confusing and depressing
–Hype and unrealistic claims
–Hard to work out exactly what products/services do
–No numbers
•Few numbers from independent sources
•Tests from Network World Fusion are great!
–http://www.nwfusion.com/reviews/2003/0915spam.html
–Real measurements on 16 vendors
–Didn’t include Isode’s M-Switch anti-spam 

17. LINX 43 November 17/18
Performance (Network World Numbers)
Vendor Delivery Rate (Messages/sec)
MailFrontier 20*
CloudMark 20*
Trend Micro 20*
Tumbleweed 10
Corvigo 7.25
Clearswift 6.7
Postini 6
Easylink 3.8
MX Logic 3.6
ActiveState 3.2
SurfControl 3
Vircom 2.6
GFI 2.4
SingleFin 1.25
Computer Mail Services 0.5

18. LINX 43 November 17/18
Spam Removal Accuracy (Network
World Numbers)
Vendor Accuracy False Positive
Postini 94.0% 0.4%
MailFrontier 89.4% 0.7%
ActiveState 89.4% (80.5%) 7.2% (2.9%)
SingleFin 86.2% 2.9%
Cloudmark 85.1% (82%) 1.6% (1.3%)
Corvigo 84.6% (77.9%) 16.5% (0.7%)
Computer Mail Services 83.4% 23.4%
Tumbleweed 81.3% (72.2%) 1.4% (1.2%)
MX Logic 77.0% 0.5%
SurfControl 76.5% 3.3%
Trend Micro 60.3% 0.8%
Clearswift 48.5% 2.3%
Easylink 23.1% 20.5%
GFI 3.6% 56.3%

19. LINX 43 November 17/18
How Isode would have fared
•Products have very wide variation in two key metrics
–Some are spectacularly poor at matching spam
–None are very good (best is 0.4% false positive)
•Isode would comfortably exceed 20 messages per
second
–30-50 messages per second on small server
•Isode’s spam removal
–Based on our data, so may not be direct comparison
–Customers suggest that these numbers are reasonable
–“Spam” –90% accuracy, 0.1% false positive
–“Possible Spam” –97% accuracy, 2% false positive
•We believe we are competitive with the best

20. LINX 43 November 17/18
Isode vs SpamAssassin
•We like SpamAssassin and have learnt a lot from it
•Isode is about ten times faster:
–SpamAssassin –10 msgs/sec
–Isode’s equivalent engine –100 msgs/sec
•Similar characteristics for matching spam
•Isode is better at separating real messages
–Significantly lower false positive rate for a given
match level
–Isode relies more on message content, and less on
generic email characteristics of spam

21. LINX 43 November 17/18
Conclusions
•Very wide variation in products (techniques and
performance)
•ISPs need good control (simple filtering is not enough)
•Focus on performance characteristics, not feature list:
–Throughput
–Amount of spam detected
–False positives
•Future quality of spam removal will depend more on
product evolution than on current product

22. LINX 43 November 17/18
Further Information
•Questions to: [email protected]
•Presentation and white papers on Isode Web site:
www.isode.com
•Isode staff (in Isode shirts) will be around the conference
and at the party tonight
•Please ask for:
–Demonstration of Isode products and more
information
–How we can help you measure performance of M-
Switch Anti-Spam with your traffic