Removing the Spam: Email Processing and Filtering (Addison-Wesley Networking Basics Series) - Softcover

Mulligan, Geoff

Softcover

ISBN 10: 0201379570 ISBN 13: 9780201379570

Publisher: Addison-Wesley, 1999

Synopsis

Written for network and system administrators, this concise, practical
guide offers step-by-step instructions for installing and configuring an
effective email system for your organization. The specific book focus is
on the growing problem of spam--unwanted electronic junk mail. This
book shows how to utilize mail-filtering tools and techniques to keep your
system and users as spam-free as possible.

Centered around real-world problems and issues, Removing the Spam
will save you hours of frustration as you get your email system up and
running. You will find in-depth coverage of major email systems and
subsystems--sendmail, Procmail, Majordomo, and SmartList--and
essential information about the following:

How to stop spam at the mail server and user levels
The most simple and effective way of generating the sendmail configuration file
Maintaining your sendmail configuration
Building and managing simple sendmail aliases and filters
Virtual domains
Filtering and sorting mail with Procmail
Procmail recipe files
Creating and administrating automated and manual mailing lists with Majordomo and SmartList
How to spot and combat mailing list abuse by spammers and list terrorists

The book's numerous practical examples (compatible with Solaris and
FreeBSD) include controlling the procmailrc file, sorting mail, working
with auto-responders, and many others.

With this book you will be well equipped to install and run an efficient
email system that won't be overrun by spam.

About this title

About the Author

Geoff Mulligan is a founder of Geocast Network Systems and works on researching emerging network technologies and network and system security products such as telecommuting tools, firewalls, and encryption. Prior to this, he was a Senior Staff Engineer at Sun Microsystems and was principal architect for Sun's premiere firewall product--SunScreen--and a founding member of the Internet Commerce Group. While at Sun, Geoff took a sabbatical to help start USA.NET and create the NetAddress product, a permanent email address. Before joining Sun, Geoff worked at Digital's Network Systems Laboratory developing the DEC SEAL firewall and Networking courseware, and researching email systems and technology. He spent 11 years in the U.S. Air Force working at the Pentagon (he brought the first system in the Pentagon up on the ARPANET) on computer and network security, building local and wide area networks and teaching computer science at the Air Force Academy. Geoff received his M.S. in Computer Science in 1988 from the University of Denver and his B.S. in Computer Science in 1979 from the United States Air Force Academy. He holds patents in network security and electronic mail technologies.

0201379570AB04062001

From the Back Cover

Written for network and system administrators, this concise, practical guide offers step-by-step instructions for installing and configuring an effective email system for your organization. The specific book focus is on the growing problem of spam--unwanted electronic junk mail. This book shows how to utilize mail-filtering tools and techniques to keep your system and users as spam-free as possible.

Centered around real-world problems and issues, Removing the Spam will save you hours of frustration as you get your email system up and running. You will find in-depth coverage of major email systems and subsystems--sendmail, Procmail, Majordomo, and SmartList--and essential information about the following:
* How to stop spam at the mail server and user levels
* The most simple and effective way of generating the sendmail configuration file
* Maintaining your sendmail configuration
* Building and managing simple sendmail aliases and filters
* Virtual domains
* Filtering and sorting mail with Procmail
* Procmail recipe files
* Creating and administrating automated and manual mailing lists with Majordomo and SmartList
* How to spot and combat mailing list abuse by spammers and list terrorists

The book's numerous practical examples (compatible with Solaris and FreeBSD) include controlling the procmailrc file, sorting mail, working with auto-responders, and many others.

With this book you will be well equipped to install and run an efficient email system that won't be overrun by spam. 0201379570B04062001

From the Inside Flap

As more companies connect to the Internet, system and network administrators are being tasked with increasingly diverse duties. Mail systems are becoming more intricate, and it is practically impossible for any individual to learn all of the complexities and idiosyncrasies of creating and maintaining a fully operational email system. When a crisis occurs, such as someone using your site as a spam relay, an administrator may spend countless hours plowing through Web pages, manuals, and numerous books to find the solution and proper configuration items. A simple and concise guide to problem fixes is necessary to save valuable time and minimize frustration.

A number of books currently exist that deal with the theory of email operation and a few cover, in excruciating detail, sendmail and mailing list configuration, but there has not been a single reference written that covers the specific tasks required to get an electronic mail system up and running. In addition, although there are many Web pages and other documents (even comments in the source code) on how to configure a system for common situations such as blocking spam, supporting virtual domains, creating custom mail filters, and adding automated mailing lists, the correct method is not always intuitive nor is which document or Web page to search to find a possible and proper fix.

Removing the Spam: Email Processing and Filtering was written to provide the system administrator with step-by-step solutions for these and other tasks required to maintain an efficient and effective Internet email site. It offers practical working examples in a comprehensive yet easy-to-use-and-understand way. After a basic overview, the types of tasks are grouped together in three chapters--each dealing with a major email subsystem. The topics for these chapters are outlined in the Organization section and here:

Email history and technology Sendmail installation and configuration Mail filtering using Procmail Mailing lists using Majordomo and SmartList

Each chapter includes subsections dealing with specific tasks. Where appropriate, some tasks may appear under multiple chapters. As an example, it is possible to filter spam using sendmail OR Procmail. Each has its own benefits and limitations and they can be used together to provide a relatively complete spam-filtering capability.

The operating systems used as examples for the step-by-step instructions are Solaris and FreeBSD. Solaris was chosen because of its wide use as the Web server of choice for many Internet Service Providers and commercial enterprise networks. FreeBSD was selected because it provides a good view of the standard BSD environment used in other operating systems, such as SunOS, BSDI, OpenBSD, and NetBSD. This leads to the question, "Why not Linux?" Ready access to a stable Linux system on which to test installation and configuration issues was not available during the writing of this book. Audience

This book is intended for both the UNIX system administrator and the email end-user to help them deal with the day-to-day tasks of managing an Internet email system. For the system administrator, the chapters cover the basic tools used to support a UNIX-based email system and the detailed tasks of getting, building, installing, and managing these tools. For the end-user, the chapters covering Procmail and mailing lists include the necessary information for configuring private filters and mailing lists. The chapters assume some basic knowledge of UNIX commands but also include step-by-step instructions that show exactly the commands to use to accomplish the tasks at hand. All instructions should be within reach of even the very novice user. Organization

This book is divided into four chapters, each detailing specific tasks required to install, configure, and maintain an email system.

Chapter 1 covers background material on the history and development of electronic mail on the Internet. This chapter discusses the evolution of mailing lists and email technology, including the workings and design of the Domain Name System, electronic mail etiquette, and the history of spam with some techniques to stop it.

Chapter 2 focuses on setting up and running sendmail. Specific sections deal with every step of the process. First is where to obtain the latest version and the step-by-step commands needed to compile and install sendmail on your system. Then comes the simplest and most effective way to generate the complex sendmail configuration file and how to build and manage simple mail aliases and filters. This is followed by instructions for stopping spam at the mail server and user levels, setting up virtual domains, hiding internal domains, and working with mail queues. Not every conceivable option is covered; instead, the focus is on the main options used to handle the most common situations facing the email administrator.

Chapter 3 delves into creating mail filters for specific tasks such as spam filtering and mail sorting. The chapter features Procmail, which is a widely used and extremely efficient mail filtering system available for most UNIX systems. Sections cover retrieving source and binary distributions, compiling and installing Procmail for your site, and the basics of building Procmail recipes. In addition, there are examples of ready-to-run filters to handle auto-reply mail information servers.

The final chapter treats obtaining, compiling, and installing mailing list servers and configuring them for use on your system. Two different mailing list packages are covered--Majordomo and SmartList--each of which has versions freely available on the Internet. The example configurations include both open and closed lists and moderated and unmoderated lists. Again, not all of the various options available when setting up any of these packages are discussed; the focus is on the most common situations.

There is also an appendix that contains on-line references to email software repositories and the Frequently Asked Questions Web locations. The Term Spam(R)

The term Spam (R) when used in the context of food and cooking is a registered trademark of Hormel Foods Corporation and in that milieu generally refers to Hormel's luncheon meat product. You can find more information about the Spam (R) food product at the Web site spam. All uses of the word "spam" in this book, except for this paragraph, refer to Unsolicited Commercial Email--the electronic mail version of junk mail. How to Use This Book

This book can be read cover to cover, but it is also meant to be used as a quick reference for specific tasks. As such, the chapters and subsections are designed to be used independently to provide the step-by-step instructions for each recommended solution. Conventions

The following conventions are used in this book:

Bold: Used in the text for emphasis of important concepts and terms. Italic: Used for RFC titles, program names, file names, email addresses, and WWW URLs. Monospaced: Used for any text that appears on the computer screen, either in files or as the output from commands or programs. Monospaced Bold: Used in examples to identify commands and text that will be typed in by the user. Monospaced Italic Used in examples to indicate variable information, for instance, example would be replaced with an actual domain name. Acknowledgements

I would first and foremost like to thank my wife, Jess, and my two girls, Katie and Jenna, for their help and support during the writing of this book--especially the final push, during which I'm sure that I was unbearable. This was in addition to the numerous weekends I spent parked in front of the computer writing and testing and missing many wonderful summer days and activities. Without their love, encouragement, and profound patience this book would not have been possible. Thank you so very much.

I would also like to thank and dedicate this book to Walt Grueninger. Walt started me on my adventure into computers some 30 years ago, when I was nine. It was his enthusiasm for the technology that sparked my interest and focused me on a career path. I appreciate the time and effort he spent teaching me about machine instructions, two-pass assemblers, and how to tell what a Data General Nova system was doing by listening to the "core" hum.

Next, great thanks to the technical reviewers who provided excellent comments, both on the structure and the technical aspects of the book. Specifically: Dustin Andrews of Information Insights, Inc., Dr. Claus Assmann, David Collier-Brown, Dave Crocker of Brandenburg Consulting, Jasbir S. Mahal, and Bob Stewart from Cisco Systems, Inc. And thanks to the folks at Geocast Network Systems, Inc., my employer, who understood my commitment to completing the book even though we are just a startup company.

Also, thank you to all of the folks at Addison Wesley Longman, especially Mary Hart, who also understood my commitment to my startup company and the inevitable delays and timing conflicts that this caused. They stuck with me, pushing me to complete the book and at least attempt to meet the deadlines. I would also like to thank them for giving me the opportunity to write this book.
--Geoff Mulligan 0201379570P04062001

Excerpt. © Reprinted by permission. All rights reserved.

Preface

As more companies connect to the Internet, system and network
administrators are being tasked with increasingly diverse duties. Mail
systems are becoming more intricate, and it is practically impossible for
any individual to learn all of the complexities and idiosyncrasies of
creating and maintaining a fully operational email system. When a crisis
occurs, such as someone using your site as a spam relay, an
administrator may spend countless hours plowing through Web pages,
manuals, and numerous books to find the solution and proper
configuration items. A simple and concise guide to problem fixes is
necessary to save valuable time and minimize frustration.

A number of books currently exist that deal with the theory of email
operation and a few cover, in excruciating detail, sendmail and mailing list
configuration, but there has not been a single reference written that
covers the specific tasks required to get an electronic mail system up and
running. In addition, although there are many Web pages and other
documents (even comments in the source code) on how to configure a
system for common situations such as blocking spam, supporting virtual
domains, creating custom mail filters, and adding automated mailing lists,
the correct method is not always intuitive nor is which document or Web
page to search to find a possible and proper fix.

Removing the Spam: Email Processing and Filtering was written to
provide the system administrator with step-by-step solutions for these and
other tasks required to maintain an efficient and effective Internet email
site. It offers practical working examples in a comprehensive yet
easy-to-use-and-understand way. After a basic overview, the types of
tasks are grouped together in three chapters--each dealing with a major
email subsystem. The topics for these chapters are outlined in the
Organization section and here:

1.Email history and technology
2.Sendmail installation and configuration
3.Mail filtering using Procmail
4.Mailing lists using Majordomo and SmartList

Each chapter includes subsections dealing with specific tasks. Where
appropriate, some tasks may appear under multiple chapters. As an
example, it is possible to filter spam using sendmail OR Procmail. Each
has its own benefits and limitations and they can be used together to
provide a relatively complete spam-filtering capability.

The operating systems used as examples for the step-by-step instructions
are Solaris and FreeBSD. Solaris was chosen because of its wide use as
the Web server of choice for many Internet Service Providers and
commercial enterprise networks. FreeBSD was selected because it
provides a good view of the standard BSD environment used in other
operating systems, such as SunOS, BSDI, OpenBSD, and NetBSD. This
leads to the question, "Why not Linux?" Ready access to a stable Linux
system on which to test installation and configuration issues was not
available during the writing of this book.

Audience

This book is intended for both the UNIX system administrator and the
email end-user to help them deal with the day-to-day tasks of managing
an Internet email system. For the system administrator, the chapters
cover the basic tools used to support a UNIX-based email system and
the detailed tasks of getting, building, installing, and managing these tools.
For the end-user, the chapters covering Procmail and mailing lists include
the necessary information for configuring private filters and mailing lists.
The chapters assume some basic knowledge of UNIX commands but
also include step-by-step instructions that show exactly the commands to
use to accomplish the tasks at hand. All instructions should be within
reach of even the very novice user.

Organization

This book is divided into four chapters, each detailing specific tasks
required to install, configure, and maintain an email system.

Chapter 1 covers background material on the history and development of
electronic mail on the Internet. This chapter discusses the evolution of
mailing lists and email technology, including the workings and design of
the Domain Name System, electronic mail etiquette, and the history of
spam with some techniques to stop it.

Chapter 2 focuses on setting up and running sendmail. Specific sections
deal with every step of the process. First is where to obtain the latest
version and the step-by-step commands needed to compile and install
sendmail on your system. Then comes the simplest and most effective
way to generate the complex sendmail configuration file and how to build
and manage simple mail aliases and filters. This is followed by
instructions for stopping spam at the mail server and user levels, setting
up virtual domains, hiding internal domains, and working with mail
queues. Not every conceivable option is covered; instead, the focus is on
the main options used to handle the most common situations facing the
email administrator.

Chapter 3 delves into creating mail filters for specific tasks such as spam
filtering and mail sorting. The chapter features Procmail, which is a
widely used and extremely efficient mail filtering system available for
most UNIX systems. Sections cover retrieving source and binary
distributions, compiling and installing Procmail for your site, and the
basics of building Procmail recipes. In addition, there are examples of
ready-to-run filters to handle auto-reply mail information servers.

The final chapter treats obtaining, compiling, and installing mailing list
servers and configuring them for use on your system. Two different
mailing list packages are covered--Majordomo and SmartList--each of
which has versions freely available on the Internet. The example
configurations include both open and closed lists and moderated and
unmoderated lists. Again, not all of the various options available when
setting up any of these packages are discussed; the focus is on the most
common situations.

There is also an appendix that contains on-line references to email
software repositories and the Frequently Asked Questions Web
locations.

The Term Spam(R)

The term Spam (R) when used in the context of food and cooking is a
registered trademark of Hormel Foods Corporation and in that milieu
generally refers to Hormel's luncheon meat product. You can find more
information about the Spam (R) food product at the Web site
http://www.spam.com. All uses of the word "spam" in this book, except
for this paragraph, refer to Unsolicited Commercial Email--the electronic
mail version of junk mail.

How to Use This Book

This book can be read cover to cover, but it is also meant to be used as a
quick reference for specific tasks. As such, the chapters and subsections
are designed to be used independently to provide the step-by-step
instructions for each recommended solution.

Conventions

The following conventions are used in this book:

Bold: Used in the text for emphasis of important concepts and terms.
Italic: Used for RFC titles, program names, file names, email addresses, and WWW URLs.
Monospaced: Used for any text that appears on the computer screen, either in files or as the output from commands or programs. Monospaced Bold: Used in examples to identify commands and text that will be typed in by the user.
Monospaced Italic Used in examples to indicate variable information, for instance, example.com would be replaced with an actual domain name.

Acknowledgements

I would first and foremost like to thank my wife, Jess, and my two girls,
Katie and Jenna, for their help and support during the writing of this
book--especially the final push, during which I'm sure that I was
unbearable. This was in addition to the numerous weekends I spent
parked in front of the computer writing and testing and missing many
wonderful summer days and activities. Without their love,
encouragement, and profound patience this book would not have been
possible. Thank you so very much.

I would also like to thank and dedicate this book to Walt Grueninger.
Walt started me on my adventure into computers some 30 years ago,
when I was nine. It was his enthusiasm for the technology that sparked
my interest and focused me on a career path. I appreciate the time and
effort he spent teaching me about machine instructions, two-pass
assemblers, and how to tell what a Data General Nova system was doing
by listening to the "core" hum.

Next, great thanks to the technical reviewers who provided excellent
comments, both on the structure and the technical aspects of the book.
Specifically: Dustin Andrews of Information Insights, Inc., Dr. Claus
Assmann, David Collier-Brown, Dave Crocker of Brandenburg
Consulting, Jasbir S. Mahal, and Bob Stewart from Cisco Systems, Inc.
And thanks to the folks at Geocast Network Systems, Inc., my employer,
who understood my commitment to completing the book even though we
are just a startup company.

Also, thank you to all of the folks at Addison Wesley Longman,
especially Mary Hart, who also understood my commitment to my startup
company and the inevitable delays and timing conflicts that this caused.
They stuck with me, pushing me to complete the book and at least
attempt to meet the deadlines. I would also like to thank them for giving
me the opportunity to write this book.

--Geoff Mulligan

Edition details

• Publisher: Addison-Wesley
• Publication date: 1999
• Language: English
• ISBN 10: 0201379570
• ISBN 13: 9780201379570
• Binding: Paperback
• Edition number: 1
• Number of pages: 190