Archive | MailScanner RSS feed for this section
November 13, 2004

DNS Blacklists – Setting up a local mirror


Introduction
I am currently mirroring a number of DNS blacklists, often referred to as RBLs, on our network in order to speed up mail processing. By keeping the queries local not only do we get a definite speed increase, faster processing and fewer timeouts but we also reduce our bandwidth usage.
In order to setup a local mirror (or caching server) you will need the following:

  • Rsync
  • Rsync access to a number of data sources
  • A DNS server - preferably BIND
  • RBLDNSD - a DNS daemon designed to serve DNSBLs (DNS blacklists). Although it is fast it uses quite a lot of memory depending on the size of the data set you are using, so make sure you run it on a machine with plenty of RAM

Rsync is available on all distros of linux but it might not be installed by default.
In order for this to work you will need to have been granted rsync access to one or more DNSBLs. Some of the DNSBLs have an "open" policy on rsync, so you can simply access it directly, however it is more common to have to ask explicitly for permission and supply the DNSBL maintainer(s) with your IP(s). In the case of SpamHaus you will need to pay a fee.
For the purposes of this document I will be looking at only one DNSBL - dsbl.org. , as they allow rsync access freely.
Setting up RBLDNSD
Grab a copy of the daemon from the site. Packages for a number of distros are available or you can install from source. The server I am using is running WhiteBox linux, so I was able to use one of the rpm packages:
wget http://www.corpit.ru/mjt/rbldnsd/rbldnsd-0.993.1-1.i386.rpm
rpm -ivh rbldnsd-0.993.1-1.i386.rpm

NB: The latest version of the packages are available here
We do not want to run the daemon as root, so we add a user for it.
adduser dnsbl
We'll need to get some data before we can start using it, so let's do that.
Setting up Rsync
DSBL provides quite clear instructions on setting up rsync with their data.
After choosing which data you want to use write a small script to "grab" the data as the user dnsbl:

su - dnsbl
vim dsblscript
#!/bin/sh
cd /home/dnsbl
rsync -tvPz rsync.dsbl.org::dsbl/rbldns-list.dsbl.org /home/dnsbl/data/

Don't forget to make the script executable:
chmod 500 dsblscript
you can test it by running it directly from the command prompt:
./dsblscript
If it is working correctly you should have some data in your "data" directory.
A DNSBL is only as good as its last update, so we'll setup a cronjob to automatically update our data:
10,40 * * * * /home/dnsbl/dsblscript
Every 30 minutes we will check to see if there are any changes. Doing it more frequently is neither required nor advisable.
Now that we have our data we need to do something with it, so let's finish setting up RBLDNSD.
For some odd reason the rpm version does not ship with a fully functional init script, so I had to put together my own based on a few documents I found online:
#!/bin/bash
#
# chkconfig: 2345 85 15
# description: rbldnsd is a DNS server designed for dnsbls.
# processname: rbldnsd
# pidfile: /var/run/rbldnsd.pid
# source function library
. /etc/init.d/functions
[ -e /etc/sysconfig/rbldnsd ] && . /etc/sysconfig/rbldnsd
RETVAL=0
start() {
echo -n $"Starting rbldnsd service: "
daemon /usr/sbin/rbldnsd $OPTIONS
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/rbldnsd
}
stop() {
echo -n $"Shutting down rbldnsd service: "
killproc rbldnsd
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/rbldnsd
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
stop
start
RETVAL=$?
;;
condrestart)
if [ -f /var/lock/subsys/rbldnsd ]; then
stop
start
RETVAL=$?
fi
;;
status)
status rbldnsd
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|restart|condrestart|status}"
exit 1
esac
exit $RETVAL

This will give you:

  • start
  • stop
  • restart
  • status
  • condrestart

which you can call as /etc/rc.d/init.d/rbldnsd $option
Before we can use it we need to tell it what data to use and where to publish it:
vim /etc/sysconfig/rbldnsd
OPTIONS="-u dnsbl -r /home/dnsbl/data -t 21600 -c 60
-p /var/run/rbldnsd.pid -b xxx.xxx.xxx.xx/53
list.dsbl.org:ip4set:rbldns-list.dsbl.org
multihop.dsbl.org:ip4set:rbldns-multihop.dsbl.org
unconfirmed.dsbl.org:ip4set:rbldns-unconfirmed.dsbl.org"

The option -u defines the user to run as, -r the data directory, -p the process ID and -b which IP and port to bind to. As I've set this to run on port 53 it could not be run on the same machine as our main nameserver.
Make sure you use the backslashes () at the end of lines as the syntax is vital.
You can now try to start your daemon:
/etc/rc.d/init.d/rbldnsd start
If you get any errors read them carefully and modify your config to fix them.
NB: It will not work if there is no data present.
Adding the Zone(s) to BIND
The last step is putting the new mirror live on your network. To do this you will create forwarding zone(s) in your BIND DNS server (it will work with other DNS servers, but I am not familiar with their configuration).
Open your named.conf in vim and go to the end of the file.
Add the following:

zone "list.dsbl.org" IN {
type forward;
forward first;
forwarders {
xxx.xxx.xxx.xx;
};
};

The example above is for the zone list.dsbl.org, so you can replace that with the zones you are using ie. create a separate entry for each one.
Replace the "xxx.xxx.xxx.xx" with the IP of the server running RBLDNSD.
Reload BIND:
rndc reload
If you want to see the queries against your DNS you can turn on logging in BIND or you could turn on logging in RBLDNSD's config.
NB: Do not leave logging on for more than a short period while verifying. The log files grow exponentially.
You should now have a working DNSBL mirror.
DISCLAIMER:
This configuration and setup works for me. Your mileage may vary.
DNS and BIND Cookbook

Related Posts:

  • No Related Posts
November 9, 2004

IRC Support

Vasiliy Boulytchev has setup a MailScanner IRC channel over on freenode:
#mailscanner
ipv4: irc.freenode.net
ipv6: irc.ipv6.freenode.net
I've also setup a "paste bin" if you need to share your config or other code with users.
If you need a good IRC client use Xchat

Related Posts:

  • No Related Posts
October 31, 2004

SPF records

Spam Assassin 3 comes with a builtin SPF record checker, so it would make sense to publish SPF records for domains. Or would it?
This domain has now got a simple set of SPF records which I setup using a couple of the online tools to generate them.
If you want to see how many domains are publishing SPF have a look here. Although it is not a definitive listing it does give some indication of the number of records published, including some of the higher profile sites.
Gmail checks for SPF, so you will see results in your headers:
Received-SPF: neutral (gmail.com: xxx.xxx.xxx.xxx is neither permitted nor denied by domain of xxxxx@xxxx.com)
The key with SPF is the scoring. If you explicitly set your SPF records to a limited number of hosts/IPs then any mail purporting to come from your domain will be checked against its SPF record. If the sending IP/hostname is not in the SPF record then the receiving MTA should not "trust" it.
Will this lead to a reduction in spam?
No, but it should help to cut down the amount of spoofed junk hitting people's mail boxes.
If you publish SPF records for your domains you *should* be able to reduce the likelihood of you r domain being used in a "joe job". At least that's my understanding of it.
If you need help in setting up SPF records then look at:

There is a lot of debate surrounding SPF in general, but some good articles like this one make it very clear.

Related Posts:

  • No Related Posts
October 30, 2004

Upgrading MailScanner – rpm based system

Upgrading MailScanner on an rpm based system is not complicated as long as you read the messages on the screen.
First off go to the MailScanner download section and get the version you want to upgrade to. The second one in the list is the rpm version for RedHat and derivatives.
Normally there is a choice of "stable" and "beta" downloads. What's the difference? The stable has been tested more thoroughly by beta testers and is unlikely to cause any issues on your system. The beta release may not be as thoroughly tested and is not really aimed at the "faint hearted". Put it another way, if you are not extremely comfortable with managing MailScanner don't use the beta release :mrgreen:
I would recommend that you download any installers into a specific directory in /home, for example I use /home/blacknight to store them. DO NOT download the installer to /etc/MailScanner
Let's begin:
cd /home/blacknight (edit this to taste)
wget http://www.sng.ecs.soton.ac.uk/mailscanner/files/4/rpm/MailScanner-4.34.8-4.rpm.tar.gz (change URL to the most recent version / the version you want to install)
tar -zxvf MailScanner-4.34.8-4.rpm.tar.gz (change to file you have downloaded)
cd MailScanner* < <- move into the MailScanner installer directory
At this point it would not be a bad idea to stop the running MS daemon, so issue the following commands:
/etc/init.d/MailScanner stop
/etc/init.d/MailScanner startin
This will stop the main daemon but then restart the incoming queue.
Next we install MS by running:
./install.sh
Depending on your setup it can take anything up to 15 minutes to complete.
At the end of the process do the following (presuming you haven't had huge errors!)
cd /etc/MailScanner << where the MailScanner scripts "live"
upgrade_MailScanner_conf << if run by itself it will give you tips on usage
upgrade_MailScanner_conf MailScanner.conf MailScanner.conf.rpmnew > MailScanner.new
this will read in the "new" config file and "intelligently" append any new option directives to your existing config which will create a new file called MailScanner.new
You can compare the two files using the "diff" command if you want, but the output of the upgrade script is usually quite verbose.
We then need to overwrite the "old" config file with the new one:
mv -f MailScanner.conf MailScanner.old
mv -f MailScanner.new MailScanner.conf
Now restart MailScanner:
/etc/init.d/MailScanner restart
If you want to see exactly what you are using try this:
MailScanner -v
which will give you a quite verbose breakdown of all the various modules and addons in use on your system
Enjoy!

Related Posts:

  • No Related Posts
October 8, 2004

MailScanner – the book

Julian's book finally arrived today. I haven't had time to read it yet, but it's nice to get a "hard copy". I really hate reading books off a screen.
Mailscanner: User Guide and Training Manual

Related Posts:

  • No Related Posts
September 27, 2004

Spam Assassin 3 problems

Spam Assassin 3 was released the other day, so I started rolling it out across as many servers as possible. We had already been testing it on a couple without encountering any issues, so putting the stable version should have been quite easy. In most cases it was, except for one Red Hat 9 server. It simply refused to go on properly.
If I got it past the install, either via CPAN, tar.gz or Julian's RPMs it would not pass the
spamassassin --lint test
After struggling with it for over an hour I gave up and rolled it back to 2.64, which works fine.
I've also ordered the book from Amazon, so I'm waiting to see if it covers anything I haven't thought of or come across:
SpamAssassin

Related Posts:

  • No Related Posts
September 25, 2004

BitDefender issue

A number of people reported severe problems with the bitdefender update scripts yesterday. Seemingly the processes were hanging and eating up CPU.
Kevin Spicer found a good interim solution:
vim /usr/lib/MailScanner/bitdefender-autoupdate
Comment out the following two lines:
LINE 190
system "$bitDCmd > $origFile ";
LINE 253
system "$bitDCmd > $destFile ";
You then need to kill any bitdefender update processes. Most of them are "bdc", so you can run:
ps auxwww|grep bdc
but I also found a few with "bitdefender" in the string.
Thanks to Kevin for solving this!
EDIT: This issue was resolved by the developers a few days later

Related Posts:

  • No Related Posts
September 7, 2004

SA3 – RC3

Julian released a set of scripts to install Spam Assassin RC3 yesterday for both rpm and non-rpm based distros.
You can get them here and maybe you'll have more luck with them than I did :mrgreen:
I couldn't get it to work on the Whitebox machine I was trying to install it on, as it would not overwrite the contents of /usr/share/spamassassin
After trying to rectify this in a number of ways I finally opted for a "clean" install using the tar.gz from the Spam Assassin site.
Word of caution: If you are using custom rulesets you may need to remove them first, as a lot of them are now part of SA3's code.
There are a number of new features in SA3 that should make it more interesting:
- Mail::SpamAssassin::Plugin::URIDNSBL - SURBL - available via a plugin that ships with it. I must have a closer look at the scoring on this
-Mail::SpamAssassin::Plugin::SPF - Checks for SPF content
-Mail::SpamAssassin::Plugin::RelayCountry - fairly obvious, though I'm not sure what we can do with the output
-Mail::SpamAssassin::Plugin::Hashcash - not 100% sure what that does :smile:
Update: I did a bit more research into Hashcash. The concept is quite interesting. In essence you "earn" the right to send email by "spending" CPU cycles. As each email requires you to "spend" resources before you can send it a spammer would not be able to send the same volume of email, while a "normal", "legitimate" user would not even notice the slight delay.
More information is available on the Hashcash site.

Related Posts:

  • No Related Posts
September 5, 2004

Throttling by IP

/usr/lib/MailScanner/MailScanner/CustomConfig.pm contains a number of handy little features that are not "on" by default. One which came up on the list today was IPBlock which allows you to control the maximum number of emails from a particular IP per hour. This might be useful for combatting infected machines spewing email at servers, but it still does not address the issue of the connection to the MTA.
Vispan can also achieve this.
It would be interesting to implement something like this that interfaced directly with IPTables instead of the MTA.

Related Posts:

  • No Related Posts
August 31, 2004

MailScanner: Stable 4.33.3 released

Julian released 4.33.3 this morning. It's basically a stable version based on the beta released a couple of days ago.
He also released some SpamAssassin v3 related scripts yesterday, but they seem to have been removed from the downloads list (or maybe I'm blind)

Related Posts:

  • No Related Posts
css.php