tony's geek stuff: 2006-11

2006-11-11

Minor Correction

I said earlier that freshclam isn't an init script and I'd have to craft my own. Not required, as there's a script in cron.daily that updates it once a day.

2006-11-10

Tweaking the anti-spam setup

Had to tweak the SA ruleset for LogWatch, specifically:
```
header TONNS_LOGWATCH  Subject =~ /^LogWatch for /
score  TONNS_LOGWATCH  -20.0
```
Eventually, it should be a real whitelist entry, but since this is a dry-run, I'll just leave it at this.

The LogWatch amavisd script is out-of-sync with the logs that amavisd generates. This fixed things:

# pwd
/etc/log.d/scripts
# diff -u services.orig/amavis services/amavis 
--- services.orig/amavis        2005-08-22 21:16:47.000000000 -0400
+++ services/amavis     2006-11-10 11:01:50.000000000 -0500
@@ -60,7 +60,7 @@
         or ($ThisLine =~ /^cached [a-zA-Z0-9]+ /)
         or ($ThisLine =~ /^starting.  amavisd at/) ) {
        # We don't care about these
-   } elsif ($ThisLine =~ /^Passed, /) {
+   } elsif ($ThisLine =~ /^Passed CLEAN, /) {
       $CleanMsgs++;
    } elsif (($FileName, $From) = ( $ThisLine =~ /^BANNED name\/type \(([^\)]+)\)\, \<([^\>]*)\>/ )) {
 
@@ -82,7 +82,18 @@
         $Viruses{$Virus}{$From}++;
       }; # if
 
-   } elsif (($Fromspam, $Towards) = ( $ThisLine =~ /^SPAM, [\(\<]([^\>\)]+)[\)\>] -\> [\(\<]([^\>\)]+)[\)\>]/ )) {
+   } elsif (($Fromspam, $Towards) = ( $ThisLine =~ /^Passed SPAM, \[[\d\.]+\] \[[\d\.]+\] [\(\<]([^\>\)]+)[\)\>] -\> [\(\<]([^\>\)]+)[\)\>]/ )) {
+      $SpamMsgs++;
+
+      if ($Detail >= 5) {
+        $Spamtypes{$Towards}++;
+      }; # if
+
+      if ($Detail >= 10) {
+        $Spams{$Towards}{$Fromspam}++;
+      }; # if
+
+   } elsif (($Fromspam, $Towards) = ( $ThisLine =~ /^Passed SPAM, [\(\<]([^\>\)]+)[\)\>] -\> [\(\<]([^\>\)]+)[\)\>]/ )) {
       $SpamMsgs++;
 
       if ($Detail >= 5) {

2006-11-05

Man, setting up an anti-spam toolsuite is a PITA. Spammers suck.

Anyway, I started off with the this HOWOTO from howtoforge.com, skipping all the SQL stuff because I'm not using a virtual setup (for now - maybe the final setup will, I'm not sure yet). Some key
differences in my setup:

In the install:

The HOWOTO is for Fedora, not CentOS. You need to add "clamd" and "razor-agents" to the install.
There's no "pyzor" package for CentOS. Need to install from source.
I didn't setup razor/pyzor exactly like the HOWTO. I can't recall all the details, but I just su'd to amavis and did it as a "standalone user". But since amavisd does all the SA processing as amavis, it's effectively global. I think I just followed the default documentation.
There's no "freshclam" init script. I still have to write something that crons (or borrow the script from Fedora). I just ran it by hand for now.
Installed "fetchmail". I'm majorly impressed by Horde on this one. IMP was working great and as soon as I installed fetchmail, all the options for remote mail retrival showed up

In amavisd.conf:

$sa_tag_level_deflt = undef; - always print the SA headers
@local_domains_maps = ( [".$mydomain", 'example.com', 'foobar.tld'] ); - if all your domains aren't here, the SA headers won't be added to the messages
$sa_spam_subject_tag = '[SPAM] '; - damn, i hate the asterisks in the subject
Uncomment the 'ClamAV-clamd' section
Make sure you add amavis to the clamd group

In clamd.conf

Add "LocalSocket /var/run/clamav/clamd"
Comment out the TCPSocket and TCPAddr sections
Make sure you add clamd to the amavis group, just to be sure.

In Postfix

Got rid of the home_mailbox directive for Maildirs from yesterday's setup
added "mailbox_command = /usr/bin/procmail"

created /etc/procmailrc with:

DEFAULT=$HOME/Maildir/
LOGFILE=/var/log/procmail.log
LOBABSTRACT=all

The logging stuff is just temporary.

The reason for the switch to procmail was the Ingo module for horde. It's pretty solid. It will update a user's .procmailrc via the FTP VFS backend, all using the single-signon password. My backend looks like this:

$backends['procmail'] = array(
    'driver' => 'vfs',
    'preferred' => 'home.tonns.net',
    'hordeauth' => true,
    'params' => array(
        'hostspec' => 'localhost',
        'filename' => '.procmailrc',
        'port' => 21,
        'vfstype' => 'ftp'
    ),
    'script' => 'procmail',
    'scriptparams' => array(
        'path_style' => 'maildir',
        'variables' => array(
            'DEFAULT' => '$HOME/Maildir/',
        )
    )
);

That last variable is redundant with the /etc/procmailrc settings, but I don't want the users to try anything crazy.

added to my .procmailrc using Ingo:
```
##### SPAM #####
:0 
* ^X-Spam-Status:.*Yes
"$DEFAULT/.SPAM/"
```
Which does as you'd expect. I also added it to /etc/skel/.procmailrc, but horde doesn't read in existing procmail rules, it just knows about the ones it created. I have to look into it - maybe I'll put it in the global procmailrc.
mkdir -p /etc/skel/Maildir/SPAM; mkdir -p /etc/skel/Maildir/.LearnAsSpam; mkdir -p /etc/skel/Maildir/.LearnAsNotSpam - if the Maildir doesn't exist IMP default to mbox in $HOME/mail - which is not what we want, so have them created by default.
Finally, I wrote a shell script that uses archivemail and sa-learn to clean out everyone's LearnAsSpam/NotSpam directories and add them to the global SA bayes filter (which is in the user amavis's home directory)

Well, I think that's about it. I might have tweaked something else... but I'll have to pick it up later.

Finally, I'm gonna have to do a major hardware upgrade - it doesn't look like 256MB is enough to run the OS and apache and php and amavisd and clamd and mysql and vsftpd and postfix - doing absolutely nothing it's swapped out 150MB. Needing more RAM and a new harddrive to replace the failed one... it's getting to the point where it's not worth using this 1Ghz P3 Coppermine system.

Installing horde...

After using the horde file manager for a project at work, I decided to give it a test drive at home. So far, so good. As this is just a dry run for the migration of tonns.com to home (as I don't have the right setup due to hardware failures). I was feeling kind-of lazy about it, so I followed this HOWTO to get postfix/SMTP-AUTH/TLS/dovecot going.

Key points on the install:

It's all about rpmforge. Dag Wieers really takes the headache out of installing all this with the rpmforge repository. Remember to send him an email thanking him - I did.
The list of packages I had to "yum install" is as follows:
apr apr-util autoconf automake curl curl-devel cyrus-sasl-devel cyrus-sasl-gssapi dovecot e2fsprogs-devel gd httpd httpd-suexec ImageMagick krb5-devel libc-client libidn libidn-devel libtool libtool-libs libxml2-devel mysql mysql-server openssl-devel pam-devel perl-DBD-MySQL perl-DBI perl-HTML-Parser perl-HTML-Tagset perl-libwww-perl perl-URI php php-devel php-domxml php-gd php-imap php-ldap php-mysql php-odbc php-pear php-pear-log php-pear-mail_mime php-pecl-fileinfo php-pecl-memcache php-xmlrpc pkgconfig postgresql-libs rpm-build unixODBC vsftpd zlib-devel
PHP and PAM don't play nice together. The pam_auth module for php exists, but damn I couldn't get it to compile as a module and fuck-no, I'm not recompiling PHP.
Instead, setup IMP and MIMP before other modules, and use thier auth (i.e. imap auth) as the horde auth using this setup: http://wiki.horde.org/MIMPHowTo adding $conf['auth']['driver'] = 'composite'; as well.

Getting the latest PEAR modules to install was also a PITA. IMP requires HTTP_Request and Auth_SASL modules, but to get them installed I had to lock-step upgrade modules to interim versions before everything would update. Specifically:

pear upgrade Archive_Tar
pear upgrade PEAR-1.3.3
pear upgrade PEAR
pear upgrade XML_RPC-1.4.0
pear upgrade-all
pear install HTTP_Request
pear install Auth_SASL

Which finally left me with:

Installed packages, channel pear.php.net:
=========================================
Package        Version State
Archive_Tar    1.3.1   stable
Auth_SASL      1.0.2   stable
Console_Getopt 1.2     stable
DB             1.7.6   stable
HTTP           1.4.0   stable
HTTP_Request   1.4.0   stable
Log            1.9.9   stable
Mail           1.1.14  stable
Mail_Mime      1.3.1   stable
Net_SMTP       1.2.8   stable
Net_Socket     1.0.6   stable
Net_URL        1.0.14  stable
PEAR           1.4.11  stable
XML_Parser     1.2.7   stable
XML_RPC        1.5.1   stable

Other than that, it was mostly following the horde INSTALL files. It's pretty simple.

Today, I'm gonna try to take a look at SpamAssassin, ClamAV, amavisd-new, CRM114, etc. etc. blah. blah.

tony's geek stuff