As my avid blog readers will be aware, I recent had to rebuild my mail server from scratch, and being the big job that it is I've been taking care of it in chunks.
SMTP first, Imap/pop second, content/spam filtering is still to come.
Postfix has sender address verification enabled, along with a few strict smtpd checks enabled and this works well to filter out the really obvious stuff from many zombies, but as expected with dspam out of the picture I was getting spam coming through to my inbox.
I use thunderbird on the desktop, and while working up the energy to have a good run at setting up proper spam filtering serverside, I decided to see if thunderbird's junkmail filtering system actually did anything useful. It was quick to setup and took minimal effort to train, so I figured it wouldn't hurt to give it a shot.
To give some context, I'm not getting a lot of spam hitting my inbox (compared to many), we're only talking about 30-40 per day. Enough to be annoying when you're used to getting only 1 every 2-3 days, as was the case with dspam in charge.
I enabled thunderbird's junkmail feature and started tagging spam by hand to train it. I configured it to move mail to a /junk folder, but I didn't mark it as read as I wanted to visually get a handle of how it was doing.
Fairly early on into the training I noticed that a lot around half of my spam had a common traight - it was all to and from my email address.
Since I only email myself (using the same to/from address pair) when testing something, this immediately lent itself to a very simple and obvious mail filtering rule:
If mail is
from: me@me.com and
to: me@me.com, then
1. Mark it is junk
2. Move it to the junk folder
3. Mark it as read
This very simple rule provided some automatic training data, and cleared my inbox of obvious junk without requiring any intelligence on the part of the junk-mail engine.
I've been Thunderbird's filtering for about a week now, and so far I've only had 1 false positive, and maybe 5 false negatives.
All things considered, and the very small subset of messages and short training periods are big factors, I'm very impressed with the performance. So much so that I'm wondering just how much work it's worth putting into a serverside anti-spam system.
Good job Mozilla!
If like me you assumed that a client side junkmail filter is likely something of a toy, I encourage you to actually give it a shot. I'm a convert!
Sunday, March 15, 2009
roundcube webmail
Hot on the heals of my success with horde, I decided to check out another webmail package - roundcubemail.
I'd come across roundcube in my surfing once before, but it was a recent positive mention on one of the blogs that I follow that prompted me to check it out properly.
Roundcube can be found at www.roundcube.net and it comes in at a teeny 1.6MB tar.gz compared to horde's 26MB. OK yes, horde does a lot more than just webmail for that 26MB, but that was after all the only thing I actually wanted....
With a working apache and mysql instance already setup, adding another vhost, and database for roundcube was really simple. 10 minute job tops.
The initial configuration & "installation" is done entirely through the web interface by accessing the /installer directory.
This takes you through a nice systems requirement check screen, followed by an easy to use 3 step wizard which creates the rather complex config files for you.
You can either download them using the provided link, or just copy/paste the contents across from the text box onscreen, which I opted for. I was already sshed into my webserver anyway, pasting into vim was quickest.
You also have to create a log and temp directories that your webserver/roundcube instance has write access to.
It really was very painless to get through all that.
At the end of the wizard you can even test the configuration, both smtp, imap, and database access. This is one seriously polished product - top marks guys!
I did run into database issues again when trying to load the front page. The roundcube error log (/logs/errors) plained pointed to the cause though the database table didn't exist.
Just like last time I seem to have managed to start off with an empty database again. I still don't know what is causing this is, but the fix was the same as with horde:
mysql -u roundcubemail -ppasswordhere roundcubemail <>
With the database initialised, roundcube was ready for action.
And that's it!
I've been playing with it for 15 minutes so far and really love it :)
Do check it out :)
I'd come across roundcube in my surfing once before, but it was a recent positive mention on one of the blogs that I follow that prompted me to check it out properly.
Roundcube can be found at www.roundcube.net and it comes in at a teeny 1.6MB tar.gz compared to horde's 26MB. OK yes, horde does a lot more than just webmail for that 26MB, but that was after all the only thing I actually wanted....
With a working apache and mysql instance already setup, adding another vhost, and database for roundcube was really simple. 10 minute job tops.
The initial configuration & "installation" is done entirely through the web interface by accessing the /installer directory.
This takes you through a nice systems requirement check screen, followed by an easy to use 3 step wizard which creates the rather complex config files for you.
You can either download them using the provided link, or just copy/paste the contents across from the text box onscreen, which I opted for. I was already sshed into my webserver anyway, pasting into vim was quickest.
You also have to create a log and temp directories that your webserver/roundcube instance has write access to.
It really was very painless to get through all that.
At the end of the wizard you can even test the configuration, both smtp, imap, and database access. This is one seriously polished product - top marks guys!
I did run into database issues again when trying to load the front page. The roundcube error log (/logs/errors) plained pointed to the cause though the database table didn't exist.
Just like last time I seem to have managed to start off with an empty database again. I still don't know what is causing this is, but the fix was the same as with horde:
mysql -u roundcubemail -ppasswordhere roundcubemail <>
With the database initialised, roundcube was ready for action.
And that's it!
I've been playing with it for 15 minutes so far and really love it :)
Do check it out :)
Wednesday, March 11, 2009
Horde webmail
Following on from my last piece, today we're going to setup webmail using horde.
My previous webmail software was squirrelmail. It worked very well, but it was also very basic. I'd seen horde, and it did look pretty darn sexy so I started there this time around.
I downloaded horde groupware webmail edition, as this version is meant to be prepackaged and easier to roll out for specifically webmail related use.... which is what I needed.
Reading through the docs/INSTALL was a bit daunting, and it really suggested that hoard requires a proper database backend such as mysql... which was really a bit more configuration than I wanted for a simple webmail platform. Oh well.
I installed apache2, php5, mysql5 using yast.
Upon starting mysql for the first time it helpfully warned me about the default mysql security, and that the user accounts needed tidying up, which I did.
Setting up the apache vhost next was dead easy! I just can't believe how easy it was compared to previous attempts.
Suse provides a very well documented template in /etc/apache2/vhosts.d/vhost.template, and it worked first try.
I next ran the horde configure script from ./scripts/setup.php.
For some reason I ran into issues creating the initial mysql database using the script, so I had to manually pump the mysql setup script into mysql using
mysql -u root -p******* < /srv/www/vhosts/horde/scripts/sql/create.mysql.sql
With that done I was able to run the setup script again to simply to create the tables (rather than the initial DB too). This worked fine.
The default horde mysql access is using a preset username and password pair, so using the mysql command line I went into mysql and created a user account specifically for horde to use, with a different password.
Finally I reran the configure script again to use the new username.
The login worked correctly. As it happens my imap server is on localhost and presumably that's where horde looks for it, which is fortunate as I wasn't asked about that at any time. It's probably in an easy to change file somewhere though.
And that's horde. Day to day use and configuration is something I'm sure you can work out on your own :)
My previous webmail software was squirrelmail. It worked very well, but it was also very basic. I'd seen horde, and it did look pretty darn sexy so I started there this time around.
I downloaded horde groupware webmail edition, as this version is meant to be prepackaged and easier to roll out for specifically webmail related use.... which is what I needed.
Reading through the docs/INSTALL was a bit daunting, and it really suggested that hoard requires a proper database backend such as mysql... which was really a bit more configuration than I wanted for a simple webmail platform. Oh well.
I installed apache2, php5, mysql5 using yast.
Upon starting mysql for the first time it helpfully warned me about the default mysql security, and that the user accounts needed tidying up, which I did.
Setting up the apache vhost next was dead easy! I just can't believe how easy it was compared to previous attempts.
Suse provides a very well documented template in /etc/apache2/vhosts.d/vhost.template, and it worked first try.
I next ran the horde configure script from ./scripts/setup.php.
For some reason I ran into issues creating the initial mysql database using the script, so I had to manually pump the mysql setup script into mysql using
mysql -u root -p******* < /srv/www/vhosts/horde/scripts/sql/create.mysql.sql
With that done I was able to run the setup script again to simply to create the tables (rather than the initial DB too). This worked fine.
The default horde mysql access is using a preset username and password pair, so using the mysql command line I went into mysql and created a user account specifically for horde to use, with a different password.
Finally I reran the configure script again to use the new username.
The login worked correctly. As it happens my imap server is on localhost and presumably that's where horde looks for it, which is fortunate as I wasn't asked about that at any time. It's probably in an easy to change file somewhere though.
And that's horde. Day to day use and configuration is something I'm sure you can work out on your own :)
Saturday, March 7, 2009
Need new mail server. Right now.
A rather unfortunate and horrible series of events resulted in the urgent need for me to build a new mail server.
The old one was poked, bad file system corruption, and just wasn't worth trying to fix it. It was very old and had been dying for a while anyway. All of the actual maildir mail was on my solaris nfs server, so nothing was lost in that sense, but the job of building a new server to serve that mail up again has been something that I've been dreading.
I've been putting of rebuild/migrating to a new mail server infrastructure because I still clearly remembered just know long it took to build the first time around. The first time around I built pretty much everything from hand, and taught myself ldap along the way.
With mail queuing up around the world, and not just for me but also for friends and family too for whom I host mail, it was time to build a new mail server.
My initial immediate reaction was that I needed to fire up a new Solaris zone and to roll some sleeves up.
Before embarking on the arduous quest, I took a break for food and hopefully some near-divine inspiration....which didn't come.
What did come however was some perspective on my planned sadism.
[Open]Solaris doesn't come with courier-imap, postfix, amavisd, spam assassin... in fact it pretty much doesn't come with anything that I actually need to build a modern day open source mail server platform.
I thought this was somewhat ironic given that Solaris has always been a proud server OS, which is why many people complain that it's not yet ready for the desktop; because there is so much desktop software still missing, and yet here I am realising that the mail server lineup wasn't looking flash. In fact it's worse than not flash, it actually ships with sendmail...*shudder*
Sun does have the monstrous Sun Java System Messaging Server, but that was way bigger and more complex than I probably needed, and while in this case it wouldn't have been a problem for me, it's not free/oss software.
So, on to building all this software by hand again, on a non-GNU environment. fun!
No this is 2009, a year where more and more people are proclaiming that the OS doesn't matter any more... and you know what, they're right.
I think the measure of a good SA is using the right tool for the job, not just stubbornly and religiously sticking your belief in a near-holy, and faultless OS of choice.
There are a number of FOSS operating systems out there and while I love Solaris, I'm big enough to not only admit when it's not the answer , but to actually proclaim when it's not the right tool for the job - and this is that job.
For shame Solaris.... FOR SHAME!
No the answer is of course to use linux.
I've been using Ubuntu quite a bit for various tasks and installs lately so in the interests of learning something new, I decided to take another look at opensuse, this time at version 11.1.
Jumping on their website I was able to quickly search for the software that I wanted to use, and was pleasantly surprised to see that everything I wanted to use was there, and with very recent versions of said software.
I downloaded the iso via bit torrent, and installed into a VM called Athena - the goddess of heroic endeavor. (She is going to be doing battle with the evil doers of the 21st century after all. The spammers!)
Installing the distro was dead easy, and adding the software was very simple too. I won't bother outlining it.
I installed courier-authdaemon, courier-imap, amavisd, clamav and went about configuring everything.
I wanted to use nfs4, and I hit some problems that eventually went away with a reboot. I'm still not quite sure what went wrong, but I think it was something to do with the required kernel module not being loaded.
With the user names matched up, the appropriate firewall rules configured (a big plus in favour of nfs4. 1 static port for all nfs traffic!), the old mail store mounted and was accessible.
Setting up courier-authdaemon took quite a few tries to get right; my authentication and mail information is all stored in an ldap directory.
Once that was done, I moved on to postfix.
Postfix is pretty easy to work with, but there are a lot of lines to change in the config file and the version that came with opensuse 11.1 (2.5.5) was a couple of major versions newer than the 2.3.x that I'd been using previously so I took the chance to familiarise myself with some of the newer config options.
Pretty soon mail was flowing through nicely, and I left it there for the moment. Webmail and content filtering to come next.
The old one was poked, bad file system corruption, and just wasn't worth trying to fix it. It was very old and had been dying for a while anyway. All of the actual maildir mail was on my solaris nfs server, so nothing was lost in that sense, but the job of building a new server to serve that mail up again has been something that I've been dreading.
I've been putting of rebuild/migrating to a new mail server infrastructure because I still clearly remembered just know long it took to build the first time around. The first time around I built pretty much everything from hand, and taught myself ldap along the way.
With mail queuing up around the world, and not just for me but also for friends and family too for whom I host mail, it was time to build a new mail server.
My initial immediate reaction was that I needed to fire up a new Solaris zone and to roll some sleeves up.
Before embarking on the arduous quest, I took a break for food and hopefully some near-divine inspiration....which didn't come.
What did come however was some perspective on my planned sadism.
[Open]Solaris doesn't come with courier-imap, postfix, amavisd, spam assassin... in fact it pretty much doesn't come with anything that I actually need to build a modern day open source mail server platform.
I thought this was somewhat ironic given that Solaris has always been a proud server OS, which is why many people complain that it's not yet ready for the desktop; because there is so much desktop software still missing, and yet here I am realising that the mail server lineup wasn't looking flash. In fact it's worse than not flash, it actually ships with sendmail...*shudder*
Sun does have the monstrous Sun Java System Messaging Server, but that was way bigger and more complex than I probably needed, and while in this case it wouldn't have been a problem for me, it's not free/oss software.
So, on to building all this software by hand again, on a non-GNU environment. fun!
No this is 2009, a year where more and more people are proclaiming that the OS doesn't matter any more... and you know what, they're right.
I think the measure of a good SA is using the right tool for the job, not just stubbornly and religiously sticking your belief in a near-holy, and faultless OS of choice.
There are a number of FOSS operating systems out there and while I love Solaris, I'm big enough to not only admit when it's not the answer , but to actually proclaim when it's not the right tool for the job - and this is that job.
For shame Solaris.... FOR SHAME!
No the answer is of course to use linux.
I've been using Ubuntu quite a bit for various tasks and installs lately so in the interests of learning something new, I decided to take another look at opensuse, this time at version 11.1.
Jumping on their website I was able to quickly search for the software that I wanted to use, and was pleasantly surprised to see that everything I wanted to use was there, and with very recent versions of said software.
I downloaded the iso via bit torrent, and installed into a VM called Athena - the goddess of heroic endeavor. (She is going to be doing battle with the evil doers of the 21st century after all. The spammers!)
Installing the distro was dead easy, and adding the software was very simple too. I won't bother outlining it.
I installed courier-authdaemon, courier-imap, amavisd, clamav and went about configuring everything.
I wanted to use nfs4, and I hit some problems that eventually went away with a reboot. I'm still not quite sure what went wrong, but I think it was something to do with the required kernel module not being loaded.
With the user names matched up, the appropriate firewall rules configured (a big plus in favour of nfs4. 1 static port for all nfs traffic!), the old mail store mounted and was accessible.
Setting up courier-authdaemon took quite a few tries to get right; my authentication and mail information is all stored in an ldap directory.
Once that was done, I moved on to postfix.
Postfix is pretty easy to work with, but there are a lot of lines to change in the config file and the version that came with opensuse 11.1 (2.5.5) was a couple of major versions newer than the 2.3.x that I'd been using previously so I took the chance to familiarise myself with some of the newer config options.
Pretty soon mail was flowing through nicely, and I left it there for the moment. Webmail and content filtering to come next.
Subscribe to:
Posts (Atom)