SQmaiL

Published: 2017 August 2

Index

Important note! I wrote this in about 2000, used it for a while, but then moved on to using IMAP instead (so I wasn't storing my mail locally). The site here is for archival purposes after migrating off SourceForge. Don't trust a word of it. You probably don't want to use this... for anything.

Introduction

SQmaiL is a MUA (mail user agent), written in Python, that stores all of its messages in a MySQL database, using Gnome. It's currently under heavy development (i.e. this morning I had to quickly add the ability to save attachments when my boss mailed me an image he wanted me to use and I suddenly realised I had no way of getting the image out of the database). Here's a quick list of features:

  • Can handle very large mail databases (>10000 messages) very quickly and easy,
  • Create VFolders based on SQL queries to painlessly sort your mail archive.
  • Ability to manually override VFolder selection for those messages that can't be automatically characterised.
  • Full-featured MIME support for both sending and receiving mail.
  • Single-click report-a-spam feature (using Spamcop).
  • Mail retrieval from POP3 servers or local spool files.
  • Picons and xface support.

It's currently in an alpha state. This means that you have been warned. I use it as my main mailer, but I know how it works and know how to avoid all the bugs. A number of features are missing (manual message filing, IMAP fetching, purging old deleted messages [on purpose, this one!]). Feel free to try it if you like; feel even more free to add enhancements. I recommend you keep backups of all your mail. Currently, SQmaiL does not have to ability to delete messages, and the actual mail handling part seems pretty stable to me; however, if you insist on using it in mission-critical situations, and it fails, I am not responsible. Read the license.


Current version: 0.3.1-alpha. New! Debian packages! Now installing SQmaiL is as simple as, um, a moderately simple thing! No more faffing around with the source to make it go. This is a huge improvement. This release is really just an interim: I'm off on holiday to Australia for three weeks, and I wanted to get the Debian patch out before then.

Also new in this release: an inline image viewer; the ability to incorporate messages from things like slocal or procmail scripts; lots of bug fixes. Get it before it's too late!

SQmaiL is released under the GNU Public License.

Frequently Asked Questions

Frequently Asked Questions

Why are all the dates displayed in a funny order?

Because SQmaiL is intended for an international audience, and YYYY-MM-DD is the only unambiguous date format. UK format (DD-MM-YYYY) is too easily confused with US (MM-DD-YYYY), and vice versa; and besides, US date format is a crime against nature and utterly abhorrent. YYYY-MM-DD also happens to be ISO standard, so there.

You keep telling me to back up my database. How?

There will soon be a nice interface for doing this, but for the time being, do:

mysqldump --opt -p sqmail | gzip -c > sqmailbackup.sql.gz

...replacing sqmail with whatever your mail database is called, of course. This will generate a compressed SQL script that will recreate the database. You can restore a backup with:

zcat sqmailbackup.sql.gz | mysql -p sqmail

You can use this to copy databases, too.

I'm upgrading from an old version and nothing seems to work any more.

Sigh. You didn't read the instructions, did you?

As SQmaiL evolves, particularly in these early stages, the database format will periodically change. (For example, the way vfolders are stored changed dramatically from 0.1 to 0.2 when I realised that the old way was deeply brain-damaged.) So, I've included a tool that will upgrade your database if it is necessary. To do this, just type:

sqmail upgrade

SQmaiL will look at the database, determine what parts need upgrading, and do it. (Yes, I could do this automatically. No, I haven't got the nerve to do it every time.) The upgrade procedure should be completely safe in that it will correctly do nothing if nothing needs to be done, but, well, hey, it's alpha.

I select some text and it crashes!

It's a bug in pygtk.

As far as I can tell, this bugs bites when you (a) select some text in a message; (b) change messages without deselecting the text; (c) select some more text. If you deselect the text before changing messages, it doesn't appear to be a problem. Sorry --- nothing I can do. Complain to the pygtk authors.

I select a message and it won't MIME decode it and stuff comes out on the console.

The message is not correctly MIME-encoded.

I'm using the Python mimetools/multifile libraries to decode the messages. They're a bit pedantic. They're also a bit buggy --- see the next question. I'm afraid there's nothing I can do at this point. I'm trying to track down the authors, but don't hold your breath. You can use the raw message display to read the message, but you won't be able to extract attachments from it. Sorry.

I select a message and it locks up.

It's a bug in the mimetools/multifile libraries. When I ask for the next part of a message, on some (rare) messages mimetools gives me the same part I already have. This seems to happen particularly with bounce messages. I've put in some code to try and protect against this, but it doesn't always work. The only thing to do is to kill the process (^Z in the console is the only way of interrupting it) and restart. Don't click on the message again. Sorry.

It won't let me send messages without a domain!

Yes. It's a feature.

Some mail servers, mine for example, will reject addresses without a domain. Also, unqualified addresses can be easily confused with mail aliases. So SQmaiL detects these and warns you. (It won't warn you if you've entered an alias. It'll just expand it and send the message.)

If you really want your old behaviour, go into the Preferences/Transport dialogue and enter your domain into the Default Domain box. Now if you enter and unqualified address, SQmaiL will silently add on the domain you've just entered and send the message.

How do I use the aliases window?

The aliases window lets you edit your current set of mail aliases. A mail alias is a single word like an email address that SQmaiL will expand it into a set of real addresses. This is useful if you want to send mail to the same set of people a lot, without setting up a mailing list.

The user interface is currently a bit strange. I'm working on it.

To create a new entry, press the New button. You can then click on the new (blank) entry, and enter text in the box. If you press New when nothing is selected, a new alias is created; if an alias or email address is selected, a new email address will be added to that alias.

Did that make sense? Um. Suggestions, please.

How do I do SQL queries?

You need to type a boolean expression into the box in the bottom-left corner. If your message matches the criterion you supply, and the criterion for that folder's parents, then it will be displayed in that vfolder. Note that messages can appear in multiple vfolders. You will probably want to use the SQL like operator, which does a glob match:

subjectfield like "%[listname]%"
...will extract messages sent by a number of major mailing list software packages. (The % is a wildcard character.) Check out the MySQL documentation for more information.

The following SQL fields are available:

  • fromfield --- the sender's email address.
  • realfromfield --- the sender's real name.
  • tofield --- the To: field, in encoded form.
  • ccfield --- the Cc: field, in encoded form.
  • subjectfield --- the message's subject.
  • date --- the arrival date of the message (in SQL datetime format).
  • annotation --- the message's annotation.
  • id --- the SQmaiL internal message ID.
  • readstatus --- one of: Read, Unread, Deleted, Sent.

Encoded form fields are stored as Python pickles. You can ignore this for all intents and purposes; like will work as expected for both real names and email addresses.

Currently it's not possible to search for arbitrary headers or body text. Support for this is planned. (Currently, the headers and the bodies are stored in different tables. This turns out to be unnecessary, and I'll collapse the two together at some point.)

What's the deal with empty SQL queries?

If you leave the query space blank, that folder is treated specially. The folder will display all the messages that aren't being shown in any of the other vfolders on the same tier. For example, if you have under Received Messages the folders foo, bar and unfiled, where the query for unfiled is blank, then only messages that are not shown in foo and bar will appear in it.

Note that messages in subfolders aren't taken into account with this.

Internally it works by taking the queries for all the other folders and combining them into (not QUERY1) and (not QUERY2) and (not QUERY3)... As a result, some rather odd things can happen. If one of your folders has a query of 1 (display all messages), then that will suppress any messages appearing in the unfiled folder (because not 1 is 0).

How can I add custom headers to outgoing messages?

Type the full header into the Extras box in the Compose window. For example, to Cc your message to me, type:

Cc: dg@tao-group.com
Anything in the Extras box will be added verbatim to the message header. There is no syntax checking. Be careful.

How do I do signatures?

You need one or more files in your home directory with a name starting with .signature. When you press the 'Sign' button in the Compose window, a menu will pop up giving you the choice of which signature you want to use. The contents of the file will be appended to the end of your message.

However, if the signature file is executable, the file will be run instead, and the output appended. This allows you to have dynamic signatures (I do this).

Installation

New installations

Installing on Debian

The recommended installation method is to use the debian package. This will install SQmaiL and the support packages:

dpkg -i sqmail_*.deb
apt-get -f install

I'm afraid the only packaging system supported so far is Debian. If any Red Hat users are reading and are interested in doing a set of control files for RPMs, please get in touch --- as I don't do Red Hat, I can't do any of this myself.

Installing on anything else

If you don't do Debian you'll have to install it manually. This is currently a pain. The first thing you need to do is to install the support software:

  • Python 1.5, available from the Python web page;
  • MySQL client (available from the MySQL page, and Python module for same (MySQLDB);
  • Gnome binaries (available from Ximian Gnome, and Python modules for same (there are two of them, PyGTK and gnome-python, both available here).
  • mail-lock and mail-unlock programs (which unfortunately I don't know where these live, but you can get the source from the Debian package).

You'll also, of course, need a MySQL server. This can run locally or remotely, SQmaiL will still work. Install this according to the documentation and create a user for yourself. Remember to set the permissions so that SQmaiL can access the database!

Now you're ready to install the SQmaiL package. Go to the Downloads page; you want the SQmaiL archive with the highest version number. Decompress this in a directory somewhere. Currently, SQmaiL doesn't install into your file system; I'm working on this. You'll have to run it from the directory you decompress it in to; the main executable is the SQmaiL file. (Use thise wherever sqmail is referenced below.)

Initial configuration

Before SQmaiL can run, you'll need to create a database for it to use. I'm assuming you've already created a user. This can be done with:

sqmail create-database

This is a simple command-line program that creates a new database. You'll need to give it your MySQL administrator password. (You did remember to set one, didn't you? Didn't you?). Follow the instructions. Warning: create-database has the ability to delete databases. Any database (you just gave it the root administrator password). Irrevocably. You may be about to delete hundreds of megabytes of crucial data. Think very carefully before giving it the go-ahead to delete the database.

(The root password is not stored anywhere, and is only used for the initial database creation. All other interactions with the MySQL server, including running SQmaiL proper, is done while logged in as the user.)

Now you can start up SQmaiL proper by doing:

sqmail gui

The first time you run it, it will ask you for details on how to contact the database server on the console. These are stored in a hidden file in your home directory. All other settings are stored in the database itself.

Upgrades

Install the new source in the normal way. You should then do:

sqmail upgrade

...to upgrade your database from the old format to the new one (if necessary). If you don't do this, some very strange things may happen (including loss of data). This is important.

Screenshots

These are snapshots from the version I'm currently using, with the Metal GTK theme. The window manager is Sawfish, with the MicroGUI theme.

[screenshot]

The main window. On the left is the vfolder list; you can see the SQL query in the box at the bottom. The message itself is shown in the notebook on the right, one attachment per pane. Want more of Thog's Favourites? Try the Thog's Masterclass page in the the Ansible archives.) Obsolete: the 0.2 version looks slightly different.

[screenshot]

The Compose window. This is where you compose new messages. Note the attachments pane.