pymta Documentation

pymta is a library to build a custom SMTP server in Python. This is useful if you want to...

• test mail-sending code against a real SMTP server even in your unit tests.
• build a custom SMTP server with non-standard behavior without reimplementing the whole SMTP protocol.
• have a low-volume SMTP server which can be easily extended using Python.

Installation + Setup

pymta is just a Python library which uses setuptools so it does not require a special setup. The only direct dependency is repoze.workflow (0.2dev). Currently pymta is only tested with Python 2.5 but probably 2.4 works too. The goal is to make pymta compatible with Python 2.3-2.6. Python 2.3 may require a custom version of asyncore.

repoze.workflow

repoze.workflow is not available in pypi (yet?) so you have to install it directly from the svn:

easy_install http://svn.repoze.org/repoze.workflow/trunk/

repoze.workflow requires zope.interface which available via pypi (and installable via the package manager for most Linux distributions).

Goals of pymta

The main goal of pymta is to provide a basic SMTP server for unit tests. It must be easy to inject custom behavior (policy checks) for every SMTP command. Furthermore the library should come with an extensive set of tests to ensure that does the right thing(tm).

Development Status

Currently (12/2008, version 0.2) the library only implements basic SMTP with very few extensions (e.g. PLAIN authentication). ‘Advanced’ features which are necessary for any decent MTA like TLS and pipelining are not implemented yet. Currently pymta is used only in the unit tests for TurboMail. Therefore it should be considered as beta software.

Related Projects

There are some other SMTP server implementations in Python available which you may want to use if you need a proven implementation right now:

• Python’s smtpd
• tmda-ofmipd
• Twisted Mail

Python’s smtpd is a module which is included in the standard distribution of Python for a long time. In the beginning I used this module for my unit tests but quite soon I had to realize that the code is quite old and messy and it is nearly impossible to implement a custom behavior (e.g. reject certain recipients). pymta evolved out of smtpd after multiple refactorings based on the idea to use a simple finite state machine (repoze.workflow).

tmda-ofmipd is another implementation which is based on Python’s smtpd. It is distributed only as part of a larger Python application which makes it harder to use if you just need a plain Python SMTP server. Furthermore the code was not cleaned up so it may be a bit hard to understand but it supports TLS (using tlslite).

Twisted Mail is probably the most featureful SMTP server implementation in Python available right now. It uses the twisted framework which can be either a huge advantage or disadvantage, depending on your point of view. When I started out with the naïve idea of just extending Python’s smtpd a bit, I dismissed Twisted Mail because it seemed to be quite hard to implement some custom behavior without writing too much code.

Architectural Overview

pytma uses asynchronous programming to handle multiple connections at the same time and is based on Python’s asyncore. There are two state machines: One for the SMTP command parsing mode (single-line commands or data) in the SMTPCommandParser and another much bigger state machine in the SMTPSession to control the correct order of commands sent by the SMTP client.

The main idea of pymta was to make it easy adding custom behavior which is considered configuration for ‘real’ SMTP servers like Exim. Therefore you should look at the DefaultMTAPolicy to add restrictions on certain SMTP commands (check recipient addresses, scan the message’s content for spam before accepting it). In order to authenticate SMTP clients (check username and password) you may implement the IAuthenticator interface.

Components

pymta consists of several main components (classes) which may be important to know.

PythonMTA

The PythonMTA is the main server component which listens on a certain port for new connections. There should be only one instance of this object. When a new connection is received, the PythonMTA spawns a new SMTPCommand parser which will handle the complete SMTP session. If a message was submitted successfully, the new_message_received() method of PythonMTA will be called so the MTA is in charge of actually doing something with the message.

You can instantiate a new server like that:

import asyncore
from pymta import PythonMTA

mta = PythonMTA('localhost', 25)
try:
    asyncore.loop()
except KeyboardInterrupt:
    pass

Interface

class PythonMTA(local_address, bind_port[, policy_class[, authenticator_class]])

local_address is a string containing either the IP oder the DNS hostname of the interface on which PythonMTA should listen. policy_class and authenticator_class are callables which can be used to add custom behavior. Every new connection gets their own instance of policy_class and authenticator_class so these classes don’t have to be thread-safe. If you ommit the policy, all syntactically valid SMTP commands are accepted. If there is not authenticator specified, authentication will not be available.

new_message_received(msg)

This method is called when a new message was submitted successfully. The mta is then in charge of delivering the message to the specified recipients. Please not that you must not reject the message anymore at this stage (if there are problems you must generate a non-delivery report aka bounce). Because there can be multiple active connections at the same time it is a good idea to make the method thread-safe and protect queue access.

DefaultMTAPolicy

The policy is asked after every SMTP command if the command should be accepted or not. You can use this to check the parameter of the command or make some features only available for selected clients.

All methods in the policy should return a boolean value to express if a command should be accepted. If you need more control you can return a tuple containing the boolean decision and either a single-line response as string or list/tuple containing the response code and response text:

def accept_helo(self, helo_string, message):
    # pymta will return the default error message for the given command if
    # you just return False
    return False

    # This will send out a '553 Bad helo string' and the command is
    # rejected. pymta won't send any additional reply because you did that
    # already.
    return (False, (553, 'Bad helo string'))

    # This is basically the same as above but now it will trigger a
    # multi-line SMTP response:
    # 553-Bad helo string
    # 553 Evil IP
    return (False, (553, ('Bad helo string', 'Evil IP'))

In the default policy you don’t have to care if the commands were given in the correct order (the state machines will take care of that). The only thing is that the message object passed into many policy methods does not contain all data at certain stages (e.g. accept_mail_from can not access the recipients list because that was not submitted yet).

Interface

accept_new_connection(peer)

This method is called directly after a new connection is received. The policy can decide if the given peer is allowed to connect to the SMTP server. If it declines, the connection will be closed immediately.

accept_helo(helo_string, message)

Decides if the HELO command with the given helo_name should be accepted.

accept_ehlo(ehlo_string, message)

Decides if the EHLO command with the given helo_name should be accepted.

accept_auth_plain(username, password, message)

Decides if we allow AUTH plain for this client. Please note that username and password are not verified before, the authenticator will check them after the policy allowed this command.

accept_from(sender, message)

Decides if the sender of this message (MAIL FROM) should be accepted.

accept_rcpt_to(new_recipient, message)

Decides if recipient of this message (RCPT TO) should be accepted. If a message should be delivered to multiple recipients this method is called for every recipient.

accept_data(message)

Decides if we allow the client to start a message transfer (the actual message contents will be transferred after this method allowed it).

accept_msgdata(message)

This method actually matches no real SMTP command. It is called after a message was transferred completely and this is the last check before the SMTP server takes the responsibility of transferring it to the recipients.

IAuthenticator

Authenticators check if the user’s credentials are actually correct. This may involve some checking against external subsystems (e.g. a database or a LDAP directory).

Interface

authenticate(username, password, peer)

This method is called after the client issued an AUTH PLAIN command and must return a boolean value (True/False).

Message

The Message is a data object contains all information about a message sent by a client. This includes not only the actual RFC822 message contents but also information about the SMTP envelope, the peer and the helo string used. The information is filled as the client sends some commands so not all information may be available at any time (e.g. the msg_data not available before the client actually sent the RFC822 message).

Peer

The Peer is another data object which contains the remote host ip address and the remote port.

SMTPSession

This class actually implements the most complicated part of the SMTP state machine and is responsible for calling the policy. If you want to extend the functionality or need to implement some custom behavior which is beyond what you can do using Policies, check this class.

Unit Test Helper Classes

The pymta package contains some helper classes which are not needed for normal operation but may be helpful your unit tests.

DebuggingMTA

MTAThread

TODO

• Size checking (only accept messages below a certain size, don’t)
• TLS implementation (probably using nss / python-nss or gnutls / python-gnutls)
• Restructure the classes, cleanup API, use interfaces and put them into the api package.