OpenBSD version: 7.0
Arch:            Any
NSFP:            Uh, better don't

The past couple of posts have been mostly about me making things ping. However, when it comes to self-hosting things, email is usually also a top priority. I have been running my own mail setup for well over a decade. Since i got off on the wrong foot with exim, i always did so using postfix, and a setup using dovecot as my MDA, and then SOGo as my webmail interface.

However, even though this has been running on postfix + OpenBSD just fine, i a) want to reduce the footprint of non-base software in my setup, and b) want to replace the funny DKIM stack around postfix. Luckily, OpenSMTPd exists, and integrates well with rspamd, which–in addition to handling DKIM signing–also has easy DMARC reporting. So, off i go.

However, running OpenSMTPd with rspamd and dovecto is already well documented. So, instead i am going to go into three issues i encountered in the migration, where OpenSMTPd did not quiet do what i was expecting:

  1. Dealing with SMTP-AUTH, given a mysql backend populated by sogo/dovecot style passwords.
  2. Having (legacy from the postfix setup) multiple tables for aliases and users (final destinations)
  3. Expanding aliases with delimiters intact (kind’a incorrect behavior… )

Dealing with passwords from SOGo and Dovecot

The first issue i ran into was OpenSMTPd’s rather strict perspective on how password hashes should look like. It ingests the most recent format of bcrypt hashes, i.e., those starting with $2b$, indicating hashes have been generated after the OpenBSD implementation issue of 2014 has been fixed. If you follow standard howtos, you will generate passwords with smtpctl, and dovecot happily uses those generated passwords. I, on the other hand, have a mysql database storing passwords, and users may update them via the SOGo web frontend…

A problem arises, when we let dovecot and/or SOGo handle passwords. First of all, dovecot (and SOGo, following that shema) support a set of algorithms. Hence, both put a {BLF-CRPT} in front of the hash when writing it to the database. Furthermore, for some reason, dovecot and SOGo are convinced those–even though generated on OpenBSD–hashes should be $2y$ version hashes, indicating they have been generated after an issue (not affecting OpenBSD) in 2011. Interestingly… dovecot uses libc’s crypt() to generate hashes, so on OpenBSD these should actually be prefixed by a $2b$… Nevertheless, it means that if a user updates their password with SOGo it goes from $2b$05$3rKAsLWBazz3d8j9OAuGcO/FgbSEbOX8h.ab/sCV8JpT6pdqHbNhS to {BLF-CRYPT}$2y$05$.xDA12VH2RhMrQUqQb1EbufqzCU./ZyWX/CCKIwAx7L02hvYOIRLC in the database. In turn, this means that users will no longer be able to authenticate to OpenSMTPd to send mail.

Essentially, given they are generated with OpenBSD’s crypt(), these hashes should be $2b$ anyway… Hence, to get rid of this, i figured that mysql is the right spot to make a difference.

I already had (abreviated) the following in my smtpd.conf:

table credentials mysql:/etc/mail/mysql-user.conf
...
listen on vio0 port 465 pki mail.aperture-labs.org smtps auth <credentials> filter "rspamd-sign" tag "DKIM"
listen on vio0 port 587 pki mail.aperture-labs.org tls-require auth <credentials> filter "rspamd-sign" tag "DKIM"

With /etc/mail/mysql-user.conf containing:

...
query_credentials select c_uid, password from vusers WHERE c_uid=?;

mysql offers a text replace function for results. Hence, I simply changed this to:

...
query_credentials select c_uid, REPLACE( REPLACE(c_password, '{BLF-CRYPT}', ''), '$2y$', '$2b$') as password from vusers WHERE c_uid=?;

I am chaining this here, as SOGo does not necessarily add the {BLF-CRYPT} in front of hashes. Hence, to make sure things work independently, i chained two REPLACE statements after each other. Luckily, this works exactly as intended. Users can change their passwords, and i can run OpenSTMPd.

Having multiple tables for aliases and final destinations

The second ‘legacy’ issue from postfix that I have with my mysql backend is that i have muliple tables. One for virtual users (including their credentials, and one for alias mappings:

MariaDB [smtpd]> show tables;
+-----------------+
| Tables_in_smtpd |
+-----------------+
| valias          |
| vdomains        |
| vusers          |
+-----------------+
3 rows in set (0.001 sec)

MariaDB [smtpd]> describe valias;
+----------+-------------+------+-----+---------+----------------+
| Field    | Type        | Null | Key | Default | Extra          |
+----------+-------------+------+-----+---------+----------------+
| alias_id | int(8)      | NO   | PRI | NULL    | auto_increment |
| addr     | varchar(42) | NO   |     | NULL    |                |
| alias    | varchar(42) | NO   |     | NULL    |                |
+----------+-------------+------+-----+---------+----------------+
3 rows in set (0.023 sec)

MariaDB [smtpd]> describe vusers;
+------------+--------------+------+-----+---------+----------------+
| Field      | Type         | Null | Key | Default | Extra          |
+------------+--------------+------+-----+---------+----------------+
| c_id       | int(11)      | NO   | PRI | NULL    | auto_increment |
| c_uid      | varchar(128) | NO   |     | NULL    |                |
| c_name     | varchar(128) | NO   |     | NULL    |                |
| c_password | varchar(255) | NO   |     | NULL    |                |
| c_cn       | varchar(255) | NO   |     | NULL    |                |
| mail       | varchar(128) | NO   |     | NULL    |                |
+------------+--------------+------+-----+---------+----------------+
6 rows in set (0.023 sec)

For OpenSMTPd, this should be one table, with virtual accounts resolving to the local virtmail user, and aliases resolving to another mail address. I now could have multiple rules in my smtpd conf… or I again use mysql. The first thing we have to do is a UNION select, so we can select from two tables into one result set. Essentially, what we want, is the following:

SELECT alias from valias where addr = ? UNION select '_virtmail' from vusers where mail = ?;

We return the union of a) the destination from the valias table if we find the lookup address there, and b) _virtmail, if the lookup address belongs to a user in the virtual users’ table.

The issue here is that OpenSMTPd only wants to fill one parameter in a prepared statement. Hence, to make this work we have tobring in an additional mysql feature: Temporary tables.

with t as (SELECT ? as addr) select alias from valias join t on valias.addr = t.addr UNION select '_virtmail' as alias from vusers join t on vusers.mail = t.addr;

What we do here is create a single temporary table (t), with a single entry, which we fill with the variable in our prepared statement. We can then use this to do a join on that table in our select queries over which we form a UNION to only have one parameter go into the statement, while being able to use the same parameter on both sub-classes of our statement.

Expanding with delimiters intact

This next one is kind of doing a stupid thing, as i am replicating technically not so correct behavior i really enjoyed on postfix. Consider the following:

foo@example.com: bar@example.com

If i am using recipient delimiters–+ in my case–an email to foo+test@example.com will get delivered to bar@example.com. I am using this quiet extensively, and actually have some sieve rules in palce, that automatically place every delimiter in a sub-folder. With the migration to OpenSMTPd, dovecot would no longer do this filing into subfolders anymore. Instead, it went straight to the INBOX.

Some digging revealed that things were actually going as they should. The alias expands to bar@example.com, and there would be no delimiter in the rcpt-to dovecot sees on LMTP based on which this extension happens… So, what we need is a way to preserve the delimiter over the alias expansion. What helps us here is that OpenSMTPD supports delimiters, but will still look up addresses including the recipient delimiter first, i.e.:

  1. Look up foo+test@example.com
  2. Look up foo@example.com This means that we can craft our mysql query to handle the delimiter ourself, i.e., we make it find bar+test@example.com if it has to look up foo+test@example.com and there is foo@example.com: bar@example.com in the database. After returning bar+test@example.com, it will deliver to bar@example.com (after another lookup), making things work as wanted (but not necessarily intended).

To make this happen, we can join together a couple of CONCAT and REGEXP_REPLACE statements (and there is most likely an easier way… also no guarrantees for this being totally secure, there are a couple of fun things you can do with regexes…) What this ultimately does to our query is this:

with t as (SELECT ? as addr) select REGEXP_REPLACE(alias, '@', CONCAT(REGEXP_REPLACE( REGEXP_REPLACE(t.addr, '^[^+]*', '' ), '@.*', '') ,'@')) as alias from valias join t on valias.addr = REGEXP_REPLACE(t.addr , '[+][^@]+', '' ) UNION select '_virtmail' as alias from vusers join t on vusers.mail = t.addr;

What we do to the first part of the UNION is first stripping the delimiter from the lookup (join t on valias.addr = REGEXP_REPLACE(t.addr , '[+][^@]+', '' )). We then select the destination of our alias, but insert the delimiter we cut out from the query parameter into the result. We do this by first extracting the raw delimiter (REGEXP_REPLACE( REGEXP_REPLACE(t.addr, '^[^+]*', '' ), '@.*', '')) and then doing a CONCAT on this with @. We then replace the @ in our result with this constructed <extracted_delimiter>@. Not pretty, but it works.

Summary

So, that’s it. Three stupid things done with OpenSMTPd, but certainly things that allow me to use it more comfortably.