modern full-featured open source secure mail server for low-maintenance self-hosted email
Find a file
Mechiel Lukkien 141637df43
when creating a mailbox subscription, don't just try to insert a record into the database and handle bstore.ErrUnique, the transaction will have been marked as botched
behaviour around failing DB calls that change data (insert/update) was changed
in bstore quite some time ago. the tx state in bstore would become inconsistent
when one or more (possibly unique) indexes had been modified, but then an
ErrUnique would occur for the next index. bstore doesn't know how to roll back
the partial changes during a transaction, so it marks the tx as botched and
refuses further operations. so, we cannot just try to insert, wait for a
possible ErrUnique, but then still try to continue with the transaction.
instead, we check if the record exists and only insert it if we couldn't find
it.

found while working on webmail.
2023-08-01 10:14:02 +02:00
.github/workflows add github workflow to build & test, exporting a coverage file 2023-05-22 18:01:21 +02:00
.go mox! 2023-01-30 14:27:06 +01:00
autotls fix typo's and old reference 2023-06-22 21:27:52 +02:00
config go fmt and updated config after make build 2023-07-23 17:08:55 +02:00
dkim add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
dmarc fix typo's 2023-02-06 11:00:11 +01:00
dmarcdb cleanup for warnings by staticcheck 2023-07-24 13:55:36 +02:00
dmarcrpt mox! 2023-01-30 14:27:06 +01:00
dns for dns resolve errors likely due to a missing name server in /etc/resolv.conf, point user to man page of systemd-resolved, the likely cause 2023-06-12 14:53:07 +02:00
dnsbl mox! 2023-01-30 14:27:06 +01:00
dsn implement decoding charsets (other than ascii and utf-8) while reading textual message parts, and improve search 2023-07-28 22:15:23 +02:00
http when creating a mailbox subscription, don't just try to insert a record into the database and handle bstore.ErrUnique, the transaction will have been marked as botched 2023-08-01 10:14:02 +02:00
imapclient add condstore & qresync imap extensions 2023-07-24 21:25:50 +02:00
imapserver when creating a mailbox subscription, don't just try to insert a record into the database and handle bstore.ErrUnique, the transaction will have been marked as botched 2023-08-01 10:14:02 +02:00
iprev mox! 2023-01-30 14:27:06 +01:00
junk implement decoding charsets (other than ascii and utf-8) while reading textual message parts, and improve search 2023-07-28 22:15:23 +02:00
message use Go's mail.ReadMessage instead of textproto.ReadMIMEHeaders and decode RFC 2047 charsets in subject header when parsing message 2023-08-01 09:50:26 +02:00
metrics make account web page configurable separately from admin, add http auth rate limiting 2023-02-13 13:53:47 +01:00
mlog add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
mox- improve deprecation warning about localpart-only destinations a bit 2023-07-03 09:48:50 +02:00
moxio implement decoding charsets (other than ascii and utf-8) while reading textual message parts, and improve search 2023-07-28 22:15:23 +02:00
moxvar add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
mtasts more helpful error message when the queue tries to deliver a message but the remote host is not listed in the mta-sts policy 2023-06-01 16:23:36 +02:00
mtastsdb more logging around smtp and mtasts tls connections 2023-06-04 17:55:55 +02:00
publicsuffix mox! 2023-01-30 14:27:06 +01:00
queue in DSNs, add a References header pointing to the message with deliverability issues 2023-07-23 17:56:39 +02:00
ratelimit fix language in comments 2023-05-22 15:04:06 +02:00
rfc new items on roadmap, mention delivered-to rfc, fix wording in comments 2023-07-26 19:23:20 +02:00
sasl new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
scram add support for SCRAM-SHA-1 2023-02-05 12:30:14 +01:00
smtp add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
smtpclient new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
smtpserver in smtpserver, allow a space after "mail from:" and "rcpt to:" commands for submission connections 2023-07-28 20:49:19 +02:00
spf fix problem with spf where we would generate errors about too many void lookups 2023-03-03 11:45:19 +01:00
store when creating a mailbox subscription, don't just try to insert a record into the database and handle bstore.ErrUnique, the transaction will have been marked as botched 2023-08-01 10:14:02 +02:00
subjectpass mox! 2023-01-30 14:27:06 +01:00
testdata merge docker-compose-based quickstart and integration tests into a single integration test 2023-07-23 23:32:02 +02:00
tlsrpt work around missing timezone in timestamps in tls reports from microsoft 2023-02-05 10:55:34 +01:00
tlsrptdb add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
updates on admin page, show warning when user hasn't enabled "check updates" 2023-02-27 15:03:37 +01:00
vendor update dependencies 2023-07-28 22:47:28 +02:00
.dockerignore don't pass git history to docker container builds 2023-07-03 09:12:25 +02:00
.gitignore for test-upgrade, import a (hopefully large) mbox file, checking for performance/memory consumption 2023-07-24 11:00:11 +02:00
.jshintrc improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config 2023-03-02 18:15:54 +01:00
backup.go add condstore & qresync imap extensions 2023-07-24 21:25:50 +02:00
checkhtmljs mox! 2023-01-30 14:27:06 +01:00
compatibility.txt add notes on tests with microsoft outlook. 2023-02-05 17:54:00 +01:00
ctl.go add missing account close, for retraining junk filter for an account with the retrain cli command 2023-07-26 19:21:58 +02:00
ctl_test.go add basic tests for the ctl subcommands, and fix two small bugs 2023-07-02 14:18:50 +02:00
develop.txt for test-upgrade, import a (hopefully large) mbox file, checking for performance/memory consumption 2023-07-24 11:00:11 +02:00
doc.go more integration tests: start "mox localserve" and submit a message with smtpclient and with "mox sendmail", check that we receive it 2023-07-01 18:48:29 +02:00
docker-compose-imaptest.yml change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user 2023-02-27 12:19:55 +01:00
docker-compose-integration.yml fix building fresh docker images for integration tests 2023-07-26 21:58:12 +02:00
docker-compose.yml add automated test for quickstart 2023-06-04 20:38:10 +02:00
docker-release.sh replace use of docker registry docker.io (hub.docker.com) with self-hosted r.xmox.nl 2023-03-20 09:35:49 +01:00
Dockerfile add timezone db files to docker images 2023-03-07 16:53:55 +01:00
Dockerfile.imaptest help run mox with docker 2023-02-24 14:16:51 +01:00
Dockerfile.moximaptest help run mox with docker 2023-02-24 14:16:51 +01:00
Dockerfile.release fix cross-compiled docker images 2023-02-27 13:46:29 +01:00
export.go add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
gendoc.sh add "mox localserve" subcommand, for running mox locally for email-related testing/developing 2023-03-12 11:40:00 +01:00
gentestdata.go in DSNs, add a References header pointing to the message with deliverability issues 2023-07-23 17:56:39 +02:00
go.mod update dependencies 2023-07-28 22:47:28 +02:00
go.sum update dependencies 2023-07-28 22:47:28 +02:00
import.go fix bug with cli import command in case the mbox/maildir had keywords, future delivery to the mailbox would fail with duplicate uid's. 2023-07-26 10:09:36 +02:00
integration_test.go add condstore & qresync imap extensions 2023-07-24 21:25:50 +02:00
junk.go add subcommand "ximport", that is like "import" but directly access files in the datadir 2023-07-01 16:43:20 +02:00
LICENSE.MIT mox! 2023-01-30 14:27:06 +01:00
LICENSE.MPLv2.0 mox! 2023-01-30 14:27:06 +01:00
localserve.go more integration tests: start "mox localserve" and submit a message with smtpclient and with "mox sendmail", check that we receive it 2023-07-01 18:48:29 +02:00
main.go fix bug with cli import command in case the mbox/maildir had keywords, future delivery to the mailbox would fail with duplicate uid's. 2023-07-26 10:09:36 +02:00
Makefile fix building fresh docker images for integration tests 2023-07-26 21:58:12 +02:00
mox.service more permissions in mox.service to ease with upgrade to v0.0.2 2023-02-27 15:48:37 +01:00
profile.go add subcommand "ximport", that is like "import" but directly access files in the datadir 2023-07-01 16:43:20 +02:00
prometheus.rules tweak alerting rule to include that it is about authentication rate limiting 2023-05-01 14:21:02 +02:00
quickstart.go in quickstart, if the host name resolves to a loopback IP, warn about it as it will likely prevent email delivery to local accounts 2023-06-12 12:19:20 +02:00
README.md new items on roadmap, mention delivered-to rfc, fix wording in comments 2023-07-26 19:23:20 +02:00
sendmail.go don't recurse into error checking function xcheckf when sendmail fails 2023-07-24 14:08:27 +02:00
serve.go add Content-Type header to message delivered for new mox releases 2023-07-25 08:24:05 +02:00
test-upgrade.sh add condstore & qresync imap extensions 2023-07-24 21:25:50 +02:00
tools.go mox! 2023-01-30 14:27:06 +01:00
updates.go remove debug print 2023-04-24 10:06:59 +02:00
updating.txt clear updating.txt for release 2023-04-22 07:52:42 +02:00
verifydata.go fix bug with cli import command in case the mbox/maildir had keywords, future delivery to the mailbox would fail with duplicate uid's. 2023-07-26 10:09:36 +02:00

Mox is a modern full-featured open source secure mail server for low-maintenance self-hosted email.

See Quickstart below to get started.

Features

  • Quick and easy to start/maintain mail server, for your own domain(s).
  • SMTP (with extensions) for receiving, submitting and delivering email.
  • IMAP4 (with extensions) for giving email clients access to email.
  • Automatic TLS with ACME, for use with Let's Encrypt and other CA's.
  • SPF, verifying that a remote host is allowed to send email for a domain.
  • DKIM, verifying that a message is signed by the claimed sender domain, and for signing emails sent by mox for others to verify.
  • DMARC, for enforcing SPF/DKIM policies set by domains. Incoming DMARC aggregate reports are analyzed.
  • Reputation tracking, learning (per user) host- and domain-based reputation from (Non-)Junk email.
  • Bayesian spam filtering that learns (per user) from (Non-)Junk email.
  • Slowing down senders with no/low reputation or questionable email content (similar to greylisting). Rejected emails are stored in a mailbox called Rejects for a short period, helping with misclassified legitimate synchronous signup/login/transactional emails.
  • Internationalized email, with unicode names in domains and usernames ("localparts").
  • TLSRPT, parsing reports about TLS usage and issues.
  • MTA-STS, for ensuring TLS is used whenever it is required. Both serving of policies, and tracking and applying policies of remote servers.
  • Web admin interface that helps you set up your domains and accounts (instructions to create DNS records, configure SPF/DKIM/DMARC/TLSRPT/MTA-STS), for status information, managing accounts/domains, and modifying the configuration file.
  • Autodiscovery (with SRV records, Microsoft-style and Thunderbird-style) for easy account setup (though not many clients support it).
  • Webserver with serving static files and forwarding requests (reverse proxy), so port 443 can also be used to serve websites.
  • Prometheus metrics and structured logging for operational insight.
  • "localserve" subcommand for running mox locally for email-related testing/developing, including pedantic mode.

Mox is available under the MIT-license and was created by Mechiel Lukkien, mechiel@ueber.net. Mox includes the Public Suffix List by Mozilla, under Mozilla Public License, v2.0.

Download

You can easily (cross) compile mox if you have a recent Go toolchain installed (see "go version", it must be >= 1.19; otherwise, see https://go.dev/dl/ or https://go.dev/doc/manage-install and $HOME/go/bin):

GOBIN=$PWD CGO_ENABLED=0 go install github.com/mjl-/mox@latest

Or you can download a binary built with the latest Go toolchain from https://beta.gobuilds.org/github.com/mjl-/mox@latest/linux-amd64-latest/, and symlink or rename it to "mox".

Verify you have a working mox binary:

./mox version

Note: Mox only compiles for/works on unix systems, not on Plan 9 or Windows.

You can also run mox with docker image r.xmox.nl/mox, with tags like v0.0.1 and v0.0.1-go1.20.1-alpine3.17.2, see https://r.xmox.nl/repo/mox/. See docker-compose.yml in this repository for instructions on starting. You must run docker with host networking, because mox needs to find your actual public IP's and get the remote IPs for incoming connections, not a local/internal NAT IP.

Quickstart

The easiest way to get started with serving email for your domain is to get a vm/machine dedicated to serving email, name it [host].[domain] (e.g. mail.example.com), login as root, and run:

# Create mox user and homedir (or pick another name or homedir):
useradd -m -d /home/mox mox

cd /home/mox
... compile or download mox to this directory, see above ...

# Generate config files for your address/domain:
./mox quickstart you@example.com

The quickstart creates an account, generates a password and configuration files, prints the DNS records you need to manually create and prints commands to start mox and optionally install mox as a service.

A dedicated machine is highly recommended because modern email requires HTTPS, and mox currently needs it for automatic TLS. You could combine mox with an existing webserver, but it requires more configuration. If you want to serve websites on the same machine, consider using the webserver built into mox. If you want to run an existing webserver on port 443/80, see "mox help quickstart", it'll tell you to run "./mox quickstart -existing-webserver you@example.com".

After starting, you can access the admin web interface on internal IPs.

Future/development

Mox has automated tests, including for interoperability with Postfix for SMTP. Mox is manually tested with email clients: Mozilla Thunderbird, mutt, iOS Mail, macOS Mail, Android Mail, Microsoft Outlook. Mox is also manually tested to interoperate with popular cloud providers: gmail.com, outlook.com, yahoo.com, proton.me.

The code is heavily cross-referenced with the RFCs for readability/maintainability.

Roadmap

  • Webmail
  • IMAP THREAD extension
  • DANE and DNSSEC
  • Sending DMARC and TLS reports (currently only receiving)
  • OAUTH2 support, for single sign on
  • Add special IMAP mailbox ("Queue?") that contains queued but not-yet-delivered messages
  • Sieve for filtering (for now see Rulesets in the account config)
  • Accepting/processing/monitoring DMARC reports for external domains
  • Calendaring
  • Privilege separation, isolating parts of the application to more restricted sandbox (e.g. new unauthenticated connections)
  • Using mox as backup MX
  • Old-style internationalization in messages
  • ARC, with forwarded email from trusted source
  • JMAP
  • Autoresponder (out of office/vacation)
  • HTTP-based API for sending messages and receiving delivery feedback
  • Milter support, for integration with external tools

There are many smaller improvements to make as well, search for "todo" in the code.

Not supported

But perhaps in the future...

  • Functioning as SMTP relay
  • Forwarding (to an external address)
  • POP3
  • Delivery to (unix) OS system users
  • Mailing list manager
  • Support for pluggable delivery mechanisms
  • iOS Mail push notifications (with XAPPLEPUSHSERVICE undocumented imap extension and hard to get APNS certificate)

FAQ - Frequently Asked Questions

Why a new mail server implementation?

Mox aims to make "running a mail server" easy and nearly effortless. Excellent quality mail server software exists, but getting a working setup typically requires you configure half a dozen services (SMTP, IMAP, SPF/DKIM/DMARC, spam filtering). That seems to lead to people no longer running their own mail servers, instead switching to one of the few centralized email providers. Email with SMTP is a long-time decentralized messaging protocol. To keep it decentralized, people need to run their own mail server. Mox aims to make that easy.

Where is the documentation?

See all commands and help text at https://pkg.go.dev/github.com/mjl-/mox/, and example config files at https://pkg.go.dev/github.com/mjl-/mox/config/.

You can get the same information by running "mox" without arguments to list its subcommands and usage, and "mox help [subcommand]" for more details.

The example config files are printed by "mox config describe-static" and "mox config describe-dynamic".

Mox is still in early stages, and documentation is still limited. Please create an issue describing what is unclear or confusing, and we'll try to improve the documentation.

How do I import/export email?

Use the import functionality on the accounts web page to import a zip/tgz with maildirs/mbox files, or use the "mox import maildir" or "mox import mbox" subcommands. You could also use your IMAP email client, add your mox account, and copy or move messages from one account to the other.

Similarly, see the export functionality on the accounts web page and the "mox export maildir" and "mox export mbox" subcommands to export email.

Importing large mailboxes may require a lot of memory (a limitation of the current database). Splitting up mailboxes in smaller parts would help.

How can I help?

Mox needs users and testing in real-life setups! So just give it a try, send and receive emails through it with your favourite email clients, and file an issue if you encounter a problem or would like to see a feature/functionality implemented.

Instead of switching email for your domain over to mox, you could simply configure mox for a subdomain, e.g. [you]@moxtest.[yourdomain].

If you have experience with how the email protocols are used in the wild, e.g. compatibility issues, limitations, anti-spam measures, specification violations, that would be interesting to hear about.

Pull requests for bug fixes and new code are welcome too. If the changes are large, it helps to start a discussion (create a ticket) before doing all the work.

By contributing (e.g. code), you agree your contributions are licensed under the MIT license (like mox), and have the rights to do so.

Where can I discuss mox?

Join #mox on irc.oftc.net, or #mox on the "Gopher slack".

For bug reports, please file an issue at https://github.com/mjl-/mox/issues/new.

How do I change my password?

Regular users (doing IMAP/SMTP with authentication) can change their password at the account page, e.g. http://localhost/. Or you can set a password with "mox setaccountpassword".

The admin can change the password of any account through the admin page, at http://localhost/admin/ by default (leave username empty when logging in).

The account and admin pages are served on localhost on your mail server. To access these from your browser, run ssh -L 8080:localhost:80 you@yourmachine locally and open http://localhost:8080/[...].

The admin password can be changed with "mox setadminpassword".

How do I configure a second mox instance as a backup MX?

Unfortunately, mox does not yet provide an option for that. Mox does spam filtering based on reputation of received messages. It will take a good amount of work to share that information with a backup MX. Without that information, spammers could use a backup MX to get their spam accepted. Until mox has a proper solution, you can simply run a single SMTP server.

How do I stay up to date?

Please set "CheckUpdates: true" in mox.conf. Mox will check for a new version through a DNS TXT request for _updates.xmox.nl once per 24h. Only if a new version is published will the changelog be fetched and delivered to the postmaster mailbox.

The changelog, including latest update instructions, is at https://updates.xmox.nl/changelog.

You can also monitor newly added releases on this repository with the github "watch" feature, or use the github RSS feed for tags (https://github.com/mjl-/mox/tags.atom) or releases (https://github.com/mjl-/mox/releases.atom), or monitor the docker images.

Keep in mind you have a responsibility to keep the internect-connected software you run up to date and secure.

How do I upgrade my mox installation?

We try to make upgrades effortless and you can typically just put a new binary in place and restart. If manual actions are required, the release notes mention them. Check the release notes of all version between your current installation and the release you're upgrading to.

Before upgrading, make a backup of the data directory with mox backup <destdir>. This writes consistent snapshots of the database files, and duplicates message files from the queue and accounts. Using the new mox binary, run mox verifydata <backupdir> (do NOT use the "live" data directory!) for a dry run. If this fails, an upgrade will probably fail too. Important: verifydata with the new mox binary can modify the database files (due to automatic schema upgrades). So make a fresh backup again before the actual upgrade. See the help output of the "backup" and "verifydata" commands for more details.

During backup, message files are hardlinked if possible. Using a destination directory like data/tmp/backup increases the odds hardlinking succeeds: the default systemd service file specifically mounts the data directory, causing attempts to outside it to fail with an error about cross-device linking.

If an upgrade fails and you have to restore (parts) of the data directory, you should run mox verifydata <datadir> (with the original binary) on the restored directory before starting mox again. If problematic files are found, for example queue or account message files that are not in the database, run mox verifydata -fix <datadir> to move away those files. After a restore, you may also want to run mox bumpuidvalidity <account> for each account for which messages in a mailbox changed, to force IMAP clients to synchronize mailbox state.

How secure is mox?

Security is high on the priority list for mox. Mox is young, so don't expect no bugs at all. Mox does have automated tests for some security aspects, e.g. for login, and uses fuzzing. Mox is written in Go, so some classes of bugs such as buffer mishandling do not typically result in privilege escalation. Of course logic bugs will still exist. If you find any security issues, please email them to mechiel@ueber.net.

I'm now running an email server, but how does email work?

Congrats and welcome to the club! Running an email server on the internet comes with some responsibilities so you should understand how it works. See https://explained-from-first-principles.com/email/ for a thorough explanation.

What are the minimum requirements to run mox?

Mox does not need much. Nowadays most machines are larger than mox needs. You can start with a machine with 512MB RAM, any CPU will do. For storage you should account for the size of the email messages (no compression currently), an additional 15% overhead for the meta data, and add some more headroom. Expand as necessary.

Can I see some screenshots?

Yes, see https://www.xmox.nl/screenshots/.

Mox has an "account" web interface where users can view their account and manage their address configuration, such as rules for automatically delivering certain incoming messages to a specific mailbox.

Mox also has an "admin" web interface where the mox instance administrator can make changes, e.g. add/remove/modify domains/accounts/addresses.

Mox does not have a webmail yet, so there are no screenshots of actual email.

Won't the big email providers block my email?

It is a common misconception that it is impossible to run your own email server nowadays. The claim is that the handful big email providers will simply block your email. However, you can run your own email server just fine, and your email will be accepted, provided you are doing it right.

If your email is rejected, it is often because your IP address has a bad email sending reputation. Email servers often use IP blocklists to reject email networks with a bad email sending reputation. These blocklists often work at the level of whole network ranges. So if you try to run an email server from a hosting provider with a bad reputation (which happens if they don't monitor their network or don't act on abuse/spam reports), your IP too will have a bad reputation and other mail servers (both large and small) may reject messages coming from you. During the quickstart, mox checks if your IPs are on a few often-used blocklists. It's typically not a good idea to host an email server on the cheapest or largest cloud providers: They often don't spend the resources necessary for a good reputation, or they simply block all outgoing SMTP traffic. It's better to look for a technically-focused local provider. They too may initially block outgoing SMTP connections on new machines to prevent spam from their networks. But they will either automatically open up outgoing SMTP traffic after a cool down period (e.g. 24 hours), or after you've contacted their support.

After you get past the IP blocklist checks, email servers use many more signals to determine if your email message could be spam and should be rejected. Mox helps you set up a system that doesn't trigger most of the technical signals (e.g. with SPF/DKIM/DMARC). But there are more signals, for example: Sending to a mail server or address for the first time. Sending from a newly registered domain. Sending messages with content that resembles known spam messages.

Should your email be rejected, you will typically get an error message that explains why. In the case of big email providers the error message often has instructions on how to prove to them you are a legimate sender.

Can I use existing TLS certificates/keys?

Yes. The quickstart command creates a config that uses ACME with Let's Encrypt, but you can change the config file to use existing certificate and key files.

You'll see "ACME: letsencrypt" in the "TLS" section of the "public" Listener. Remove or comment out the ACME-line, and add a "KeyCerts" section like in the example config file in https://pkg.go.dev/github.com/mjl-/mox/config#hdr-mox_conf. You can have multiple certificates and keys: The line with the "-" (dash) is the start of a list item. Duplicate that line up to and including the line with KeyFile for each certificate/key you have. Mox makes a TLS config that holds all specified certificates/keys, and uses it for all services for that Listener (including a webserver), choosing the correct certificate for incoming requests.

Keep in mind that for each email domain you host, you will need a certificate for mta-sts.<domain> and autoconfig.<domain>, unless you disable MTA-STS and autoconfig for that domain.

Mox opens the key and certificate files during initial startup, as root (and passes file descriptors to the unprivileged process). No special permissions are needed on the key and certificate files.