modern full-featured open source secure mail server for low-maintenance self-hosted email
Find a file
Mechiel Lukkien 40163bd145
implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap
the mailbox select/examine responses now return all flags used in a mailbox in
the FLAGS response. and indicate in the PERMANENTFLAGS response that clients
can set new keywords. we store these values on the new Message.Keywords field.
system/well-known flags are still in Message.Flags, so we're recognizing those
and handling them separately.

the imap store command handles the new flags. as does the append command, and
the search command.

we store keywords in a mailbox when a message in that mailbox gets the keyword.
we don't automatically remove the keywords from a mailbox. there is currently
no way at all to remove a keyword from a mailbox.

the import commands now handle non-system/well-known keywords too, when
importing from mbox/maildir.

jmap requires keyword support, so best to get it out of the way now.
2023-06-24 00:24:43 +02:00
.github/workflows add github workflow to build & test, exporting a coverage file 2023-05-22 18:01:21 +02:00
.go mox! 2023-01-30 14:27:06 +01:00
autotls fix typo's and old reference 2023-06-22 21:27:52 +02:00
config new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
dkim add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
dmarc fix typo's 2023-02-06 11:00:11 +01:00
dmarcdb add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
dmarcrpt mox! 2023-01-30 14:27:06 +01:00
dns for dns resolve errors likely due to a missing name server in /etc/resolv.conf, point user to man page of systemd-resolved, the likely cause 2023-06-12 14:53:07 +02:00
dnsbl mox! 2023-01-30 14:27:06 +01:00
dsn unbreak the subcommands that talk to the mox instance of the ctl socket 2023-06-16 13:27:27 +02:00
http implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
imapclient implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
imapserver implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
iprev mox! 2023-01-30 14:27:06 +01:00
junk add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
message tweak time format for added date headers 2023-06-04 21:04:10 +02:00
metrics make account web page configurable separately from admin, add http auth rate limiting 2023-02-13 13:53:47 +01:00
mlog add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
mox- new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
moxio change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user 2023-02-27 12:19:55 +01:00
moxvar add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
mtasts more helpful error message when the queue tries to deliver a message but the remote host is not listed in the mta-sts policy 2023-06-01 16:23:36 +02:00
mtastsdb more logging around smtp and mtasts tls connections 2023-06-04 17:55:55 +02:00
publicsuffix mox! 2023-01-30 14:27:06 +01:00
queue new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
ratelimit fix language in comments 2023-05-22 15:04:06 +02:00
rfc add reverse proxying websocket connections 2023-05-30 22:11:31 +02:00
sasl new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
scram add support for SCRAM-SHA-1 2023-02-05 12:30:14 +01:00
smtp add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
smtpclient new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
smtpserver new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
spf fix problem with spf where we would generate errors about too many void lookups 2023-03-03 11:45:19 +01:00
store implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
subjectpass mox! 2023-01-30 14:27:06 +01:00
testdata implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
tlsrpt work around missing timezone in timestamps in tls reports from microsoft 2023-02-05 10:55:34 +01:00
tlsrptdb add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
updates on admin page, show warning when user hasn't enabled "check updates" 2023-02-27 15:03:37 +01:00
vendor implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
.dockerignore mox! 2023-01-30 14:27:06 +01:00
.gitignore add automated test for quickstart 2023-06-04 20:38:10 +02:00
.jshintrc improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config 2023-03-02 18:15:54 +01:00
backup.go tweaks to backup & verifydata tool to make a typical backup+verifydata produce no output 2023-06-01 11:34:28 +02:00
checkhtmljs mox! 2023-01-30 14:27:06 +01:00
compatibility.txt add notes on tests with microsoft outlook. 2023-02-05 17:54:00 +01:00
ctl.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
develop.txt unbreak "mox localserve" 2023-06-12 14:59:40 +02:00
doc.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
docker-compose-imaptest.yml change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user 2023-02-27 12:19:55 +01:00
docker-compose-integration.yml change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user 2023-02-27 12:19:55 +01:00
docker-compose-quickstart.yml add automated test for quickstart 2023-06-04 20:38:10 +02:00
docker-compose.yml add automated test for quickstart 2023-06-04 20:38:10 +02:00
docker-release.sh replace use of docker registry docker.io (hub.docker.com) with self-hosted r.xmox.nl 2023-03-20 09:35:49 +01:00
Dockerfile add timezone db files to docker images 2023-03-07 16:53:55 +01:00
Dockerfile.imaptest help run mox with docker 2023-02-24 14:16:51 +01:00
Dockerfile.moximaptest help run mox with docker 2023-02-24 14:16:51 +01:00
Dockerfile.release fix cross-compiled docker images 2023-02-27 13:46:29 +01:00
export.go add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
gendoc.sh add "mox localserve" subcommand, for running mox locally for email-related testing/developing 2023-03-12 11:40:00 +01:00
gentestdata.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
go.mod implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
go.sum implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
import.go implement storing non-system/well-known flags (keywords) for messages and mailboxes, with imap 2023-06-24 00:24:43 +02:00
integration_test.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
junk.go update to latest bstore (with support for an index on a []string: Message.DKIMDomains), and cyclic data types (to be used for Message.Part soon); also adds a context.Context to database operations. 2023-05-22 14:40:36 +02:00
LICENSE.MIT mox! 2023-01-30 14:27:06 +01:00
LICENSE.MPLv2.0 mox! 2023-01-30 14:27:06 +01:00
localserve.go unbreak the subcommands that talk to the mox instance of the ctl socket 2023-06-16 13:27:27 +02:00
main.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
Makefile restore checking integration_test.go with go vet 2023-06-16 12:55:57 +02:00
mox.service more permissions in mox.service to ease with upgrade to v0.0.2 2023-02-27 15:48:37 +01:00
prometheus.rules tweak alerting rule to include that it is about authentication rate limiting 2023-05-01 14:21:02 +02:00
quickstart.go in quickstart, if the host name resolves to a loopback IP, warn about it as it will likely prevent email delivery to local accounts 2023-06-12 12:19:20 +02:00
quickstart_test.go fix typo's and old reference 2023-06-22 21:27:52 +02:00
README.md fix typo's and old reference 2023-06-22 21:27:52 +02:00
sendmail.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
serve.go new feature: when delivering messages from the queue, make it possible to use a "transport" 2023-06-16 18:57:05 +02:00
test-upgrade.sh add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades 2023-05-26 19:26:51 +02:00
tools.go mox! 2023-01-30 14:27:06 +01:00
updates.go remove debug print 2023-04-24 10:06:59 +02:00
updating.txt clear updating.txt for release 2023-04-22 07:52:42 +02:00
verifydata.go tweaks to backup & verifydata tool to make a typical backup+verifydata produce no output 2023-06-01 11:34:28 +02:00

Mox is a modern full-featured open source secure mail server for low-maintenance self-hosted email.

See Quickstart below to get started.

Features

  • Quick and easy to start/maintain mail server, for your own domain(s).
  • SMTP (with extensions) for receiving, submitting and delivering email.
  • IMAP4 (with extensions) for giving email clients access to email.
  • Automatic TLS with ACME, for use with Let's Encrypt and other CA's.
  • SPF, verifying that a remote host is allowed to sent email for a domain.
  • DKIM, verifying that a message is signed by the claimed sender domain, and for signing emails sent by mox for others to verify.
  • DMARC, for enforcing SPF/DKIM policies set by domains. Incoming DMARC aggregate reports are analyzed.
  • Reputation tracking, learning (per user) host- and domain-based reputation from (Non-)Junk email.
  • Bayesian spam filtering that learns (per user) from (Non-)Junk email.
  • Slowing down senders with no/low reputation or questionable email content (similar to greylisting). Rejected emails are stored in a mailbox called Rejects for a short period, helping with misclassified legitimate synchronous signup/login/transactional emails.
  • Internationalized email, with unicode names in domains and usernames ("localparts").
  • TLSRPT, parsing reports about TLS usage and issues.
  • MTA-STS, for ensuring TLS is used whenever it is required. Both serving of policies, and tracking and applying policies of remote servers.
  • Web admin interface that helps you set up your domains and accounts (instructions to create DNS records, configure SPF/DKIM/DMARC/TLSRPT/MTA-STS), for status information, managing accounts/domains, and modifying the configuration file.
  • Autodiscovery (with SRV records, Microsoft-style and Thunderbird-style) for easy account setup (though not many clients support it).
  • Webserver with serving static files and forwarding requests (reverse proxy), so port 443 can also be used to serve websites.
  • Prometheus metrics and structured logging for operational insight.
  • "localserve" subcommand for running mox locally for email-related testing/developing, including pedantic mode.

Mox is available under the MIT-license and was created by Mechiel Lukkien, mechiel@ueber.net. Mox includes the Public Suffix List by Mozilla, under Mozilla Public License, v2.0.

Download

You can easily (cross) compile mox if you have a recent Go toolchain installed (see "go version", it must be >= 1.19; otherwise, see https://go.dev/dl/ or https://go.dev/doc/manage-install and $HOME/go/bin):

GOBIN=$PWD CGO_ENABLED=0 go install github.com/mjl-/mox@latest

Or you can download a binary built with the latest Go toolchain from https://beta.gobuilds.org/github.com/mjl-/mox, and symlink or rename it to "mox".

Verify you have a working mox binary:

./mox version

Note: Mox only compiles for/works on unix systems, not on Plan 9 or Windows.

You can also run mox with docker image r.xmox.nl/mox, with tags like v0.0.1 and v0.0.1-go1.20.1-alpine3.17.2, see https://r.xmox.nl/repo/mox/. See docker-compose.yml in this repository for instructions on starting. You must run docker with host networking, because mox needs to find your actual public IP's and get the remote IPs for incoming connections, not a local/internal NAT IP.

Quickstart

The easiest way to get started with serving email for your domain is to get a vm/machine dedicated to serving email, name it [host].[domain] (e.g. mail.example.com), login as root, and run:

# Create mox user and homedir (or pick another name or homedir):
useradd -m -d /home/mox mox

cd /home/mox
... compile or download mox to this directory, see above ...

# Generate config files for your address/domain:
./mox quickstart you@example.com

The quickstart creates an account, generates a password and configuration files, prints the DNS records you need to manually create and prints commands to start mox and optionally install mox as a service.

A dedicated machine is highly recommended because modern email requires HTTPS, and mox currently needs it for automatic TLS. You could combine mox with an existing webserver, but it requires more configuration. If you want to serve websites on the same machine, consider using the webserver built into mox. If you want to run an existing webserver on port 443/80, see "mox help quickstart", it'll tell you to run "./mox quickstart -existing-webserver you@example.com".

After starting, you can access the admin web interface on internal IPs.

Future/development

Mox has automated tests, including for interoperability with Postfix for SMTP. Mox is manually tested with email clients: Mozilla Thunderbird, mutt, iOS Mail, macOS Mail, Android Mail, Microsoft Outlook. Mox is also manually tested to interoperate with popular cloud providers: gmail.com, outlook.com, yahoo.com, proton.me.

The code is heavily cross-referenced with the RFCs for readability/maintainability.

Roadmap

  • Privilege separation, isolating parts of the application to more restricted sandbox (e.g. new unauthenticated connections).
  • DANE and DNSSEC.
  • Sending DMARC and TLS reports (currently only receiving).
  • OAUTH2 support, for single sign on.
  • Add special IMAP mailbox ("Queue?") that contains queued but not-yet-delivered messages.
  • Sieve for filtering (for now see Rulesets in the account config)
  • Calendaring
  • IMAP CONDSTORE and QRESYNC extensions
  • IMAP THREAD extension
  • Using mox as backup MX.
  • Old-style internationalization in messages.
  • JMAP
  • Webmail

There are many smaller improvements to make as well, search for "todo" in the code.

Not supported

But perhaps in the future...

  • HTTP-based API for sending messages and receiving delivery feedback
  • Functioning as SMTP relay
  • Forwarding (to an external address)
  • Autoresponders
  • POP3
  • Delivery to (unix) OS system users
  • Mailing list manager
  • Support for pluggable delivery mechanisms
  • iOS Mail push notifications (with XAPPLEPUSHSERVICE undocumented imap extension and hard to get APNS certificate)

FAQ - Frequently Asked Questions

Why a new mail server implementation?

Mox aims to make "running a mail server" easy and nearly effortless. Excellent quality mail server software exists, but getting a working setup typically requires you configure half a dozen services (SMTP, IMAP, SPF/DKIM/DMARC, spam filtering). That seems to lead to people no longer running their own mail servers, instead switching to one of the few centralized email providers. Email with SMTP is a long-time decentralized messaging protocol. To keep it decentralized, people need to run their own mail server. Mox aims to make that easy.

Where is the documentation?

See all commands and help text at https://pkg.go.dev/github.com/mjl-/mox/, and example config files at https://pkg.go.dev/github.com/mjl-/mox/config/.

You can get the same information by running "mox" without arguments to list its subcommands and usage, and "mox help [subcommand]" for more details.

The example config files are printed by "mox config describe-static" and "mox config describe-dynamic".

Mox is still in early stages, and documentation is still limited. Please create an issue describing what is unclear or confusing, and we'll try to improve the documentation.

How do I import/export email?

Use the import functionality on the accounts web page to import a zip/tgz with maildirs/mbox files, or use the "mox import maildir" or "mox import mbox" subcommands. You could also use your IMAP email client, add your mox account, and copy or move messages from one account to the other.

Similarly, see the export functionality on the accounts web page and the "mox export maildir" and "mox export mbox" subcommands to export email.

How can I help?

Mox needs users and testing in real-life setups! So just give it a try, send and receive emails through it with your favourite email clients, and file an issue if you encounter a problem or would like to see a feature/functionality implemented.

Instead of switching email for your domain over to mox, you could simply configure mox for a subdomain, e.g. [you]@moxtest.[yourdomain].

If you have experience with how the email protocols are used in the wild, e.g. compatibility issues, limitations, anti-spam measures, specification violations, that would be interesting to hear about.

Pull requests for bug fixes and new code are welcome too. If the changes are large, it helps to start a discussion (create a ticket) before doing all the work.

By contributing (e.g. code), you agree your contributions are licensed under the MIT license (like mox), and have the rights to do so.

Where can I discuss mox?

Join #mox on irc.oftc.net, or #mox on the "Gopher slack".

For bug reports, please file an issue at https://github.com/mjl-/mox/issues/new.

How do I change my password?

Regular users (doing IMAP/SMTP with authentication) can change their password at the account page, e.g. http://localhost/. Or you can set a password with "mox setaccountpassword".

The admin can change the password of any account through the admin page, at http://localhost/admin/ by default (leave username empty when logging in).

The account and admin pages are served on localhost on your mail server. To access these from your browser, run ssh -L 8080:localhost:80 you@yourmachine locally and open http://localhost:8080/[...].

The admin password can be changed with "mox setadminpassword".

How do I configure a second mox instance as a backup MX?

Unfortunately, mox does not yet provide an option for that. Mox does spam filtering based on reputation of received messages. It will take a good amount of work to share that information with a backup MX. Without that information, spammers could use a backup MX to get their spam accepted. Until mox has a proper solution, you can simply run a single SMTP server.

How do I stay up to date?

Please set "CheckUpdates: true" in mox.conf. Mox will check for a new version through a DNS TXT request for _updates.xmox.nl once per 24h. Only if a new version is published will the changelog be fetched and delivered to the postmaster mailbox.

The changelog, including latest update instructions, is at https://updates.xmox.nl/changelog.

You can also monitor newly added releases on this repository with the github "watch" feature, or use the github RSS feed for tags (https://github.com/mjl-/mox/tags.atom) or releases (https://github.com/mjl-/mox/releases.atom), or monitor the docker images.

Keep in mind you have a responsibility to keep the internect-connected software you run up to date and secure.

How do I upgrade my mox installation?

We try to make upgrades effortless and you can typically just put a new binary in place and restart. If manual actions are required, the release notes mention them. Check the release notes of all version between your current installation and the release you're upgrading to.

Before upgrading, make a backup of the data directory with mox backup <destdir>. This writes consistent snapshots of the database files, and duplicates message files from the queue and accounts. Using the new mox binary, run mox verifydata <backupdir> (do NOT use the "live" data directory!) for a dry run. If this fails, an upgrade will probably fail too. Important: verifydata with the new mox binary can modify the database files (due to automatic schema upgrades). So make a fresh backup again before the actual upgrade. See the help output of the "backup" and "verifydata" commands for more details.

During backup, message files are hardlinked if possible. Using a destination directory like data/tmp/backup increases the odds hardlinking succeeds: the default systemd service file specifically mounts the data directory, causing attempts to outside it to fail with an error about cross-device linking.

If an upgrade fails and you have to restore (parts) of the data directory, you should run mox verifydata <datadir> (with the original binary) on the restored directory before starting mox again. If problematic files are found, for example queue or account message files that are not in the database, run mox verifydata -fix <datadir> to move away those files. After a restore, you may also want to run mox bumpuidvalidity <account> for each account for which messages in a mailbox changed, to force IMAP clients to synchronize mailbox state.

How secure is mox?

Security is high on the priority list for mox. Mox is young, so don't expect no bugs at all. Mox does have automated tests for some security aspects, e.g. for login, and uses fuzzing. Mox is written in Go, so some classes of bugs such as buffer mishandling do not typically result in privilege escalation. Of course logic bugs will still exist. If you find any security issues, please email them to mechiel@ueber.net.

I'm now running an email server, but how does email work?

Congrats and welcome to the club! Running an email server on the internet comes with some responsibilities so you should understand how it works. See https://explained-from-first-principles.com/email/ for a thorough explanation.

What are the minimum requirements to run mox?

Mox does not need much. Nowadays most machines are larger than mox needs. You can start with a machine with 512MB RAM, any CPU will do. For storage you should account for the size of the email messages (no compression currently), an additional 15% overhead for the meta data, and add some more headroom. Expand as necessary.

Can I see some screenshots?

Yes, see https://www.xmox.nl/screenshots/.

Mox has an "account" web interface where users can view their account and manage their address configuration, such as rules for automatically delivering certain incoming messages to a specific mailbox.

Mox also has an "admin" web interface where the mox instance administrator can make changes, e.g. add/remove/modify domains/accounts/addresses.

Mox does not have a webmail yet, so there are no screenshots of actual email.

Won't the big email providers block my email?

It is a common misconception that it is impossible to run your own email server nowadays. The claim is that the handful big email providers will simply block your email. However, you can run your own email server just fine, and your email will be accepted, provided you are doing it right.

If your email is rejected, it is often because your IP address has a bad email sending reputation. Email servers often use IP blocklists to reject email networks with a bad email sending reputation. These blocklists often work at the level of whole network ranges. So if you try to run an email server from a hosting provider with a bad reputation (which happens if they don't monitor their network or don't act on abuse/spam reports), your IP too will have a bad reputation and other mail servers (both large and small) may reject messages coming from you. During the quickstart, mox checks if your IPs are on a few often-used blocklists. It's typically not a good idea to host an email server on the cheapest or largest cloud providers: They often don't spend the resources necessary for a good reputation, or they simply block all outgoing SMTP traffic. It's better to look for a technically-focused local provider. They too may initially block outgoing SMTP connections on new machines to prevent spam from their networks. But they will either automatically open up outgoing SMTP traffic after a cool down period (e.g. 24 hours), or after you've contacted their support.

After you get past the IP blocklist checks, email servers use many more signals to determine if your email message could be spam and should be rejected. Mox helps you set up a system that doesn't trigger most of the technical signals (e.g. with SPF/DKIM/DMARC). But there are more signals, for example: Sending to a mail server or address for the first time. Sending from a newly registered domain. Sending messages with content that resembles known spam messages.

Should your email be rejected, you will typically get an error message that explains why. In the case of big email providers the error message often has instructions on how to prove to them you are a legimate sender.