2023-01-30 16:43:50 +03:00
|
|
|
Mox is a modern full-featured open source secure mail server for low-maintenance self-hosted email.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
See Quickstart below to get started.
|
|
|
|
|
2023-02-25 15:38:07 +03:00
|
|
|
## Features
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-10 21:43:02 +03:00
|
|
|
- Quick and easy to start/maintain mail server, for your own domain(s).
|
new feature: when delivering messages from the queue, make it possible to use a "transport"
the default transport is still just "direct delivery", where we connect to the
destination domain's MX servers.
other transports are:
- regular smtp without authentication, this is relaying to a smarthost.
- submission with authentication, e.g. to a third party email sending service.
- direct delivery, but with with connections going through a socks proxy. this
can be helpful if your ip is blocked, you need to get email out, and you have
another IP that isn't blocked.
keep in mind that for all of the above, appropriate SPF/DKIM settings have to
be configured. the "dnscheck" for a domain does a check for any SOCKS IP in the
SPF record. SPF for smtp/submission (ranges? includes?) and any DKIM
requirements cannot really be checked.
which transport is used can be configured through routes. routes can be set on
an account, a domain, or globally. the routes are evaluated in that order, with
the first match selecting the transport. these routes are evaluated for each
delivery attempt. common selection criteria are recipient domain and sender
domain, but also which delivery attempt this is. you could configured mox to
attempt sending through a 3rd party from the 4th attempt onwards.
routes and transports are optional. if no route matches, or an empty/zero
transport is selected, normal direct delivery is done.
we could already "submit" emails with 3rd party accounts with "sendmail". but
we now support more SASL authentication mechanisms with SMTP (not only PLAIN,
but also SCRAM-SHA-256, SCRAM-SHA-1 and CRAM-MD5), which sendmail now also
supports. sendmail will use the most secure mechanism supported by the server,
or the explicitly configured mechanism.
for issue #36 by dmikushin. also based on earlier discussion on hackernews.
2023-06-16 19:38:28 +03:00
|
|
|
- SMTP (with extensions) for receiving, submitting and delivering email.
|
2023-02-16 12:09:33 +03:00
|
|
|
- IMAP4 (with extensions) for giving email clients access to email.
|
2023-09-24 13:34:46 +03:00
|
|
|
- Webmail for reading/sending email from the browser.
|
2023-11-01 19:55:40 +03:00
|
|
|
- SPF/DKIM/DMARC for authenticating messages/delivery, also DMARC aggregate
|
|
|
|
reports.
|
2023-09-24 13:34:46 +03:00
|
|
|
- Reputation tracking, learning (per user) host-, domain- and
|
|
|
|
sender address-based reputation from (Non-)Junk email classification.
|
2023-01-30 16:27:06 +03:00
|
|
|
- Bayesian spam filtering that learns (per user) from (Non-)Junk email.
|
2023-02-08 23:45:32 +03:00
|
|
|
- Slowing down senders with no/low reputation or questionable email content
|
|
|
|
(similar to greylisting). Rejected emails are stored in a mailbox called Rejects
|
2023-02-10 21:43:02 +03:00
|
|
|
for a short period, helping with misclassified legitimate synchronous
|
2023-01-30 16:27:06 +03:00
|
|
|
signup/login/transactional emails.
|
2023-09-24 13:34:46 +03:00
|
|
|
- Internationalized email, with unicode in email address usernames
|
|
|
|
("localparts"), and in domain names (IDNA).
|
|
|
|
- Automatic TLS with ACME, for use with Let's Encrypt and other CA's.
|
implement dnssec-awareness throughout code, and dane for incoming/outgoing mail delivery
the vendored dns resolver code is a copy of the go stdlib dns resolver, with
awareness of the "authentic data" (i.e. dnssec secure) added, as well as support
for enhanced dns errors, and looking up tlsa records (for dane). ideally it
would be upstreamed, but the chances seem slim.
dnssec-awareness is added to all packages, e.g. spf, dkim, dmarc, iprev. their
dnssec status is added to the Received message headers for incoming email.
but the main reason to add dnssec was for implementing dane. with dane, the
verification of tls certificates can be done through certificates/public keys
published in dns (in the tlsa records). this only makes sense (is trustworthy)
if those dns records can be verified to be authentic.
mox now applies dane to delivering messages over smtp. mox already implemented
mta-sts for webpki/pkix-verification of certificates against the (large) pool
of CA's, and still enforces those policies when present. but it now also checks
for dane records, and will verify those if present. if dane and mta-sts are
both absent, the regular opportunistic tls with starttls is still done. and the
fallback to plaintext is also still done.
mox also makes it easy to setup dane for incoming deliveries, so other servers
can deliver with dane tls certificate verification. the quickstart now
generates private keys that are used when requesting certificates with acme.
the private keys are pre-generated because they must be static and known during
setup, because their public keys must be published in tlsa records in dns.
autocert would generate private keys on its own, so had to be forked to add the
option to provide the private key when requesting a new certificate. hopefully
upstream will accept the change and we can drop the fork.
with this change, using the quickstart to setup a new mox instance, the checks
at internet.nl result in a 100% score, provided the domain is dnssec-signed and
the network doesn't have any issues.
2023-10-10 13:09:35 +03:00
|
|
|
- DANE and MTA-STS for inbound and outbound delivery over SMTP with STARTTLS,
|
implement outgoing tls reports
we were already accepting, processing and displaying incoming tls reports. now
we start tracking TLS connection and security-policy-related errors for
outgoing message deliveries as well. we send reports once a day, to the
reporting addresses specified in TLSRPT records (rua) of a policy domain. these
reports are about MTA-STS policies and/or DANE policies, and about
STARTTLS-related failures.
sending reports is enabled by default, but can be disabled through setting
NoOutgoingTLSReports in mox.conf.
only at the end of the implementation process came the realization that the
TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy
for the recipient domain, and that MTA-STS and DANE TLS/policy results are
typically delivered in separate reports. so MX hosts need their own TLSRPT
policies.
config for the per-host TLSRPT policy should be added to mox.conf for existing
installs, in field HostTLSRPT. it is automatically configured by quickstart for
new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin
pages now suggest the per-host TLSRPT record. by creating that record, you're
requesting TLS reports about your MX host.
gathering all the TLS/policy results is somewhat tricky. the tentacles go
throughout the code. the positive result is that the TLS/policy-related code
had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect
reality better, with independent settings about whether PKIX and/or DANE
verification has to be done, and/or whether verification errors have to be
ignored (e.g. for tls-required: no header). also, cached mtasts policies of
mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
|
|
|
including REQUIRETLS and with incoming/outgoing TLSRPT reporting.
|
2023-01-30 16:27:06 +03:00
|
|
|
- Web admin interface that helps you set up your domains and accounts
|
|
|
|
(instructions to create DNS records, configure
|
|
|
|
SPF/DKIM/DMARC/TLSRPT/MTA-STS), for status information, managing
|
|
|
|
accounts/domains, and modifying the configuration file.
|
implement dnssec-awareness throughout code, and dane for incoming/outgoing mail delivery
the vendored dns resolver code is a copy of the go stdlib dns resolver, with
awareness of the "authentic data" (i.e. dnssec secure) added, as well as support
for enhanced dns errors, and looking up tlsa records (for dane). ideally it
would be upstreamed, but the chances seem slim.
dnssec-awareness is added to all packages, e.g. spf, dkim, dmarc, iprev. their
dnssec status is added to the Received message headers for incoming email.
but the main reason to add dnssec was for implementing dane. with dane, the
verification of tls certificates can be done through certificates/public keys
published in dns (in the tlsa records). this only makes sense (is trustworthy)
if those dns records can be verified to be authentic.
mox now applies dane to delivering messages over smtp. mox already implemented
mta-sts for webpki/pkix-verification of certificates against the (large) pool
of CA's, and still enforces those policies when present. but it now also checks
for dane records, and will verify those if present. if dane and mta-sts are
both absent, the regular opportunistic tls with starttls is still done. and the
fallback to plaintext is also still done.
mox also makes it easy to setup dane for incoming deliveries, so other servers
can deliver with dane tls certificate verification. the quickstart now
generates private keys that are used when requesting certificates with acme.
the private keys are pre-generated because they must be static and known during
setup, because their public keys must be published in tlsa records in dns.
autocert would generate private keys on its own, so had to be forked to add the
option to provide the private key when requesting a new certificate. hopefully
upstream will accept the change and we can drop the fork.
with this change, using the quickstart to setup a new mox instance, the checks
at internet.nl result in a 100% score, provided the domain is dnssec-signed and
the network doesn't have any issues.
2023-10-10 13:09:35 +03:00
|
|
|
- Account autodiscovery (with SRV records, Microsoft-style, Thunderbird-style,
|
|
|
|
and Apple device management profiles) for easy account setup (though client
|
|
|
|
support is limited).
|
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config
- make builtin http handlers serve on specific domains, such as for mta-sts, so
e.g. /.well-known/mta-sts.txt isn't served on all domains.
- add logging of a few more fields in access logging.
- small tweaks/bug fixes in webserver request handling.
- add config option for redirecting entire domains to another (common enough).
- split httpserver metric into two: one for duration until writing header (i.e.
performance of server), another for duration until full response is sent to
client (i.e. performance as perceived by users).
- add admin ui, a new page for managing the configs. after making changes
and hitting "save", the changes take effect immediately. the page itself
doesn't look very well-designed (many input fields, makes it look messy). i
have an idea to improve it (explained in admin.html as todo) by making the
layout look just like the config file. not urgent though.
i've already changed my websites/webapps over.
the idea of adding a webserver is to take away a (the) reason for folks to want
to complicate their mox setup by running an other webserver on the same machine.
i think the current webserver implementation can already serve most common use
cases. with a few more tweaks (feedback needed!) we should be able to get to 95%
of the use cases. the reverse proxy can take care of the remaining 5%.
nevertheless, a next step is still to change the quickstart to make it easier
for folks to run with an existing webserver, with existing tls certs/keys.
that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
|
|
|
- Webserver with serving static files and forwarding requests (reverse
|
|
|
|
proxy), so port 443 can also be used to serve websites.
|
2023-01-30 16:27:06 +03:00
|
|
|
- Prometheus metrics and structured logging for operational insight.
|
2023-09-24 13:34:46 +03:00
|
|
|
- "mox localserve" subcommand for running mox locally for email-related
|
2023-03-12 17:16:01 +03:00
|
|
|
testing/developing, including pedantic mode.
|
2023-12-11 17:01:56 +03:00
|
|
|
- Most non-server Go packages mox consists of are written to be reusable.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-25 15:38:07 +03:00
|
|
|
Mox is available under the MIT-license and was created by Mechiel Lukkien,
|
2023-09-23 13:05:40 +03:00
|
|
|
mechiel@ueber.net. Mox includes BSD-3-claused code from the Go Authors, and the
|
|
|
|
Public Suffix List by Mozilla under Mozilla Public License, v2.0.
|
2023-02-11 15:35:07 +03:00
|
|
|
|
2023-09-24 13:42:19 +03:00
|
|
|
Mox has automated tests, including for interoperability with Postfix for SMTP.
|
|
|
|
Mox is manually tested with email clients: Mozilla Thunderbird, mutt, iOS Mail,
|
|
|
|
macOS Mail, Android Mail, Microsoft Outlook. Mox is also manually tested to
|
|
|
|
interoperate with popular cloud providers: gmail.com, outlook.com, yahoo.com,
|
|
|
|
proton.me.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-09-24 13:42:19 +03:00
|
|
|
The code is heavily cross-referenced with the RFCs for readability/maintainability.
|
2023-02-24 16:16:51 +03:00
|
|
|
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
# Quickstart
|
|
|
|
|
|
|
|
The easiest way to get started with serving email for your domain is to get a
|
2023-09-24 13:34:46 +03:00
|
|
|
(virtual) machine dedicated to serving email, name it [host].[domain] (e.g.
|
2023-03-04 02:49:02 +03:00
|
|
|
mail.example.com), login as root, and run:
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-03-04 02:49:02 +03:00
|
|
|
# Create mox user and homedir (or pick another name or homedir):
|
|
|
|
useradd -m -d /home/mox mox
|
|
|
|
|
|
|
|
cd /home/mox
|
2023-09-24 13:42:19 +03:00
|
|
|
... compile or download mox to this directory, see below ...
|
2023-03-04 02:49:02 +03:00
|
|
|
|
|
|
|
# Generate config files for your address/domain:
|
2023-01-30 16:27:06 +03:00
|
|
|
./mox quickstart you@example.com
|
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
The quickstart creates configuration files for the domain and account,
|
|
|
|
generates an admin and account password, prints the DNS records you need to add
|
|
|
|
and prints commands to start mox and optionally install mox as a service.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
A machine that doesn't already run a webserver is highly recommended because
|
|
|
|
modern email requires HTTPS, and mox currently needs it for automatic TLS. You
|
|
|
|
could combine mox with an existing webserver, but it requires a lot more
|
|
|
|
configuration. If you want to serve websites on the same machine, consider using
|
|
|
|
the webserver built into mox. It's pretty good! If you want to run an existing
|
2023-09-24 13:42:19 +03:00
|
|
|
webserver on port 443/80, see "mox help quickstart".
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
After starting, you can access the admin web interface on internal IPs.
|
|
|
|
|
2023-09-24 13:42:19 +03:00
|
|
|
# Download
|
|
|
|
|
|
|
|
You can easily (cross) compile mox if you have a recent Go toolchain installed
|
|
|
|
(see "go version", it must be >= 1.20; otherwise, see https://go.dev/dl/ or
|
|
|
|
https://go.dev/doc/manage-install and $HOME/go/bin):
|
|
|
|
|
|
|
|
GOBIN=$PWD CGO_ENABLED=0 go install github.com/mjl-/mox@latest
|
|
|
|
|
|
|
|
Or you can download a binary built with the latest Go toolchain from
|
|
|
|
https://beta.gobuilds.org/github.com/mjl-/mox@latest/linux-amd64-latest/, and
|
|
|
|
symlink or rename it to "mox".
|
|
|
|
|
|
|
|
Verify you have a working mox binary:
|
|
|
|
|
|
|
|
./mox version
|
|
|
|
|
make mox compile on windows, without "mox serve" but with working "mox localserve"
getting mox to compile required changing code in only a few places where
package "syscall" was used: for accessing file access times and for umask
handling. an open problem is how to start a process as an unprivileged user on
windows. that's why "mox serve" isn't implemented yet. and just finding a way
to implement it now may not be good enough in the near future: we may want to
starting using a more complete privilege separation approach, with a process
handling sensitive tasks (handling private keys, authentication), where we may
want to pass file descriptors between processes. how would that work on
windows?
anyway, getting mox to compile for windows doesn't mean it works properly on
windows. the largest issue: mox would normally open a file, rename or remove
it, and finally close it. this happens during message delivery. that doesn't
work on windows, the rename/remove would fail because the file is still open.
so this commit swaps many "remove" and "close" calls. renames are a longer
story: message delivery had two ways to deliver: with "consuming" the
(temporary) message file (which would rename it to its final destination), and
without consuming (by hardlinking the file, falling back to copying). the last
delivery to a recipient of a message (and the only one in the common case of a
single recipient) would consume the message, and the earlier recipients would
not. during delivery, the already open message file was used, to parse the
message. we still want to use that open message file, and the caller now stays
responsible for closing it, but we no longer try to rename (consume) the file.
we always hardlink (or copy) during delivery (this works on windows), and the
caller is responsible for closing and removing (in that order) the original
temporary file. this does cost one syscall more. but it makes the delivery code
(responsibilities) a bit simpler.
there is one more obvious issue: the file system path separator. mox already
used the "filepath" package to join paths in many places, but not everywhere.
and it still used strings with slashes for local file access. with this commit,
the code now uses filepath.FromSlash for path strings with slashes, uses
"filepath" in a few more places where it previously didn't. also switches from
"filepath" to regular "path" package when handling mailbox names in a few
places, because those always use forward slashes, regardless of local file
system conventions. windows can handle forward slashes when opening files, so
test code that passes path strings with forward slashes straight to go stdlib
file i/o functions are left unchanged to reduce code churn. the regular
non-test code, or test code that uses path strings in places other than
standard i/o functions, does have the paths converted for consistent paths
(otherwise we would end up with paths with mixed forward/backward slashes in
log messages).
windows cannot dup a listening socket. for "mox localserve", it isn't
important, and we can work around the issue. the current approach for "mox
serve" (forking a process and passing file descriptors of listening sockets on
"privileged" ports) won't work on windows. perhaps it isn't needed on windows,
and any user can listen on "privileged" ports? that would be welcome.
on windows, os.Open cannot open a directory, so we cannot call Sync on it after
message delivery. a cursory internet search indicates that directories cannot
be synced on windows. the story is probably much more nuanced than that, with
long deep technical details/discussions/disagreement/confusion, like on unix.
for "mox localserve" we can get away with making syncdir a no-op.
2023-10-14 11:54:07 +03:00
|
|
|
Mox only compiles for and fully works on unix systems. Mox also compiles for
|
|
|
|
Windows, but "mox serve" does not yet work, though "mox localserve" (for a
|
|
|
|
local test instance) and most other subcommands do. Mox does not compile for
|
|
|
|
Plan 9.
|
2023-09-24 13:42:19 +03:00
|
|
|
|
|
|
|
You can also run mox with docker image `r.xmox.nl/mox`, with tags like `v0.0.1`
|
|
|
|
and `v0.0.1-go1.20.1-alpine3.17.2`, see https://r.xmox.nl/r/mox/. Though new
|
|
|
|
docker images aren't (automatically) generated for new Go runtime/compile
|
|
|
|
releases. See docker-compose.yml in this repository for instructions on
|
|
|
|
starting. It is important to run with docker host networking, so mox can use
|
|
|
|
the public IPs and has correct remote IP information for incoming connections
|
|
|
|
(important for junk filtering and rate-limiting). Given these caveats, it's
|
|
|
|
recommended to run mox without docker.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-25 15:38:07 +03:00
|
|
|
# Future/development
|
|
|
|
|
2023-09-21 17:08:43 +03:00
|
|
|
Mox will receive funding for essentially full-time continued work from August
|
|
|
|
2023 to August 2024 through NLnet/EU's NGI0 Entrust, see
|
|
|
|
https://nlnet.nl/project/Mox/.
|
|
|
|
|
2023-02-25 15:38:07 +03:00
|
|
|
## Roadmap
|
|
|
|
|
2023-12-24 01:07:21 +03:00
|
|
|
- Improve documentation
|
2023-12-14 22:34:44 +03:00
|
|
|
- Improve SMTP delivery from queue
|
|
|
|
- Webmail improvements
|
2023-09-24 13:34:46 +03:00
|
|
|
- HTTP-based API for sending messages and receiving delivery feedback
|
|
|
|
- Calendaring with CalDAV/iCal
|
|
|
|
- More IMAP extensions (PREVIEW, WITHIN, IMPORTANT, COMPRESS=DEFLATE,
|
|
|
|
CREATE-SPECIAL-USE, SAVEDATE, UNAUTHENTICATE, REPLACE, QUOTA, NOTIFY,
|
|
|
|
MULTIAPPEND, OBJECTID, MULTISEARCH)
|
|
|
|
- ARC, with forwarded email from trusted source
|
|
|
|
- Forwarding (to an external address)
|
2023-02-25 15:38:07 +03:00
|
|
|
- Add special IMAP mailbox ("Queue?") that contains queued but
|
2023-12-11 17:01:56 +03:00
|
|
|
undelivered messages, updated with IMAP flags/keywords/tags and message headers.
|
2023-02-27 17:04:53 +03:00
|
|
|
- Sieve for filtering (for now see Rulesets in the account config)
|
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
|
|
|
- Expose threading through IMAP extension
|
2023-09-24 13:34:46 +03:00
|
|
|
- Autoresponder (out of office/vacation)
|
|
|
|
- OAUTH2 support, for single sign on
|
2023-07-24 12:03:53 +03:00
|
|
|
- Privilege separation, isolating parts of the application to more restricted
|
|
|
|
sandbox (e.g. new unauthenticated connections)
|
|
|
|
- Using mox as backup MX
|
2023-02-27 17:04:53 +03:00
|
|
|
- JMAP
|
2023-07-24 12:03:53 +03:00
|
|
|
- Milter support, for integration with external tools
|
2023-09-24 13:34:46 +03:00
|
|
|
- IMAP extensions for "online"/non-syncing/webmail clients (SORT (including
|
|
|
|
DISPLAYFROM, DISPLAYTO), THREAD, PARTIAL, CONTEXT=SEARCH CONTEXT=SORT ESORT,
|
|
|
|
FILTERS)
|
|
|
|
- IMAP Sieve extension, to run Sieve scripts after message changes (not only
|
|
|
|
new deliveries)
|
|
|
|
- Improve support for mobile clients with extensions: IMAP URLAUTH, SMTP
|
|
|
|
CHUNKING and BINARYMIME, IMAP CATENATE
|
2023-02-25 15:38:07 +03:00
|
|
|
|
|
|
|
There are many smaller improvements to make as well, search for "todo" in the code.
|
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
## Not supported/planned
|
2023-02-25 15:38:07 +03:00
|
|
|
|
|
|
|
But perhaps in the future...
|
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
- Mailing list manager
|
2023-02-25 15:38:07 +03:00
|
|
|
- Functioning as SMTP relay
|
|
|
|
- POP3
|
|
|
|
- Delivery to (unix) OS system users
|
|
|
|
- Support for pluggable delivery mechanisms
|
2023-06-22 22:27:52 +03:00
|
|
|
- iOS Mail push notifications (with XAPPLEPUSHSERVICE undocumented imap
|
2023-05-31 11:24:48 +03:00
|
|
|
extension and hard to get APNS certificate)
|
2023-02-25 15:38:07 +03:00
|
|
|
|
|
|
|
|
2023-01-30 16:27:06 +03:00
|
|
|
# FAQ - Frequently Asked Questions
|
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## Why a new mail server implementation?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
Mox aims to make "running a mail server" easy and nearly effortless. Excellent
|
2023-09-24 13:34:46 +03:00
|
|
|
quality (open source) mail server software exists, but getting a working setup
|
|
|
|
typically requires you configure half a dozen services (SMTP, IMAP,
|
|
|
|
SPF/DKIM/DMARC, spam filtering), which are often written in C (where small bugs
|
|
|
|
often have large consequences). That seems to lead to people no longer running
|
|
|
|
their own mail servers, instead switching to one of the few centralized email
|
|
|
|
providers. Email with SMTP is a long-time decentralized messaging protocol. To
|
|
|
|
keep it decentralized, people need to run their own mail server. Mox aims to
|
|
|
|
make that easy.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## Where is the documentation?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
See all commands and help output at https://pkg.go.dev/github.com/mjl-/mox/.
|
|
|
|
|
|
|
|
See the commented example config files at
|
|
|
|
https://pkg.go.dev/github.com/mjl-/mox/config/. They often contain enough
|
|
|
|
documentation about a feature and how to configure it.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-01-30 16:43:50 +03:00
|
|
|
You can get the same information by running "mox" without arguments to list its
|
2023-02-10 21:43:02 +03:00
|
|
|
subcommands and usage, and "mox help [subcommand]" for more details.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-01-30 16:43:50 +03:00
|
|
|
The example config files are printed by "mox config describe-static" and "mox
|
|
|
|
config describe-dynamic".
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
Mox is still in early stages, and documentation is still limited. Please create
|
|
|
|
an issue describing what is unclear or confusing, and we'll try to improve the
|
|
|
|
documentation.
|
|
|
|
|
2024-01-01 20:30:31 +03:00
|
|
|
## Is Mox affected by SMTP smuggling?
|
|
|
|
|
|
|
|
Mox itself is not affected: it only treats "\r\n.\r\n" as SMTP end-of-message.
|
|
|
|
But read on for caveats.
|
|
|
|
|
|
|
|
SMTP smuggling exploits differences in handling by SMTP servers of: carriage
|
|
|
|
returns (CR, or "\r"), newlines (line feeds, LF, "\n") in the context of "dot
|
|
|
|
stuffing". SMTP is a text-based protocol. An SMTP transaction to send a
|
|
|
|
message is finalized with a "\r\n.\r\n" sequence. This sequence could occur in
|
|
|
|
the message being transferred, so any verbatim "." at the start of a line in a
|
|
|
|
message is "escaped" with another dot ("dot stuffing"), to not trigger the SMTP
|
|
|
|
end-of-message. SMTP smuggling takes advantage of bugs in some mail servers
|
|
|
|
that interpret other sequences than "\r\n.\r\n" as SMTP end-of-message. For
|
|
|
|
example "\n.\n" or even "\r.\r", and perhaps even other magic character
|
|
|
|
combinations.
|
|
|
|
|
|
|
|
Before v0.0.9, mox accepted SMTP transactions with bare carriage returns
|
|
|
|
(without newline) for compatibility with real-world email messages, considering
|
|
|
|
them meaningless and therefore innocuous.
|
|
|
|
|
|
|
|
Since v0.0.9, SMTP transactions with bare carriage returns are rejected.
|
|
|
|
Sending messages with bare carriage returns to buggy mail servers can cause
|
|
|
|
those mail servers to materialize non-existent messages. Now that mox rejects
|
|
|
|
messages with bare carriage returns, sending a message through mox can no
|
|
|
|
longer be used to trigger those bugs.
|
|
|
|
|
|
|
|
Mox can still handle bare carriage returns in email messages, e.g. those
|
|
|
|
imported from mbox files or Maildirs, or from messages added over IMAP. Mox
|
|
|
|
still fixes up messages with bare newlines by adding the missing carriage
|
|
|
|
returns.
|
|
|
|
|
|
|
|
Before v0.0.9, an SMTP transaction for a message containing "\n.\n" would
|
|
|
|
result in a non-specific error message, and "\r\n.\n" would result in the dot
|
|
|
|
being dropped. Since v0.0.9, these sequences are rejected with a message
|
|
|
|
mentioning SMTP smuggling.
|
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## How do I import/export email?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-16 11:57:27 +03:00
|
|
|
Use the import functionality on the accounts web page to import a zip/tgz with
|
|
|
|
maildirs/mbox files, or use the "mox import maildir" or "mox import mbox"
|
|
|
|
subcommands. You could also use your IMAP email client, add your mox account,
|
|
|
|
and copy or move messages from one account to the other.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-16 11:57:27 +03:00
|
|
|
Similarly, see the export functionality on the accounts web page and the "mox
|
|
|
|
export maildir" and "mox export mbox" subcommands to export email.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-07-24 12:00:11 +03:00
|
|
|
Importing large mailboxes may require a lot of memory (a limitation of the
|
2023-09-24 13:34:46 +03:00
|
|
|
current database). Splitting up mailboxes in smaller parts (e.g. 100k messages)
|
|
|
|
would help.
|
2023-07-24 12:00:11 +03:00
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## How can I help?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
Mox needs users and testing in real-life setups! So just give it a try, send
|
|
|
|
and receive emails through it with your favourite email clients, and file an
|
|
|
|
issue if you encounter a problem or would like to see a feature/functionality
|
|
|
|
implemented.
|
|
|
|
|
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config
- make builtin http handlers serve on specific domains, such as for mta-sts, so
e.g. /.well-known/mta-sts.txt isn't served on all domains.
- add logging of a few more fields in access logging.
- small tweaks/bug fixes in webserver request handling.
- add config option for redirecting entire domains to another (common enough).
- split httpserver metric into two: one for duration until writing header (i.e.
performance of server), another for duration until full response is sent to
client (i.e. performance as perceived by users).
- add admin ui, a new page for managing the configs. after making changes
and hitting "save", the changes take effect immediately. the page itself
doesn't look very well-designed (many input fields, makes it look messy). i
have an idea to improve it (explained in admin.html as todo) by making the
layout look just like the config file. not urgent though.
i've already changed my websites/webapps over.
the idea of adding a webserver is to take away a (the) reason for folks to want
to complicate their mox setup by running an other webserver on the same machine.
i think the current webserver implementation can already serve most common use
cases. with a few more tweaks (feedback needed!) we should be able to get to 95%
of the use cases. the reverse proxy can take care of the remaining 5%.
nevertheless, a next step is still to change the quickstart to make it easier
for folks to run with an existing webserver, with existing tls certs/keys.
that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
|
|
|
Instead of switching email for your domain over to mox, you could simply
|
2023-02-10 21:43:02 +03:00
|
|
|
configure mox for a subdomain, e.g. [you]@moxtest.[yourdomain].
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
If you have experience with how the email protocols are used in the wild, e.g.
|
|
|
|
compatibility issues, limitations, anti-spam measures, specification
|
|
|
|
violations, that would be interesting to hear about.
|
|
|
|
|
|
|
|
Pull requests for bug fixes and new code are welcome too. If the changes are
|
2023-09-24 13:34:46 +03:00
|
|
|
large, it helps to start a discussion (create an "issue") before doing all the
|
|
|
|
work. In practice, starting with a small contribution and growing from there has
|
|
|
|
the highest chance of success.
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-03-02 22:22:26 +03:00
|
|
|
By contributing (e.g. code), you agree your contributions are licensed under the
|
|
|
|
MIT license (like mox), and have the rights to do so.
|
|
|
|
|
2023-02-25 15:38:07 +03:00
|
|
|
## Where can I discuss mox?
|
|
|
|
|
|
|
|
Join #mox on irc.oftc.net, or #mox on the "Gopher slack".
|
|
|
|
|
|
|
|
For bug reports, please file an issue at https://github.com/mjl-/mox/issues/new.
|
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## How do I change my password?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
Regular users (doing IMAP/SMTP with authentication) can change their password
|
2023-03-20 14:49:40 +03:00
|
|
|
at the account page, e.g. http://localhost/. Or you can set a password with "mox
|
2023-02-16 11:57:27 +03:00
|
|
|
setaccountpassword".
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-03-20 14:49:40 +03:00
|
|
|
The admin can change the password of any account through the admin page, at
|
|
|
|
http://localhost/admin/ by default (leave username empty when logging in).
|
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
The account and admin pages are served on localhost for configs created with
|
|
|
|
the quickstart. To access these from your browser, run
|
2023-03-20 14:49:40 +03:00
|
|
|
`ssh -L 8080:localhost:80 you@yourmachine` locally and open
|
|
|
|
http://localhost:8080/[...].
|
|
|
|
|
2023-01-30 16:27:06 +03:00
|
|
|
The admin password can be changed with "mox setadminpassword".
|
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## How do I configure a second mox instance as a backup MX?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
|
|
|
Unfortunately, mox does not yet provide an option for that. Mox does spam
|
|
|
|
filtering based on reputation of received messages. It will take a good amount
|
|
|
|
of work to share that information with a backup MX. Without that information,
|
2023-09-24 13:34:46 +03:00
|
|
|
spammers could use a backup MX to get their spam accepted.
|
|
|
|
|
|
|
|
Until mox has a proper solution, you can simply run a single SMTP server. The
|
|
|
|
author has run a single mail server for over a decade without issues. Machines
|
|
|
|
and network connectivity are stable nowadays, and email delivery will be
|
|
|
|
retried for many hours during temporary errors (e.g. when rebooting a machine
|
|
|
|
after updates).
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-27 17:04:53 +03:00
|
|
|
## How do I stay up to date?
|
|
|
|
|
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config
- make builtin http handlers serve on specific domains, such as for mta-sts, so
e.g. /.well-known/mta-sts.txt isn't served on all domains.
- add logging of a few more fields in access logging.
- small tweaks/bug fixes in webserver request handling.
- add config option for redirecting entire domains to another (common enough).
- split httpserver metric into two: one for duration until writing header (i.e.
performance of server), another for duration until full response is sent to
client (i.e. performance as perceived by users).
- add admin ui, a new page for managing the configs. after making changes
and hitting "save", the changes take effect immediately. the page itself
doesn't look very well-designed (many input fields, makes it look messy). i
have an idea to improve it (explained in admin.html as todo) by making the
layout look just like the config file. not urgent though.
i've already changed my websites/webapps over.
the idea of adding a webserver is to take away a (the) reason for folks to want
to complicate their mox setup by running an other webserver on the same machine.
i think the current webserver implementation can already serve most common use
cases. with a few more tweaks (feedback needed!) we should be able to get to 95%
of the use cases. the reverse proxy can take care of the remaining 5%.
nevertheless, a next step is still to change the quickstart to make it easier
for folks to run with an existing webserver, with existing tls certs/keys.
that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
|
|
|
Please set "CheckUpdates: true" in mox.conf. Mox will check for a new version
|
|
|
|
through a DNS TXT request for `_updates.xmox.nl` once per 24h. Only if a new
|
|
|
|
version is published will the changelog be fetched and delivered to the
|
2023-02-27 17:04:53 +03:00
|
|
|
postmaster mailbox.
|
|
|
|
|
2023-03-09 18:50:24 +03:00
|
|
|
The changelog, including latest update instructions, is at
|
|
|
|
https://updates.xmox.nl/changelog.
|
2023-02-27 17:04:53 +03:00
|
|
|
|
2023-03-09 18:50:24 +03:00
|
|
|
You can also monitor newly added releases on this repository with the github
|
|
|
|
"watch" feature, or use the github RSS feed for tags
|
|
|
|
(https://github.com/mjl-/mox/tags.atom) or releases
|
|
|
|
(https://github.com/mjl-/mox/releases.atom), or monitor the docker images.
|
2023-02-27 17:04:53 +03:00
|
|
|
|
2023-10-04 08:39:44 +03:00
|
|
|
Keep in mind you have a responsibility to keep the internet-connected software
|
2023-02-27 17:04:53 +03:00
|
|
|
you run up to date and secure.
|
|
|
|
|
2023-05-31 11:30:34 +03:00
|
|
|
## How do I upgrade my mox installation?
|
|
|
|
|
|
|
|
We try to make upgrades effortless and you can typically just put a new binary
|
|
|
|
in place and restart. If manual actions are required, the release notes mention
|
|
|
|
them. Check the release notes of all version between your current installation
|
|
|
|
and the release you're upgrading to.
|
|
|
|
|
|
|
|
Before upgrading, make a backup of the data directory with `mox backup
|
|
|
|
<destdir>`. This writes consistent snapshots of the database files, and
|
2023-09-24 13:34:46 +03:00
|
|
|
duplicates message files from the outgoing queue and accounts. Using the new
|
|
|
|
mox binary, run `mox verifydata <backupdir>` (do NOT use the "live" data
|
2023-05-31 11:30:34 +03:00
|
|
|
directory!) for a dry run. If this fails, an upgrade will probably fail too.
|
2023-09-24 13:34:46 +03:00
|
|
|
Important: verifydata with the new mox binary can modify the database files (due
|
|
|
|
to automatic schema upgrades). So make a fresh backup again before the actual
|
|
|
|
upgrade. See the help output of the "backup" and "verifydata" commands for more
|
|
|
|
details.
|
2023-05-31 11:30:34 +03:00
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
During backup, message files are hardlinked if possible, and copied otherwise.
|
|
|
|
Using a destination directory like `data/tmp/backup` increases the odds
|
|
|
|
hardlinking succeeds: the default mox systemd service file mounts
|
|
|
|
the data directory separately, so hardlinks to outside the data directory are
|
|
|
|
cross-device and will fail.
|
2023-05-31 11:30:34 +03:00
|
|
|
|
|
|
|
If an upgrade fails and you have to restore (parts) of the data directory, you
|
|
|
|
should run `mox verifydata <datadir>` (with the original binary) on the
|
|
|
|
restored directory before starting mox again. If problematic files are found,
|
|
|
|
for example queue or account message files that are not in the database, run
|
|
|
|
`mox verifydata -fix <datadir>` to move away those files. After a restore, you may
|
|
|
|
also want to run `mox bumpuidvalidity <account>` for each account for which
|
|
|
|
messages in a mailbox changed, to force IMAP clients to synchronize mailbox
|
|
|
|
state.
|
|
|
|
|
2023-02-11 15:35:07 +03:00
|
|
|
## How secure is mox?
|
2023-01-30 16:27:06 +03:00
|
|
|
|
2023-02-10 21:43:02 +03:00
|
|
|
Security is high on the priority list for mox. Mox is young, so don't expect no
|
2023-01-30 16:27:06 +03:00
|
|
|
bugs at all. Mox does have automated tests for some security aspects, e.g. for
|
|
|
|
login, and uses fuzzing. Mox is written in Go, so some classes of bugs such as
|
|
|
|
buffer mishandling do not typically result in privilege escalation. Of course
|
|
|
|
logic bugs will still exist. If you find any security issues, please email them
|
|
|
|
to mechiel@ueber.net.
|
2023-02-27 17:04:53 +03:00
|
|
|
|
|
|
|
## I'm now running an email server, but how does email work?
|
|
|
|
|
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config
- make builtin http handlers serve on specific domains, such as for mta-sts, so
e.g. /.well-known/mta-sts.txt isn't served on all domains.
- add logging of a few more fields in access logging.
- small tweaks/bug fixes in webserver request handling.
- add config option for redirecting entire domains to another (common enough).
- split httpserver metric into two: one for duration until writing header (i.e.
performance of server), another for duration until full response is sent to
client (i.e. performance as perceived by users).
- add admin ui, a new page for managing the configs. after making changes
and hitting "save", the changes take effect immediately. the page itself
doesn't look very well-designed (many input fields, makes it look messy). i
have an idea to improve it (explained in admin.html as todo) by making the
layout look just like the config file. not urgent though.
i've already changed my websites/webapps over.
the idea of adding a webserver is to take away a (the) reason for folks to want
to complicate their mox setup by running an other webserver on the same machine.
i think the current webserver implementation can already serve most common use
cases. with a few more tweaks (feedback needed!) we should be able to get to 95%
of the use cases. the reverse proxy can take care of the remaining 5%.
nevertheless, a next step is still to change the quickstart to make it easier
for folks to run with an existing webserver, with existing tls certs/keys.
that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
|
|
|
Congrats and welcome to the club! Running an email server on the internet comes
|
|
|
|
with some responsibilities so you should understand how it works. See
|
2023-02-27 17:04:53 +03:00
|
|
|
https://explained-from-first-principles.com/email/ for a thorough explanation.
|
2023-03-04 03:04:54 +03:00
|
|
|
|
|
|
|
## What are the minimum requirements to run mox?
|
|
|
|
|
|
|
|
Mox does not need much. Nowadays most machines are larger than mox needs. You
|
|
|
|
can start with a machine with 512MB RAM, any CPU will do. For storage you
|
|
|
|
should account for the size of the email messages (no compression currently),
|
|
|
|
an additional 15% overhead for the meta data, and add some more headroom.
|
|
|
|
Expand as necessary.
|
2023-04-20 17:44:07 +03:00
|
|
|
|
|
|
|
## Can I see some screenshots?
|
|
|
|
|
|
|
|
Yes, see https://www.xmox.nl/screenshots/.
|
|
|
|
|
2023-08-16 11:16:48 +03:00
|
|
|
Mox has a webmail for reading/writing messages.
|
|
|
|
|
|
|
|
Mox also has an "account" web interface where users can view their account and
|
2023-04-20 17:44:07 +03:00
|
|
|
manage their address configuration, such as rules for automatically delivering
|
|
|
|
certain incoming messages to a specific mailbox.
|
|
|
|
|
2023-08-16 11:16:48 +03:00
|
|
|
And mox has an "admin" web interface where the administrator can make changes,
|
|
|
|
e.g. add/remove/modify domains/accounts/addresses.
|
2023-06-12 17:25:35 +03:00
|
|
|
|
|
|
|
## Won't the big email providers block my email?
|
|
|
|
|
|
|
|
It is a common misconception that it is impossible to run your own email server
|
|
|
|
nowadays. The claim is that the handful big email providers will simply block
|
|
|
|
your email. However, you can run your own email server just fine, and your
|
|
|
|
email will be accepted, provided you are doing it right.
|
|
|
|
|
|
|
|
If your email is rejected, it is often because your IP address has a bad email
|
|
|
|
sending reputation. Email servers often use IP blocklists to reject email
|
|
|
|
networks with a bad email sending reputation. These blocklists often work at
|
|
|
|
the level of whole network ranges. So if you try to run an email server from a
|
|
|
|
hosting provider with a bad reputation (which happens if they don't monitor
|
|
|
|
their network or don't act on abuse/spam reports), your IP too will have a bad
|
|
|
|
reputation and other mail servers (both large and small) may reject messages
|
|
|
|
coming from you. During the quickstart, mox checks if your IPs are on a few
|
|
|
|
often-used blocklists. It's typically not a good idea to host an email server
|
|
|
|
on the cheapest or largest cloud providers: They often don't spend the
|
|
|
|
resources necessary for a good reputation, or they simply block all outgoing
|
|
|
|
SMTP traffic. It's better to look for a technically-focused local provider.
|
2023-06-12 17:35:03 +03:00
|
|
|
They too may initially block outgoing SMTP connections on new machines to
|
|
|
|
prevent spam from their networks. But they will either automatically open up
|
|
|
|
outgoing SMTP traffic after a cool down period (e.g. 24 hours), or after you've
|
|
|
|
contacted their support.
|
2023-06-12 17:25:35 +03:00
|
|
|
|
|
|
|
After you get past the IP blocklist checks, email servers use many more signals
|
|
|
|
to determine if your email message could be spam and should be rejected. Mox
|
|
|
|
helps you set up a system that doesn't trigger most of the technical signals
|
|
|
|
(e.g. with SPF/DKIM/DMARC). But there are more signals, for example: Sending to
|
|
|
|
a mail server or address for the first time. Sending from a newly registered
|
|
|
|
domain. Sending messages with content that resembles known spam messages.
|
|
|
|
|
2023-09-24 13:34:46 +03:00
|
|
|
Should your email be rejected, you will typically get an error message during
|
|
|
|
the SMTP transaction that explains why. In the case of big email providers the
|
|
|
|
error message often has instructions on how to prove to them you are a legimate
|
|
|
|
sender.
|
2023-07-02 16:05:55 +03:00
|
|
|
|
|
|
|
## Can I use existing TLS certificates/keys?
|
|
|
|
|
|
|
|
Yes. The quickstart command creates a config that uses ACME with Let's Encrypt,
|
|
|
|
but you can change the config file to use existing certificate and key files.
|
|
|
|
|
|
|
|
You'll see "ACME: letsencrypt" in the "TLS" section of the "public" Listener.
|
|
|
|
Remove or comment out the ACME-line, and add a "KeyCerts" section like in the
|
|
|
|
example config file in
|
|
|
|
https://pkg.go.dev/github.com/mjl-/mox/config#hdr-mox_conf. You can have
|
|
|
|
multiple certificates and keys: The line with the "-" (dash) is the start of a
|
|
|
|
list item. Duplicate that line up to and including the line with KeyFile for
|
|
|
|
each certificate/key you have. Mox makes a TLS config that holds all specified
|
|
|
|
certificates/keys, and uses it for all services for that Listener (including a
|
|
|
|
webserver), choosing the correct certificate for incoming requests.
|
|
|
|
|
|
|
|
Keep in mind that for each email domain you host, you will need a certificate
|
|
|
|
for `mta-sts.<domain>` and `autoconfig.<domain>`, unless you disable MTA-STS
|
|
|
|
and autoconfig for that domain.
|
|
|
|
|
|
|
|
Mox opens the key and certificate files during initial startup, as root (and
|
|
|
|
passes file descriptors to the unprivileged process). No special permissions
|
|
|
|
are needed on the key and certificate files.
|