Commit graph

18 commits

Author SHA1 Message Date
Mechiel Lukkien
8096441f67
new feature: when delivering messages from the queue, make it possible to use a "transport"
the default transport is still just "direct delivery", where we connect to the
destination domain's MX servers.

other transports are:

- regular smtp without authentication, this is relaying to a smarthost.
- submission with authentication, e.g. to a third party email sending service.
- direct delivery, but with with connections going through a socks proxy. this
  can be helpful if your ip is blocked, you need to get email out, and you have
  another IP that isn't blocked.

keep in mind that for all of the above, appropriate SPF/DKIM settings have to
be configured. the "dnscheck" for a domain does a check for any SOCKS IP in the
SPF record. SPF for smtp/submission (ranges? includes?) and any DKIM
requirements cannot really be checked.

which transport is used can be configured through routes. routes can be set on
an account, a domain, or globally. the routes are evaluated in that order, with
the first match selecting the transport. these routes are evaluated for each
delivery attempt. common selection criteria are recipient domain and sender
domain, but also which delivery attempt this is. you could configured mox to
attempt sending through a 3rd party from the 4th attempt onwards.

routes and transports are optional. if no route matches, or an empty/zero
transport is selected, normal direct delivery is done.

we could already "submit" emails with 3rd party accounts with "sendmail". but
we now support more SASL authentication mechanisms with SMTP (not only PLAIN,
but also SCRAM-SHA-256, SCRAM-SHA-1 and CRAM-MD5), which sendmail now also
supports. sendmail will use the most secure mechanism supported by the server,
or the explicitly configured mechanism.

for issue #36 by dmikushin. also based on earlier discussion on hackernews.
2023-06-16 18:57:05 +02:00
Mechiel Lukkien
77d78191f8
more helpful error message when the queue tries to deliver a message but the remote host is not listed in the mta-sts policy
based on questions on irc by Nemain where this better error message would
probably have made the problem easier to find and fix.
2023-06-01 16:23:36 +02:00
Mechiel Lukkien
aad5a5bcb9
add a "backup" subcommand to make consistent backups, and a "verifydata" subcommand to verify a backup before restoring, and add tests for future upgrades
the backup command will make consistent snapshots of all the database files. i
had been copying the db files before, and it usually works. but if the file is
modified during the backup, it is inconsistent and is likely to generate errors
when reading (can be at any moment in the future, when reading some db page).
"mox backup" opens the database file and writes out a copy in a transaction.
it also duplicates the message files.

before doing a restore, you could run "mox verifydata" on the to-be-restored
"data" directory. it check the database files, and compares the message files
with the database.

the new "gentestdata" subcommand generates a basic "data" directory, with a
queue and a few accounts. we will use it in the future along with "verifydata"
to test upgrades from old version to the latest version. both when going to the
next version, and when skipping several versions. the script test-upgrades.sh
executes these tests and doesn't do anything at the moment, because no releases
have this subcommand yet.

inspired by a failed upgrade attempt of a pre-release version.
2023-05-26 19:26:51 +02:00
Mechiel Lukkien
b0623e6038
in queue.Drop, to drop a message from the outgoing queue, not only remove it from the database, but also its contents from the file system 2023-05-22 15:03:23 +02:00
Mechiel Lukkien
88fd775ec4
if we encounter an error fetching an mta-sts policy as part of a delivery attempt, properly continue with delivery with strict tls checking 2023-05-22 14:46:20 +02:00
Mechiel Lukkien
e81930ba20
update to latest bstore (with support for an index on a []string: Message.DKIMDomains), and cyclic data types (to be used for Message.Part soon); also adds a context.Context to database operations. 2023-05-22 14:40:36 +02:00
Mechiel Lukkien
74dab5fc39
fix sending to address where the domain does not have an mx record (but where we should connect directly to the host, or follow cname records)
such deliveries would fail because a canceled "context" was reused, so the dns
lookups would fail.

the tests didn't catch it before because they ignored their context parameters.
2023-04-24 10:34:19 +02:00
Mechiel Lukkien
936a0d5afe
bugfix: when dkim-signing submitted messages, use the domain from the "message from header" instead of "smtp mail from"
dmarc verifiers will only accept a dkim signature if the domain the message From
header matches the domain of the signature (i.e. it is "aligned").

i hadn't run into this before and when testing because thunderbird sets the
"smtp mail from" to the same address as a custom "message from" header. but
other mail clients don't have to do that.

should fix issue #22
2023-03-30 10:38:36 +02:00
Mechiel Lukkien
0099197d00
add "mox localserve" subcommand, for running mox locally for email-related testing/developing
localserve creates a config for listening on localhost for
imap/smtp/submission/http, on port numbers 1000 + the common service port
numbers. all incoming email is accepted (if checks pass), and a few pattern in
localparts are recognized and result in delivery errors.
2023-03-12 11:40:00 +01:00
Mechiel Lukkien
5742ed1537
when logging email addresses with IDNA domain and/or special characters or utf8 in localpart, log both native utf8 form and form with escape localpart and ascii-only domain
the idea is to make it clear from the logging if non-ascii characters are used.

this is implemented by making mlog recognize if a field value that will be
logged has a LogString method. if so, that value is logged. dns.Domain,
smtp.Address, smtp.Localpart, smtp.Path now have a LogString method.

some explicit calls to String have been replaced to LogString, and some %q
formatting have been replaced with %s, because the escaped localpart would
already have double quotes, and double doublequotes aren't easy to read.
2023-03-09 20:18:34 +01:00
Mechiel Lukkien
26fcaa17f5
consistently use lower-cased field names for logging lines 2023-02-25 12:37:59 +01:00
Mechiel Lukkien
ad900b74e7
on mtasts error during delivery from queue, continue delivery but with strict tls
no need to not even try delivering in that case. if anyone is preventing mtasts
to work, nothing was achieved, because our fallback is the strictest behaviour
that could have been specified in the policy.
2023-02-17 17:19:41 +01:00
Mechiel Lukkien
5c33640aea
consistently use log.Check for logging errors that "should not happen", don't influence application flow
sooner or later, someone will notice one of these messages, which will lead us
to a bug.
2023-02-16 13:22:00 +01:00
Mechiel Lukkien
5336032088
add funtionality to import zip/tgz with maildirs/mboxes to account page
so users can easily take their email out of somewhere else, and import it into mox.

this goes a little way to give feedback as the import progresses: upload
progress is shown (surprisingly, browsers aren't doing this...), imported
mailboxes/messages are counted (batched) and import issues/warnings are
displayed, all sent over an SSE connection. an import token is stored in
sessionstorage. if you reload the page (e.g. after a connection error), the
browser will reconnect to the running import and show its progress again. and
you can just abort the import before it is finished and committed, and nothing
will have changed.

this also imports flags/keywords from mbox files.
2023-02-16 09:57:27 +01:00
Mechiel Lukkien
87854cfde3
change some log levels from info to debug, and use lower case log messages 2023-02-11 23:54:22 +01:00
Mechiel Lukkien
8bdca09b7b
on admin index page, show number of messages in queue next to link to the queue list 2023-02-08 19:42:21 +01:00
Mechiel Lukkien
1617b7c0d6
add jitter to the retries by the queue 2023-02-06 16:08:21 +01:00
Mechiel Lukkien
cb229cb6cf
mox! 2023-01-30 14:27:06 +01:00