Commit graph

160 commits

Author SHA1 Message Date
Mechiel Lukkien
dcc051e149
for fuzzing the imapserver and smtpserver use different config files than regular tests
otherwise they cannot be running at the same time, they could overwrite each
other's files.
2023-05-22 15:37:03 +02:00
Mechiel Lukkien
1f5ab1b795
fix language in comments
found through goreportcard.com
2023-05-22 15:04:06 +02:00
Mechiel Lukkien
b0623e6038
in queue.Drop, to drop a message from the outgoing queue, not only remove it from the database, but also its contents from the file system 2023-05-22 15:03:23 +02:00
Mechiel Lukkien
88fd775ec4
if we encounter an error fetching an mta-sts policy as part of a delivery attempt, properly continue with delivery with strict tls checking 2023-05-22 14:46:20 +02:00
Mechiel Lukkien
e81930ba20
update to latest bstore (with support for an index on a []string: Message.DKIMDomains), and cyclic data types (to be used for Message.Part soon); also adds a context.Context to database operations. 2023-05-22 14:40:36 +02:00
Kohei Watanabe
f6ed860ccb
Fixed MTASTSHTTPS.NonTLS option (#29)
AutoconfigHTTPS.NonTLS option was being used.
Fixed to use MTASTSHTTPS.NonTLS option.
2023-05-03 16:26:04 +02:00
Mechiel Lukkien
70ab9a7d4c
tweak alerting rule to include that it is about authentication rate limiting 2023-05-01 14:21:02 +02:00
Mechiel Lukkien
c1753b369d
in smtpserver, accept delivery to postmaster@<hostname>, and also postmaster@ addresses for domains that don't have a postmaster address configured. 2023-04-24 12:04:46 +02:00
Mechiel Lukkien
74dab5fc39
fix sending to address where the domain does not have an mx record (but where we should connect directly to the host, or follow cname records)
such deliveries would fail because a canceled "context" was reused, so the dns
lookups would fail.

the tests didn't catch it before because they ignored their context parameters.
2023-04-24 10:34:19 +02:00
Mechiel Lukkien
1f4df30019
remove debug print 2023-04-24 10:06:59 +02:00
Mechiel Lukkien
517fb31212
clear updating.txt for release 2023-04-22 07:52:42 +02:00
Mechiel Lukkien
b3f3c0a056
in smtpclient, when delivering with pipelining, don't return a unhelpful read error when the remote server closes the connection early after writing an error response
e.g. when outlook.com puts your IP on a blocklist, it will respond with 550 to
MAIL FROM, then closes the connection (without responding to the remaining
commands). we were reading the 550 response, not yet acting on it, then reading
the response to RCPT TO. that read failed, and we would return that error. now,
we will properly return the 550 (permanent error, instead of the temporary read
error) from the first MAIL FROM (but we do still always try to read the
response for RCPT TO and DATA).
2023-04-20 22:29:18 +02:00
Mechiel Lukkien
0b364862ed
add link to screenshots
the screenshots are not in the git repo, they may change quite a bit and are
larger binary blobs. i don't want to make the repo too big (the code with its
dependencies is already pretty big!).

the website with the screenshots is in github.com/mjl-/mox.

for github issue #26
2023-04-20 16:47:32 +02:00
Mechiel Lukkien
dae73eb32d
update golang.org/x dependencies 2023-04-20 15:07:24 +02:00
Mechiel Lukkien
6eff832d09
fix fuzzing imapserver by not triggering the rate limiter 2023-04-20 14:58:05 +02:00
Mechiel Lukkien
08eb1a5472
in store/, change functions from calling panic to returning errors
this is a library package, errors should be explicit. callers had to be careful
when calling these "X" functions. now it's explicit.
2023-04-20 14:16:56 +02:00
Mechiel Lukkien
936a0d5afe
bugfix: when dkim-signing submitted messages, use the domain from the "message from header" instead of "smtp mail from"
dmarc verifiers will only accept a dkim signature if the domain the message From
header matches the domain of the signature (i.e. it is "aligned").

i hadn't run into this before and when testing because thunderbird sets the
"smtp mail from" to the same address as a custom "message from" header. but
other mail clients don't have to do that.

should fix issue #22
2023-03-30 10:38:36 +02:00
Mechiel Lukkien
0989e59567
remove pgp and s/mime from possible future feature list
for issue #23
2023-03-29 21:24:59 +02:00
Mechiel Lukkien
df6956bed8
the mbox export also contains all message flags, so fix the text along the export links 2023-03-29 21:21:43 +02:00
Mechiel Lukkien
b571dd4b28
implement a catchall address for a domain
by specifying a "destination" in an account that is just "@" followed by the
domain, e.g. "@example.org". messages are only delivered to the catchall
address when no regular destination matches (taking the per-domain
catchall-separator and case-sensisitivity into account).

for issue #18
2023-03-29 21:11:43 +02:00
Mechiel Lukkien
51ad345dbb
refuse to add an address when its localpart contains the domains catchall separator, or when its canonicalized address (e.g. lower cased when case-insensitive) is already present, and check at startup as well
such configurations are certainly errors, but were silently accepted and highly
likely not doing what you may have hoped. i suspect no one has configured mox
this way.
2023-03-29 20:58:50 +02:00
Mechiel Lukkien
9b57c69c1c
implement limits on outgoing messages for an account
by default 1000 messages per day, and to max 200 first-time receivers.
i don't think a person would reach those limits. a compromised account abused
by spammers could easily reach that limit. this prevents further damage.

the error message you will get is quite clear, pointing to the configuration
parameter that should be changed.
2023-03-29 09:36:06 +02:00
Mechiel Lukkien
9bd497b836
set timeouts for webserver, for idle connections and reading http request header 2023-03-28 17:16:05 +02:00
Mechiel Lukkien
00ea31f2f6
do not generate http status 502 for canceled http requests
do log them with level debug, and print the error in the http access log line.
2023-03-21 09:25:49 +01:00
Mechiel Lukkien
f531a9bf35
don't serve static file when requested as dir, and fix 500 internal server errors when a file below such a file-as-dir is requested
e.g. when /index.html/ returned content. and /index.html/image.png would result
in 500 internal server errors. now they all return 404 not found.
2023-03-20 13:48:17 +01:00
Mechiel Lukkien
c5fdb7309f
document that mox sendmail -t does not treat cc and bcc headers specially 2023-03-20 13:25:38 +01:00
Mechiel Lukkien
d36419170b
also use -loglevel in localserve, and fail if the loglevel is invalid 2023-03-20 13:01:49 +01:00
Mechiel Lukkien
98b5a27fd2
mention where the admin interface can be accessed
at the end of the quickstart. also hint at it during startup, when printing the
listener. and mention it in the FAQ.

another recent commit make the admin and account http path configurable, and
that expanded the config docs with a mention of the default path.

based on feedback from stroyselmash in issue #20, thanks!
2023-03-20 12:49:40 +01:00
Mechiel Lukkien
a9b2bc8cec
replace use of docker registry docker.io (hub.docker.com) with self-hosted r.xmox.nl
when setting up the docker organization account for mox it already felt off.
depending on such a party to serve binaries didn't feel great to begin with.
after clarifying online discussions of docker's vague announcement of removing
free team organizations it was clearly time to move off hub.docker.com.  best to
self-host.

r.xmox.nl is hosted with the new github.com/mjl-/vex.

pinging issue #3
2023-03-20 09:35:49 +01:00
Mechiel Lukkien
a6e603e1df
another rfc 2023-03-20 08:52:45 +01:00
Mechiel Lukkien
9d03e2b135
add prometheus rules for mox 2023-03-12 22:14:42 +01:00
Mechiel Lukkien
ad49941302
add notes on release process 2023-03-12 21:52:07 +01:00
Mechiel Lukkien
7681f8bdc8
for localserve, validate the incoming message
it must be completely parsable.
normally, if we receive a message that we cannot fully parse, we accept it and
treat it as opaque application/octet-stream.

also make it more clear that localserve accepts email intended for any email
address.
2023-03-12 21:42:28 +01:00
Mechiel Lukkien
19ea0d9a58
make all helpall output go to the same file descriptor 2023-03-12 15:25:16 +01:00
Mechiel Lukkien
5535515fcb
move sendmail to separate file 2023-03-12 15:22:23 +01:00
Mechiel Lukkien
317dc78397
add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility 2023-03-12 15:16:01 +01:00
Mechiel Lukkien
132f08913b
log msgfrom for incoming delivered messages 2023-03-12 13:22:51 +01:00
Mechiel Lukkien
10daf3cb81
make http(s) path for serving the account and admin pages configurable
so you can use the host (domain) name of the mail server for serving other
resources too. the default is is still that account is served on /, and so
takes all incoming requests before giving webhandlers a chance.

mox localserve now serves the account pages on /account/
2023-03-12 11:52:15 +01:00
Mechiel Lukkien
0099197d00
add "mox localserve" subcommand, for running mox locally for email-related testing/developing
localserve creates a config for listening on localhost for
imap/smtp/submission/http, on port numbers 1000 + the common service port
numbers. all incoming email is accepted (if checks pass), and a few pattern in
localparts are recognized and result in delivery errors.
2023-03-12 11:40:00 +01:00
Mechiel Lukkien
bddc8e4062
also configure acme validation with http-01, and fix a bug that caused tls cert refresh at startup to not always run
we already do acme tls-alpn-01 validation, and still require it (we could relax
this at some point). http-01 is easy to add.

the bug was that the list of acme managers and hosts to refresh was overwritten
by another listener. the listeners are a map, and we range over it, so the
order we handle them is random. if the public listener was handled first, and
an internal handler later, the list was reset again.
2023-03-10 17:55:37 +01:00
Mechiel Lukkien
f60ad1452f
use configured tls ca config for all tls connections, so https as well
and add documentation for developers for setting up certificates with manual
local CA (with cfssl) or local ACME CA (with pebble).
2023-03-10 16:25:18 +01:00
Mechiel Lukkien
47b88550be
add a little explanation about sconf, the config file syntax 2023-03-10 11:42:50 +01:00
Mechiel Lukkien
f9eae88aba
for imap/smtp syntax errors, only echo the remaining buffer if the connection is authenticated 2023-03-10 11:32:34 +01:00
Mechiel Lukkien
e413c906b1
if the first smtp or imap command is invalid, shut down the connection instead of trying to read more
this is quite common on the internet. the other side may be trying some other
protocol, e.g. http, or some common vulnerability. we don't want to spam our
own logs with multiple invalid lines. if the first command is valid, but later
are not, we'll keep trying to process them. so this only affects protocol
sessions that are very likely not smtp/imap.

also remove a few more sleeps during tests, making imapserver and smtpserver tests a bit faster.
2023-03-10 10:23:43 +01:00
Mechiel Lukkien
2c07645ab4
deprecate having only localparts in an Account's Destinations, it should always be a full email address
current behaviour isn't intuitive. it's not great to have to attempt parsing
the strings as both localpart and email address. so we deprecate the
localpart-only behaviour. when we load the config file, and it has
localpart-only Destinations keys, we'll change them to full addresses in
memory. when an admin causes a write of domains.conf, it'll automatically be
fixed. we log an error with a deprecated notice for each localpart-only
destinations key.

sometime in the future, we can remove the old localpart-only destination
support. will be in the release notes then.

also start keeping track of update notes that need to make it in the release
notes of the next release.

for issue #18
2023-03-09 22:13:56 +01:00
Mechiel Lukkien
5742ed1537
when logging email addresses with IDNA domain and/or special characters or utf8 in localpart, log both native utf8 form and form with escape localpart and ascii-only domain
the idea is to make it clear from the logging if non-ascii characters are used.

this is implemented by making mlog recognize if a field value that will be
logged has a LogString method. if so, that value is logged. dns.Domain,
smtp.Address, smtp.Localpart, smtp.Path now have a LogString method.

some explicit calls to String have been replaced to LogString, and some %q
formatting have been replaced with %s, because the escaped localpart would
already have double quotes, and double doublequotes aren't easy to read.
2023-03-09 20:18:34 +01:00
Mechiel Lukkien
eb26e9b921
when rejecting a message, add field "msgfrom" to the logging line 2023-03-09 17:21:27 +01:00
Mechiel Lukkien
b0641a1901
mention the github watch release feature to stay up to date, and link to the rss feeds
if you watch for a release, you will get an email when a new release is created.
when a new tag is created, github does not notify you. so i will be creating
github releases for newly added tags, so it's easier to stay up to date with
that feature.
2023-03-09 16:50:24 +01:00
Mechiel Lukkien
e6df84a8de
add config field "IPsNATed" to listener, indicating the IPs are not the actual public IPs but are NATed, to skip a few DNS checks
the dns check was returning errors that could not be fixed with that setup,
which makes the checks much less useful.

for issue #17
2023-03-09 15:24:06 +01:00
Mechiel Lukkien
8b0706e02d
for WebRedirect, don't "match" when the destination URL has the same scheme,host,path, for doing http -> https redirects without loops
you can already get most http to https redirects through DontRedirectPlainHTTP
in WebHandler, but that needs handlers for all paths.

now you can just set up a redirect for a domain and all its path to baseurl
https://domain (leaving other webdirect fields empty). when the request comes
in with plain http, the redirect to https is done. that next request will also
evaluate the same redirect rule. but it will not cause a match because it would
redirect to the same scheme,host,path. so next webhandlers get a chance to
serve.

also clarify in webhandlers docs that also account & admin built-in handlers
run first.

related to issue #16
2023-03-08 23:29:44 +01:00