Commit graph

462 commits

Author SHA1 Message Date
Mechiel Lukkien
e7478ed6ac
implement the plus variants of scram, to bind the authentication exchange to the tls connection
to get the security benefits (detecting mitm attempts), explicitly configure
clients to use a scram plus variant, e.g. scram-sha-256-plus. unfortunately,
not many clients support it yet.

imapserver scram plus support seems to work with the latest imtest (imap test
client) from cyrus-sasl. no success yet with mutt (with gsasl) though.
2023-12-23 23:19:36 +01:00
Mechiel Lukkien
4701857d7f
at startup, request missing acme tls certificates more quickly/silently 2023-12-22 13:41:00 +01:00
Mechiel Lukkien
dbd6773f6b
quickstart: don't print logging line about new password 2023-12-22 12:00:05 +01:00
Mechiel Lukkien
ee1094e1cb
implement ACME external account binding (EAB)
where a new acme account is created with a reference to an existing non-acme
account known by the acme provider. some acme providers require this.
2023-12-22 11:50:50 +01:00
Mechiel Lukkien
db3fef4981
when suggesting CAA records for a domain, suggest variants that bind to the account id and with validation methods used by mox
should prevent potential mitm attacks. especially when done close to the
machine itself (where a http/tls challenge is intercepted to get a valid
certificate), as seen on the internet last month.
2023-12-21 15:53:32 +01:00
Mechiel Lukkien
ca97293cb2
add last commit date to cross-reference page 2023-12-21 09:46:01 +01:00
Mechiel Lukkien
802dcef192
webmail: for messages in designated Sent mailbox, show To/Cc/Bcc in italics, and show all correspondents in collapsed thread
showing addressees for Sent messages for issue #104 by mattfbacon, thanks for the report!
2023-12-21 09:23:06 +01:00
Mechiel Lukkien
57fc37af22
if an smtp-submitted message has a return-path header, only fail in pedantic mode
some software sends messages with return-path header.

for issue #103 by Halyul, thanks for reporting!
2023-12-20 21:04:03 +01:00
Mechiel Lukkien
d73bda7511
add per-account quota for total message size disk usage
so a single user cannot fill up the disk.
by default, there is (still) no limit. a default can be set in the config file
for all accounts, and a per-account max size can be set that would override any
global setting.

this does not take into account disk usage of the index database. and also not
of any file system overhead.
2023-12-20 20:54:12 +01:00
Mechiel Lukkien
e048d0962b
small fixes
a typo, using ongoing tx instead of making a new one, don't pass literal string
to formatting function.

found while working on quota support.
2023-12-16 11:53:14 +01:00
Mechiel Lukkien
dfddf0e874
for webapi requests, make canceled contexts a user instead of server error
no need to trigger alerts for user-initiated errors
2023-12-15 15:47:54 +01:00
Mechiel Lukkien
1be0cf485e
add more short-term todo's to the roadmap 2023-12-14 20:34:44 +01:00
Mechiel Lukkien
1abadc5499
add "warn" log level
now that we are using slog, which has them.
and we already could use them for a deprecation warning.
2023-12-14 20:26:06 +01:00
Mechiel Lukkien
41e3d1af10
imapserver: only send OLDNAME in LIST responses when IMAP4rev2 was enabled
OLDNAME is included in IMAP4rev2, but not in IMAP4rev1. it is also included in
the NOTIFY extension, but we don't implement that yet.

found by Damian Poddebniak with https://github.com/duesee/imap-flow, thanks!
2023-12-14 20:20:17 +01:00
Mechiel Lukkien
fbc18d522d
smtpserver: when writing slow responses, don't take so long the remote smtp client regards it as timeout
when writing the 4xx temporary error line, we were taking 1s in between each
byte. the total line could take longer than 30 seconds, which is the timeout we
use for reading a whole line (regardless of individual bytes). so mox as
deliverer was timing out to mox as slow rejecter. this causes slow writes to
not take longer than the 30s timeout: if we are 2s before the 30s, we write the
remainder in one go.

based on a debug log from naturalethic, thanks!
2023-12-14 20:20:17 +01:00
Mechiel Lukkien
2710a5b971
when generating Authentication-Results, put each method on a new line for better readability 2023-12-14 20:20:17 +01:00
Mechiel Lukkien
406fdc312d
when autocompleting, abort previous still pending request
should prevent a long list of "Autocompleting address" mentions in the status
bar at the top in case of non-responsive network
2023-12-14 20:20:17 +01:00
Mechiel Lukkien
22f46aa174
when logging version, also log go version and goos and goarch 2023-12-14 20:20:17 +01:00
Mechiel Lukkien
6d081f38fc
update to latest github.com/prometheus/common to drop dependency on github.com/golang/protobuf 2023-12-14 20:20:17 +01:00
Mechiel Lukkien
920b858da7
when logging, format timestamps more compactly, without needing quoting 2023-12-14 20:20:17 +01:00
Mechiel Lukkien
d1b66035a9
add more documentation, examples with tests to illustrate reusable components 2023-12-14 20:20:17 +01:00
Mechiel Lukkien
810cbdc61d
document that we keep some packages reusable 2023-12-14 20:20:12 +01:00
Mechiel Lukkien
19d1a8059b
smtpclient: expose entire tls connectionstate, not just whether tls was enabled
for moxtools
2023-12-14 15:39:47 +01:00
Mechiel Lukkien
f3a35a6766
keep track of the exposed api for reusable packages using apidiff 2023-12-14 15:39:47 +01:00
Mechiel Lukkien
72ac1fde29
expose fewer internals in packages, for easier software reuse
- prometheus is now behind an interface, they aren't dependencies for the
  reusable components anymore.
- some dependencies have been inverted: instead of packages importing a main
  package to get configuration, the main package now sets configuration in
  these packages. that means fewer internals are pulled in.
- some functions now have new parameters for values that were retrieved from
  package "mox-".
2023-12-14 15:39:36 +01:00
Mechiel Lukkien
fcaa504878
wrap long lines with many logging parameters to multiple lines
for improved readability
2023-12-14 13:45:52 +01:00
Mechiel Lukkien
5b20cba50a
switch to slog.Logger for logging, for easier reuse of packages by external software
we don't want external software to include internal details like mlog.
slog.Logger is/will be the standard.

we still have mlog for its helper functions, and its handler that logs in
concise logfmt used by mox.

packages that are not meant for reuse still pass around mlog.Log for
convenience.

we use golang.org/x/exp/slog because we also support the previous Go toolchain
version. with the next Go release, we'll switch to the builtin slog.
2023-12-14 13:45:52 +01:00
Mechiel Lukkien
56b2a9d980
help user run "mox localserve" using docker
based on feedback from damian poddebniak
2023-12-11 15:56:29 +01:00
Mechiel Lukkien
af5da17623
smtpserver: also allow space after "MAIL FROM:" and "RCPT TO:" command for SMTP delivery (unless in pedantic mode)
we already allowed it for (authenticated) SMTP submission. it turns out also
legitimate senders can use this invalid syntax to deliver messages.

for issue #101 by Fell, thanks for reporting & explaining!
2023-12-11 15:34:11 +01:00
Mechiel Lukkien
02eb7b5033
bugfix: imapserver "append" command: properly account for message size when bare newlines ("\n") are converted to crlf ("\r\n")
the original size, with bare newlines, was stored in the database, not the
actual adjusted file size. this caused failures when reading the message.

users may want to run "mox fixmsgsize <account>" if they imported messages from
another account over IMAP.

reported by daftaupe, thanks!
2023-12-11 15:18:06 +01:00
Mechiel Lukkien
7c1879da82
webmail: when replying to message we sent, don't compose the reply to ourselve, but copy the original to/cc/bcc headers 2023-11-27 12:26:31 +01:00
Mechiel Lukkien
fb81effe45
webmail: for domain in From address, show if domain is dmarc(-like) validated
i'm not sure this is good enough.
this is based on field MsgFromValidation, but it doesn't hold the full DMARC information.
we also don't know mailing list-status for all historic messages.
so the red underline can occur too often.
2023-11-27 12:11:05 +01:00
Mechiel Lukkien
2ff87a0f9c
more strict junk checks for some first-time senders: when TLS isn't used and when recipient address isn't in To/Cc header
both cases are quite typical for spammers, and not for legitimate senders.
this doesn't apply to known senders. and it only requires that the content look
more like ham instead of spam. so legitimate mail can still get through with
these properties.
2023-11-27 10:34:01 +01:00
Mechiel Lukkien
8e37fadc13
webmail: in initial start (sse) event, send the version, and ask user to reload if it changes
will prevent showing errors to users about new unknown fields that may be added
in the new version.
2023-11-27 08:06:27 +01:00
Mechiel Lukkien
416113af72
webmail: do not automatically mark read messages in Rejects mailbox as nonjunk 2023-11-27 07:34:18 +01:00
Mechiel Lukkien
9d2e761494
turns out the esearch tag is a string before imap4rev2, so can't blame new outlook 2023-11-22 22:01:23 +01:00
Mechiel Lukkien
2ae121e400
work around bug in microsoft outlook "new", which fails when the tag in an esearch response doesn't have quotes 2023-11-22 21:51:04 +01:00
Mechiel Lukkien
91b7d3dda8
implement the obsolete sasl login mechanism for smtp
so microsoft outlook "new" can login. that's the "new" email client that logs
in from cloud servers.
2023-11-22 21:44:55 +01:00
Mechiel Lukkien
c66fa64b8b
wrap long dkim dns records at 100 characters instead of 255 for better display (no line-wrap) 2023-11-22 14:02:24 +01:00
Mechiel Lukkien
361bc2b516
when accepting an incoming message, turn any bare newlines (without carriage return) into crlf
because that is what most of the code expects. we could work around having bare
lf, but it would complicate too much code.

currently, a message with bare lf is accepted (in smtpserver delivery,
imapserver append, etc), but when an imap session would try to fetch parsed
parts, that would fail because and even cause a imapserver panic (closing the
connection).

in message imports we would already convert bare lf to crlf (because it is
expected those messages are all lf-only-ending).

we store messages with crlf-ending instead of lf-ending so the imapserver has
all correct information at hand (line counts, byte counts).

found by using emclient with mox. it adds a message to the inbox that can have
mixed crlf and bare lf line endings in a few header fields (in some
localization, emclient authors explained how that happened, thanks!).  we can
now convert those lines and read those messages over imap. emclient already
switched to all-crlf line endings in newer (development) versions.
2023-11-21 13:19:54 +01:00
Mechiel Lukkien
3d80c05423
webmail: for long to/cc/bcc address list (>5) show the first 4 and a button to show the rest
for issue #98 by mattfbacon, thanks
2023-11-20 21:36:40 +01:00
Mechiel Lukkien
73a2a09711
better handling of outgoing tls reports to recipient domains vs hosts
based on discussion on uta mailing list. it seems the intention of the tlsrpt
is to only send reports to recipient domains. but i was able to interpret the
tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it
makes sense given how DANE works per MX host, not recipient domain). this
change makes the behaviour of outgoing reports to recipient domains work more
in line with expectations most folks may have about tls reporting (i.e. also
include per-mx host tlsa policies in the report). this also keeps reports to mx
hosts working, and makes them more useful by including the recipient domains of
affected deliveries.
2023-11-20 11:31:46 +01:00
Mechiel Lukkien
e5f77a0411
update to latest bstore, with fix for a bug that was triggered by an upcoming commit 2023-11-20 11:01:15 +01:00
Mechiel Lukkien
bdd8fa078e
rfc/xr: tweak, committed previous too soon... 2023-11-14 14:21:02 +01:00
Mechiel Lukkien
5b62013f27
rfc/xr: be more careful about which urls we load in iframes
anything that looks like it specifies a different host should not be loaded.
www.xmox.nl also has a CSP policy that should prevent resources from other
domains from being loaded.
2023-11-14 14:09:35 +01:00
Mechiel Lukkien
51e314f65a
for external domains (for which we only accept external dmarc reports), don't try to fetch tls certificates at startup for autoconfig host 2023-11-14 00:26:18 +01:00
Mechiel Lukkien
651fa68067
webadmin: in list with dmarc evaluations, add the dispositions applied
to easily spot rejects
2023-11-13 14:44:40 +01:00
Mechiel Lukkien
bcb80c3598
tweaks to cross-referenced code/rfc html pages
- show commit hash, with a link to the commit
- highlight if this is the dev or released version page
- sort the rfc's, the list in rfc/index.txt has the major rfc's at the topic, but this nuance is lost in the html page
2023-11-13 14:12:40 +01:00
Mechiel Lukkien
e24e1bee19
add suppression list for outgoing dmarc and tls reports
for reporting addresses that cause DSNs to be returned. that just adds noise.
the admin can add/remove/extend addresses through the webadmin.

in the future, we could send reports with a smtp mail from of
"postmaster+<signed-encoded-recipient>@...", and add the reporting recipient
on the suppression list automatically when a DSN comes in on that address, but
for now this will probably do.
2023-11-13 13:48:52 +01:00
Mechiel Lukkien
6ce69d5425
in starttls command in smtp & imap server, add the cid in the "ok, go ahead with tls" response
so facilitate debugging. a remote client that logs details about failing
connections can give the cid to the mox operator to find the relevant logging.
2023-11-13 10:26:31 +01:00