tls servers send a list of certificates for the connection. the first is the
leaf certificate. that's the one for the server itself. that's the one we want
to verify. the others are intermediate CA's. and possibly even the root CA
certificate that it hopes is trusted at the client (though sending it doesn't
make it trusted). with dane-ta, the public key of an intermediate or root CA
certificate is listed in the TSLA record. when verifying, we add any
intermediate/root CA that matches a dane-ta tlsa record to the trusted root CA
certs. we should also have added CA certs that didn't match a TLSA record to
the "intermediates" of x509.VerifyOptions. because we didn't,
x509.Certificate.Verify couldn't verify the chain from the trusted dane-ta ca
cert to the leaf cert. we would only properly verify a dane-ta connection
correctly if the dane-ta-trusted ca cert was the one immediately following the
leaf cert. not when there were one or more additional intermediate certs.
this showed when connecting to mx.runbox.com.
problem reported by robbo5000 on matrix, thanks!
quite some output is printed. you could remember to tee it all to a file. but
that's probably often realized only after having run the quickstart. you can
also copy/paste it all from the terminal, but that's sometimes annoying to do.
writing to a file is more helpful to users.
this has been requested a few times in the past on irc/matrix (i forgot who).
after seeing some junk messages pass the filter, i investigated word counts in
junkfilter.db. i had seen suspicious counts that were just around powers of
two. did not make sense at the time. more investigating makes it clear: instead
of setting new word counts when updating the junk filter, we were adding the
new value to the current value (instead of just setting the new value). so the
counts got approximately doubled when being updated.
users should retrain the junk filter after this update using the "retrain"
subcommand.
this also adds logging for the hypothetical case where numbers would get
decreased below zero (which would wrap around due to uints).
and this fixes junk filter tests that were passing wrong parameters to
train/untrain...
the types in webhook should be subjected to apidiff'ing, this was a shared
function. it is better off in package queue. also change the apidiff script so
it leaves apidiff/next.txt empty when there aren't any changes. makes it easier
to rotate the files after releases where nothing changed (a common occurrence).
these settings are applied anywhere the webmail is open. the settings are for
showing keyboard shortcuts in the lower right after a mouse interaction, and
showing additional headers. the shorcuts were configurable in the "help" popup
before. the additional headers were only configurable through the developer
console before.
the "mailto:" (un)register buttons are now in the settings popup too.
ctrl-l is commonly "focus on browser address bar".
ctrl-u is commonly "view source".
ctrl-I (shift i) is commonly "open developer console".
these keys are more useful to leave for the browser. ctrl-l and ctrl-u (moving
to a message without opening it) can still be had by using also pressing shift.
the previous ctrl-shift-i (show all headers) is now just ctrl-i.
this has been requested in the past on irc/matrix (i forgot who).
the field is optional. if absent, the default behaviour is currently to disable
session tickets. users can set the option if they want to try if delivery from
microsoft is working again. in a future version, we can switch the default to
enabling session tickets.
the previous fix was to disable session tickets for all tls connections,
including https. that was a bit much.
for issue #237
for each message part. The ContentDisposition value is the base value without
header key/value parameters. the Filename field is the likely filename of the
part. the different email clients encode filenames differently. there is a
standard mime mechanism from rfc 2231. and there is the q/b-word-encoding from
rfc 2047. instead of letting users of the webhook api deal with those
differences, we provide just the parsed filename.
for issue #258 by morki, thanks for reporting!
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
needed for upcoming changes, where (now) package admin needs to import package
store. before, because package store imports mox- (for accessing the active
config), that would lead to a cyclic import. package mox- keeps its active
config, package admin has the higher-level config-changing functions.
this allows removing some ugly instantiations of an rng based on the current
time.
Intn is now IntN for our concurrency-safe prng wrapper to match the randv2 api.
v2 exists since go1.22, which we already require.
and add an alerting rule if the failure rate becomes >10% (e.g. expired
certificate).
the prometheus metrics includes a reason, including potential tls alerts, if
remote smtp clients would send those (openssl s_client -starttls does).
inspired by issue #237, where incoming connections were aborted by remote. such
errors would show up as "eof" in the metrics.
if files {webmail,webaccount,webadmin}.{css,js} exist in the configdir (where
the mox.conf file lives), their contents are included in the web apps.
the webmail now uses css variables, mostly for colors. so you can write a
custom webmail.css that changes the variables, e.g.:
:root {
--color: blue
}
you can also look at css class names and override their styles.
in the future, we may want to make some css variables configurable in the
per-user settings in the webmail. should reduce the number of variables first.
any custom javascript is loaded first. if it defines a global function
"moxBeforeDisplay", that is called each time a page loads (after
authentication) with the DOM element of the page content as parameter. the
webmail is a single persistent page. this can be used to make some changes to
the DOM, e.g. inserting some elements. we'll have to see how well this works in
practice. perhaps some patterns emerge (e.g. adding a logo), and we can make
those use-cases easier to achieve.
helps partially with issue #114, and based on questions from laura-lilly on
matrix.
the pointer events for moving the mouse would be consumed by the iframe. that
broke resizing of the compose popup. we now disable pointerevents on the main
ui when we are dragging the corner of the compose popup.
this is similar to an earlier change about the draggable split bar between the
message list and the message view (when showing an html message).
we kept the "save draft" promise, and would wait for it again for other
operations (eg close, save again, send), which wouldn't make progress.
can easily be reproduced by saving a message with a control character in an
address or the subject. saving the draft will fail.
for issue #256 by ally9335, thanks for reporting
since ios18, downloaded files don't go immediately to the settings (which is
somewhat understandable given potential for abuse), but go to the Files app.
opening them in the Files app then adds them to the settings where they can be
installed.
This PR adds support for negotiating IMAP, SMTP & HTTP on the configured HTTPS port using TLS ALPN. This is intended to be useful for deploying Mox as a chatmail server. The upstream implementation of chatmail servers uses `imap` and `smtp` as the “next protocol” values for IMAP and SMTP, respectively: https://github.com/deltachat/chatmail/blob/main/cmdeploy/src/cmdeploy/nginx/nginx.conf.j2#L16-L17
To test, configure Mox as is standard for the `mox localserve` mode, but add this block under `Listeners.local`:
```sconf
AutoconfigHTTPS:
Enabled: true
Port: 1443
```
Then run Mox and use OpenSSL’s s_client mode to connect to the local instance:
```
> openssl s_client -quiet -connect localhost:1443 -alpn smtp
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
220 localhost ESMTP mox 7f5e1087d42d5d7bf57649c5c6a2155c3706ce39+modifications
HELO test
550 5.5.0 your ehlo domain does not resolve to an IP address (htqp11_GJOmHEhSs_Y03eg)
QUIT
221 2.0.0 okay thanks bye
^C
> openssl s_client -quiet -crlf -connect localhost:1443 -alpn imap
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
* OK [CAPABILITY IMAP4rev2 IMAP4rev1 ENABLE LITERAL+ IDLE SASL-IR BINARY UNSELECT UIDPLUS ESEARCH SEARCHRES MOVE UTF8=ACCEPT LIST-EXTENDED SPECIAL-USE LIST-STATUS AUTH=SCRAM-SHA-256-PLUS AUTH=SCRAM-SHA-256 AUTH=SCRAM-SHA-1-PLUS AUTH=SCRAM-SHA-1 AUTH=CRAM-MD5 ID APPENDLIMIT=9223372036854775807 CONDSTORE QRESYNC STATUS=SIZE QUOTA QUOTA=RES-STORAGE AUTH=PLAIN] mox imap
c1 STARTTLS
c1 BAD STARTTLS unrecognized syntax/command: tls already active
^C
> openssl s_client -quiet -crlf -connect localhost:1443
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
GET / HTTP/1.1
Host: localhost
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Mon, 25 Nov 2024 07:28:00 GMT
Content-Length: 19
404 page not found
^C
```
As part of this change, I had to `go get golang.org/x/net/http2`. (The default `http.Server` supports HTTP2 if you leave it alone, but that built-in support is deactivated if I set `TLSNextProto` to a non-nil value. To ensure that Mox continues to support HTTP2, [the Go documentation](https://pkg.go.dev/net/http@go1.23.3#hdr-HTTP_2) directs folks with "…more complex configurations…" to import x/net/http2.) Go decided that it also wanted to update a bunch of other dependencies while it was at it. This has caused the large number of dependency updates. I can revert these and attempt a more surgical addition of the http2 library, if you’d like.
There are also two major deficiencies in this code that I’d like advice on correcting:
1. Right now, this ALPN feature is enabled when Mox is configured to provide Autoconfigure services to mail clients. I chose to do this because it was relatively straightforward to implement. However, the resulting behavior is extremely non-obvious. **How would you recommend exposing the ALPN feature in the configuration?**
2. I’m not sure what the best way to expose the private `serve()` functions in `imapserver` and `smtpserver` are. The current implementation creates a public function called `ServeConn()` in each module that just calls the private `serve()` function with all the same arguments, but this feels redundant. **Would you recommend making the `serve()` functions public, using public wrappers but with a more limited set of parameters, or something else entirely?**
Thanks!
if admin has disabled mta-sts for a domain, we still check for records &
policies, but won't mark it as error when they don't exist. we do now keep
warning that mta-sts isn't enabled, otherwise we would start showing a green
"ok".
this also fixes the mta-sts code returning ErrNoPolicy when mtasts.<domain>
doesn't exist. the dns lookup is done with the reguler "net" package dns lookup
code, not through adns, so we look for two types of DNSError's.
noticed a while ago when testing with MTA-STS while debugging TLS connection
issues with MS.
