the http basic auth we had was very simple to reason about, and to implement.
but it has a major downside:
there is no way to logout, browsers keep sending credentials. ideally, browsers
themselves would show a button to stop sending credentials.
a related downside: the http auth mechanism doesn't indicate for which server
paths the credentials are.
another downside: the original password is sent to the server with each
request. though sending original passwords to web servers seems to be
considered normal.
our new approach uses session cookies, along with csrf values when we can. the
sessions are server-side managed, automatically extended on each use. this
makes it easy to invalidate sessions and keeps the frontend simpler (than with
long- vs short-term sessions and refreshing). the cookies are httponly,
samesite=strict, scoped to the path of the web interface. cookies are set
"secure" when set over https. the cookie is set by a successful call to Login.
a call to Logout invalidates a session. changing a password invalidates all
sessions for a user, but keeps the session with which the password was changed
alive. the csrf value is also random, and associated with the session cookie.
the csrf must be sent as header for api calls, or as parameter for direct form
posts (where we cannot set a custom header). rest-like calls made directly by
the browser, e.g. for images, don't have a csrf protection. the csrf value is
returned by the Login api call and stored in localstorage.
api calls without credentials return code "user:noAuth", and with bad
credentials return "user:badAuth". the api client recognizes this and triggers
a login. after a login, all auth-failed api calls are automatically retried.
only for "user:badAuth" is an error message displayed in the login form (e.g.
session expired).
in an ideal world, browsers would take care of most session management. a
server would indicate authentication is needed (like http basic auth), and the
browsers uses trusted ui to request credentials for the server & path. the
browser could use safer mechanism than sending original passwords to the
server, such as scram, along with a standard way to create sessions. for now,
web developers have to do authentication themselves: from showing the login
prompt, ensuring the right session/csrf cookies/localstorage/headers/etc are
sent with each request.
webauthn is a newer way to do authentication, perhaps we'll implement it in the
future. though hardware tokens aren't an attractive option for many users, and
it may be overkill as long as we still do old-fashioned authentication in smtp
& imap where passwords can be sent to the server.
for issue #58
mox was already strict in its "\r\n.\r\n" handling for end-of-message in an
smtp transaction.
due to a mostly unrelated bug, sequences of "\nX\n", including "\n.\n" were
rejected with a "local processing error".
the sequence "\r\n.\n" dropped the dot, not necessarily a big problem, this is
unlikely to happen in a legimate transaction and the behaviour not
unreasonable.
we take this opportunity to reject all bare \r. we detect all slightly
incorrect combinations of "\r\n.\r\n" with an error mentioning smtp smuggling,
in part to appease the tools checking for it.
smtp errors are 500 "bad syntax", and mention smtp smuggling.
the imapserver started with imap4rev2-only and utf8=only. to prevent potential
issues with imaputf7, which makes "&" special, we refused any mailbox with an
"&" in the name. we already tried decoding utf7, falling back to using a
mailbox name verbatim. that behaviour wasn't great. we now treat the enabled
extensions IMAP4rev2 and/or UTF8=ACCEPT as indication whether mailbox names are
in imaputf7. if they are, the encoding must be correct.
we now also send mailbox names in imaputf7 when imap4rev2/utf8=accept isn't
enabled.
and we now allow "*" and "%" (wildcard characters for matching) in mailbox
names. not ideal for IMAP LIST with patterns, but not enough reason to refuse
them in mailbox names. people that migrate may run into this, possibly as
blocker.
we also allow "#" in mailbox names, but not as first character, to prevent
potential clashes with IMAP namespaces in the future.
based on report from Damian Poddebniak using
https://github.com/duesee/imap-flow and issue #110, thanks for reporting!
all ui frontend code is now in typescript. we no longer need jshint, and we
build the frontend code during "make build".
this also changes tlsrpt types for a Report, not encoding field names with
dashes, but to keep them valid identifiers in javascript. this makes it more
conveniently to work with in the frontend, and works around a sherpats
limitation.
the autoconfig/autodiscover endpoints, and the printed client settings (in
quickstart, in the admin interface) now all point to the cname record (called
"client settings domain"). it is configurable per domain, and set to
"mail.<domain>" by default. for existing mox installs, the domain can be added
by editing the config file.
this makes it easier for a domain to migrate to another server in the future.
client settings don't have to be updated, the cname can just be changed.
before, the hostname of the mail server was configured in email clients.
migrating away would require changing settings in all clients.
if a client settings domain is configured, a TLS certificate for the name will
be requested through ACME, or must be configured manually.
to get the security benefits (detecting mitm attempts), explicitly configure
clients to use a scram plus variant, e.g. scram-sha-256-plus. unfortunately,
not many clients support it yet.
imapserver scram plus support seems to work with the latest imtest (imap test
client) from cyrus-sasl. no success yet with mutt (with gsasl) though.
should prevent potential mitm attacks. especially when done close to the
machine itself (where a http/tls challenge is intercepted to get a valid
certificate), as seen on the internet last month.
so a single user cannot fill up the disk.
by default, there is (still) no limit. a default can be set in the config file
for all accounts, and a per-account max size can be set that would override any
global setting.
this does not take into account disk usage of the index database. and also not
of any file system overhead.
OLDNAME is included in IMAP4rev2, but not in IMAP4rev1. it is also included in
the NOTIFY extension, but we don't implement that yet.
found by Damian Poddebniak with https://github.com/duesee/imap-flow, thanks!
when writing the 4xx temporary error line, we were taking 1s in between each
byte. the total line could take longer than 30 seconds, which is the timeout we
use for reading a whole line (regardless of individual bytes). so mox as
deliverer was timing out to mox as slow rejecter. this causes slow writes to
not take longer than the 30s timeout: if we are 2s before the 30s, we write the
remainder in one go.
based on a debug log from naturalethic, thanks!
- prometheus is now behind an interface, they aren't dependencies for the
reusable components anymore.
- some dependencies have been inverted: instead of packages importing a main
package to get configuration, the main package now sets configuration in
these packages. that means fewer internals are pulled in.
- some functions now have new parameters for values that were retrieved from
package "mox-".
we don't want external software to include internal details like mlog.
slog.Logger is/will be the standard.
we still have mlog for its helper functions, and its handler that logs in
concise logfmt used by mox.
packages that are not meant for reuse still pass around mlog.Log for
convenience.
we use golang.org/x/exp/slog because we also support the previous Go toolchain
version. with the next Go release, we'll switch to the builtin slog.
we already allowed it for (authenticated) SMTP submission. it turns out also
legitimate senders can use this invalid syntax to deliver messages.
for issue #101 by Fell, thanks for reporting & explaining!
the original size, with bare newlines, was stored in the database, not the
actual adjusted file size. this caused failures when reading the message.
users may want to run "mox fixmsgsize <account>" if they imported messages from
another account over IMAP.
reported by daftaupe, thanks!
i'm not sure this is good enough.
this is based on field MsgFromValidation, but it doesn't hold the full DMARC information.
we also don't know mailing list-status for all historic messages.
so the red underline can occur too often.
both cases are quite typical for spammers, and not for legitimate senders.
this doesn't apply to known senders. and it only requires that the content look
more like ham instead of spam. so legitimate mail can still get through with
these properties.
