the previous default, marking the messages as junk had the interesting effect
of training the junk filter. rejecting could have been the result of the
sending IP being in the DNSBL. so the DNSBL helped to automatically train the
junk filter. perhaps we can keep that in the future and just not take messages
from the rejects mailbox into account when evaluating the reputation for
incoming deliveries.
before, we used heuristics to decide when to train/untrain a message as junk or
nonjunk: the message had to be seen, be in certain mailboxes. then if a message
was marked as junk, it was junk. and otherwise it was nonjunk. this wasn't good
enough: you may want to keep some messages around as neither junk or nonjunk.
and that wasn't possible.
ideally, we would just look at the imap $Junk and $NotJunk flags. the problem
is that mail clients don't set these flags, or don't make it easy. thunderbird
can set the flags based on its own bayesian filter. it has a shortcut for
marking Junk and moving it to the junk folder (good), but the counterpart of
notjunk only marks a message as notjunk without showing in the UI that it was
marked as notjunk. there is also no "move and mark as notjunk" mechanism. e.g.
"archive" does not mark a message as notjunk. ios mail and mutt don't appear to
have any way to see or change the $Junk and $NotJunk flags.
what email clients do have is the ability to move messages to other
mailboxes/folders. so mox now has a mechanism that allows you to configure
mailboxes that automatically set $Junk or $NotJunk (or clear both) when a
message is moved/copied/delivered to that folder. e.g. a mailbox called junk or
spam or rejects marks its messags as junk. inbox, postmaster, dmarc, tlsrpt,
neutral* mark their messages as neither junk or notjunk. other folders mark
their messages as notjunk. e.g. list/*, archive. this functionality is
optional, but enabled with the quickstart and for new accounts.
also, mox now keeps track of the previous training of a message and will only
untrain/train if needed. before, there probably have been duplicate or missing
(un)trainings.
this also includes a new subcommand "retrain" to recreate the junkfilter for an
account. you should run it after updating to this version. and you should
probably also modify your account config to include the AutomaticJunkFlags.
similar to greylisting, but not quite the same: with greylisting you would
always reject the first delivery attempt with a temporary failure. with the
hope that spammers won't retry their deliveries. the spams i've been receiving
seem to be quite consistent though. and we would keep rejecting them anyway.
we slow down the spammy connections to waste some of the resources of a
spammer. this may slow their campaigns down a bit, leaving a bit more time to
take measures.
we do the same with connections that have their 3rd authentication failure,
typically password guess attempts.
when we accept a message by a first-time sender, we sleep for 15 seconds before
actually delivering them. known-good senders don't have to wait. if the message
turns out to be a spammer, at least we've consumed one of their connections,
and they cannot deliver at too high a rate to us because of the max open
connection limit.
limiting is done based on remote ip's, with 3 ip mask variants to limit networks
of machines. often with two windows, enabling short bursts of activity, but not
sustained high activity. currently only for imap and smtp, not yet http.
limits are currently based on:
- number of open connections
- connection rate
- limits after authentication failures. too many failures, and new connections will be dropped.
- rate of delivery in total number of messages
- rate of delivery in total size of messages
the limits on connections and authentication failures are in-memory. the limits
on delivery of messages are based on stored messages.
the limits themselves are not yet configurable, let's use this first.
in the future, we may also want to have stricter limits for senders without any
reputation.
and change thunderbird autoconfiguration to use it.
unfortunately, for microsoft autodiscover, there appears to be no way to
request secure password negotiation. so it will default to plain text auth.
cram-md5 is less secure than scram-sha-*, but thunderbird does not yet support
scram auth. it currently chooses "plain", sending the literal password over the
connection (which is TLS-protected, but we don't want to receive clear text
passwords). in short, cram-md5 is better than nothing...
for cram-md5 to work, a new set of derived credentials need to be stored in the
database. so you need to save your password again to make it work. this was
also the case with the scram-sha-1 addition, but i forgot to mention it then.
the idea is that clients may not support SCRAM-SHA-256, but may support
SCRAM-SHA-1. if they do support the 256 variant, they'll use it.
unfortunately, thunderbird does not support scram-sha-1 either.
named "traceauth" and "tracedata".
with this, you can (almost) enable trace logging without fear of logging
sensitive data or ddos'ing your log server.
the caveat is that the imap login command has already printed the line as
regular trace before we can decide it should not be. can be fixed soon.
