Commit graph

744 commits

Author SHA1 Message Date
Mechiel Lukkien
8e6fe7459b
normalize localparts with unicode nfc when parsing
both when parsing our configs, and for incoming on smtp or in messages.
so we properly compare things like é and e+accent as equal, and accept the
different encodings of that same address.
2024-03-08 21:08:40 +01:00
Mechiel Lukkien
4fbd7abb57
update to latest adns, synced with Go's net 2024-03-08 15:31:54 +01:00
Mechiel Lukkien
a00b0ba6cd
add note about testing localserve on various OSes before release 2024-03-08 15:31:34 +01:00
Mechiel Lukkien
372585de72
build before running test-upgrade 2024-03-08 09:28:39 +01:00
Mechiel Lukkien
03e220c749
update dependencies 2024-03-08 09:28:09 +01:00
Mechiel Lukkien
a9f11b8fa3
fix changing domains.conf through admin with new MonitorDNSBLs present
by not clearing the existing derived info, we would detect duplicate domains
and refuse the changed config.
2024-03-07 11:26:53 +01:00
Mechiel Lukkien
df105a028c
unbreak enforcing dane since previous commits
by using the correct variable.
should have automated tests for this.
found it by manual test through email-security-scans.org, useful service!
2024-03-07 11:19:08 +01:00
Mechiel Lukkien
484ffa67d1
fix new reference to smtp limits rfc 2024-03-07 10:56:58 +01:00
Mechiel Lukkien
85f72582c6
mention matrix channel, add moxtools to things to check for a release 2024-03-07 10:51:48 +01:00
Mechiel Lukkien
b541646275
be more helpful about instructions for installing unbound and dnssec
by mentioning the dnssec root keys, mentioning which unbound version has EDE,
giving a "dig" invocation to check for dnssec results.

based on issue #131 by romner-set, thanks for reporting
2024-03-07 10:47:48 +01:00
Mechiel Lukkien
4db1f5593c
better check for dnssec-verifying resolver
check the authentic data bit for the NS records of "com.", not for ".": some
dnssec-verifying resolvers return unauthentic data for ".".

for issue #139 by triatic, thanks!
2024-03-07 10:34:13 +01:00
Mechiel Lukkien
9e7d6b85b7
queue: deliver to multiple recipients in a single smtp transaction
transferring the data only once. we only do this when the recipient domains
are the same. when queuing, we now take care to set the same NextAttempt
timestamp, so queued messages are actually eligable for combined delivery.

this adds a DeliverMultiple to the smtp client. for pipelined requests, it will
send all RCPT TO (and MAIL and DATA) in one go, and handles the various
responses and error conditions, returning either an overal error, or per
recipient smtp responses. the results of the smtp LIMITS extension are also
available in the smtp client now.

this also takes the "LIMITS RCPTMAX" smtp extension into account: if the server
only accepts a single recipient, we won't send multiple.
if a server doesn't announce a RCPTMAX limit, but still has one (like mox does
for non-spf-verified transactions), we'll recognize code 452 and 552 (for
historic reasons) as temporary error, and try again in a separate transaction
immediately after. we don't yet implement "LIMITS MAILMAX", doesn't seem likely
in practice.
2024-03-07 10:07:53 +01:00
Mechiel Lukkien
8550a5af45
don't expose functions on the prng that aren't mutex-protected
the current Intn calls in queue could be called concurrently, found by the race
detector with upcoming new tests.  best to just prevent any possible concurrent
access.
2024-03-07 10:05:35 +01:00
Mechiel Lukkien
47ebfa8152
queue: implement adding a message to the queue that gets sent to multiple recipients
and in a way that allows us to send that message to multiple recipients in a
single smtp transaction.
2024-03-05 20:10:28 +01:00
Mechiel Lukkien
15e450df61
implement only monitoring dns blocklists, without using them for incoming deliveries
so you can still know when someone has put you on their blocklist (which may
affect delivery), without using them.

also query dnsbls for our ips more often when we do more outgoing connections
for delivery: once every 100 messages, but at least 5 mins and at most 3 hours
since the previous check.
2024-03-05 19:37:48 +01:00
Mechiel Lukkien
e0c36edb8f
accept tls reports with both host & recipient domains, and with multiple recipient domains
embarrassingly, we didn't accept all reports we generated. after the changed
handling of reports about mx/mail host vs recipient domains, would send reports
to mail hosts about multiple recipient domains + the mail host. and we included
a policy domain of the mail host when sending to a recipient domain. we were
still being strict in what we accepted: only a single domain in total in the
entire report, and we still enforced that a report sent to the mx host tlsrpt
address only contained the mx host as policy domain. and likewise for recipient
domains and their tls reporting addresses. those checks would reject reports
generated by a mox instance. this probably only happens with dane configured,
probably most users haven't seen it because of that.

somewhat related to issue #125
2024-03-05 11:43:49 +01:00
Mechiel Lukkien
a9cb6f9d0a
webadmin: add single-line form for looking up a cid for a received id 2024-03-05 10:50:56 +01:00
Mechiel Lukkien
5738d9e7b8
when auth fails due to missing derived secrets, don't hold it against connection
smtp & imap can only indicate which mechanisms the server software supports.
individual accounts may not have derived secrets for all those mechanisms. imap
& smtp cannot indicate that a client should try another (specific) mechanism.
but at least we shouldn't slow the connection down due to failed auth attempts
in that case.

heard from ben that this is a common source for trouble when setting up email
accounts.
2024-03-05 10:40:40 +01:00
Mechiel Lukkien
caa4931d35
tweak faq about email being rejected 2024-03-05 09:41:44 +01:00
Mechiel Lukkien
af968f7614
webmail: for junk/rejects messages, show sender address instead of name in list 2024-03-05 09:04:59 +01:00
Mechiel Lukkien
79f91ebd87
webmail: don't switch back focus after autocompleting address
actually, this fix can reduce focus changes for more operations. withStatus is
often used to show an operation in progress in the status bar, only when the
operation isn't done within 1 second. we would restore focus to the element
before the operation started. that was done because we disable elements
sometimes (preventing duplicate form submission). for things like the
autocomplete, with the tab key, which also moves focus to the next element, we
don't want that focus switched back again.
2024-03-05 08:46:56 +01:00
Mechiel Lukkien
63c3c1fd6a
webmail: leave out own address in reply all when we have addresses remaining 2024-03-04 20:21:41 +01:00
Mechiel Lukkien
26ff0c9417
increase memory limit during tests for upgrade 2024-03-04 19:11:53 +01:00
Mechiel Lukkien
13923e4b7b
better thread matching for dsns
keep track of whether a message is a dsn, and match dsn's against their sent
message by ignoring the message subject.
2024-03-04 16:40:27 +01:00
Mechiel Lukkien
f6497b1aaf
when parsing a dsn, actually set the Action field
noticed when writing dsn-processing code
2024-02-21 21:19:52 +01:00
Mechiel Lukkien
79da4faaa1
add Delivered-To header when locally delivering a DSN
so tools can pick it up and find the original "MAIL FROM", and take the encode
destination address or message id from its localpart.
2024-02-20 16:39:49 +01:00
Mechiel Lukkien
1c934f0103
improve dsn handling
have the full smtp reply in the Diagnostic-Code field, not something that
resembles it but isn't quite the same.

include any additional error message in the Status field as comment.

before, we ended up having an Diagnostic-Code that didn't include the original
smtp code. it only had the enhanced error code.
2024-02-20 16:31:15 +01:00
Mechiel Lukkien
dc83ad1df5
set correct local account when adding a message to the queue
all dsns were going to the postmaster account...
2024-02-20 15:02:47 +01:00
Mechiel Lukkien
cb5097714b
add a few more rfc 2024-02-20 14:58:16 +01:00
Mechiel Lukkien
37de8de1c5
fix incorrect error about bare cr/lf when sending a message over smtp
we weren't properly tracking the cr's and lf's when being strict about message
lines when sending data.

we are reading buffered data from a Reader. if that chunk happens to start with
a newline, we weren't looking at the previously written data, which could be a
cr. instead, in that case, we would always claim the cr/lf wasn't correct.

the new test case triggered the behaviour before having the fix.

should solve issue #129 by x8x, thanks for the report!
2024-02-16 20:20:58 +01:00
Mechiel Lukkien
fd359d5973
add to previous commit, adding multiline smtp responses in dsn
also include api change.
2024-02-16 20:13:05 +01:00
Mechiel Lukkien
50c13965a7
include full smtp response in dsn on errors
we now keep track of the full smtp error responses, potentially multi-line. and
we include it in a dsn in the first free-form human-readable text.

it can have multiple lines in practice, e.g. when a destination mail server
tries to be helpful in explaining what the problem is.
2024-02-14 23:37:43 +01:00
Mechiel Lukkien
39bfa4338a
smtpclient: only obey SIZE= of server if it isn't 0
since that means there is no explicit limit.
2024-02-14 17:46:01 +01:00
Mechiel Lukkien
8046b323fb
fix and ensure consistent lines 2024-02-14 17:43:21 +01:00
Mechiel Lukkien
67300969c1
don't use bash if not needed
from mteege
2024-02-11 21:46:45 +01:00
Mechiel Lukkien
93c52b01a0
implement "future release"
the smtp extension, rfc 4865.
also implement in the webmail.
the queueing/delivery part hardly required changes: we just set the first
delivery time in the future instead of immediately.

still have to find the first client that implements it.
2024-02-10 17:55:56 +01:00
Mechiel Lukkien
17734196e3
add rfc 9078, "Reaction: Indicating Summary Reaction to a Message" to the list
about emoji responses to messages.

no concrete plans (lack of time), but would be fun to experiment with in the
webmail.
2024-02-10 12:14:36 +01:00
Mechiel Lukkien
49c8dbf47e
add FAQ about directly accessing mailboxes through the file system
commonly asked, again at fosdem.
2024-02-10 11:39:31 +01:00
Mechiel Lukkien
ee1db2dde7
webmail: implement registering and handling "mailto:" links
to start composing a message.

the help popup now has a button to register the "mailto:" links with the mox
webmail (typically only works over https, not all browsers support it).

the mailto links are specified in 6068. we support the to/cc/bcc/subject/body
parameters. other parameters should be seen as custom headers, but we don't
support messages with custom headers at all at the moment, so we ignore them.

we now also turn text of the form "mailto:user@host" into a clickable link
(will not be too common). we could be recognizing any "x@x.x" as email address
and make them clickable in the future.

thanks to Hans-Jörg for explaining this functionality.
2024-02-09 11:21:33 +01:00
Mechiel Lukkien
f3bf348214
webmail: show unicode for internationalized email addresses by default
before, we showed the xn-- ascii names, along with the unicode name. but users
of internationalized email don't want to see any xn-- names. we now put those
in an html title attribute for some cases, so you can still see them if you
really want to, by hovering.

after talking to arnt at fosdem.
2024-02-08 18:03:48 +01:00
Mechiel Lukkien
39f4800290
xr: unbreak following links, they were now being opened in a new window
broken in previous update. the tricky part keeps being about when browsers fire
'load' and 'hashchange' events for the outer and two inner documents. the
previous change attempted to prevent a history item being set on the first
load. that behaviour seems to be kept.
2024-02-08 16:25:33 +01:00
Mechiel Lukkien
4ea9e9e978
run more of go vet on the special-purpose tools
tools that are behind build constraints
2024-02-08 15:12:43 +01:00
Mechiel Lukkien
61836f6d00
don't shadow variables, no empty "else" blocks
from go vet and staticcheck
2024-02-08 15:12:06 +01:00
Mechiel Lukkien
5f40d23c1c
remove unused build constraint 2024-02-08 15:10:32 +01:00
Mechiel Lukkien
e75419aeaf
unbreak rfc/xr.go after changing golang.org/x/exp/maps
shouldn't have changed this one.
2024-02-08 15:08:26 +01:00
Mechiel Lukkien
d1b87cdb0d
replace packages slog and slices from golang.org/x/exp with stdlib
since we are now at go1.21 as minimum.
2024-02-08 14:49:01 +01:00
Mechiel Lukkien
c698cd07d9
apidiff: properly check against actual previous version
not hardcoded v0.0.8...
2024-02-08 14:46:31 +01:00
Mechiel Lukkien
ecf60568b4
fix: don't insert spurious \r when fixing up crlf line endings when writing a message
message.Writer.Write() adds missing \r's, but the buffer of "last bytes
written" was only being updated while writing the message headers, not while
writing the body. so for Write()'s in the body section (depending on
buffering), we were compensating based on the "last bytes written" as set
during the last write in the header section. that could cause a spurious \r to
be added when a Write starts with \n while the previous Write did properly
end with \r.

for issue #117, thanks haraldrudell for reporting and investigating
2024-02-08 12:33:19 +01:00
Mechiel Lukkien
dd540e401a
replace another "/bin/bash" with "/usr/bin/env bash" and remove old file 2024-02-01 09:03:32 +01:00
Pierre-Alain TORET
5f297ce54c Improve portability of build scripts 2024-02-01 09:00:21 +01:00