Commit graph

808 commits

Author SHA1 Message Date
Mechiel Lukkien
3d52efbdf9
fix apidiff.sh to always generate a new apidiff/next.txt file
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
2025-01-23 23:02:36 +01:00
Mechiel Lukkien
6aa2139a54
do not use results from junk filter if we have less than 50 positive classifications to base the decision on
useful for new accounts. we don't want to start rejecting incoming messages for
having a score near 0.5 because of too little training material. we err on the
side of allowing messages in. the user will mark them as junk, training the
filter. once enough non-junk has come in, we'll start the actual filtering.

for issue #64 by x8x, and i've also seen this concern on matrix
2025-01-23 22:55:50 +01:00
Mechiel Lukkien
8fac9f862b
attempt to fix workflow again
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
sigh, this is why you don't you use cloud tools that you can't run locally...
2025-01-23 18:40:05 +01:00
Mechiel Lukkien
7df54071d7
update to github action actions/upload-artifact@v4 from v3
we'll now get a coverage file artifact for each of the builds. we do two
builds, and the last was likely overwriting the coverage file "artifact" of the
first.

hopefully fixes the test. can't test it locally...
2025-01-23 18:29:43 +01:00
Mechiel Lukkien
acc1c133b0
admin check: do not raise error when forward-confirmed reverse dns does not match hostname
this should be relatively common with setups involving NAT.
so we do warn about it when NAT isn't active since it could highlight potential
misconfiguration.

for issue #239 by exander77
2025-01-23 18:11:00 +01:00
s0ph0s
3c77e076e2
Add support for negotiating IMAP and SMTP on the HTTPS port 443 using TLS ALPN "imap" and "smtp"
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
Intended for future use with chatmail servers. Standard email ports may be
blocked on some networks, while the HTTPS port may be accessible.

This is a squashed commit of PR #255 by s0ph0s-dog.
2025-01-23 11:16:20 +01:00
Mechiel Lukkien
0203dfa9d9
webmail: fix nil pointer dereference when searching for attachment types, eg "a:spreadsheet"
for issue #272 by mattfbacon
2025-01-23 11:03:08 +01:00
Mechiel Lukkien
008de1cafb
webmail: in message view, under More, add button to open currently displayed part (either text or html) as raw text (but decoded if in base64/quoted-printable/etc).
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
2025-01-22 21:19:24 +01:00
Mechiel Lukkien
7647264a72
web interfaces: when there is no login session, and a non-existent path is requested, mention the web interface this is about
may help users understand when /admin/ isn't enabled on a hostname but the
account web interface is at /. the error will now say: no session for "account"
web interface. it hopefully tells users that their request isn't going to an
admin interface, but ends up at the account web interface.

for issue #268
2025-01-22 20:15:14 +01:00
Mechiel Lukkien
f15f2d68fc
webmail: more helpful error message when emptying a mailbox that is already empty
and mention in a tooltip too that "empty mailbox" only affects messages in the
mailbox, not submailboxes or their messages.

prompted by a question on matrix/irc
2025-01-22 20:09:19 +01:00
Mechiel Lukkien
315f10d5f2
add release to website
Some checks failed
Build and test / build-test (oldstable) (push) Has been cancelled
Build and test / build-test (stable) (push) Has been cancelled
2025-01-20 12:54:45 +01:00
Mechiel Lukkien
5fcea1eb3b
rotate apidiff/next.txt for release 2025-01-20 12:49:20 +01:00
Mechiel Lukkien
be1065a6c4
add another makefile testing target 2025-01-13 23:23:00 +01:00
Mechiel Lukkien
b85401a83d
fix command gentestdata for testing upgrades
not working since tlspubkey auth
2025-01-13 23:22:14 +01:00
Mechiel Lukkien
dd92ed5117
update to latest golang.org/x dependencies
Some checks failed
Build and test / build-test (oldstable) (push) Has been cancelled
Build and test / build-test (stable) (push) Has been cancelled
2025-01-13 22:29:42 +01:00
Mechiel Lukkien
871f70151c
smtpserver: allow using an "message from" address from an allowed alias as smtp mail from during submission
mail clients will use these message from addresses also for smtp mail from, so
sending over smtp would fail for these cases. for the webmail and webapi they
already succeeded since we just took the "message from" address as "smtp mail
from" address.

for issue #266 by Robby-, thanks for reporting!
2025-01-13 21:34:59 +01:00
Mechiel Lukkien
d4d2a0fd99
webmail: when listing messages in backend to send to frontend, don't error out when there's a large plain text part
by not trying to parse the full message for the MessageItem, but only reading
headers when needed.

before previous commit, we wouldn't try reading such messages in full either.
2025-01-13 16:13:25 +01:00
Mechiel Lukkien
1e15a10b66
webmail: fix js error rerendering additional headers after updated keywords
i've seen the error a few times:

	msgheaderElem.children[(msgheaderElem.children.length - 1)] is undefined

i've seen it happen after sending a reply (with the "answered" flag added).
the updateKeywords callback would render the message again, but the code for
rendering the "additional headers" table rows again was making invalid
assumptions.

the approach is now changed. the backend now just immediately sends the
additional headers to the frontend. before, the frontend would first render the
base message, then render again once the headers came in for the parsed
message. this also prevents a reflow for the (quite common) case that one of
the additional headers are present in the message.
2025-01-13 14:53:43 +01:00
Mechiel Lukkien
f7193bd4c3
webmail: fix css to not show text on button (actually html "a" element for links) for downloaded (visited) attachments in blue
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
2025-01-13 11:22:44 +01:00
Mechiel Lukkien
5a14a5b067
smtpserver: when doing slow writes due to spammy incoming delivery, try a bit harder to prevent a timeout for the other side (if it is mox/itself!)
based on question from wneessen
2025-01-13 11:13:26 +01:00
Mechiel Lukkien
b8bf99e082
ensure kind "acme-tls-alpn-01" is registered on the http handler
previous code couldn't possibly be triggered by my reading.

encountered during PR #255
2025-01-13 10:43:55 +01:00
Mechiel Lukkien
eb88e2651a
dkim: add reference to rfc that says not to accept rsa keys < 1024 bits
saw it mentioned on HN recently
2025-01-13 10:35:25 +01:00
Mechiel Lukkien
e5eaf4d46f
fix race in imapserver tests
Some checks failed
Build and test / build-test (oldstable) (push) Has been cancelled
Build and test / build-test (stable) (push) Has been cancelled
2024-12-25 16:50:23 +01:00
Mechiel Lukkien
9b429cce4f
try harder to start docker integration tests with clean slate
for some reason "docker-compose down" takes a very long time, and doesn't
actually stop containers if you add a timeout.
2024-12-25 16:44:54 +01:00
Mechiel Lukkien
965a2b426f
webadmin: when loading page with webserver routes, internal services would always be shown with "admin" as internal services, and saving the handler would overwrite the correct setting
Some checks are pending
Build and test / build-test (oldstable) (push) Waiting to run
Build and test / build-test (stable) (push) Waiting to run
fix this by properly loading the correct internal service.

for issue #264 reported by kiekerjan, thanks!
2024-12-24 22:02:28 +01:00
Mechiel Lukkien
f7666d1582
fix verifying dane-ta connections for outgoing email where the dane-ta record is not for the first certificate in the chain after the leaf certifiate.
tls servers send a list of certificates for the connection. the first is the
leaf certificate. that's the one for the server itself. that's the one we want
to verify. the others are intermediate CA's. and possibly even the root CA
certificate that it hopes is trusted at the client (though sending it doesn't
make it trusted). with dane-ta, the public key of an intermediate or root CA
certificate is listed in the TSLA record. when verifying, we add any
intermediate/root CA that matches a dane-ta tlsa record to the trusted root CA
certs. we should also have added CA certs that didn't match a TLSA record to
the "intermediates" of x509.VerifyOptions. because we didn't,
x509.Certificate.Verify couldn't verify the chain from the trusted dane-ta ca
cert to the leaf cert. we would only properly verify a dane-ta connection
correctly if the dane-ta-trusted ca cert was the one immediately following the
leaf cert. not when there were one or more additional intermediate certs.

this showed when connecting to mx.runbox.com.

problem reported by robbo5000 on matrix, thanks!
2024-12-21 16:09:53 +01:00
Mechiel Lukkien
aa9a06680f
update to golang.org/x/net/html (slow parsing fixed) and other golang.org/x deps 2024-12-21 09:44:11 +01:00
Mechiel Lukkien
d082aaada8
only use constant strings in string formatting
builds with go1.24rc1 fail on these.
only the case in smtpserver could be triggered externally.
2024-12-14 09:38:56 +01:00
Mechiel Lukkien
5320ec1c5b
quickstart: for -existing-webserver, also tls key/cert placeholder for mail.$domain
unless mail.$domain is the mx hostname.

after question about which tls certs are needed from robbo5000 on matrix
2024-12-08 10:18:57 +01:00
Mechiel Lukkien
2255ebcf11
quickstart: write all output to a file "quickstart.log" for later reference
quite some output is printed. you could remember to tee it all to a file. but
that's probably often realized only after having run the quickstart. you can
also copy/paste it all from the terminal, but that's sometimes annoying to do.
writing to a file is more helpful to users.

this has been requested a few times in the past on irc/matrix (i forgot who).
2024-12-07 21:14:43 +01:00
Mechiel Lukkien
35af7e30a6
do not try to get a tls cert for autoconfig.<domain> at startup if there is no listener with autoconfig enabled
reduces needless logging in setups that don't use autoconfig.
2024-12-07 20:28:52 +01:00
Mechiel Lukkien
cbe418ec59
try clarifying that aliases are lists, not to be used for simply adding an address to an account
for issue #244 by exander77
2024-12-07 19:10:02 +01:00
Mechiel Lukkien
f7b58c87b1
instead of using loglevel error for printing a warning, just log it as "warn" error level, and don't log message parsing errors as loglevel error 2024-12-07 19:07:16 +01:00
Mechiel Lukkien
94fb48c2dc
mox retrain: make the parameter, for account, optional and retrain all accounts when absent
for more easily retraining all accounts. users should be retraining their
accounts with the next release, due to the fix in the previous commit.
2024-12-07 17:00:00 +01:00
Mechiel Lukkien
17baf9a883
junk filter: fix adjusting word counts after train/untrain
after seeing some junk messages pass the filter, i investigated word counts in
junkfilter.db. i had seen suspicious counts that were just around powers of
two. did not make sense at the time. more investigating makes it clear: instead
of setting new word counts when updating the junk filter, we were adding the
new value to the current value (instead of just setting the new value). so the
counts got approximately doubled when being updated.

users should retrain the junk filter after this update using the "retrain"
subcommand.

this also adds logging for the hypothetical case where numbers would get
decreased below zero (which would wrap around due to uints).

and this fixes junk filter tests that were passing wrong parameters to
train/untrain...
2024-12-07 16:53:53 +01:00
Mechiel Lukkien
69a4995449
move func PartStructure from webhook to queue, so it isn't tracked anymore for apidiff changes
the types in webhook should be subjected to apidiff'ing, this was a shared
function. it is better off in package queue. also change the apidiff script so
it leaves apidiff/next.txt empty when there aren't any changes. makes it easier
to rotate the files after releases where nothing changed (a common occurrence).
2024-12-07 13:57:07 +01:00
Mechiel Lukkien
0871bf5219
move checking whether a message needs smtputf8 (has utf8 in any of the header sections) to package message 2024-12-07 13:05:09 +01:00
Mechiel Lukkien
3f727cf380
webmail: move 2 config options from localstorage to the settings popup, storing their values on the server
these settings are applied anywhere the webmail is open.  the settings are for
showing keyboard shortcuts in the lower right after a mouse interaction, and
showing additional headers.  the shorcuts were configurable in the "help" popup
before.  the additional headers were only configurable through the developer
console before.

the "mailto:" (un)register buttons are now in the settings popup too.
2024-12-07 12:32:54 +01:00
Mechiel Lukkien
4d3c4115f8
webmail: don't bind to shortcuts ctrl-l, ctrl-u and ctrl-I
ctrl-l is commonly "focus on browser address bar".
ctrl-u is commonly "view source".
ctrl-I (shift i) is commonly "open developer console".

these keys are more useful to leave for the browser.  ctrl-l and ctrl-u (moving
to a message without opening it) can still be had by using also pressing shift.
the previous ctrl-shift-i (show all headers) is now just ctrl-i.

this has been requested in the past on irc/matrix (i forgot who).
2024-12-07 12:29:12 +01:00
Mechiel Lukkien
0a77bc5955
tweak doucmentation for sasl and scram 2024-12-06 15:59:22 +01:00
Mechiel Lukkien
ce75852b7c
add missing space in x-mox-reason that's been bothering me for a while 2024-12-06 15:49:22 +01:00
Mechiel Lukkien
b750668152
add metrics that track how many error/warn/info logging is happening 2024-12-06 15:07:42 +01:00
Mechiel Lukkien
056b571fb6
webmail: don't consume keyboard events while login form is open
e.g. ctrl-l, for going to address bar to go to another site.
2024-12-06 14:57:20 +01:00
Mechiel Lukkien
e59f894a94
add an option for the smtp delivery listener to enable/disable tls session tickets
the field is optional. if absent, the default behaviour is currently to disable
session tickets. users can set the option if they want to try if delivery from
microsoft is working again. in a  future version, we can switch the default to
enabling session tickets.

the previous fix was to disable session tickets for all tls connections,
including https. that was a bit much.

for issue #237
2024-12-06 14:50:02 +01:00
Mechiel Lukkien
42793834f8
add Content-Disposition and Filename to the payload of incoming webhooks
for each message part. The ContentDisposition value is the base value without
header key/value parameters. the Filename field is the likely filename of the
part. the different email clients encode filenames differently. there is a
standard mime mechanism from rfc 2231. and there is the q/b-word-encoding from
rfc 2047. instead of letting users of the webhook api deal with those
differences, we provide just the parsed filename.

for issue #258 by morki, thanks for reporting!
2024-12-06 14:19:39 +01:00
Mechiel Lukkien
8804d6b60e
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client.  with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.

for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.

for smtp (submission), we don't require an explicit auth command.

both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.

only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.

users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.

the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.

there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 10:08:17 +01:00
Mechiel Lukkien
5f7831a7f0
move config-changing code from package mox-/ to admin/
needed for upcoming changes, where (now) package admin needs to import package
store. before, because package store imports mox- (for accessing the active
config), that would lead to a cyclic import. package mox- keeps its active
config, package admin has the higher-level config-changing functions.
2024-12-02 22:03:18 +01:00
Mechiel Lukkien
de435fceba
switch to math/rand/v2 in most places
this allows removing some ugly instantiations of an rng based on the current
time.

Intn is now IntN for our concurrency-safe prng wrapper to match the randv2 api.

v2 exists since go1.22, which we already require.
2024-11-29 13:45:19 +01:00
Mechiel Lukkien
96a3ecd52c
use reflect.TypeFor instead of kludgy reflect.TypeOf
TypeFor was introduced in go1.22, which we already require.
2024-11-29 13:17:13 +01:00
Mechiel Lukkien
afb182cb14
smtpserver: add prometheus metric for failing starttls handshakes for incoming deliveries
and add an alerting rule if the failure rate becomes >10% (e.g. expired
certificate).

the prometheus metrics includes a reason, including potential tls alerts, if
remote smtp clients would send those (openssl s_client -starttls does).

inspired by issue #237, where incoming connections were aborted by remote. such
errors would show up as "eof" in the metrics.
2024-11-29 12:43:21 +01:00