Commit graph

132 commits

Author SHA1 Message Date
Mechiel Lukkien
e2e2c71212
minor tweaks to suggested dns records 2023-02-27 15:04:32 +01:00
Mechiel Lukkien
67c97c6499
on admin page, show warning when user hasn't enabled "check updates"
and send Accept-header when fetching changelog, so we can show an html version
instead of json when requested by regular browsers.
2023-02-27 15:03:37 +01:00
Mechiel Lukkien
d3590caa2e
do not require a user "mox" to be present for tests
uid 1000 doesn't actually have to be present (except for the special imaptest),
but this prevents looking up the user named mox.
2023-02-27 14:54:16 +01:00
Mechiel Lukkien
9e280221b6
merge start.go into serve.go 2023-02-27 14:12:58 +01:00
Mechiel Lukkien
44a3f9b1bc
in imapserver, do not advertise STARTTLS if TLS isn't configured 2023-02-27 14:10:43 +01:00
Mechiel Lukkien
0639c396b9
add CGO_ENABLED=0 in go install command
because users may run this on their local machine, then copy the binary the
their server. there is a good chance that binary won't start on linux/amd64 when
compiled with cgo due to missing glibc symbols.
2023-02-27 14:05:10 +01:00
Mechiel Lukkien
259729e03d
use full docker hub path in docker-compose.yml 2023-02-27 14:04:45 +01:00
Mechiel Lukkien
e20677cfd6
add missing capability CAP_FSETID to mox.service
without it, process cannot create setgid directories.
2023-02-27 14:04:20 +01:00
Mechiel Lukkien
f3f2c6f8ea
fix cross-compiled docker images
binaries for linux/amd64 were build for each target platform. apparently the
--platform in the first-stage overrides the $TARGET* variables of the build.
docker behaviour always manages to surprise me...
2023-02-27 13:46:29 +01:00
Mechiel Lukkien
92e018e463
change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user
makes it easier to run on bsd's, where you cannot (easily?) let non-root users
bind to ports <1024. starting as root also paves the way for future improvements
with privilege separation.

unfortunately, this requires changes to how you start mox. though mox will help
by automatically fix up dir/file permissions/ownership.

if you start mox from the systemd unit file, you should update it so it starts
as root and adds a few additional capabilities:

        # first update the mox binary, then, as root:
        ./mox config printservice >mox.service
        systemctl daemon-reload
        systemctl restart mox
        journalctl -f -u mox &
        # you should see mox start up, with messages about fixing permissions on dirs/files.

if you used the recommended config/ and data/ directory, in a directory just for
mox, and with the mox user called "mox", this should be enough.

if you don't want mox to modify dir/file permissions, set "NoFixPermissions:
true" in mox.conf.

if you named the mox user something else than mox, e.g. "_mox", add "User: _mox"
to mox.conf.

if you created a shared service user as originally suggested, you may want to
get rid of that as it is no longer useful and may get in the way. e.g. if you
had /home/service/mox with a "service" user, that service user can no longer
access any files: only mox and root can.

this also adds scripts for building mox docker images for alpine-supported
platforms.

the "restart" subcommand has been removed. it wasn't all that useful and got in
the way.

and another change: when adding a domain while mtasts isn't enabled, don't add
the per-domain mtasts config, as it would cause failure to add the domain.

based on report from setting up mox on openbsd from mteege.
and based on issue #3. thanks for the feedback!
2023-02-27 12:19:55 +01:00
Mechiel Lukkien
eda907fc86
better error message if import fails, only stack traces for unexpected panics (i.e. not the special sential panic value) 2023-02-26 22:25:57 +01:00
Mechiel Lukkien
9b3a170cc2
fix nil pointer deref when importing a message that the junkfilter could not parse (e.g. malformed header)
import does its own batched junkfilter training, so the deliver function
doesn't have to do it one message at a time, writing the updated filter each
time. however, if the message cannot be parsed, it isn't trained during import,
and deliver would try to train it again. it would try to open the junk filter
to do so, but that would fail because the import function already has the
junkfilter open (and the timeout is reached). a missing error check would
continue with a nil junkfilter, resulting in the nil pointer deref.

this adds the missing error check, and makes sure the deliver function does not
also try to train unparseable imported messages.

report from Jens Hilligsøe
2023-02-26 22:21:13 +01:00
Mechiel Lukkien
7e56ed9e8d
Merge pull request #9 from belst/main
fix config options
2023-02-26 20:45:10 +01:00
belst
8e178d9a1f fix config options 2023-02-26 15:57:28 +01:00
Mechiel Lukkien
e07fb5c0f5
add roadmap to readme, and shuffle some sections 2023-02-25 13:38:07 +01:00
Mechiel Lukkien
b80d85fd3f
for smtp error responses, add the encrypted cid for easier debugging 2023-02-25 13:07:32 +01:00
Mechiel Lukkien
b8fa918d74
add a bit more logging for non-SNI tls requests
for peace of mind.

these are probably requests to port 443 without SNI.
2023-02-25 12:56:25 +01:00
Mechiel Lukkien
26fcaa17f5
consistently use lower-cased field names for logging lines 2023-02-25 12:37:59 +01:00
Mechiel Lukkien
0ede7f78c1
add option to handle autoconfig and mta-sts requests without TLS, for when it is reverse proxied
for #5 with hints from belst & idnovic
2023-02-25 11:28:15 +01:00
Mechiel Lukkien
ac3a3e496e
make api urls relative, so they can work then being reverse proxied
for #5, found by belst
2023-02-25 11:20:07 +01:00
Mechiel Lukkien
b1dcd73ebe
help run mox with docker
in the Dockerfile, allow running on privileged ports and expose those ports.

add a docker-compose.yml with instructions for the quickstart.

fix running imaptest somewhat. after a short while it will hit the rate limiter.

in quickstart, recognize we are running under docker, and print slightly
different commands to set permissions, and skip generating the systemd service
file. als fix cleaning up the right paths during failure in quickstart.

for issue #3
2023-02-24 14:16:51 +01:00
Mechiel Lukkien
210fd34702
umask logging line tweak 2023-02-22 23:26:00 +01:00
Mechiel Lukkien
f2fd6241a0
do not bind to port 443 for tls-alpn01 if there is no ACME configured
for #2
2023-02-22 23:22:42 +01:00
Mechiel Lukkien
54981fbc93
add a todo for preventing message content from being logged on common imap append to Sent 2023-02-22 10:43:21 +01:00
Mechiel Lukkien
a480bcd583
use standard useradd options only, so the command works on bsds too, and no need for users to add an explicit -config flag anymore 2023-02-22 10:42:20 +01:00
Mechiel Lukkien
8affa12c39
prevent panic when starting with a mox.conf that references an ACME provider that isn't configured
the error handling code attempts to collect all error messages it encounters,
to print them all at the end, so you can fix them all before trying again. so
we try to continue preparing the config after an error. in this case, we
continued as if acme.Manager was properly set and nil-dereferenced it.

for issue #2, from hismailbulut
2023-02-21 23:06:11 +01:00
Mechiel Lukkien
cc3a2c9bc8
make it possible to serve https on a different port than 443 through configuration
so you can run mox on openbsd with port redirects in pf.conf.

in the future, starting as root, binding the sockets, and passing the bound
sockets to a new unprivileged process should be implemented, but this should
get openbsd users going.

from discussion with mteege
2023-02-18 16:53:06 +01:00
Mechiel Lukkien
a289a3e771
when delivery fails due to missing 8bitmime/smtputf8 extensions, make it temporary failure
if you start delivering, openbsd's spamd will not announce 8bitmime support.
but once you get patched through to the actual mail server, it will likely
announce 8bitmime support.

perhaps we should also just attempt to deliver 8bit email without 8bitmime
extension. probably better for users than not even trying to send the message.
perhaps in a non-strict mode.
2023-02-17 21:58:05 +01:00
Mechiel Lukkien
4e5030ff38
fix output of "mox checkupdate", and specify changes to be from newest to oldest 2023-02-17 20:14:26 +01:00
Mechiel Lukkien
cafccefad1
fix integration test, the delay for new senders was causing our db open to fail 2023-02-17 19:30:30 +01:00
Mechiel Lukkien
6df4b454d5
update dependencies, including bolt with stability fixes 2023-02-17 18:55:01 +01:00
Mechiel Lukkien
fb3794e31b
only send \NonExistent for IMAP4rev2, and automatically subscribe to imported mailboxes 2023-02-17 18:35:11 +01:00
Mechiel Lukkien
ad900b74e7
on mtasts error during delivery from queue, continue delivery but with strict tls
no need to not even try delivering in that case. if anyone is preventing mtasts
to work, nothing was achieved, because our fallback is the strictest behaviour
that could have been specified in the policy.
2023-02-17 17:19:41 +01:00
Mechiel Lukkien
3608d0e246
more testdata to ignore 2023-02-17 17:06:27 +01:00
Mechiel Lukkien
f76fe26976
when exporting mbox files, set new status,x-status,x-keywords headers, and remove any content-length header 2023-02-17 17:04:48 +01:00
Mechiel Lukkien
c65731ac56
fix 3 cases of shadowed variables (other than "err")
2 of these were actual bugs.
2023-02-16 13:24:51 +01:00
Mechiel Lukkien
5c33640aea
consistently use log.Check for logging errors that "should not happen", don't influence application flow
sooner or later, someone will notice one of these messages, which will lead us
to a bug.
2023-02-16 13:22:00 +01:00
Mechiel Lukkien
ef8e5fa1a8
on accounts page, fix opening destination with utf8
by decoding location.hash
2023-02-16 10:26:28 +01:00
Mechiel Lukkien
8d867490d1
readme tweak 2023-02-16 10:09:33 +01:00
Mechiel Lukkien
5336032088
add funtionality to import zip/tgz with maildirs/mboxes to account page
so users can easily take their email out of somewhere else, and import it into mox.

this goes a little way to give feedback as the import progresses: upload
progress is shown (surprisingly, browsers aren't doing this...), imported
mailboxes/messages are counted (batched) and import issues/warnings are
displayed, all sent over an SSE connection. an import token is stored in
sessionstorage. if you reload the page (e.g. after a connection error), the
browser will reconnect to the running import and show its progress again. and
you can just abort the import before it is finished and committed, and nothing
will have changed.

this also imports flags/keywords from mbox files.
2023-02-16 09:57:27 +01:00
Mechiel Lukkien
23b530ae36
use config/mox.conf as default path for the config file
this matches how the quickstart recommends setting up mox.
i've been running "./mox -config/mox.conf" from there for admin commands, but
"mox" is enough now.
2023-02-13 22:40:45 +01:00
Mechiel Lukkien
4a58b8f434
export more imap flags (eg $Junk/$NotJunk/$Forwarded) with maildirs, in dovecot-keywords file
and let the subcommand "export" use the same export code as the accounts page.
2023-02-13 22:37:25 +01:00
Mechiel Lukkien
b349010e3d
fix build with go1.19 2023-02-13 19:28:28 +01:00
Mechiel Lukkien
3de6642b3a
implement exporting of all mailboxes/messages as zip/tgz of mbox/maildir 2023-02-13 18:04:05 +01:00
Mechiel Lukkien
ad51ffc365
make account web page configurable separately from admin, add http auth rate limiting
ideally both account & admin web pages should be on non-public ips (e.g. a
wireguard tunnel). but during setup, users may not have that set up, and they
may want to configure the admin/account pages on their public ip's. the auth
rate limiting should make it less of issue.

users can now also only put the account web page publicly available. useful for
if you're the admin and you have a vpn connection, but your other/external
users do not have a vpn into your mail server. to make the account page more
easily findable, the http root serves the account page. the admin page is still
at /admin/, to prevent clash with potential account pages, but if no account
page is present, you are helpfully redirected from / to /admin/.

this also adds a prometheus metric counting how often auth attempts have been
rate limited.
2023-02-13 13:53:47 +01:00
Mechiel Lukkien
2601766c2f
when cleaning up messages in rejects mailbox, remove the on-disk message file too
we were leaving files behind
2023-02-13 11:06:16 +01:00
Mechiel Lukkien
707d3a3fa0
store rejects for 14 days, and don't keep them as neutral by default so they won't cause outright rejects for repeated delivery attempts of spam messages
the previous default, marking the messages as junk had the interesting effect
of training the junk filter. rejecting could have been the result of the
sending IP being in the DNSBL. so the DNSBL helped to automatically train the
junk filter. perhaps we can keep that in the future and just not take messages
from the rejects mailbox into account when evaluating the reputation for
incoming deliveries.
2023-02-13 10:47:20 +01:00
Mechiel Lukkien
87854cfde3
change some log levels from info to debug, and use lower case log messages 2023-02-11 23:54:22 +01:00
Mechiel Lukkien
bf04fb8a1a
improve training of junk filter
before, we used heuristics to decide when to train/untrain a message as junk or
nonjunk: the message had to be seen, be in certain mailboxes. then if a message
was marked as junk, it was junk. and otherwise it was nonjunk. this wasn't good
enough: you may want to keep some messages around as neither junk or nonjunk.
and that wasn't possible.

ideally, we would just look at the imap $Junk and $NotJunk flags. the problem
is that mail clients don't set these flags, or don't make it easy. thunderbird
can set the flags based on its own bayesian filter. it has a shortcut for
marking Junk and moving it to the junk folder (good), but the counterpart of
notjunk only marks a message as notjunk without showing in the UI that it was
marked as notjunk. there is also no "move and mark as notjunk" mechanism. e.g.
"archive" does not mark a message as notjunk. ios mail and mutt don't appear to
have any way to see or change the $Junk and $NotJunk flags.

what email clients do have is the ability to move messages to other
mailboxes/folders. so mox now has a mechanism that allows you to configure
mailboxes that automatically set $Junk or $NotJunk (or clear both) when a
message is moved/copied/delivered to that folder. e.g. a mailbox called junk or
spam or rejects marks its messags as junk. inbox, postmaster, dmarc, tlsrpt,
neutral* mark their messages as neither junk or notjunk. other folders mark
their messages as notjunk. e.g. list/*, archive. this functionality is
optional, but enabled with the quickstart and for new accounts.

also, mox now keeps track of the previous training of a message and will only
untrain/train if needed. before, there probably have been duplicate or missing
(un)trainings.

this also includes a new subcommand "retrain" to recreate the junkfilter for an
account. you should run it after updating to this version. and you should
probably also modify your account config to include the AutomaticJunkFlags.
2023-02-11 23:00:12 +01:00
Mechiel Lukkien
a4306ef783
let's attempt irc & the gopher slack as places of discussion 2023-02-11 13:35:07 +01:00