Commit graph

241 commits

Author SHA1 Message Date
Mechiel Lukkien
6abee87aa3
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config
- make builtin http handlers serve on specific domains, such as for mta-sts, so
  e.g. /.well-known/mta-sts.txt isn't served on all domains.
- add logging of a few more fields in access logging.
- small tweaks/bug fixes in webserver request handling.
- add config option for redirecting entire domains to another (common enough).
- split httpserver metric into two: one for duration until writing header (i.e.
  performance of server), another for duration until full response is sent to
  client (i.e. performance as perceived by users).
- add admin ui, a new page for managing the configs. after making changes
  and hitting "save", the changes take effect immediately. the page itself
  doesn't look very well-designed (many input fields, makes it look messy). i
  have an idea to improve it (explained in admin.html as todo) by making the
  layout look just like the config file. not urgent though.

i've already changed my websites/webapps over.

the idea of adding a webserver is to take away a (the) reason for folks to want
to complicate their mox setup by running an other webserver on the same machine.
i think the current webserver implementation can already serve most common use
cases. with a few more tweaks (feedback needed!) we should be able to get to 95%
of the use cases. the reverse proxy can take care of the remaining 5%.
nevertheless, a next step is still to change the quickstart to make it easier
for folks to run with an existing webserver, with existing tls certs/keys.
that's how this relates to issue #5.
2023-03-02 18:15:54 +01:00
Mechiel Lukkien
6706c5c84a
add basic webserver that can do most of what i need
- serve static files, serving index.html or optionally listings for directories
- redirects
- reverse-proxy, forwarding requests to a backend

these are configurable through the config file. a domain and path regexp have to
be configured. path prefixes can be stripped.  configured domains are added to
the autotls allowlist, so acme automatically fetches certificates for them.

all webserver requests now have (access) logging, metrics, rate limiting.
on http errors, the error message prints an encrypted cid for relating with log files.

this also adds a new mechanism for example config files.
2023-02-28 22:19:24 +01:00
Mechiel Lukkien
fbfbd97947
connecction -> connection
from mteege
2023-02-28 20:43:31 +01:00
Mechiel Lukkien
994b5b9a0a
fix serving mta-sts policy files
i broke it 3 days ago when adding support for serving it through external reverse proxy.

report by mteege, thanks!
2023-02-28 20:40:52 +01:00
Mechiel Lukkien
c9190337e6
another rfc (we don't implement it) 2023-02-27 22:35:07 +01:00
Mechiel Lukkien
d670e68745
make error message about listeners with a missing tls config more helpful for users
by pointing out which section is causing the need for a tls config.
2023-02-27 21:42:27 +01:00
Mechiel Lukkien
da3fefd42e
explain to user they should pick a random, unguessable password
and help with a button to generate one.
2023-02-27 21:29:27 +01:00
Mechiel Lukkien
aed8360002
more permissions in mox.service to ease with upgrade to v0.0.2
- CAP_FSETID looks needed to make accessing setgid dirs work.
- CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH are needed to access the existing config/mox.conf.
- CAP_FOWNER seems needed to chmod the config/ and data/ files.
- RestrictSUIDSGID=yes has to be off for chmod to work
2023-02-27 15:48:37 +01:00
Mechiel Lukkien
cfa6554d03
move sieve up to roadmap, and add jmap there as well, and add questions about staying up to date and learning more about email 2023-02-27 15:04:53 +01:00
Mechiel Lukkien
e2e2c71212
minor tweaks to suggested dns records 2023-02-27 15:04:32 +01:00
Mechiel Lukkien
67c97c6499
on admin page, show warning when user hasn't enabled "check updates"
and send Accept-header when fetching changelog, so we can show an html version
instead of json when requested by regular browsers.
2023-02-27 15:03:37 +01:00
Mechiel Lukkien
d3590caa2e
do not require a user "mox" to be present for tests
uid 1000 doesn't actually have to be present (except for the special imaptest),
but this prevents looking up the user named mox.
2023-02-27 14:54:16 +01:00
Mechiel Lukkien
9e280221b6
merge start.go into serve.go 2023-02-27 14:12:58 +01:00
Mechiel Lukkien
44a3f9b1bc
in imapserver, do not advertise STARTTLS if TLS isn't configured 2023-02-27 14:10:43 +01:00
Mechiel Lukkien
0639c396b9
add CGO_ENABLED=0 in go install command
because users may run this on their local machine, then copy the binary the
their server. there is a good chance that binary won't start on linux/amd64 when
compiled with cgo due to missing glibc symbols.
2023-02-27 14:05:10 +01:00
Mechiel Lukkien
259729e03d
use full docker hub path in docker-compose.yml 2023-02-27 14:04:45 +01:00
Mechiel Lukkien
e20677cfd6
add missing capability CAP_FSETID to mox.service
without it, process cannot create setgid directories.
2023-02-27 14:04:20 +01:00
Mechiel Lukkien
f3f2c6f8ea
fix cross-compiled docker images
binaries for linux/amd64 were build for each target platform. apparently the
--platform in the first-stage overrides the $TARGET* variables of the build.
docker behaviour always manages to surprise me...
2023-02-27 13:46:29 +01:00
Mechiel Lukkien
92e018e463
change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user
makes it easier to run on bsd's, where you cannot (easily?) let non-root users
bind to ports <1024. starting as root also paves the way for future improvements
with privilege separation.

unfortunately, this requires changes to how you start mox. though mox will help
by automatically fix up dir/file permissions/ownership.

if you start mox from the systemd unit file, you should update it so it starts
as root and adds a few additional capabilities:

        # first update the mox binary, then, as root:
        ./mox config printservice >mox.service
        systemctl daemon-reload
        systemctl restart mox
        journalctl -f -u mox &
        # you should see mox start up, with messages about fixing permissions on dirs/files.

if you used the recommended config/ and data/ directory, in a directory just for
mox, and with the mox user called "mox", this should be enough.

if you don't want mox to modify dir/file permissions, set "NoFixPermissions:
true" in mox.conf.

if you named the mox user something else than mox, e.g. "_mox", add "User: _mox"
to mox.conf.

if you created a shared service user as originally suggested, you may want to
get rid of that as it is no longer useful and may get in the way. e.g. if you
had /home/service/mox with a "service" user, that service user can no longer
access any files: only mox and root can.

this also adds scripts for building mox docker images for alpine-supported
platforms.

the "restart" subcommand has been removed. it wasn't all that useful and got in
the way.

and another change: when adding a domain while mtasts isn't enabled, don't add
the per-domain mtasts config, as it would cause failure to add the domain.

based on report from setting up mox on openbsd from mteege.
and based on issue #3. thanks for the feedback!
2023-02-27 12:19:55 +01:00
Mechiel Lukkien
eda907fc86
better error message if import fails, only stack traces for unexpected panics (i.e. not the special sential panic value) 2023-02-26 22:25:57 +01:00
Mechiel Lukkien
9b3a170cc2
fix nil pointer deref when importing a message that the junkfilter could not parse (e.g. malformed header)
import does its own batched junkfilter training, so the deliver function
doesn't have to do it one message at a time, writing the updated filter each
time. however, if the message cannot be parsed, it isn't trained during import,
and deliver would try to train it again. it would try to open the junk filter
to do so, but that would fail because the import function already has the
junkfilter open (and the timeout is reached). a missing error check would
continue with a nil junkfilter, resulting in the nil pointer deref.

this adds the missing error check, and makes sure the deliver function does not
also try to train unparseable imported messages.

report from Jens Hilligsøe
2023-02-26 22:21:13 +01:00
Mechiel Lukkien
7e56ed9e8d
Merge pull request #9 from belst/main
fix config options
2023-02-26 20:45:10 +01:00
belst
8e178d9a1f fix config options 2023-02-26 15:57:28 +01:00
Mechiel Lukkien
e07fb5c0f5
add roadmap to readme, and shuffle some sections 2023-02-25 13:38:07 +01:00
Mechiel Lukkien
b80d85fd3f
for smtp error responses, add the encrypted cid for easier debugging 2023-02-25 13:07:32 +01:00
Mechiel Lukkien
b8fa918d74
add a bit more logging for non-SNI tls requests
for peace of mind.

these are probably requests to port 443 without SNI.
2023-02-25 12:56:25 +01:00
Mechiel Lukkien
26fcaa17f5
consistently use lower-cased field names for logging lines 2023-02-25 12:37:59 +01:00
Mechiel Lukkien
0ede7f78c1
add option to handle autoconfig and mta-sts requests without TLS, for when it is reverse proxied
for #5 with hints from belst & idnovic
2023-02-25 11:28:15 +01:00
Mechiel Lukkien
ac3a3e496e
make api urls relative, so they can work then being reverse proxied
for #5, found by belst
2023-02-25 11:20:07 +01:00
Mechiel Lukkien
b1dcd73ebe
help run mox with docker
in the Dockerfile, allow running on privileged ports and expose those ports.

add a docker-compose.yml with instructions for the quickstart.

fix running imaptest somewhat. after a short while it will hit the rate limiter.

in quickstart, recognize we are running under docker, and print slightly
different commands to set permissions, and skip generating the systemd service
file. als fix cleaning up the right paths during failure in quickstart.

for issue #3
2023-02-24 14:16:51 +01:00
Mechiel Lukkien
210fd34702
umask logging line tweak 2023-02-22 23:26:00 +01:00
Mechiel Lukkien
f2fd6241a0
do not bind to port 443 for tls-alpn01 if there is no ACME configured
for #2
2023-02-22 23:22:42 +01:00
Mechiel Lukkien
54981fbc93
add a todo for preventing message content from being logged on common imap append to Sent 2023-02-22 10:43:21 +01:00
Mechiel Lukkien
a480bcd583
use standard useradd options only, so the command works on bsds too, and no need for users to add an explicit -config flag anymore 2023-02-22 10:42:20 +01:00
Mechiel Lukkien
8affa12c39
prevent panic when starting with a mox.conf that references an ACME provider that isn't configured
the error handling code attempts to collect all error messages it encounters,
to print them all at the end, so you can fix them all before trying again. so
we try to continue preparing the config after an error. in this case, we
continued as if acme.Manager was properly set and nil-dereferenced it.

for issue #2, from hismailbulut
2023-02-21 23:06:11 +01:00
Mechiel Lukkien
cc3a2c9bc8
make it possible to serve https on a different port than 443 through configuration
so you can run mox on openbsd with port redirects in pf.conf.

in the future, starting as root, binding the sockets, and passing the bound
sockets to a new unprivileged process should be implemented, but this should
get openbsd users going.

from discussion with mteege
2023-02-18 16:53:06 +01:00
Mechiel Lukkien
a289a3e771
when delivery fails due to missing 8bitmime/smtputf8 extensions, make it temporary failure
if you start delivering, openbsd's spamd will not announce 8bitmime support.
but once you get patched through to the actual mail server, it will likely
announce 8bitmime support.

perhaps we should also just attempt to deliver 8bit email without 8bitmime
extension. probably better for users than not even trying to send the message.
perhaps in a non-strict mode.
2023-02-17 21:58:05 +01:00
Mechiel Lukkien
4e5030ff38
fix output of "mox checkupdate", and specify changes to be from newest to oldest 2023-02-17 20:14:26 +01:00
Mechiel Lukkien
cafccefad1
fix integration test, the delay for new senders was causing our db open to fail 2023-02-17 19:30:30 +01:00
Mechiel Lukkien
6df4b454d5
update dependencies, including bolt with stability fixes 2023-02-17 18:55:01 +01:00
Mechiel Lukkien
fb3794e31b
only send \NonExistent for IMAP4rev2, and automatically subscribe to imported mailboxes 2023-02-17 18:35:11 +01:00
Mechiel Lukkien
ad900b74e7
on mtasts error during delivery from queue, continue delivery but with strict tls
no need to not even try delivering in that case. if anyone is preventing mtasts
to work, nothing was achieved, because our fallback is the strictest behaviour
that could have been specified in the policy.
2023-02-17 17:19:41 +01:00
Mechiel Lukkien
3608d0e246
more testdata to ignore 2023-02-17 17:06:27 +01:00
Mechiel Lukkien
f76fe26976
when exporting mbox files, set new status,x-status,x-keywords headers, and remove any content-length header 2023-02-17 17:04:48 +01:00
Mechiel Lukkien
c65731ac56
fix 3 cases of shadowed variables (other than "err")
2 of these were actual bugs.
2023-02-16 13:24:51 +01:00
Mechiel Lukkien
5c33640aea
consistently use log.Check for logging errors that "should not happen", don't influence application flow
sooner or later, someone will notice one of these messages, which will lead us
to a bug.
2023-02-16 13:22:00 +01:00
Mechiel Lukkien
ef8e5fa1a8
on accounts page, fix opening destination with utf8
by decoding location.hash
2023-02-16 10:26:28 +01:00
Mechiel Lukkien
8d867490d1
readme tweak 2023-02-16 10:09:33 +01:00
Mechiel Lukkien
5336032088
add funtionality to import zip/tgz with maildirs/mboxes to account page
so users can easily take their email out of somewhere else, and import it into mox.

this goes a little way to give feedback as the import progresses: upload
progress is shown (surprisingly, browsers aren't doing this...), imported
mailboxes/messages are counted (batched) and import issues/warnings are
displayed, all sent over an SSE connection. an import token is stored in
sessionstorage. if you reload the page (e.g. after a connection error), the
browser will reconnect to the running import and show its progress again. and
you can just abort the import before it is finished and committed, and nothing
will have changed.

this also imports flags/keywords from mbox files.
2023-02-16 09:57:27 +01:00
Mechiel Lukkien
23b530ae36
use config/mox.conf as default path for the config file
this matches how the quickstart recommends setting up mox.
i've been running "./mox -config/mox.conf" from there for admin commands, but
"mox" is enough now.
2023-02-13 22:40:45 +01:00