serv/mox
1
0
Fork 0
mirror of https://github.com/mjl-/mox.git synced 2024-12-27 08:53:48 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions
mox/config/config.go

407 lines
35 KiB
Go
Raw Normal View History

mox!
2023-01-30 16:27:06 +03:00
package config
import (
"crypto"
"crypto/tls"
"crypto/x509"
"net"
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
"net/url"
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
"reflect"
mox!
2023-01-30 16:27:06 +03:00
"regexp"
"time"
"github.com/mjl-/mox/autotls"
"github.com/mjl-/mox/dns"
"github.com/mjl-/mox/junk"
"github.com/mjl-/mox/mtasts"
"github.com/mjl-/mox/smtp"
)
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
// todo: better default values, so less has to be specified in the config file.
mox!
2023-01-30 16:27:06 +03:00
// Port returns port if non-zero, and fallback otherwise.
func Port(port, fallback int) int {
if port == 0 {
return fallback
}
return port
}
// Static is a parsed form of the mox.conf configuration file, before converting it
// into a mox.Config after additional processing.
type Static struct {
DataDir string `sconf-doc:"Directory where all data is stored, e.g. queue, accounts and messages, ACME TLS certs/keys. If this is a relative path, it is relative to the directory of mox.conf."`
add two new log levels for tracing sensitive auth protocol messages, and bulk data messages named "traceauth" and "tracedata". with this, you can (almost) enable trace logging without fear of logging sensitive data or ddos'ing your log server. the caveat is that the imap login command has already printed the line as regular trace before we can decide it should not be. can be fixed soon.
2023-02-03 22:33:19 +03:00
LogLevel string `sconf-doc:"Default log level, one of: error, info, debug, trace, traceauth, tracedata. Trace logs SMTP and IMAP protocol transcripts, with traceauth also messages with passwords, and tracedata on top of that also the full data exchanges (full messages), which can be a large amount of data."`
mox!
2023-01-30 16:27:06 +03:00
PackageLogLevels map[string]string `sconf:"optional" sconf-doc:"Overrides of log level per package (e.g. queue, smtpclient, smtpserver, imapserver, spf, dkim, dmarc, dmarcdb, autotls, junk, mtasts, tlsrpt)."`
change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user makes it easier to run on bsd's, where you cannot (easily?) let non-root users bind to ports <1024. starting as root also paves the way for future improvements with privilege separation. unfortunately, this requires changes to how you start mox. though mox will help by automatically fix up dir/file permissions/ownership. if you start mox from the systemd unit file, you should update it so it starts as root and adds a few additional capabilities: # first update the mox binary, then, as root: ./mox config printservice >mox.service systemctl daemon-reload systemctl restart mox journalctl -f -u mox & # you should see mox start up, with messages about fixing permissions on dirs/files. if you used the recommended config/ and data/ directory, in a directory just for mox, and with the mox user called "mox", this should be enough. if you don't want mox to modify dir/file permissions, set "NoFixPermissions: true" in mox.conf. if you named the mox user something else than mox, e.g. "_mox", add "User: _mox" to mox.conf. if you created a shared service user as originally suggested, you may want to get rid of that as it is no longer useful and may get in the way. e.g. if you had /home/service/mox with a "service" user, that service user can no longer access any files: only mox and root can. this also adds scripts for building mox docker images for alpine-supported platforms. the "restart" subcommand has been removed. it wasn't all that useful and got in the way. and another change: when adding a domain while mtasts isn't enabled, don't add the per-domain mtasts config, as it would cause failure to add the domain. based on report from setting up mox on openbsd from mteege. and based on issue #3. thanks for the feedback!
2023-02-27 14:19:55 +03:00
User string `sconf:"optional" sconf-doc:"User to switch to after binding to all sockets as root. Default: mox. If the value is not a known user, it is parsed as integer and used as uid and gid."`
NoFixPermissions bool `sconf:"optional" sconf-doc:"If true, do not automatically fix file permissions when starting up. By default, mox will ensure reasonable owner/permissions on the working, data and config directories (and files), and mox binary (if present)."`
mox!
2023-01-30 16:27:06 +03:00
Hostname string `sconf-doc:"Full hostname of system, e.g. mail.<domain>"`
HostnameDomain dns.Domain `sconf:"-" json:"-"` // Parsed form of hostname.
CheckUpdates bool `sconf:"optional" sconf-doc:"If enabled, a single DNS TXT lookup of _updates.xmox.nl is done every 24h to check for a new release. Each time a new release is found, a changelog is fetched from https://updates.xmox.nl and delivered to the postmaster mailbox."`
add pedantic mode (used by localserve) that refuses some behaviour that is invalid according to specifications and that we normally accept for compatibility
2023-03-12 17:16:01 +03:00
Pedantic bool `sconf:"optional" sconf-doc:"In pedantic mode protocol violations (that happen in the wild) for SMTP/IMAP/etc result in errors instead of accepting such behaviour."`
mox!
2023-01-30 16:27:06 +03:00
TLS struct {
CA *struct {
AdditionalToSystem bool `sconf:"optional"`
CertFiles []string `sconf:"optional"`
} `sconf:"optional"`
CertPool *x509.CertPool `sconf:"-" json:"-"`
use configured tls ca config for all tls connections, so https as well and add documentation for developers for setting up certificates with manual local CA (with cfssl) or local ACME CA (with pebble).
2023-03-10 18:25:18 +03:00
} `sconf:"optional" sconf-doc:"Global TLS configuration, e.g. for additional Certificate Authorities. Used for outgoing SMTP connections, HTTPS requests."`
mox!
2023-01-30 16:27:06 +03:00
ACME map[string]ACME `sconf:"optional" sconf-doc:"Automatic TLS configuration with ACME, e.g. through Let's Encrypt. The key is a name referenced in TLS configs, e.g. letsencrypt."`
AdminPasswordFile string `sconf:"optional" sconf-doc:"File containing hash of admin password, for authentication in the web admin pages (if enabled)."`
at startup, with acme, if the config has explicitly configured public ips (the default with the quickstart), lookup the host names allowed for acme validation and warn about ips that mox is not configured to listen on i've seen this cause acme validation failures 3 times now, so give a hint in the logs to new users. also for issue #13.
2023-03-05 18:22:23 +03:00
Listeners map[string]Listener `sconf-doc:"Listeners are groups of IP addresses and services enabled on those IP addresses, such as SMTP/IMAP or internal endpoints for administration or Prometheus metrics. All listeners with SMTP/IMAP services enabled will serve all configured domains. If the listener is named 'public', it will get a few helpful additional configuration checks, for acme automatic tls certificates and monitoring of ips in dnsbls if those are configured."`
mox!
2023-01-30 16:27:06 +03:00
Postmaster struct {
Account string
Mailbox string `sconf-doc:"E.g. Postmaster or Inbox."`
in smtpserver, accept delivery to postmaster@<hostname>, and also postmaster@ addresses for domains that don't have a postmaster address configured.
2023-04-24 13:04:46 +03:00
} `sconf-doc:"Destination for emails delivered to postmaster addresses: a plain 'postmaster' without domain, 'postmaster@<hostname>' (also for each listener with SMTP enabled), and as fallback for each domain without explicitly configured postmaster destination."`
mox!
2023-01-30 16:27:06 +03:00
DefaultMailboxes []string `sconf:"optional" sconf-doc:"Mailboxes to create when adding an account. Inbox is always created. If no mailboxes are specified, the following are automatically created: Sent, Archive, Trash, Drafts and Junk."`
// All IPs that were explicitly listen on for external SMTP. Only set when there
add reverse ip checks during quickstart and in "check dns" admin page/subcommand - and don't have a global variable "d" in the big checkDomain function in http/admin.go. - and set loglevel from command-line flag again after loading the config file, for all subcommands except "serve".
2023-02-03 17:54:34 +03:00
// are no unspecified external SMTP listeners and there is at most one for IPv4 and
mox!
2023-01-30 16:27:06 +03:00
// at most one for IPv6. Used for setting the local address when making outgoing
// connections. Those IPs are assumed to be in an SPF record for the domain,
// potentially unlike other IPs on the machine. If there is only one address
// family, outgoing connections with the other address family are still made if
// possible.
SpecifiedSMTPListenIPs []net.IP `sconf:"-" json:"-"`
change mox to start as root, bind to network sockets, then drop to regular unprivileged mox user makes it easier to run on bsd's, where you cannot (easily?) let non-root users bind to ports <1024. starting as root also paves the way for future improvements with privilege separation. unfortunately, this requires changes to how you start mox. though mox will help by automatically fix up dir/file permissions/ownership. if you start mox from the systemd unit file, you should update it so it starts as root and adds a few additional capabilities: # first update the mox binary, then, as root: ./mox config printservice >mox.service systemctl daemon-reload systemctl restart mox journalctl -f -u mox & # you should see mox start up, with messages about fixing permissions on dirs/files. if you used the recommended config/ and data/ directory, in a directory just for mox, and with the mox user called "mox", this should be enough. if you don't want mox to modify dir/file permissions, set "NoFixPermissions: true" in mox.conf. if you named the mox user something else than mox, e.g. "_mox", add "User: _mox" to mox.conf. if you created a shared service user as originally suggested, you may want to get rid of that as it is no longer useful and may get in the way. e.g. if you had /home/service/mox with a "service" user, that service user can no longer access any files: only mox and root can. this also adds scripts for building mox docker images for alpine-supported platforms. the "restart" subcommand has been removed. it wasn't all that useful and got in the way. and another change: when adding a domain while mtasts isn't enabled, don't add the per-domain mtasts config, as it would cause failure to add the domain. based on report from setting up mox on openbsd from mteege. and based on issue #3. thanks for the feedback!
2023-02-27 14:19:55 +03:00
// To switch to after initialization as root.
UID uint32 `sconf:"-" json:"-"`
GID uint32 `sconf:"-" json:"-"`
mox!
2023-01-30 16:27:06 +03:00
}
// Dynamic is the parsed form of domains.conf, and is automatically reloaded when changed.
type Dynamic struct {
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
Domains map[string]Domain `sconf-doc:"Domains for which email is accepted. For internationalized domains, use their IDNA names in UTF-8."`
Accounts map[string]Account `sconf-doc:"Accounts to which email can be delivered. An account can accept email for multiple domains, for multiple localparts, and deliver to multiple mailboxes."`
WebDomainRedirects map[string]string `sconf:"optional" sconf-doc:"Redirect all requests from domain (key) to domain (value). Always redirects to HTTPS. For plain HTTP redirects, use a WebHandler with a WebRedirect."`
for WebRedirect, don't "match" when the destination URL has the same scheme,host,path, for doing http -> https redirects without loops you can already get most http to https redirects through DontRedirectPlainHTTP in WebHandler, but that needs handlers for all paths. now you can just set up a redirect for a domain and all its path to baseurl https://domain (leaving other webdirect fields empty). when the request comes in with plain http, the redirect to https is done. that next request will also evaluate the same redirect rule. but it will not cause a match because it would redirect to the same scheme,host,path. so next webhandlers get a chance to serve. also clarify in webhandlers docs that also account & admin built-in handlers run first. related to issue #16
2023-03-09 01:29:44 +03:00
WebHandlers []WebHandler `sconf:"optional" sconf-doc:"Handle webserver requests by serving static files, redirecting or reverse-proxying HTTP(s). The first matching WebHandler will handle the request. Built-in handlers, e.g. for account, admin, autoconfig and mta-sts always run first. If no handler matches, the response status code is file not found (404). If functionality you need is missng, simply forward the requests to an application that can provide the needed functionality."`
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
WebDNSDomainRedirects map[dns.Domain]dns.Domain `sconf:"-"`
mox!
2023-01-30 16:27:06 +03:00
}
type ACME struct {
DirectoryURL string `sconf-doc:"For letsencrypt, use https://acme-v02.api.letsencrypt.org/directory."`
RenewBefore time.Duration `sconf:"optional" sconf-doc:"How long before expiration to renew the certificate. Default is 30 days."`
ContactEmail string `sconf-doc:"Email address to register at ACME provider. The provider can email you when certificates are about to expire. If you configure an address for which email is delivered by this server, keep in mind that TLS misconfigurations could result in such notification emails not arriving."`
make it possible to serve https on a different port than 443 through configuration so you can run mox on openbsd with port redirects in pf.conf. in the future, starting as root, binding the sockets, and passing the bound sockets to a new unprivileged process should be implemented, but this should get openbsd users going. from discussion with mteege
2023-02-18 18:53:06 +03:00
Port int `sconf:"optional" sconf-doc:"TLS port for ACME validation, 443 by default. You should only override this if you cannot listen on port 443 directly. ACME will make requests to port 443, so you'll have to add an external mechanism to get the connection here, e.g. by configuring port forwarding."`
mox!
2023-01-30 16:27:06 +03:00
Manager *autotls.Manager `sconf:"-" json:"-"`
}
type Listener struct {
in quickstart, add -hostname flag and check public ips with 2 dnsbl's - if the guessed hostname is not correct, you can specify one yourself. useful if you generate a config locally and deploy to a different machine. - if explicit public ips are found, check them with spamhaus and spamcop DNSBLs and warn if they are listed, with links to check more DNSBLs. should prevent disappointment later on.
2023-03-05 17:40:26 +03:00
IPs []string `sconf-doc:"Use 0.0.0.0 to listen on all IPv4 and/or :: to listen on all IPv6 addresses, but it is better to explicitly specify the IPs you want to use for email, as mox will make sure outgoing connections will only be made from one of those IPs."`
add config field "IPsNATed" to listener, indicating the IPs are not the actual public IPs but are NATed, to skip a few DNS checks the dns check was returning errors that could not be fixed with that setup, which makes the checks much less useful. for issue #17
2023-03-09 17:24:06 +03:00
IPsNATed bool `sconf:"optional" sconf-doc:"Set this if the specified IPs are not the public IPs, but are NATed. This makes the DNS check skip a few checks related to IPs, such as for iprev, mx, spf, autoconfig, autodiscover."`
mox!
2023-01-30 16:27:06 +03:00
Hostname string `sconf:"optional" sconf-doc:"If empty, the config global Hostname is used."`
HostnameDomain dns.Domain `sconf:"-" json:"-"` // Set when parsing config.
TLS *TLS `sconf:"optional" sconf-doc:"For SMTP/IMAP STARTTLS, direct TLS and HTTPS connections."`
SMTPMaxMessageSize int64 `sconf:"optional" sconf-doc:"Maximum size in bytes accepted incoming and outgoing messages. Default is 100MB."`
SMTP struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 25."`
NoSTARTTLS bool `sconf:"optional" sconf-doc:"Do not offer STARTTLS to secure the connection. Not recommended."`
RequireSTARTTLS bool `sconf:"optional" sconf-doc:"Do not accept incoming messages if STARTTLS is not active. Can be used in combination with a strict MTA-STS policy. A remote SMTP server may not support TLS and may not be able to deliver messages."`
DNSBLs []string `sconf:"optional" sconf-doc:"Addresses of DNS block lists for incoming messages. Block lists are only consulted for connections/messages without enough reputation to make an accept/reject decision. This prevents sending IPs of all communications to the block list provider. If any of the listed DNSBLs contains a requested IP address, the message is rejected as spam. The DNSBLs are checked for healthiness before use, at most once per 4 hours. Example DNSBLs: sbl.spamhaus.org, bl.spamcop.net"`
DNSBLZones []dns.Domain `sconf:"-"`
} `sconf:"optional"`
Submission struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 587."`
NoRequireSTARTTLS bool `sconf:"optional" sconf-doc:"Do not require STARTTLS. Since users must login, this means password may be sent without encryption. Not recommended."`
} `sconf:"optional" sconf-doc:"SMTP for submitting email, e.g. by email applications. Starts out in plain text, can be upgraded to TLS with the STARTTLS command. Prefer using Submissions which is always a TLS connection."`
Submissions struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 465."`
} `sconf:"optional" sconf-doc:"SMTP over TLS for submitting email, by email applications. Requires a TLS config."`
IMAP struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 143."`
NoRequireSTARTTLS bool `sconf:"optional" sconf-doc:"Enable this only when the connection is otherwise encrypted (e.g. through a VPN)."`
} `sconf:"optional" sconf-doc:"IMAP for reading email, by email applications. Starts out in plain text, can be upgraded to TLS with the STARTTLS command. Prefer using IMAPS instead which is always a TLS connection."`
IMAPS struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 993."`
} `sconf:"optional" sconf-doc:"IMAP over TLS for reading email, by email applications. Requires a TLS config."`
make account web page configurable separately from admin, add http auth rate limiting ideally both account & admin web pages should be on non-public ips (e.g. a wireguard tunnel). but during setup, users may not have that set up, and they may want to configure the admin/account pages on their public ip's. the auth rate limiting should make it less of issue. users can now also only put the account web page publicly available. useful for if you're the admin and you have a vpn connection, but your other/external users do not have a vpn into your mail server. to make the account page more easily findable, the http root serves the account page. the admin page is still at /admin/, to prevent clash with potential account pages, but if no account page is present, you are helpfully redirected from / to /admin/. this also adds a prometheus metric counting how often auth attempts have been rate limited.
2023-02-13 15:53:47 +03:00
AccountHTTP struct {
Enabled bool
make http(s) path for serving the account and admin pages configurable so you can use the host (domain) name of the mail server for serving other resources too. the default is is still that account is served on /, and so takes all incoming requests before giving webhandlers a chance. mox localserve now serves the account pages on /account/
2023-03-12 13:52:15 +03:00
Port int `sconf:"optional" sconf-doc:"Default 80."`
Path string `sconf:"optional" sconf-doc:"Path to serve account requests on, e.g. /mox/. Useful if domain serves other resources. Default is /."`
add hint on how to access your admin endpoint
2023-03-05 15:48:24 +03:00
} `sconf:"optional" sconf-doc:"Account web interface, for email users wanting to change their accounts, e.g. set new password, set new delivery rulesets. Served at /."`
make account web page configurable separately from admin, add http auth rate limiting ideally both account & admin web pages should be on non-public ips (e.g. a wireguard tunnel). but during setup, users may not have that set up, and they may want to configure the admin/account pages on their public ip's. the auth rate limiting should make it less of issue. users can now also only put the account web page publicly available. useful for if you're the admin and you have a vpn connection, but your other/external users do not have a vpn into your mail server. to make the account page more easily findable, the http root serves the account page. the admin page is still at /admin/, to prevent clash with potential account pages, but if no account page is present, you are helpfully redirected from / to /admin/. this also adds a prometheus metric counting how often auth attempts have been rate limited.
2023-02-13 15:53:47 +03:00
AccountHTTPS struct {
Enabled bool
make http(s) path for serving the account and admin pages configurable so you can use the host (domain) name of the mail server for serving other resources too. the default is is still that account is served on /, and so takes all incoming requests before giving webhandlers a chance. mox localserve now serves the account pages on /account/
2023-03-12 13:52:15 +03:00
Port int `sconf:"optional" sconf-doc:"Default 80."`
Path string `sconf:"optional" sconf-doc:"Path to serve account requests on, e.g. /mox/. Useful if domain serves other resources. Default is /."`
make account web page configurable separately from admin, add http auth rate limiting ideally both account & admin web pages should be on non-public ips (e.g. a wireguard tunnel). but during setup, users may not have that set up, and they may want to configure the admin/account pages on their public ip's. the auth rate limiting should make it less of issue. users can now also only put the account web page publicly available. useful for if you're the admin and you have a vpn connection, but your other/external users do not have a vpn into your mail server. to make the account page more easily findable, the http root serves the account page. the admin page is still at /admin/, to prevent clash with potential account pages, but if no account page is present, you are helpfully redirected from / to /admin/. this also adds a prometheus metric counting how often auth attempts have been rate limited.
2023-02-13 15:53:47 +03:00
} `sconf:"optional" sconf-doc:"Account web interface listener for HTTPS. Requires a TLS config."`
mox!
2023-01-30 16:27:06 +03:00
AdminHTTP struct {
Enabled bool
make http(s) path for serving the account and admin pages configurable so you can use the host (domain) name of the mail server for serving other resources too. the default is is still that account is served on /, and so takes all incoming requests before giving webhandlers a chance. mox localserve now serves the account pages on /account/
2023-03-12 13:52:15 +03:00
Port int `sconf:"optional" sconf-doc:"Default 80."`
Path string `sconf:"optional" sconf-doc:"Path to serve admin requests on, e.g. /moxadmin/. Useful if domain serves other resources. Default is /admin/."`
fix language in comments found through goreportcard.com
2023-05-22 16:04:06 +03:00
} `sconf:"optional" sconf-doc:"Admin web interface, for managing domains, accounts, etc. Served at /admin/. Preferably only enable on non-public IPs. Hint: use 'ssh -L 8080:localhost:80 you@yourmachine' and open http://localhost:8080/admin/, or set up a tunnel (e.g. WireGuard) and add its IP to the mox 'internal' listener."`
mox!
2023-01-30 16:27:06 +03:00
AdminHTTPS struct {
Enabled bool
make http(s) path for serving the account and admin pages configurable so you can use the host (domain) name of the mail server for serving other resources too. the default is is still that account is served on /, and so takes all incoming requests before giving webhandlers a chance. mox localserve now serves the account pages on /account/
2023-03-12 13:52:15 +03:00
Port int `sconf:"optional" sconf-doc:"Default 443."`
Path string `sconf:"optional" sconf-doc:"Path to serve admin requests on, e.g. /moxadmin/. Useful if domain serves other resources. Default is /admin/."`
fix language in comments found through goreportcard.com
2023-05-22 16:04:06 +03:00
} `sconf:"optional" sconf-doc:"Admin web interface listener for HTTPS. Requires a TLS config. Preferably only enable on non-public IPs."`
mox!
2023-01-30 16:27:06 +03:00
MetricsHTTP struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 8010."`
} `sconf:"optional" sconf-doc:"Serve prometheus metrics, for monitoring. You should not enable this on a public IP."`
PprofHTTP struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Default 8011."`
} `sconf:"optional" sconf-doc:"Serve /debug/pprof/ for profiling a running mox instance. Do not enable this on a public IP!"`
AutoconfigHTTPS struct {
Enabled bool
add option to handle autoconfig and mta-sts requests without TLS, for when it is reverse proxied for #5 with hints from belst & idnovic
2023-02-25 13:28:15 +03:00
Port int `sconf:"optional" sconf-doc:"TLS port, 443 by default. You should only override this if you cannot listen on port 443 directly. Autoconfig requests will be made to port 443, so you'll have to add an external mechanism to get the connection here, e.g. by configuring port forwarding."`
NonTLS bool `sconf:"optional" sconf-doc:"If set, plain HTTP instead of HTTPS is spoken on the configured port. Can be useful when the autoconfig domain is reverse proxied."`
mox!
2023-01-30 16:27:06 +03:00
} `sconf:"optional" sconf-doc:"Serve autoconfiguration/autodiscovery to simplify configuring email applications, will use port 443. Requires a TLS config."`
MTASTSHTTPS struct {
Enabled bool
add option to handle autoconfig and mta-sts requests without TLS, for when it is reverse proxied for #5 with hints from belst & idnovic
2023-02-25 13:28:15 +03:00
Port int `sconf:"optional" sconf-doc:"TLS port, 443 by default. You should only override this if you cannot listen on port 443 directly. MTA-STS requests will be made to port 443, so you'll have to add an external mechanism to get the connection here, e.g. by configuring port forwarding."`
NonTLS bool `sconf:"optional" sconf-doc:"If set, plain HTTP instead of HTTPS is spoken on the configured port. Can be useful when the mta-sts domain is reverse proxied."`
make it possible to serve https on a different port than 443 through configuration so you can run mox on openbsd with port redirects in pf.conf. in the future, starting as root, binding the sockets, and passing the bound sockets to a new unprivileged process should be implemented, but this should get openbsd users going. from discussion with mteege
2023-02-18 18:53:06 +03:00
} `sconf:"optional" sconf-doc:"Serve MTA-STS policies describing SMTP TLS requirements. Requires a TLS config."`
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
WebserverHTTP struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Port for plain HTTP (non-TLS) webserver."`
} `sconf:"optional" sconf-doc:"All configured WebHandlers will serve on an enabled listener."`
WebserverHTTPS struct {
Enabled bool
Port int `sconf:"optional" sconf-doc:"Port for HTTPS webserver."`
} `sconf:"optional" sconf-doc:"All configured WebHandlers will serve on an enabled listener. Either ACME must be configured, or for each WebHandler domain a TLS certificate must be configured."`
mox!
2023-01-30 16:27:06 +03:00
}
type Domain struct {
Description string `sconf:"optional" sconf-doc:"Free-form description of domain."`
LocalpartCatchallSeparator string `sconf:"optional" sconf-doc:"If not empty, only the string before the separator is used to for email delivery decisions. For example, if set to \"+\", you+anything@example.com will be delivered to you@example.com."`
LocalpartCaseSensitive bool `sconf:"optional" sconf-doc:"If set, upper/lower case is relevant for email delivery."`
DKIM DKIM `sconf:"optional" sconf-doc:"With DKIM signing, a domain is taking responsibility for (content of) emails it sends, letting receiving mail servers build up a (hopefully positive) reputation of the domain, which can help with mail delivery."`
DMARC *DMARC `sconf:"optional" sconf-doc:"With DMARC, a domain publishes, in DNS, a policy on how other mail servers should handle incoming messages with the From-header matching this domain and/or subdomain (depending on the configured alignment). Receiving mail servers use this to build up a reputation of this domain, which can help with mail delivery. A domain can also publish an email address to which reports about DMARC verification results can be sent by verifying mail servers, useful for monitoring. Incoming DMARC reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
MTASTS *MTASTS `sconf:"optional" sconf-doc:"With MTA-STS a domain publishes, in DNS, presence of a policy for using/requiring TLS for SMTP connections. The policy is served over HTTPS."`
TLSRPT *TLSRPT `sconf:"optional" sconf-doc:"With TLSRPT a domain specifies in DNS where reports about encountered SMTP TLS behaviour should be sent. Useful for monitoring. Incoming TLS reports are automatically parsed, validated, added to metrics and stored in the reporting database for later display in the admin web pages."`
Domain dns.Domain `sconf:"-" json:"-"`
}
type DMARC struct {
Localpart string `sconf-doc:"Address-part before the @ that accepts DMARC reports. Must be non-internationalized. Recommended value: dmarc-reports."`
Account string `sconf-doc:"Account to deliver to."`
Mailbox string `sconf-doc:"Mailbox to deliver to, e.g. DMARC."`
ParsedLocalpart smtp.Localpart `sconf:"-"`
}
type MTASTS struct {
PolicyID string `sconf-doc:"Policies are versioned. The version must be specified in the DNS record. If you change a policy, first change it in mox, then update the DNS record."`
Mode mtasts.Mode `sconf-doc:"testing, enforce or none. If set to enforce, a remote SMTP server will not deliver email to us if it cannot make a TLS connection."`
MaxAge time.Duration `sconf-doc:"How long a remote mail server is allowed to cache a policy. Typically 1 or several weeks."`
MX []string `sconf:"optional" sconf-doc:"List of server names allowed for SMTP. If empty, the configured hostname is set. Host names can contain a wildcard (*) as a leading label (matching a single label, e.g. *.example matches host.example, not sub.host.example)."`
// todo: parse mx as valid mtasts.Policy.MX, with dns.ParseDomain but taking wildcard into account
}
type TLSRPT struct {
Localpart string `sconf-doc:"Address-part before the @ that accepts TLSRPT reports. Recommended value: tls-reports."`
Account string `sconf-doc:"Account to deliver to."`
Mailbox string `sconf-doc:"Mailbox to deliver to, e.g. TLSRPT."`
ParsedLocalpart smtp.Localpart `sconf:"-"`
}
type Selector struct {
Hash string `sconf:"optional" sconf-doc:"sha256 (default) or (older, not recommended) sha1"`
HashEffective string `sconf:"-"`
Canonicalization struct {
HeaderRelaxed bool `sconf-doc:"If set, some modifications to the headers (mostly whitespace) are allowed."`
BodyRelaxed bool `sconf-doc:"If set, some whitespace modifications to the message body are allowed."`
} `sconf:"optional"`
Headers []string `sconf:"optional" sconf-doc:"Headers to sign with DKIM. If empty, a reasonable default set of headers is selected."`
HeadersEffective []string `sconf:"-"`
DontSealHeaders bool `sconf:"optional" sconf-doc:"If set, don't prevent duplicate headers from being added. Not recommended."`
Expiration string `sconf:"optional" sconf-doc:"Period a signature is valid after signing, as duration, e.g. 72h. The period should be enough for delivery at the final destination, potentially with several hops/relays. In the order of days at least."`
PrivateKeyFile string `sconf-doc:"Either an RSA or ed25519 private key file in PKCS8 PEM form."`
ExpirationSeconds int `sconf:"-" json:"-"` // Parsed from Expiration.
Key crypto.Signer `sconf:"-" json:"-"` // As parsed with x509.ParsePKCS8PrivateKey.
Domain dns.Domain `sconf:"-" json:"-"` // Of selector only, not FQDN.
}
type DKIM struct {
Selectors map[string]Selector `sconf-doc:"Emails can be DKIM signed. Config parameters are per selector. A DNS record must be created for each selector. Add the name to Sign to use the selector for signing messages."`
Sign []string `sconf:"optional" sconf-doc:"List of selectors that emails will be signed with."`
}
type Account struct {
deprecate having only localparts in an Account's Destinations, it should always be a full email address current behaviour isn't intuitive. it's not great to have to attempt parsing the strings as both localpart and email address. so we deprecate the localpart-only behaviour. when we load the config file, and it has localpart-only Destinations keys, we'll change them to full addresses in memory. when an admin causes a write of domains.conf, it'll automatically be fixed. we log an error with a deprecated notice for each localpart-only destinations key. sometime in the future, we can remove the old localpart-only destination support. will be in the release notes then. also start keeping track of update notes that need to make it in the release notes of the next release. for issue #18
2023-03-10 00:07:37 +03:00
Domain string `sconf-doc:"Default domain for account. Deprecated behaviour: If a destination is not a full address but only a localpart, this domain is added to form a full address."`
mox!
2023-01-30 16:27:06 +03:00
Description string `sconf:"optional" sconf-doc:"Free form description, e.g. full name or alternative contact info."`
implement a catchall address for a domain by specifying a "destination" in an account that is just "@" followed by the domain, e.g. "@example.org". messages are only delivered to the catchall address when no regular destination matches (taking the per-domain catchall-separator and case-sensisitivity into account). for issue #18
2023-03-29 22:11:43 +03:00
Destinations map[string]Destination `sconf-doc:"Destinations, keys are email addresses (with IDNA domains). If the address is of the form '@domain', i.e. with localpart missing, it serves as a catchall for the domain, matching all messages that are not explicitly configured. Deprecated behaviour: If the address is not a full address but a localpart, it is combined with Domain to form a full address."`
mox!
2023-01-30 16:27:06 +03:00
SubjectPass struct {
Period time.Duration `sconf-doc:"How long unique values are accepted after generating, e.g. 12h."` // todo: have a reasonable default for this?
} `sconf:"optional" sconf-doc:"If configured, messages classified as weakly spam are rejected with instructions to retry delivery, but this time with a signed token added to the subject. During the next delivery attempt, the signed token will bypass the spam filter. Messages with a clear spam signal, such as a known bad reputation, are rejected/delayed without a signed token."`
improve training of junk filter before, we used heuristics to decide when to train/untrain a message as junk or nonjunk: the message had to be seen, be in certain mailboxes. then if a message was marked as junk, it was junk. and otherwise it was nonjunk. this wasn't good enough: you may want to keep some messages around as neither junk or nonjunk. and that wasn't possible. ideally, we would just look at the imap $Junk and $NotJunk flags. the problem is that mail clients don't set these flags, or don't make it easy. thunderbird can set the flags based on its own bayesian filter. it has a shortcut for marking Junk and moving it to the junk folder (good), but the counterpart of notjunk only marks a message as notjunk without showing in the UI that it was marked as notjunk. there is also no "move and mark as notjunk" mechanism. e.g. "archive" does not mark a message as notjunk. ios mail and mutt don't appear to have any way to see or change the $Junk and $NotJunk flags. what email clients do have is the ability to move messages to other mailboxes/folders. so mox now has a mechanism that allows you to configure mailboxes that automatically set $Junk or $NotJunk (or clear both) when a message is moved/copied/delivered to that folder. e.g. a mailbox called junk or spam or rejects marks its messags as junk. inbox, postmaster, dmarc, tlsrpt, neutral* mark their messages as neither junk or notjunk. other folders mark their messages as notjunk. e.g. list/*, archive. this functionality is optional, but enabled with the quickstart and for new accounts. also, mox now keeps track of the previous training of a message and will only untrain/train if needed. before, there probably have been duplicate or missing (un)trainings. this also includes a new subcommand "retrain" to recreate the junkfilter for an account. you should run it after updating to this version. and you should probably also modify your account config to include the AutomaticJunkFlags.
2023-02-12 01:00:12 +03:00
RejectsMailbox string `sconf:"optional" sconf-doc:"Mail that looks like spam will be rejected, but a copy can be stored temporarily in a mailbox, e.g. Rejects. If mail isn't coming in when you expect, you can look there. The mail still isn't accepted, so the remote mail server may retry (hopefully, if legitimate), or give up (hopefully, if indeed a spammer). Messages are automatically removed from this mailbox, so do not set it to a mailbox that has messages you want to keep."`
AutomaticJunkFlags struct {
store rejects for 14 days, and don't keep them as neutral by default so they won't cause outright rejects for repeated delivery attempts of spam messages the previous default, marking the messages as junk had the interesting effect of training the junk filter. rejecting could have been the result of the sending IP being in the DNSBL. so the DNSBL helped to automatically train the junk filter. perhaps we can keep that in the future and just not take messages from the rejects mailbox into account when evaluating the reputation for incoming deliveries.
2023-02-13 12:47:20 +03:00
Enabled bool `sconf-doc:"If enabled, flags will be set automatically if they match a regular expression below. When two of the three mailbox regular expressions are set, the remaining one will match all unmatched messages. Messages are matched in the order specified and the search stops on the first match. Mailboxes are lowercased before matching."`
JunkMailboxRegexp string `sconf:"optional" sconf-doc:"Example: ^(junk|spam)."`
NeutralMailboxRegexp string `sconf:"optional" sconf-doc:"Example: ^(inbox|neutral|postmaster|dmarc|tlsrpt|rejects), and you may wish to add trash depending on how you use it, or leave this empty."`
improve training of junk filter before, we used heuristics to decide when to train/untrain a message as junk or nonjunk: the message had to be seen, be in certain mailboxes. then if a message was marked as junk, it was junk. and otherwise it was nonjunk. this wasn't good enough: you may want to keep some messages around as neither junk or nonjunk. and that wasn't possible. ideally, we would just look at the imap $Junk and $NotJunk flags. the problem is that mail clients don't set these flags, or don't make it easy. thunderbird can set the flags based on its own bayesian filter. it has a shortcut for marking Junk and moving it to the junk folder (good), but the counterpart of notjunk only marks a message as notjunk without showing in the UI that it was marked as notjunk. there is also no "move and mark as notjunk" mechanism. e.g. "archive" does not mark a message as notjunk. ios mail and mutt don't appear to have any way to see or change the $Junk and $NotJunk flags. what email clients do have is the ability to move messages to other mailboxes/folders. so mox now has a mechanism that allows you to configure mailboxes that automatically set $Junk or $NotJunk (or clear both) when a message is moved/copied/delivered to that folder. e.g. a mailbox called junk or spam or rejects marks its messags as junk. inbox, postmaster, dmarc, tlsrpt, neutral* mark their messages as neither junk or notjunk. other folders mark their messages as notjunk. e.g. list/*, archive. this functionality is optional, but enabled with the quickstart and for new accounts. also, mox now keeps track of the previous training of a message and will only untrain/train if needed. before, there probably have been duplicate or missing (un)trainings. this also includes a new subcommand "retrain" to recreate the junkfilter for an account. you should run it after updating to this version. and you should probably also modify your account config to include the AutomaticJunkFlags.
2023-02-12 01:00:12 +03:00
NotJunkMailboxRegexp string `sconf:"optional" sconf-doc:"Example: .* or an empty string."`
} `sconf:"optional" sconf-doc:"Automatically set $Junk and $NotJunk flags based on mailbox messages are delivered/moved/copied to. Email clients typically have too limited functionality to conveniently set these flags, especially $NonJunk, but they can all move messages to a different mailbox, so this helps them."`
implement limits on outgoing messages for an account by default 1000 messages per day, and to max 200 first-time receivers. i don't think a person would reach those limits. a compromised account abused by spammers could easily reach that limit. this prevents further damage. the error message you will get is quite clear, pointing to the configuration parameter that should be changed.
2023-03-28 21:50:36 +03:00
JunkFilter *JunkFilter `sconf:"optional" sconf-doc:"Content-based filtering, using the junk-status of individual messages to rank words in such messages as spam or ham. It is recommended you always set the applicable (non)-junk status on messages, and that you do not empty your Trash because those messages contain valuable ham/spam training information."` // todo: sane defaults for junkfilter
MaxOutgoingMessagesPerDay int `sconf:"optional" sconf-doc:"Maximum number of outgoing messages for this account in a 24 hour window. This limits the damage to recipients and the reputation of this mail server in case of account compromise. Default 1000."`
MaxFirstTimeRecipientsPerDay int `sconf:"optional" sconf-doc:"Maximum number of first-time recipients in outgoing messages for this account in a 24 hour window. This limits the damage to recipients and the reputation of this mail server in case of account compromise. Default 200."`
improve training of junk filter before, we used heuristics to decide when to train/untrain a message as junk or nonjunk: the message had to be seen, be in certain mailboxes. then if a message was marked as junk, it was junk. and otherwise it was nonjunk. this wasn't good enough: you may want to keep some messages around as neither junk or nonjunk. and that wasn't possible. ideally, we would just look at the imap $Junk and $NotJunk flags. the problem is that mail clients don't set these flags, or don't make it easy. thunderbird can set the flags based on its own bayesian filter. it has a shortcut for marking Junk and moving it to the junk folder (good), but the counterpart of notjunk only marks a message as notjunk without showing in the UI that it was marked as notjunk. there is also no "move and mark as notjunk" mechanism. e.g. "archive" does not mark a message as notjunk. ios mail and mutt don't appear to have any way to see or change the $Junk and $NotJunk flags. what email clients do have is the ability to move messages to other mailboxes/folders. so mox now has a mechanism that allows you to configure mailboxes that automatically set $Junk or $NotJunk (or clear both) when a message is moved/copied/delivered to that folder. e.g. a mailbox called junk or spam or rejects marks its messags as junk. inbox, postmaster, dmarc, tlsrpt, neutral* mark their messages as neither junk or notjunk. other folders mark their messages as notjunk. e.g. list/*, archive. this functionality is optional, but enabled with the quickstart and for new accounts. also, mox now keeps track of the previous training of a message and will only untrain/train if needed. before, there probably have been duplicate or missing (un)trainings. this also includes a new subcommand "retrain" to recreate the junkfilter for an account. you should run it after updating to this version. and you should probably also modify your account config to include the AutomaticJunkFlags.
2023-02-12 01:00:12 +03:00
DNSDomain dns.Domain `sconf:"-"` // Parsed form of Domain.
JunkMailbox *regexp.Regexp `sconf:"-" json:"-"`
NeutralMailbox *regexp.Regexp `sconf:"-" json:"-"`
NotJunkMailbox *regexp.Regexp `sconf:"-" json:"-"`
mox!
2023-01-30 16:27:06 +03:00
}
type JunkFilter struct {
Threshold float64 `sconf-doc:"Approximate spaminess score between 0 and 1 above which emails are rejected as spam. Each delivery attempt adds a little noise to make it slightly harder for spammers to identify words that strongly indicate non-spaminess and use it to bypass the filter. E.g. 0.95."`
junk.Params
}
type Destination struct {
Mailbox string `sconf:"optional" sconf-doc:"Mailbox to deliver to if none of Rulesets match. Default: Inbox."`
Rulesets []Ruleset `sconf:"optional" sconf-doc:"Delivery rules based on message and SMTP transaction. You may want to match each mailing list by SMTP MailFrom address, VerifiedDomain and/or List-ID header (typically <listname.example.org> if the list address is listname@example.org), delivering them to their own mailbox."`
DMARCReports bool `sconf:"-" json:"-"`
TLSReports bool `sconf:"-" json:"-"`
}
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
// Equal returns whether d and o are equal, only looking at their user-changeable fields.
func (d Destination) Equal(o Destination) bool {
if d.Mailbox != o.Mailbox || len(d.Rulesets) != len(o.Rulesets) {
return false
}
for i, rs := range d.Rulesets {
if !rs.Equal(o.Rulesets[i]) {
return false
}
}
return true
}
mox!
2023-01-30 16:27:06 +03:00
type Ruleset struct {
SMTPMailFromRegexp string `sconf:"optional" sconf-doc:"Matches if this regular expression matches (a substring of) the SMTP MAIL FROM address (not the message From-header). E.g. user@example.org."`
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
VerifiedDomain string `sconf:"optional" sconf-doc:"Matches if this domain matches an SPF- and/or DKIM-verified (sub)domain."`
HeadersRegexp map[string]string `sconf:"optional" sconf-doc:"Matches if these header field/value regular expressions all match (substrings of) the message headers. Header fields and valuees are converted to lower case before matching. Whitespace is trimmed from the value before matching. A header field can occur multiple times in a message, only one instance has to match. For mailing lists, you could match on ^list-id$ with the value typically the mailing list address in angled brackets with @ replaced with a dot, e.g. <name\\.lists\\.example\\.org>."`
mox!
2023-01-30 16:27:06 +03:00
// todo: add a SMTPRcptTo check, and MessageFrom that works on a properly parsed From header.
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
ListAllowDomain string `sconf:"optional" sconf-doc:"Influence the spam filtering, this does not change whether this ruleset applies to a message. If this domain matches an SPF- and/or DKIM-verified (sub)domain, the message is accepted without further spam checks, such as a junk filter or DMARC reject evaluation. DMARC rejects should not apply for mailing lists that are not configured to rewrite the From-header of messages that don't have a passing DKIM signature of the From-domain. Otherwise, by rejecting messages, you may be automatically unsubscribed from the mailing list. The assumption is that mailing lists do their own spam filtering/moderation."`
mox!
2023-01-30 16:27:06 +03:00
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
Mailbox string `sconf-doc:"Mailbox to deliver to if this ruleset matches."`
mox!
2023-01-30 16:27:06 +03:00
SMTPMailFromRegexpCompiled *regexp.Regexp `sconf:"-" json:"-"`
VerifiedDNSDomain dns.Domain `sconf:"-"`
HeadersRegexpCompiled [][2]*regexp.Regexp `sconf:"-" json:"-"`
ListAllowDNSDomain dns.Domain `sconf:"-"`
}
in account web page, allow user to configure rulesets for delivery for example, by matching incoming messags on smtp mail from, verified domains (spf/dkim), headers. then delivering to a configured mailbox. for mailing lists, if a verified domain matches, regular spam checks can be skipped. this was already possible by editing the configuration file, but only admins can edit that file. now users can manage their own rulesets.
2023-02-11 01:47:19 +03:00
// Equal returns whether r and o are equal, only looking at their user-changeable fields.
func (r Ruleset) Equal(o Ruleset) bool {
if r.SMTPMailFromRegexp != o.SMTPMailFromRegexp || r.VerifiedDomain != o.VerifiedDomain || r.ListAllowDomain != o.ListAllowDomain || r.Mailbox != o.Mailbox {
return false
}
if !reflect.DeepEqual(r.HeadersRegexp, o.HeadersRegexp) {
return false
}
return true
}
make it easier to run with existing webserver - make it easier to run with an existing webserver. the quickstart now has a new option for that, it generates a different mox.conf, and further instructions such as configuring the tls keys/certs and reverse proxy urls. and changes to make autoconfig work in that case too. - when starting up, request a tls cert for the hostname and for the autoconfig endpoint. the first will be requested soon anyway, and the autoconfig cert is needed early so the first autoconfig request doesn't time out (without helpful message to the user by at least thunderbird). and don't request the certificate before the servers are online. the root process was now requesting the certs, before the child process was serving on the tls port. - add examples of configs generated by the quickstart. - enable debug logging in config from quickstart, to give user more info. for issue #5
2023-03-04 02:49:02 +03:00
type KeyCert struct {
CertFile string `sconf-doc:"Certificate including intermediate CA certificates, in PEM format."`
KeyFile string `sconf-doc:"Private key for certificate, in PEM format. PKCS8 is recommended, but PKCS1 and EC private keys are recognized as well."`
}
mox!
2023-01-30 16:27:06 +03:00
type TLS struct {
make it easier to run with existing webserver - make it easier to run with an existing webserver. the quickstart now has a new option for that, it generates a different mox.conf, and further instructions such as configuring the tls keys/certs and reverse proxy urls. and changes to make autoconfig work in that case too. - when starting up, request a tls cert for the hostname and for the autoconfig endpoint. the first will be requested soon anyway, and the autoconfig cert is needed early so the first autoconfig request doesn't time out (without helpful message to the user by at least thunderbird). and don't request the certificate before the servers are online. the root process was now requesting the certs, before the child process was serving on the tls port. - add examples of configs generated by the quickstart. - enable debug logging in config from quickstart, to give user more info. for issue #5
2023-03-04 02:49:02 +03:00
ACME string `sconf:"optional" sconf-doc:"Name of provider from top-level configuration to use for ACME, e.g. letsencrypt."`
KeyCerts []KeyCert `sconf:"optional"`
MinVersion string `sconf:"optional" sconf-doc:"Minimum TLS version. Default: TLSv1.2."`
mox!
2023-01-30 16:27:06 +03:00
add a bit more logging for non-SNI tls requests for peace of mind. these are probably requests to port 443 without SNI.
2023-02-25 14:56:25 +03:00
Config *tls.Config `sconf:"-" json:"-"` // TLS config for non-ACME-verification connections, i.e. SMTP and IMAP, and not port 443.
ACMEConfig *tls.Config `sconf:"-" json:"-"` // TLS config that handles ACME verification, for serving on port 443.
mox!
2023-01-30 16:27:06 +03:00
}
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
type WebHandler struct {
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
LogName string `sconf:"optional" sconf-doc:"Name to use in logging and metrics."`
Domain string `sconf-doc:"Both Domain and PathRegexp must match for this WebHandler to match a request. Exactly one of WebStatic, WebRedirect, WebForward must be set."`
PathRegexp string `sconf-doc:"Regular expression matched against request path, must always start with ^ to ensure matching from the start of the path. The matching prefix can optionally be stripped by WebForward. The regular expression does not have to end with $."`
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
DontRedirectPlainHTTP bool `sconf:"optional" sconf-doc:"If set, plain HTTP requests are not automatically permanently redirected (308) to HTTPS. If you don't have a HTTPS webserver configured, set this to true."`
WebStatic *WebStatic `sconf:"optional" sconf-doc:"Serve static files."`
WebRedirect *WebRedirect `sconf:"optional" sconf-doc:"Redirect requests to configured URL."`
WebForward *WebForward `sconf:"optional" sconf-doc:"Forward requests to another webserver, i.e. reverse proxy."`
Name string `sconf:"-"` // Either LogName, or numeric index if LogName was empty. Used instead of LogName in logging/metrics.
DNSDomain dns.Domain `sconf:"-"`
Path *regexp.Regexp `sconf:"-" json:"-"`
}
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
// Equal returns if wh and o are equal, only looking at fields in the configuration file, not the derived fields.
func (wh WebHandler) Equal(o WebHandler) bool {
clean := func(x WebHandler) WebHandler {
x.Name = ""
x.DNSDomain = dns.Domain{}
x.Path = nil
x.WebStatic = nil
x.WebRedirect = nil
x.WebForward = nil
return x
}
cwh := clean(wh)
co := clean(o)
if cwh != co {
return false
}
if (wh.WebStatic == nil) != (o.WebStatic == nil) || (wh.WebRedirect == nil) != (o.WebRedirect == nil) || (wh.WebForward == nil) != (o.WebForward == nil) {
return false
}
if wh.WebStatic != nil {
return reflect.DeepEqual(wh.WebStatic, o.WebStatic)
}
if wh.WebRedirect != nil {
return wh.WebRedirect.equal(*o.WebRedirect)
}
if wh.WebForward != nil {
return wh.WebForward.equal(*o.WebForward)
}
return true
}
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
type WebStatic struct {
StripPrefix string `sconf:"optional" sconf-doc:"Path to strip from the request URL before evaluating to a local path. If the requested URL path does not start with this prefix and ContinueNotFound it is considered non-matching and next WebHandlers are tried. If ContinueNotFound is not set, a file not found (404) is returned in that case."`
Root string `sconf-doc:"Directory to serve files from for this handler. Keep in mind that relative paths are relative to the working directory of mox."`
ListFiles bool `sconf:"optional" sconf-doc:"If set, and a directory is requested, and no index.html is present that can be served, a file listing is returned. Results in 403 if ListFiles is not set. If a directory is requested and the URL does not end with a slash, the response is a redirect to the path with trailing slash."`
ContinueNotFound bool `sconf:"optional" sconf-doc:"If a requested URL does not exist, don't return a file not found (404) response, but consider this handler non-matching and continue attempts to serve with later WebHandlers, which may be a reverse proxy generating dynamic content, possibly even writing a static file for a next request to serve statically. If ContinueNotFound is set, HTTP requests other than GET and HEAD do not match. This mechanism can be used to implement the equivalent of 'try_files' in other webservers."`
ResponseHeaders map[string]string `sconf:"optional" sconf-doc:"Headers to add to the response. Useful for cache-control, content-type, etc. By default, Content-Type headers are automatically added for recognized file types, unless added explicitly through this setting. For directory listings, a content-type header is skipped."`
}
type WebRedirect struct {
for WebRedirect, don't "match" when the destination URL has the same scheme,host,path, for doing http -> https redirects without loops you can already get most http to https redirects through DontRedirectPlainHTTP in WebHandler, but that needs handlers for all paths. now you can just set up a redirect for a domain and all its path to baseurl https://domain (leaving other webdirect fields empty). when the request comes in with plain http, the redirect to https is done. that next request will also evaluate the same redirect rule. but it will not cause a match because it would redirect to the same scheme,host,path. so next webhandlers get a chance to serve. also clarify in webhandlers docs that also account & admin built-in handlers run first. related to issue #16
2023-03-09 01:29:44 +03:00
BaseURL string `sconf:"optional" sconf-doc:"Base URL to redirect to. The path must be empty and will be replaced, either by the request URL path, or by OrigPathRegexp/ReplacePath. Scheme, host, port and fragment stay intact, and query strings are combined. If empty, the response redirects to a different path through OrigPathRegexp and ReplacePath, which must then be set. Use a URL without scheme to redirect without changing the protocol, e.g. //newdomain/. If a redirect would send a request to a URL with the same scheme, host and path, the WebRedirect does not match so a next WebHandler can be tried. This can be used to redirect all plain http traffic to https."`
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
OrigPathRegexp string `sconf:"optional" sconf-doc:"Regular expression for matching path. If set and path does not match, a 404 is returned. The HTTP path used for matching always starts with a slash."`
ReplacePath string `sconf:"optional" sconf-doc:"Replacement path for destination URL based on OrigPathRegexp. Implemented with Go's Regexp.ReplaceAllString: $1 is replaced with the text of the first submatch, etc. If both OrigPathRegexp and ReplacePath are empty, BaseURL must be set and all paths are redirected unaltered."`
StatusCode int `sconf:"optional" sconf-doc:"Status code to use in redirect, e.g. 307. By default, a permanent redirect (308) is returned."`
URL *url.URL `sconf:"-" json:"-"`
OrigPath *regexp.Regexp `sconf:"-" json:"-"`
}
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
func (wr WebRedirect) equal(o WebRedirect) bool {
wr.URL = nil
wr.OrigPath = nil
o.URL = nil
o.OrigPath = nil
return reflect.DeepEqual(wr, o)
}
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
type WebForward struct {
StripPath bool `sconf:"optional" sconf-doc:"Strip the matching WebHandler path from the WebHandler before forwarding the request."`
add reverse proxying websocket connections if we recognize that a request for a WebForward is trying to turn the connection into a websocket, we forward it to the backend and check if the backend understands the websocket request. if so, we pass back the upgrade response and get out of the way, copying bytes between the two. we do log the total amount of bytes read from the client and written to the client. if the backend doesn't respond with a websocke response, or an invalid one, we respond with a regular non-websocket response. and we log details about the failed connection, should help with debugging and any bug reports. we don't try to parse the websocket framing, that's between the client and the backend. we could try to parse it, in part to protect the backend from bad frames, but it would be a lot of work and could be brittle in the face of extensions. this doesn't yet handle websocket connections when a http proxy is configured. we'll implement it when someone needs it. we do recognize it and fail the connection. for issue #25
2023-05-30 23:11:31 +03:00
URL string `sconf-doc:"URL to forward HTTP requests to, e.g. http://127.0.0.1:8123/base. If StripPath is false the full request path is added to the URL. Host headers are sent unmodified. New X-Forwarded-{For,Host,Proto} headers are set. Any query string in the URL is ignored. Requests are made using Go's net/http.DefaultTransport that takes environment variables HTTP_PROXY and HTTPS_PROXY into account. Websocket connections are forwarded and data is copied between client and backend without looking at the framing. The websocket 'version' and 'key'/'accept' headers are verified during the handshake, but other websocket headers, including 'origin', 'protocol' and 'extensions' headers, are not inspected and the backend is responsible for verifying/interpreting them."`
add basic webserver that can do most of what i need - serve static files, serving index.html or optionally listings for directories - redirects - reverse-proxy, forwarding requests to a backend these are configurable through the config file. a domain and path regexp have to be configured. path prefixes can be stripped. configured domains are added to the autotls allowlist, so acme automatically fetches certificates for them. all webserver requests now have (access) logging, metrics, rate limiting. on http errors, the error message prints an encrypted cid for relating with log files. this also adds a new mechanism for example config files.
2023-03-01 00:12:27 +03:00
ResponseHeaders map[string]string `sconf:"optional" sconf-doc:"Headers to add to the response. Useful for adding security- and cache-related headers."`
TargetURL *url.URL `sconf:"-" json:"-"`
}
improve webserver, add domain redirects (aliases), add tests and admin page ui to manage the config - make builtin http handlers serve on specific domains, such as for mta-sts, so e.g. /.well-known/mta-sts.txt isn't served on all domains. - add logging of a few more fields in access logging. - small tweaks/bug fixes in webserver request handling. - add config option for redirecting entire domains to another (common enough). - split httpserver metric into two: one for duration until writing header (i.e. performance of server), another for duration until full response is sent to client (i.e. performance as perceived by users). - add admin ui, a new page for managing the configs. after making changes and hitting "save", the changes take effect immediately. the page itself doesn't look very well-designed (many input fields, makes it look messy). i have an idea to improve it (explained in admin.html as todo) by making the layout look just like the config file. not urgent though. i've already changed my websites/webapps over. the idea of adding a webserver is to take away a (the) reason for folks to want to complicate their mox setup by running an other webserver on the same machine. i think the current webserver implementation can already serve most common use cases. with a few more tweaks (feedback needed!) we should be able to get to 95% of the use cases. the reverse proxy can take care of the remaining 5%. nevertheless, a next step is still to change the quickstart to make it easier for folks to run with an existing webserver, with existing tls certs/keys. that's how this relates to issue #5.
2023-03-02 20:15:54 +03:00
func (wf WebForward) equal(o WebForward) bool {
wf.TargetURL = nil
o.TargetURL = nil
return reflect.DeepEqual(wf, o)
}
Reference in a new issue Copy permalink