diff --git a/README.md b/README.md index 5e5978f..724245a 100644 --- a/README.md +++ b/README.md @@ -61,28 +61,36 @@ Note: Mox only compiles for/works on unix systems, not on Plan 9 or Windows. You can also run mox with docker image "docker.io/moxmail/mox", with tags like "latest", "0.0.1" and "0.0.1-go1.20.1-alpine3.17.2", see https://hub.docker.com/r/moxmail/mox. See docker-compose.yml in this -repository for instructions on starting. +repository for instructions on starting. You must run docker with host +networking, because mox needs to find your actual public IP's and get the +remote IPs for incoming connections, not a local/internal NAT IP. # Quickstart The easiest way to get started with serving email for your domain is to get a vm/machine dedicated to serving email, name it [host].[domain] (e.g. -mail.example.com), login as root, create user "mox" and its homedir by running -`useradd -d /home/mox mox && mkdir /home/mox` (or pick another directory), -download mox to that directory, and generate a configuration for your desired -email address at your domain: +mail.example.com), login as root, and run: + # Create mox user and homedir (or pick another name or homedir): + useradd -m -d /home/mox mox + + cd /home/mox + ... compile or download mox to this directory, see above ... + + # Generate config files for your address/domain: ./mox quickstart you@example.com -This creates an account, generates a password and configuration files, prints -the DNS records you need to manually create and prints commands to start mox and -optionally install mox as a service. +The quickstart creates an account, generates a password and configuration +files, prints the DNS records you need to manually create and prints commands +to start mox and optionally install mox as a service. A dedicated machine is highly recommended because modern email requires HTTPS, and mox currently needs it for automatic TLS. You could combine mox with an existing webserver, but it requires more configuration. If you want to serve -websites on the same machine, use the webserver built into mox. +websites on the same machine, consider using the webserver built into mox. If +you want to run an existing webserver on port 443/80, see "mox help quickstart", +it'll tell you to run "./mox quickstart -existing-webserver you@example.com". After starting, you can access the admin web interface on internal IPs. @@ -110,13 +118,15 @@ The code is heavily cross-referenced with the RFCs for readability/maintainabili - DANE and DNSSEC. - Sending DMARC and TLS reports (currently only receiving). - OAUTH2 support, for single sign on. -- Using mox as backup MX. - ACME verification over HTTP (in addition to current tls-alpn01). - Add special IMAP mailbox ("Queue?") that contains queued but not-yet-delivered messages. -- Old-style internationalization in messages. - Sieve for filtering (for now see Rulesets in the account config) - Calendaring +- IMAP CONDSTORE and QRESYNC extensions +- IMAP THREAD extension +- Using mox as backup MX. +- Old-style internationalization in messages. - JMAP - Webmail diff --git a/config/config.go b/config/config.go index fa542f1..3c52f6c 100644 --- a/config/config.go +++ b/config/config.go @@ -298,13 +298,15 @@ func (r Ruleset) Equal(o Ruleset) bool { return true } +type KeyCert struct { + CertFile string `sconf-doc:"Certificate including intermediate CA certificates, in PEM format."` + KeyFile string `sconf-doc:"Private key for certificate, in PEM format. PKCS8 is recommended, but PKCS1 and EC private keys are recognized as well."` +} + type TLS struct { - ACME string `sconf:"optional" sconf-doc:"Name of provider from top-level configuration to use for ACME, e.g. letsencrypt."` - KeyCerts []struct { - CertFile string `sconf-doc:"Certificate including intermediate CA certificates, in PEM format."` - KeyFile string `sconf-doc:"Private key for certificate, in PEM format. PKCS8 is recommended, but PKCS1 and EC private keys are recognized as well."` - } `sconf:"optional"` - MinVersion string `sconf:"optional" sconf-doc:"Minimum TLS version. Default: TLSv1.2."` + ACME string `sconf:"optional" sconf-doc:"Name of provider from top-level configuration to use for ACME, e.g. letsencrypt."` + KeyCerts []KeyCert `sconf:"optional"` + MinVersion string `sconf:"optional" sconf-doc:"Minimum TLS version. Default: TLSv1.2."` Config *tls.Config `sconf:"-" json:"-"` // TLS config for non-ACME-verification connections, i.e. SMTP and IMAP, and not port 443. ACMEConfig *tls.Config `sconf:"-" json:"-"` // TLS config that handles ACME verification, for serving on port 443. diff --git a/config/doc.go b/config/doc.go index ee5b885..0dedf1e 100644 --- a/config/doc.go +++ b/config/doc.go @@ -670,7 +670,7 @@ describe-static" and "mox config describe-domains": # Examples Mox includes configuration files to illustrate common setups. You can see these -examples with "mox examples", and print a specific example with "mox examples +examples with "mox example", and print a specific example with "mox example ". Below are all examples included in mox. # Example webhandlers diff --git a/doc.go b/doc.go index c12b06a..90c5a9c 100644 --- a/doc.go +++ b/doc.go @@ -14,7 +14,7 @@ low-maintenance self-hosted email. mox [-config config/mox.conf] ... mox serve - mox quickstart user@domain [user | uid] + mox quickstart [-existing-webserver] user@domain [user | uid] mox stop mox setaccountpassword address mox setadminpassword @@ -41,7 +41,7 @@ low-maintenance self-hosted email. mox config domain rm domain mox config describe-sendmail >/etc/moxsubmit.conf mox config printservice >mox.service - mox examples [name] + mox example [name] mox checkupdate mox cid cid mox clientconfig domain @@ -91,7 +91,25 @@ systemd service file and prints commands to enable and start mox as service. The user or uid is optional, defaults to "mox", and is the user or uid/gid mox will run as after initialization. - usage: mox quickstart user@domain [user | uid] +Mox is by far easiest to operate if you let it listen on port 443 (HTTPS) and +80 (HTTP). TLS will be fully automatic with ACME with Let's Encrypt. + +You can run mox along with an existing webserver, but because of MTA-STS and +autoconfig, you'll need to forward HTTPS traffic for two domains to mox. Run +"mox quickstart -existing-webserver ..." to generate configuration files and +instructions for configuring mox along with an existing webserver. + +But please first consider configuring mox on port 443. It can itself serve +domains with HTTP/HTTPS, including with automatic TLS with ACME, is easily +configured through both configuration files and admin web interface, and can act +as a reverse proxy (and static file server for that matter), so you can forward +traffic to your existing backend applications. Look for "WebHandlers:" in the +output of "mox config describe-domains" and see the output of "mox example +webhandlers". + + usage: mox quickstart [-existing-webserver] user@domain [user | uid] + -existing-webserver + use if a webserver is already running, so mox won't listen on port 80 and 443; you'll have to provide tls certificates/keys, and configure the existing webserver as reverse proxy, forwarding requests to mox. # mox stop @@ -389,11 +407,11 @@ date version. usage: mox config printservice >mox.service -# mox examples +# mox example List available examples, or print a specific example. - usage: mox examples [name] + usage: mox example [name] # mox checkupdate diff --git a/docker-compose.yml b/docker-compose.yml index 6eee61c..830cba5 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,6 +1,7 @@ # Before launching mox, run the quickstart to create config files for running as # user the mox user (create it on the host system first, e.g. "useradd -d $PWD mox"): # +# mkdir config data web # docker-compose run mox mox quickstart you@yourdomain.example $(id -u mox) # # After following the quickstart instructions you can start mox: @@ -13,7 +14,7 @@ services: # Replace latest with the version you want to run. image: docker.io/moxmail/mox:latest environment: - - MOX_DOCKER=... # Quickstart won't try to write systemd service file. + - MOX_DOCKER=yes # Quickstart won't try to write systemd service file. # Mox needs host networking because it needs access to the IPs of the # machine, and the IPs of incoming connections for spam filtering. network_mode: 'host' diff --git a/gendoc.sh b/gendoc.sh index 91faa10..013735e 100755 --- a/gendoc.sh +++ b/gendoc.sh @@ -66,15 +66,15 @@ cat <". Below are all examples included in mox. EOF -for ex in $(./mox examples); do +for ex in $(./mox example); do echo '# Example '$ex echo - ./mox examples $ex | sed 's/^/\t/' + ./mox example $ex | sed 's/^/\t/' echo done diff --git a/http/autoconf.go b/http/autoconf.go index d46d9bd..7e2ebc1 100644 --- a/http/autoconf.go +++ b/http/autoconf.go @@ -51,100 +51,103 @@ var ( // Autoconfiguration for Mozilla Thunderbird. // User should create a DNS record: autoconfig. (CNAME or A). // See https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat -func autoconfHandle(l config.Listener) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - log := xlog.WithContext(r.Context()) +func autoconfHandle(w http.ResponseWriter, r *http.Request) { + log := xlog.WithContext(r.Context()) - var addrDom string - defer func() { - metricAutoconf.WithLabelValues(addrDom).Inc() - }() + var addrDom string + defer func() { + metricAutoconf.WithLabelValues(addrDom).Inc() + }() - email := r.FormValue("emailaddress") - log.Debug("autoconfig request", mlog.Field("email", email)) - addr, err := smtp.ParseAddress(email) - if err != nil { - http.Error(w, "400 - bad request - invalid parameter emailaddress", http.StatusBadRequest) - return - } + email := r.FormValue("emailaddress") + log.Debug("autoconfig request", mlog.Field("email", email)) + addr, err := smtp.ParseAddress(email) + if err != nil { + http.Error(w, "400 - bad request - invalid parameter emailaddress", http.StatusBadRequest) + return + } - if _, ok := mox.Conf.Domain(addr.Domain); !ok { - http.Error(w, "400 - bad request - unknown domain", http.StatusBadRequest) - return - } - addrDom = addr.Domain.Name() + if _, ok := mox.Conf.Domain(addr.Domain); !ok { + http.Error(w, "400 - bad request - unknown domain", http.StatusBadRequest) + return + } + addrDom = addr.Domain.Name() - hostname := l.HostnameDomain - if hostname.IsZero() { - hostname = mox.Conf.Static.HostnameDomain - } + hostname := mox.Conf.Static.HostnameDomain - // Thunderbird doesn't seem to allow U-labels, always return ASCII names. - var resp autoconfigResponse - resp.Version = "1.1" - resp.EmailProvider.ID = addr.Domain.ASCII - resp.EmailProvider.Domain = addr.Domain.ASCII - resp.EmailProvider.DisplayName = email - resp.EmailProvider.DisplayShortName = addr.Domain.ASCII + // Thunderbird doesn't seem to allow U-labels, always return ASCII names. + var resp autoconfigResponse + resp.Version = "1.1" + resp.EmailProvider.ID = addr.Domain.ASCII + resp.EmailProvider.Domain = addr.Domain.ASCII + resp.EmailProvider.DisplayName = email + resp.EmailProvider.DisplayShortName = addr.Domain.ASCII - var imapPort int - var imapSocket string + var imapPort int + var imapSocket string + for _, l := range mox.Conf.Static.Listeners { if l.IMAPS.Enabled { - imapPort = config.Port(l.IMAPS.Port, 993) imapSocket = "SSL" + imapPort = config.Port(l.IMAPS.Port, 993) } else if l.IMAP.Enabled { - imapPort = config.Port(l.IMAP.Port, 143) - if l.TLS != nil { + if l.TLS != nil && imapSocket != "SSL" { imapSocket = "STARTTLS" - } else { + imapPort = config.Port(l.IMAP.Port, 143) + } else if imapSocket == "" { imapSocket = "plain" + imapPort = config.Port(l.IMAP.Port, 143) } - } else { - log.Error("autoconfig: no imap configured?") } + } + if imapPort == 0 { + log.Error("autoconfig: no imap configured?") + } - // todo: specify SCRAM-SHA-256 once thunderbird and autoconfig supports it. or perhaps that will fall under "password-encrypted" by then. + // todo: specify SCRAM-SHA-256 once thunderbird and autoconfig supports it. or perhaps that will fall under "password-encrypted" by then. - resp.EmailProvider.IncomingServer.Type = "imap" - resp.EmailProvider.IncomingServer.Hostname = hostname.ASCII - resp.EmailProvider.IncomingServer.Port = imapPort - resp.EmailProvider.IncomingServer.SocketType = imapSocket - resp.EmailProvider.IncomingServer.Username = email - resp.EmailProvider.IncomingServer.Authentication = "password-encrypted" + resp.EmailProvider.IncomingServer.Type = "imap" + resp.EmailProvider.IncomingServer.Hostname = hostname.ASCII + resp.EmailProvider.IncomingServer.Port = imapPort + resp.EmailProvider.IncomingServer.SocketType = imapSocket + resp.EmailProvider.IncomingServer.Username = email + resp.EmailProvider.IncomingServer.Authentication = "password-encrypted" - var smtpPort int - var smtpSocket string + var smtpPort int + var smtpSocket string + for _, l := range mox.Conf.Static.Listeners { if l.Submissions.Enabled { - smtpPort = config.Port(l.Submissions.Port, 465) smtpSocket = "SSL" + smtpPort = config.Port(l.Submissions.Port, 465) } else if l.Submission.Enabled { - smtpPort = config.Port(l.Submission.Port, 587) - if l.TLS != nil { + if l.TLS != nil && smtpSocket != "SSL" { smtpSocket = "STARTTLS" - } else { + smtpPort = config.Port(l.Submission.Port, 587) + } else if smtpSocket == "" { smtpSocket = "plain" + smtpPort = config.Port(l.Submission.Port, 587) } - } else { - log.Error("autoconfig: no smtp submission configured?") } + } + if smtpPort == 0 { + log.Error("autoconfig: no smtp submission configured?") + } - resp.EmailProvider.OutgoingServer.Type = "smtp" - resp.EmailProvider.OutgoingServer.Hostname = hostname.ASCII - resp.EmailProvider.OutgoingServer.Port = smtpPort - resp.EmailProvider.OutgoingServer.SocketType = smtpSocket - resp.EmailProvider.OutgoingServer.Username = email - resp.EmailProvider.OutgoingServer.Authentication = "password-encrypted" + resp.EmailProvider.OutgoingServer.Type = "smtp" + resp.EmailProvider.OutgoingServer.Hostname = hostname.ASCII + resp.EmailProvider.OutgoingServer.Port = smtpPort + resp.EmailProvider.OutgoingServer.SocketType = smtpSocket + resp.EmailProvider.OutgoingServer.Username = email + resp.EmailProvider.OutgoingServer.Authentication = "password-encrypted" - // todo: should we put the email address in the URL? - resp.ClientConfigUpdate.URL = fmt.Sprintf("https://%s/mail/config-v1.1.xml", hostname.ASCII) + // todo: should we put the email address in the URL? + resp.ClientConfigUpdate.URL = fmt.Sprintf("https://%s/mail/config-v1.1.xml", hostname.ASCII) - w.Header().Set("Content-Type", "application/xml; charset=utf-8") - enc := xml.NewEncoder(w) - enc.Indent("", "\t") - fmt.Fprint(w, xml.Header) - if err := enc.Encode(resp); err != nil { - log.Errorx("marshal autoconfig response", err) - } + w.Header().Set("Content-Type", "application/xml; charset=utf-8") + enc := xml.NewEncoder(w) + enc.Indent("", "\t") + fmt.Fprint(w, xml.Header) + if err := enc.Encode(resp); err != nil { + log.Errorx("marshal autoconfig response", err) } } @@ -158,125 +161,129 @@ func autoconfHandle(l config.Listener) http.HandlerFunc { // errors. // // Thunderbird does understand autodiscover. -func autodiscoverHandle(l config.Listener) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - log := xlog.WithContext(r.Context()) +func autodiscoverHandle(w http.ResponseWriter, r *http.Request) { + log := xlog.WithContext(r.Context()) - var addrDom string - defer func() { - metricAutodiscover.WithLabelValues(addrDom).Inc() - }() + var addrDom string + defer func() { + metricAutodiscover.WithLabelValues(addrDom).Inc() + }() - if r.Method != "POST" { - http.Error(w, "405 - method not allowed - post required", http.StatusMethodNotAllowed) - return - } + if r.Method != "POST" { + http.Error(w, "405 - method not allowed - post required", http.StatusMethodNotAllowed) + return + } - var req autodiscoverRequest - if err := xml.NewDecoder(r.Body).Decode(&req); err != nil { - http.Error(w, "400 - bad request - parsing autodiscover request: "+err.Error(), http.StatusMethodNotAllowed) - return - } + var req autodiscoverRequest + if err := xml.NewDecoder(r.Body).Decode(&req); err != nil { + http.Error(w, "400 - bad request - parsing autodiscover request: "+err.Error(), http.StatusMethodNotAllowed) + return + } - log.Debug("autodiscover request", mlog.Field("email", req.Request.EmailAddress)) + log.Debug("autodiscover request", mlog.Field("email", req.Request.EmailAddress)) - addr, err := smtp.ParseAddress(req.Request.EmailAddress) - if err != nil { - http.Error(w, "400 - bad request - invalid parameter emailaddress", http.StatusBadRequest) - return - } + addr, err := smtp.ParseAddress(req.Request.EmailAddress) + if err != nil { + http.Error(w, "400 - bad request - invalid parameter emailaddress", http.StatusBadRequest) + return + } - if _, ok := mox.Conf.Domain(addr.Domain); !ok { - http.Error(w, "400 - bad request - unknown domain", http.StatusBadRequest) - return - } - addrDom = addr.Domain.Name() + if _, ok := mox.Conf.Domain(addr.Domain); !ok { + http.Error(w, "400 - bad request - unknown domain", http.StatusBadRequest) + return + } + addrDom = addr.Domain.Name() - hostname := l.HostnameDomain - if hostname.IsZero() { - hostname = mox.Conf.Static.HostnameDomain - } + hostname := mox.Conf.Static.HostnameDomain - // The docs are generated and fragmented in many tiny pages, hard to follow. - // High-level starting point, https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/78530279-d042-4eb0-a1f4-03b18143cd19 - // Request: https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/2096fab2-9c3c-40b9-b123-edf6e8d55a9b - // Response, protocol: https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/f4238db6-a983-435c-807a-b4b4a624c65b - // It appears autodiscover does not allow specifying SCRAM-SHA-256 as - // authentication method, or any authentication method that real clients actually - // use. See - // https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/21fd2dd5-c4ee-485b-94fb-e7db5da93726 + // The docs are generated and fragmented in many tiny pages, hard to follow. + // High-level starting point, https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/78530279-d042-4eb0-a1f4-03b18143cd19 + // Request: https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/2096fab2-9c3c-40b9-b123-edf6e8d55a9b + // Response, protocol: https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/f4238db6-a983-435c-807a-b4b4a624c65b + // It appears autodiscover does not allow specifying SCRAM-SHA-256 as + // authentication method, or any authentication method that real clients actually + // use. See + // https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxdscli/21fd2dd5-c4ee-485b-94fb-e7db5da93726 - var imapPort int - imapSSL := "off" - var imapEncryption string + var imapPort int + imapSSL := "off" + var imapEncryption string + + var smtpPort int + smtpSSL := "off" + var smtpEncryption string + for _, l := range mox.Conf.Static.Listeners { if l.IMAPS.Enabled { imapPort = config.Port(l.IMAPS.Port, 993) imapSSL = "on" imapEncryption = "TLS" // Assuming this means direct TLS. } else if l.IMAP.Enabled { - imapPort = config.Port(l.IMAP.Port, 143) - if l.TLS != nil { + if l.TLS != nil && imapEncryption != "TLS" { imapSSL = "on" + imapPort = config.Port(l.IMAP.Port, 143) + } else if imapSSL == "" { + imapPort = config.Port(l.IMAP.Port, 143) } - } else { - log.Error("autoconfig: no imap configured?") } - var smtpPort int - smtpSSL := "off" - var smtpEncryption string if l.Submissions.Enabled { smtpPort = config.Port(l.Submissions.Port, 465) smtpSSL = "on" smtpEncryption = "TLS" // Assuming this means direct TLS. } else if l.Submission.Enabled { - smtpPort = config.Port(l.Submission.Port, 587) - if l.TLS != nil { + if l.TLS != nil && smtpEncryption != "TLS" { smtpSSL = "on" + smtpPort = config.Port(l.Submission.Port, 587) + } else if smtpSSL == "" { + smtpPort = config.Port(l.Submission.Port, 587) } - } else { - log.Error("autoconfig: no smtp submission configured?") } + } + if imapPort == 0 { + log.Error("autoconfig: no smtp submission configured?") + } + if smtpPort == 0 { + log.Error("autoconfig: no imap configured?") + } - w.Header().Set("Content-Type", "application/xml; charset=utf-8") + w.Header().Set("Content-Type", "application/xml; charset=utf-8") - resp := autodiscoverResponse{} - resp.XMLName.Local = "Autodiscover" - resp.XMLName.Space = "http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006" - resp.Response.XMLName.Local = "Response" - resp.Response.XMLName.Space = "http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a" - resp.Response.Account = autodiscoverAccount{ - AccountType: "email", - Action: "settings", - Protocol: []autodiscoverProtocol{ - { - Type: "IMAP", - Server: hostname.ASCII, - Port: imapPort, - LoginName: req.Request.EmailAddress, - SSL: imapSSL, - Encryption: imapEncryption, - SPA: "off", // Override default "on", this is Microsofts proprietary authentication protocol. - AuthRequired: "on", - }, - { - Type: "SMTP", - Server: hostname.ASCII, - Port: smtpPort, - LoginName: req.Request.EmailAddress, - SSL: smtpSSL, - Encryption: smtpEncryption, - SPA: "off", // Override default "on", this is Microsofts proprietary authentication protocol. - AuthRequired: "on", - }, + resp := autodiscoverResponse{} + resp.XMLName.Local = "Autodiscover" + resp.XMLName.Space = "http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006" + resp.Response.XMLName.Local = "Response" + resp.Response.XMLName.Space = "http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a" + resp.Response.Account = autodiscoverAccount{ + AccountType: "email", + Action: "settings", + Protocol: []autodiscoverProtocol{ + { + Type: "IMAP", + Server: hostname.ASCII, + Port: imapPort, + LoginName: req.Request.EmailAddress, + SSL: imapSSL, + Encryption: imapEncryption, + SPA: "off", // Override default "on", this is Microsofts proprietary authentication protocol. + AuthRequired: "on", }, - } - enc := xml.NewEncoder(w) - enc.Indent("", "\t") - fmt.Fprint(w, xml.Header) - if err := enc.Encode(resp); err != nil { - log.Errorx("marshal autodiscover response", err) - } + { + Type: "SMTP", + Server: hostname.ASCII, + Port: smtpPort, + LoginName: req.Request.EmailAddress, + SSL: smtpSSL, + Encryption: smtpEncryption, + SPA: "off", // Override default "on", this is Microsofts proprietary authentication protocol. + AuthRequired: "on", + }, + }, + } + enc := xml.NewEncoder(w) + enc.Indent("", "\t") + fmt.Fprint(w, xml.Header) + if err := enc.Encode(resp); err != nil { + log.Errorx("marshal autodiscover response", err) } } diff --git a/http/web.go b/http/web.go index ead5142..afe557a 100644 --- a/http/web.go +++ b/http/web.go @@ -22,6 +22,7 @@ import ( "github.com/prometheus/client_golang/prometheus/promauto" "github.com/prometheus/client_golang/prometheus/promhttp" + "github.com/mjl-/mox/autotls" "github.com/mjl-/mox/config" "github.com/mjl-/mox/dns" "github.com/mjl-/mox/mlog" @@ -371,8 +372,8 @@ func Listen() { // todo: may want to check this against the configured domains, could in theory be just a webserver. return strings.HasPrefix(dom.ASCII, "autoconfig.") } - srv.HandleFunc("autoconfig", autoconfigMatch, "/mail/config-v1.1.xml", safeHeaders(autoconfHandle(l))) - srv.HandleFunc("autodiscover", autoconfigMatch, "/autodiscover/autodiscover.xml", safeHeaders(autodiscoverHandle(l))) + srv.HandleFunc("autoconfig", autoconfigMatch, "/mail/config-v1.1.xml", safeHeaders(autoconfHandle)) + srv.HandleFunc("autodiscover", autoconfigMatch, "/autodiscover/autodiscover.xml", safeHeaders(autodiscoverHandle)) } if l.MTASTSHTTPS.Enabled { port := config.Port(l.MTASTSHTTPS.Port, 443) @@ -405,51 +406,32 @@ func Listen() { } // We'll explicitly ensure these TLS certs exist (e.g. are created with ACME) - // immediately after startup. We only do so for our explicitly hostnames, not for - // autoconfig or mta-sts DNS records, they can be requested on demand (perhaps - // never). - ensureHosts := map[dns.Domain]struct{}{} + // immediately after startup. We only do so for our explicit listener hostnames, + // not for mta-sts DNS records, it can be requested on demand (perhaps never). We + // do request autoconfig, otherwise clients may run into their timeouts waiting for + // the certificate to be given during the first https connection. + ensureManagerHosts = map[*autotls.Manager]map[dns.Domain]struct{}{} if l.TLS != nil && l.TLS.ACME != "" { m := mox.Conf.Static.ACME[l.TLS.ACME].Manager - ensureHosts[mox.Conf.Static.HostnameDomain] = struct{}{} + hosts := map[dns.Domain]struct{}{ + mox.Conf.Static.HostnameDomain: {}, + } if l.HostnameDomain.ASCII != "" { - ensureHosts[l.HostnameDomain] = struct{}{} + hosts[l.HostnameDomain] = struct{}{} + } + // All domains are served on all listeners. + for _, name := range mox.Conf.Domains() { + dom, err := dns.ParseDomain("autoconfig." + name) + if err != nil { + xlog.Errorx("parsing domain from config for autoconfig", err) + } else { + hosts[dom] = struct{}{} + } } - go func() { - // Just in case someone adds quite some domains to their config. We don't want to - // hit any ACME rate limits. - if len(ensureHosts) > 10 { - return - } - - time.Sleep(1 * time.Second) - i := 0 - for hostname := range ensureHosts { - if i > 0 { - // Sleep just a little. We don't want to hammer our ACME provider, e.g. Let's Encrypt. - time.Sleep(10 * time.Second) - } - i++ - - hello := &tls.ClientHelloInfo{ - ServerName: hostname.ASCII, - - // Make us fetch an ECDSA P256 cert. - // We add TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 to get around the ecDSA check in autocert. - CipherSuites: []uint16{tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_AES_128_GCM_SHA256}, - SupportedCurves: []tls.CurveID{tls.CurveP256}, - SignatureSchemes: []tls.SignatureScheme{tls.ECDSAWithP256AndSHA256}, - SupportedVersions: []uint16{tls.VersionTLS13}, - } - xlog.Print("ensuring certificate availability", mlog.Field("hostname", hostname)) - if _, err := m.Manager.GetCertificate(hello); err != nil { - xlog.Errorx("requesting automatic certificate", err, mlog.Field("hostname", hostname)) - } - } - }() + ensureManagerHosts[m] = hosts } for port, srv := range portServe { @@ -485,6 +467,7 @@ func adminIndex(w http.ResponseWriter, r *http.Request) { // functions to be launched in goroutine that will serve on a listener. var servers []func() +var ensureManagerHosts map[*autotls.Manager]map[dns.Domain]struct{} // listen prepares a listener, and adds it to "servers", to be launched (if not running as root) through Serve. func listen1(ip string, port int, tlsConfig *tls.Config, name string, kinds []string, handler http.Handler) { @@ -535,4 +518,38 @@ func Serve() { go serve() } servers = nil + + go func() { + time.Sleep(1 * time.Second) + i := 0 + for m, hosts := range ensureManagerHosts { + for host := range hosts { + if i >= 10 { + // Just in case someone adds quite some domains to their config. We don't want to + // hit any ACME rate limits. + return + } + if i > 0 { + // Sleep just a little. We don't want to hammer our ACME provider, e.g. Let's Encrypt. + time.Sleep(10 * time.Second) + } + i++ + + hello := &tls.ClientHelloInfo{ + ServerName: host.ASCII, + + // Make us fetch an ECDSA P256 cert. + // We add TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 to get around the ecDSA check in autocert. + CipherSuites: []uint16{tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_AES_128_GCM_SHA256}, + SupportedCurves: []tls.CurveID{tls.CurveP256}, + SignatureSchemes: []tls.SignatureScheme{tls.ECDSAWithP256AndSHA256}, + SupportedVersions: []uint16{tls.VersionTLS13}, + } + xlog.Print("ensuring certificate availability", mlog.Field("hostname", host)) + if _, err := m.Manager.GetCertificate(hello); err != nil { + xlog.Errorx("requesting automatic certificate", err, mlog.Field("hostname", host)) + } + } + } + }() } diff --git a/main.go b/main.go index 8cdc673..c63fae2 100644 --- a/main.go +++ b/main.go @@ -105,7 +105,7 @@ var commands = []struct { {"config domain rm", cmdConfigDomainRemove}, {"config describe-sendmail", cmdConfigDescribeSendmail}, {"config printservice", cmdConfigPrintservice}, - {"examples", cmdExamples}, + {"example", cmdExample}, {"checkupdate", cmdCheckupdate}, {"cid", cmdCid}, @@ -477,7 +477,7 @@ are printed. c.Usage() } - _, errs := mox.ParseConfig(context.Background(), mox.ConfigStaticPath, true) + _, errs := mox.ParseConfig(context.Background(), mox.ConfigStaticPath, true, false) if len(errs) > 1 { log.Printf("multiple errors:") for _, err := range errs { @@ -842,7 +842,7 @@ WebHandlers: }, } -func cmdExamples(c *cmd) { +func cmdExample(c *cmd) { c.params = "[name]" c.help = `List available examples, or print a specific example.` @@ -866,7 +866,6 @@ func cmdExamples(c *cmd) { log.Fatalln("not found") } fmt.Print(match()) - return } func cmdLoglevels(c *cmd) { diff --git a/mox-/config.go b/mox-/config.go index 7aed72a..1dbb696 100644 --- a/mox-/config.go +++ b/mox-/config.go @@ -336,7 +336,7 @@ func MustLoadConfig() { // LoadConfig attempts to parse and load a config, returning any errors // encountered. func LoadConfig(ctx context.Context) []error { - c, errs := ParseConfig(ctx, ConfigStaticPath, false) + c, errs := ParseConfig(ctx, ConfigStaticPath, false, false) if len(errs) > 0 { return errs } @@ -352,9 +352,11 @@ func SetConfig(c *Config) { Conf = Config{c.Static, sync.Mutex{}, c.Log, sync.Mutex{}, c.Dynamic, c.dynamicMtime, c.DynamicLastCheck, c.accountDestinations} } -// ParseConfig parses the static config at path p. If checkOnly is true, no -// changes are made, such as registering ACME identities. -func ParseConfig(ctx context.Context, p string, checkOnly bool) (c *Config, errs []error) { +// ParseConfig parses the static config at path p. If checkOnly is true, no changes +// are made, such as registering ACME identities. If skipCheckTLSKeyCerts is true, +// the TLS KeyCerts configuration is not checked. This is used during the +// quickstart in the case the user is going to provide their own certificates. +func ParseConfig(ctx context.Context, p string, checkOnly, skipCheckTLSKeyCerts bool) (c *Config, errs []error) { c = &Config{ Static: config.Static{ DataDir: ".", @@ -373,7 +375,7 @@ func ParseConfig(ctx context.Context, p string, checkOnly bool) (c *Config, errs return nil, []error{fmt.Errorf("parsing %s: %v", p, err)} } - if xerrs := PrepareStaticConfig(ctx, p, c, checkOnly); len(xerrs) > 0 { + if xerrs := PrepareStaticConfig(ctx, p, c, checkOnly, skipCheckTLSKeyCerts); len(xerrs) > 0 { return nil, xerrs } @@ -390,7 +392,7 @@ func ParseConfig(ctx context.Context, p string, checkOnly bool) (c *Config, errs // PrepareStaticConfig parses the static config file and prepares data structures // for starting mox. If checkOnly is set no substantial changes are made, like // creating an ACME registration. -func PrepareStaticConfig(ctx context.Context, configFile string, config *Config, checkOnly bool) (errs []error) { +func PrepareStaticConfig(ctx context.Context, configFile string, config *Config, checkOnly, skipCheckTLSKeyCerts bool) (errs []error) { addErrorf := func(format string, args ...any) { errs = append(errs, fmt.Errorf(format, args...)) } @@ -428,7 +430,7 @@ func PrepareStaticConfig(ctx context.Context, configFile string, config *Config, if err != nil && errors.As(err, &userErr) { uid, err := strconv.ParseUint(c.User, 10, 32) if err != nil { - addErrorf("parsing unknown user %s as uid: %v", c.User, err) + addErrorf("parsing unknown user %s as uid: %v (hint: add user mox with \"useradd -d $PWD mox\" or specify a different username on the quickstart command-line)", c.User, err) } else { // We assume the same gid as uid. c.UID = uint32(uid) @@ -514,8 +516,10 @@ func PrepareStaticConfig(ctx context.Context, configFile string, config *Config, } l.TLS.Config = tlsconfig } else if len(l.TLS.KeyCerts) != 0 { - if err := loadTLSKeyCerts(configFile, "listener "+name, l.TLS); err != nil { - addErrorf("%w", err) + if !skipCheckTLSKeyCerts { + if err := loadTLSKeyCerts(configFile, "listener "+name, l.TLS); err != nil { + addErrorf("%w", err) + } } } else { addErrorf("listener %q: cannot have TLS config without ACME and without static keys/certificates", name) @@ -562,9 +566,6 @@ func PrepareStaticConfig(ctx context.Context, configFile string, config *Config, addErrorf("listener %q does not specify tls config, but requires tls for %s", name, strings.Join(needsTLS, ", ")) } } - if l.AutoconfigHTTPS.Enabled && (!l.IMAP.Enabled && !l.IMAPS.Enabled || !l.Submission.Enabled && !l.Submissions.Enabled) { - addErrorf("listener %q with autoconfig enabled must have SMTP submission or submissions and IMAP or IMAPS enabled", name) - } if l.AutoconfigHTTPS.Enabled && l.MTASTSHTTPS.Enabled && l.AutoconfigHTTPS.Port == l.MTASTSHTTPS.Port && l.AutoconfigHTTPS.NonTLS != l.MTASTSHTTPS.NonTLS { addErrorf("listener %q tries to enable autoconfig and mta-sts enabled on same port but with both http and https", name) } diff --git a/quickstart.go b/quickstart.go index a961df9..66e5a68 100644 --- a/quickstart.go +++ b/quickstart.go @@ -41,7 +41,7 @@ func pwgen() string { } func cmdQuickstart(c *cmd) { - c.params = "user@domain [user | uid]" + c.params = "[-existing-webserver] user@domain [user | uid]" c.help = `Quickstart generates configuration files and prints instructions to quickly set up a mox instance. Quickstart writes configuration files, prints initial admin and account @@ -50,7 +50,25 @@ systemd service file and prints commands to enable and start mox as service. The user or uid is optional, defaults to "mox", and is the user or uid/gid mox will run as after initialization. + +Mox is by far easiest to operate if you let it listen on port 443 (HTTPS) and +80 (HTTP). TLS will be fully automatic with ACME with Let's Encrypt. + +You can run mox along with an existing webserver, but because of MTA-STS and +autoconfig, you'll need to forward HTTPS traffic for two domains to mox. Run +"mox quickstart -existing-webserver ..." to generate configuration files and +instructions for configuring mox along with an existing webserver. + +But please first consider configuring mox on port 443. It can itself serve +domains with HTTP/HTTPS, including with automatic TLS with ACME, is easily +configured through both configuration files and admin web interface, and can act +as a reverse proxy (and static file server for that matter), so you can forward +traffic to your existing backend applications. Look for "WebHandlers:" in the +output of "mox config describe-domains" and see the output of "mox example +webhandlers". ` + var existingWebserver bool + c.flag.BoolVar(&existingWebserver, "existing-webserver", false, "use if a webserver is already running, so mox won't listen on port 80 and 443; you'll have to provide tls certificates/keys, and configure the existing webserver as reverse proxy, forwarding requests to mox.") args := c.Parse() if len(args) != 1 && len(args) != 2 { c.Usage() @@ -352,17 +370,19 @@ This likely means one of two things: dc := config.Dynamic{} sc := config.Static{ - DataDir: "../data", - User: user, - LogLevel: "info", - Hostname: hostname.Name(), - ACME: map[string]config.ACME{ + DataDir: "../data", + User: user, + LogLevel: "debug", // Help new users, they'll bring it back to info when it all works. + Hostname: hostname.Name(), + AdminPasswordFile: "adminpasswd", + } + if !existingWebserver { + sc.ACME = map[string]config.ACME{ "letsencrypt": { DirectoryURL: "https://acme-v02.api.letsencrypt.org/directory", ContactEmail: args[0], // todo: let user specify an alternative fallback address? }, - }, - AdminPasswordFile: "adminpasswd", + } } dataDir := "data" // ../data is relative to config/ os.MkdirAll(dataDir, 0770) @@ -376,15 +396,31 @@ This likely means one of two things: public := config.Listener{ IPs: publicListenerIPs, - TLS: &config.TLS{ - ACME: "letsencrypt", - }, } public.SMTP.Enabled = true public.Submissions.Enabled = true public.IMAPS.Enabled = true - public.AutoconfigHTTPS.Enabled = true - public.MTASTSHTTPS.Enabled = true + + if existingWebserver { + hostbase := fmt.Sprintf("path/to/%s", hostname.Name()) + mtastsbase := fmt.Sprintf("path/to/mta-sts.%s", domain.Name()) + autoconfigbase := fmt.Sprintf("path/to/autoconfig.%s", domain.Name()) + public.TLS = &config.TLS{ + KeyCerts: []config.KeyCert{ + {CertFile: hostbase + "-chain.crt.pem", KeyFile: hostbase + ".key.pem"}, + {CertFile: mtastsbase + "-chain.crt.pem", KeyFile: mtastsbase + ".key.pem"}, + {CertFile: autoconfigbase + "-chain.crt.pem", KeyFile: autoconfigbase + ".key.pem"}, + }, + } + } else { + public.TLS = &config.TLS{ + ACME: "letsencrypt", + } + public.AutoconfigHTTPS.Enabled = true + public.MTASTSHTTPS.Enabled = true + public.WebserverHTTP.Enabled = true + public.WebserverHTTPS.Enabled = true + } // Suggest blocklists, but we'll comment them out after generating the config. public.SMTP.DNSBLs = []string{"sbl.spamhaus.org", "bl.spamcop.net"} @@ -396,6 +432,18 @@ This likely means one of two things: internal.AccountHTTP.Enabled = true internal.AdminHTTP.Enabled = true internal.MetricsHTTP.Enabled = true + if existingWebserver { + internal.AccountHTTP.Port = 1080 + internal.AdminHTTP.Port = 1080 + internal.AutoconfigHTTPS.Enabled = true + internal.AutoconfigHTTPS.Port = 81 + internal.AutoconfigHTTPS.NonTLS = true + internal.MTASTSHTTPS.Enabled = true + internal.MTASTSHTTPS.Port = 81 + internal.MTASTSHTTPS.NonTLS = true + internal.WebserverHTTP.Enabled = true + internal.WebserverHTTP.Port = 81 + } sc.Listeners = map[string]config.Listener{ "public": public, @@ -478,7 +526,8 @@ This likely means one of two things: xwritefile("config/domains.conf", []byte(dconfstr), 0660) // Verify config. - mc, errs := mox.ParseConfig(context.Background(), "config/mox.conf", true) + skipCheckTLSKeyCerts := existingWebserver + mc, errs := mox.ParseConfig(context.Background(), "config/mox.conf", true, skipCheckTLSKeyCerts) if len(errs) > 0 { if len(errs) > 1 { log.Printf("checking generated config, multiple errors:") @@ -518,12 +567,41 @@ autoconfig/autodiscover does not work, use the settings below.`) fmt.Println("") printClientConfig(domain) - fmt.Println("") - fmt.Println(`Configuration files have been written to config/mox.conf and + if existingWebserver { + fmt.Printf(` +Configuration files have been written to config/mox.conf and +config/domains.conf. + +Create the DNS records below. The admin interface can show these same records, and +has a page to check they have been configured correctly. + +You must configure your existing webserver to forward requests for: + + https://mta-sts.%s/ + https://autoconfig.%s/ + +To mox, at: + + http://127.0.0.1:81 + +If it makes it easier to get a TLS certificate for %s, you can add a +reverse proxy for that hostname too. + +You must edit mox.conf and configure the paths to the TLS certificates and keys. +The paths are relative to config/ directory that holds mox.conf! To test if your +config is valid, run: + + ./mox config test +`, domain.ASCII, domain.ASCII, hostname.ASCII) + } else { + fmt.Printf(` +Configuration files have been written to config/mox.conf and config/domains.conf. You should review them. Then create the DNS records below. You can also skip creating the DNS records and start mox immediately. The admin interface can show these same records, and has a page to check they have been -configured correctly.`) +configured correctly. +`) + } // We do not verify the records exist: If they don't exist, we would only be // priming dns caches with negative/absent records, causing our "quick setup" to @@ -533,17 +611,26 @@ configured correctly.`) if err != nil { fatalf("making required DNS records") } - fmt.Print("\n\n\n" + strings.Join(records, "\n") + "\n\n\n\n") + fmt.Print("\n\n" + strings.Join(records, "\n") + "\n\n\n\n") fmt.Printf(`WARNING: The configuration and DNS records above assume you do not currently have email configured for your domain. If you do already have email configured, or if you are sending email for your domain from other machines/services, you should understand the consequences of the DNS records above before continuing! - -You can now start mox with "./mox serve", as root. File ownership and -permissions are automatically set correctly by mox when starting up. On linux, -you may want to enable mox as a systemd service. +`) + if os.Getenv("MOX_DOCKER") == "" { + fmt.Printf(` +You can now start mox with "./mox serve", as root. +`) + } else { + fmt.Printf(` +You can now start the mox container. +`) + } + fmt.Printf(` +File ownership and permissions are automatically set correctly by mox when +starting up. On linux, you may want to enable mox as a systemd service. `) @@ -567,23 +654,21 @@ you may want to enable mox as a systemd service. `) } - fmt.Println(`For secure email exchange you should have a strictly validating DNSSEC + fmt.Printf(`For secure email exchange you should have a strictly validating DNSSEC resolver. An easy and the recommended way is to install unbound. -Enjoy! +If you run into problem, have questions/feedback or found a bug, please let us +know. Mox needs your help! -PS: If port 443 is not available on this machine, automatic TLS with Let's -Encrypt will not work. You can configure existing TLS certificates/keys in mox -(run "mox config describe-static" for examples, and don't forget to renew the -certificates!), or disable TLS (not secure, but perhaps you are just evaluating -mox). If you disable TLS, you must also remove the DNS records about mta-sts, -autoconfig, autodiscover and the SRV records. You also have to edit -config/mox.conf and disable (comment out) TLS in the "public" listener, replace -field "Submissions" with "Submission" and add a sub field "NoRequireSTARTTLS: -true", replace field "IMAPS" with "IMAP" add add a sub field "NoRequireSTARTTLS: -true", and set the "Enabled" field of "AutoconfigHTTPS" and "MTASTSHTTPS" to -false. Final warning: If you disable TLS, your email messages, and user name and -potentially password will be transferred over the internet in plain text!`) +Enjoy! +`) + + if !existingWebserver { + fmt.Printf(` +PS: If you want to run mox along side an existing webserver that uses port 443 +and 80, see "mox help quickstart" with the -existing-webserver option. +`) + } cleanupPaths = nil }