serv/mox
1
0
Fork 0
mirror of https://github.com/mjl-/mox.git synced 2024-12-26 16:33:47 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
mox/tlsrptsend/send_test.go

519 lines
16 KiB
Go
Raw Normal View History

implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
package tlsrptsend
import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"reflect"
"sync"
"testing"
"time"
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
"github.com/mjl-/bstore"
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
"github.com/mjl-/mox/dns"
"github.com/mjl-/mox/mlog"
"github.com/mjl-/mox/mox-"
"github.com/mjl-/mox/moxio"
"github.com/mjl-/mox/queue"
"github.com/mjl-/mox/tlsrpt"
"github.com/mjl-/mox/tlsrptdb"
)
var ctxbg = context.Background()
func tcheckf(t *testing.T, err error, format string, args ...any) {
t.Helper()
if err != nil {
t.Fatalf("%s: %s", fmt.Sprintf(format, args...), err)
}
}
func tcompare(t *testing.T, got, expect any) {
t.Helper()
if !reflect.DeepEqual(got, expect) {
t.Fatalf("got:\n%v\nexpected:\n%v", got, expect)
}
}
func TestSendReports(t *testing.T) {
os.RemoveAll("../testdata/tlsrptsend/data")
mox.Context = ctxbg
mox.ConfigStaticPath = filepath.FromSlash("../testdata/tlsrptsend/mox.conf")
mox.MustLoadConfig(true, false)
err := tlsrptdb.Init()
tcheckf(t, err, "init database")
db := tlsrptdb.ResultDB
resolver := dns.MockResolver{
TXT: map[string][]string{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"_smtp._tls.xn--74h.example.": {
"v=TLSRPTv1; rua=mailto:tls-reports@xn--74h.example,https://ignored.example/",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"_smtp._tls.mailhost.xn--74h.example.": {
"v=TLSRPTv1; rua=mailto:tls-reports1@mailhost.xn--74h.example,mailto:tls-reports2@mailhost.xn--74h.example; rua=mailto:tls-reports3@mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
"_smtp._tls.noreport.example.": {
"v=TLSRPTv1; rua=mailto:tls-reports@noreport.example",
},
"_smtp._tls.mailhost.norua.example.": {
"v=TLSRPTv1;",
},
},
}
endUTC := midnightUTC(time.Now())
dayUTC := endUTC.Add(-12 * time.Hour).Format("20060102")
tlsResults := []tlsrptdb.TLSResult{
// For report1 below.
{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
PolicyDomain: "☺.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
DayUTC: dayUTC,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
RecipientDomain: "☺.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
IsHost: false,
SendReport: true,
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.STS,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
String: []string{"... mtasts policy ..."},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
MXHost: []string{"*.xn--74h.example"},
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 3,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultCertificateExpired,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 3,
},
},
},
},
},
// For report2 below.
{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
PolicyDomain: "mailhost.☺.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
DayUTC: dayUTC,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
RecipientDomain: "☺.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
IsHost: true,
SendReport: false, // Would be ignored if on its own, but we have another result for this policy domain.
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
String: []string{"... tlsa record ..."},
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 1,
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
},
},
{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
PolicyDomain: "mailhost.☺.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
DayUTC: dayUTC,
RecipientDomain: "sharedsender.example",
IsHost: true,
SendReport: true, // Causes previous result to be included in this report.
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
String: []string{"... tlsa record ..."},
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 1,
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
},
},
// No report due to SendReport false.
{
PolicyDomain: "mailhost.noreport.example",
DayUTC: dayUTC,
RecipientDomain: "noreport.example",
IsHost: true,
SendReport: false, // No report.
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.NoPolicyFound,
Domain: "mailhost.noreport.example",
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 2,
TotalFailureSessionCount: 1,
},
},
},
},
// No report due to no mailto rua.
{
PolicyDomain: "mailhost.norua.example",
DayUTC: dayUTC,
RecipientDomain: "norua.example",
IsHost: true,
SendReport: false, // No report.
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.NoPolicyFound,
Domain: "mailhost.norua.example",
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 2,
TotalFailureSessionCount: 1,
},
},
},
},
// No report due to no TLSRPT record.
{
PolicyDomain: "mailhost.notlsrpt.example",
DayUTC: dayUTC,
RecipientDomain: "notlsrpt.example",
IsHost: true,
SendReport: true,
Results: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.NoPolicyFound,
Domain: "mailhost.notlsrpt.example",
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 2,
TotalFailureSessionCount: 1,
},
},
},
},
}
report1 := tlsrpt.Report{
outgoing dmarc/tls reporting improvements - dmarc reports: add a cid to the log line about one run of sending reports, and log line for each report - in smtpclient, also handle tls errors from the first read after a handshake. we appear to sometimes get tls alerts about bad certificates on the first read. - for messages to dmarc/tls reporting addresses that we think should/can not be processed as reports, add an X-Mox- header explaining the reason. - tls reports: send report messages with From address of postmaster at an actually configured domain for the mail host. and only send reports when dkim signing is configured for that domain. the domain is also the submitter domain. the rfc seems to require dkim-signing with an exact match with the message from and submitter. - for incoming tls reports, in the smtp server, we do allow a dkim-signature domain that is higher-level (up to publicsuffix) of the message from domain. so we are stricter in what we send than what we receive.
2023-11-10 21:34:00 +03:00
OrganizationName: "mox.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
DateRange: tlsrpt.TLSRPTDateRange{
Start: endUTC.Add(-24 * time.Hour),
End: endUTC.Add(-time.Second),
},
outgoing dmarc/tls reporting improvements - dmarc reports: add a cid to the log line about one run of sending reports, and log line for each report - in smtpclient, also handle tls errors from the first read after a handshake. we appear to sometimes get tls alerts about bad certificates on the first read. - for messages to dmarc/tls reporting addresses that we think should/can not be processed as reports, add an X-Mox- header explaining the reason. - tls reports: send report messages with From address of postmaster at an actually configured domain for the mail host. and only send reports when dkim signing is configured for that domain. the domain is also the submitter domain. the rfc seems to require dkim-signing with an exact match with the message from and submitter. - for incoming tls reports, in the smtp server, we do allow a dkim-signature domain that is higher-level (up to publicsuffix) of the message from domain. so we are stricter in what we send than what we receive.
2023-11-10 21:34:00 +03:00
ContactInfo: "postmaster@mox.example",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReportID: endUTC.Add(-12*time.Hour).Format("20060102") + ".xn--74h.example@mox.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
Policies: []tlsrpt.Result{
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.STS,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
String: []string{"... mtasts policy ..."},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
MXHost: []string{"*.xn--74h.example"},
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 3,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultCertificateExpired,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 3,
},
},
},
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
// Includes reports about MX target, for DANE policies.
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "mailhost.xn--74h.example",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
String: []string{"... tlsa record ..."},
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 1,
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
}
report2 := tlsrpt.Report{
outgoing dmarc/tls reporting improvements - dmarc reports: add a cid to the log line about one run of sending reports, and log line for each report - in smtpclient, also handle tls errors from the first read after a handshake. we appear to sometimes get tls alerts about bad certificates on the first read. - for messages to dmarc/tls reporting addresses that we think should/can not be processed as reports, add an X-Mox- header explaining the reason. - tls reports: send report messages with From address of postmaster at an actually configured domain for the mail host. and only send reports when dkim signing is configured for that domain. the domain is also the submitter domain. the rfc seems to require dkim-signing with an exact match with the message from and submitter. - for incoming tls reports, in the smtp server, we do allow a dkim-signature domain that is higher-level (up to publicsuffix) of the message from domain. so we are stricter in what we send than what we receive.
2023-11-10 21:34:00 +03:00
OrganizationName: "mox.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
DateRange: tlsrpt.TLSRPTDateRange{
Start: endUTC.Add(-24 * time.Hour),
End: endUTC.Add(-time.Second),
},
outgoing dmarc/tls reporting improvements - dmarc reports: add a cid to the log line about one run of sending reports, and log line for each report - in smtpclient, also handle tls errors from the first read after a handshake. we appear to sometimes get tls alerts about bad certificates on the first read. - for messages to dmarc/tls reporting addresses that we think should/can not be processed as reports, add an X-Mox- header explaining the reason. - tls reports: send report messages with From address of postmaster at an actually configured domain for the mail host. and only send reports when dkim signing is configured for that domain. the domain is also the submitter domain. the rfc seems to require dkim-signing with an exact match with the message from and submitter. - for incoming tls reports, in the smtp server, we do allow a dkim-signature domain that is higher-level (up to publicsuffix) of the message from domain. so we are stricter in what we send than what we receive.
2023-11-10 21:34:00 +03:00
ContactInfo: "postmaster@mox.example",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReportID: endUTC.Add(-12*time.Hour).Format("20060102") + ".mailhost.xn--74h.example@mox.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
Policies: []tlsrpt.Result{
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
// The MX target policies are per-recipient domain, so the MX operator can see the
// affected recipient domains.
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "sharedsender.example", // Recipient domain.
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
String: []string{"... tlsa record ..."},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
MXHost: []string{"mailhost.xn--74h.example"}, // Original policy domain.
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
Summary: tlsrpt.Summary{
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
ReceivingIP: "4.3.2.1",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
FailedSessionCount: 1,
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
Domain: "xn--74h.example", // Recipient domain.
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
String: []string{"... tlsa record ..."},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
MXHost: []string{"mailhost.xn--74h.example"}, // Original policy domain.
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 1,
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
},
}
report3 := tlsrpt.Report{
OrganizationName: "mox.example",
DateRange: tlsrpt.TLSRPTDateRange{
Start: endUTC.Add(-24 * time.Hour),
End: endUTC.Add(-time.Second),
},
ContactInfo: "postmaster@mox.example",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReportID: endUTC.Add(-12*time.Hour).Format("20060102") + ".mailhost.xn--74h.example@mox.example",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
Policies: []tlsrpt.Result{
// The MX target policies are per-recipient domain, so the MX operator can see the
// affected recipient domains.
{
Policy: tlsrpt.ResultPolicy{
Type: tlsrpt.TLSA,
Domain: "sharedsender.example", // Recipient domain.
String: []string{"... tlsa record ..."},
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
MXHost: []string{"mailhost.xn--74h.example"}, // Original policy domain.
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
},
Summary: tlsrpt.Summary{
TotalSuccessfulSessionCount: 10,
TotalFailureSessionCount: 1,
},
FailureDetails: []tlsrpt.FailureDetails{
{
ResultType: tlsrpt.ResultValidationFailure,
SendingMTAIP: "1.2.3.4",
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
ReceivingMXHostname: "mailhost.xn--74h.example",
ReceivingMXHelo: "mailhost.xn--74h.example",
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
ReceivingIP: "4.3.2.1",
FailedSessionCount: 1,
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
FailureReasonCode: "dns-extended-error-7-signature-expired",
},
},
},
},
}
// Set a timeUntil that we steplock and that causes the actual sleep to return
// immediately when we want to.
wait := make(chan struct{})
step := make(chan time.Duration)
jitteredTimeUntil = func(_ time.Time) time.Duration {
wait <- struct{}{}
return <-step
}
sleepBetween = func(ctx context.Context, d time.Duration) (ok bool) { return true }
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
test := func(results []tlsrptdb.TLSResult, expReports map[string][]tlsrpt.Report) {
t.Helper()
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
mox.Shutdown, mox.ShutdownCancel = context.WithCancel(ctxbg)
for _, r := range results {
err := db.Insert(ctxbg, &r)
tcheckf(t, err, "inserting tlsresult")
}
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
haveReports := map[string][]tlsrpt.Report{}
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
var mutex sync.Mutex
var index int
queue: implement adding a message to the queue that gets sent to multiple recipients and in a way that allows us to send that message to multiple recipients in a single smtp transaction.
2024-03-05 22:10:28 +03:00
queueAdd = func(ctx context.Context, log mlog.Log, senderAccount string, msgFile *os.File, qml ...queue.Msg) error {
if len(qml) != 1 {
return fmt.Errorf("queued %d messages, expect 1", len(qml))
}
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
mutex.Lock()
defer mutex.Unlock()
// Read message file. Also write copy to disk for inspection.
buf, err := io.ReadAll(&moxio.AtReader{R: msgFile})
tcheckf(t, err, "read report message")
p := fmt.Sprintf("../testdata/tlsrptsend/data/report%d.eml", index)
index++
queue: implement adding a message to the queue that gets sent to multiple recipients and in a way that allows us to send that message to multiple recipients in a single smtp transaction.
2024-03-05 22:10:28 +03:00
err = os.WriteFile(p, append(append([]byte{}, qml[0].MsgPrefix...), buf...), 0600)
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
tcheckf(t, err, "write report message")
change javascript into typescript for webaccount and webadmin interface all ui frontend code is now in typescript. we no longer need jshint, and we build the frontend code during "make build". this also changes tlsrpt types for a Report, not encoding field names with dashes, but to keep them valid identifiers in javascript. this makes it more conveniently to work with in the frontend, and works around a sherpats limitation.
2023-12-31 13:55:22 +03:00
reportJSON, err := tlsrpt.ParseMessage(log.Logger, msgFile)
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
tcheckf(t, err, "parsing generated report message")
queue: implement adding a message to the queue that gets sent to multiple recipients and in a way that allows us to send that message to multiple recipients in a single smtp transaction.
2024-03-05 22:10:28 +03:00
addr := qml[0].Recipient().String()
change javascript into typescript for webaccount and webadmin interface all ui frontend code is now in typescript. we no longer need jshint, and we build the frontend code during "make build". this also changes tlsrpt types for a Report, not encoding field names with dashes, but to keep them valid identifiers in javascript. this makes it more conveniently to work with in the frontend, and works around a sherpats limitation.
2023-12-31 13:55:22 +03:00
haveReports[addr] = append(haveReports[addr], reportJSON.Convert())
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
return nil
}
Start(resolver)
// Run first loop.
<-wait
step <- 0
<-wait
tcompare(t, haveReports, expReports)
// Second loop. Evaluations cleaned, should not result in report messages.
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
haveReports = map[string][]tlsrpt.Report{}
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
step <- 0
<-wait
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
tcompare(t, haveReports, map[string][]tlsrpt.Report{})
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
// Caus Start to stop.
mox.ShutdownCancel()
step <- time.Minute
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
leftover, err := bstore.QueryDB[tlsrptdb.TLSResult](ctxbg, db).List()
tcheckf(t, err, "querying database")
if len(leftover) != 0 {
t.Fatalf("leftover results in database after sending reports: %v", leftover)
}
_, err = bstore.QueryDB[tlsrptdb.TLSResult](ctxbg, db).Delete()
tcheckf(t, err, "cleaning from database")
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
}
// Multiple results, some are combined into a single report, another result
// generates a separate report to multiple rua's, and the last don't send a report.
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
test(tlsResults, map[string][]tlsrpt.Report{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"tls-reports@xn--74h.example": {report1},
"tls-reports1@mailhost.xn--74h.example": {report2},
"tls-reports2@mailhost.xn--74h.example": {report2},
"tls-reports3@mailhost.xn--74h.example": {report2},
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
})
add suppression list for outgoing dmarc and tls reports for reporting addresses that cause DSNs to be returned. that just adds noise. the admin can add/remove/extend addresses through the webadmin. in the future, we could send reports with a smtp mail from of "postmaster+<signed-encoded-recipient>@...", and add the reporting recipient on the suppression list automatically when a DSN comes in on that address, but for now this will probably do.
2023-11-13 15:48:52 +03:00
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
// If MX target has same reporting addresses as recipient domain, only recipient
// domain should get a report.
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
resolver.TXT["_smtp._tls.mailhost.xn--74h.example."] = []string{"v=TLSRPTv1; rua=mailto:tls-reports@xn--74h.example"}
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
test(tlsResults[:2], map[string][]tlsrpt.Report{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"tls-reports@xn--74h.example": {report1},
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
})
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
resolver.TXT["_smtp._tls.sharedsender.example."] = []string{"v=TLSRPTv1; rua=mailto:tls-reports@xn--74h.example"}
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
test(tlsResults, map[string][]tlsrpt.Report{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"tls-reports@xn--74h.example": {report1, report3},
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
})
// Suppressed addresses don't get a report.
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
resolver.TXT["_smtp._tls.mailhost.xn--74h.example."] = []string{"v=TLSRPTv1; rua=mailto:tls-reports1@mailhost.xn--74h.example,mailto:tls-reports2@mailhost.xn--74h.example; rua=mailto:tls-reports3@mailhost.xn--74h.example"}
add suppression list for outgoing dmarc and tls reports for reporting addresses that cause DSNs to be returned. that just adds noise. the admin can add/remove/extend addresses through the webadmin. in the future, we could send reports with a smtp mail from of "postmaster+<signed-encoded-recipient>@...", and add the reporting recipient on the suppression list automatically when a DSN comes in on that address, but for now this will probably do.
2023-11-13 15:48:52 +03:00
db.Insert(ctxbg,
use new sherpadoc rename mechanism to remove some typename stuttering the stuttering was introduced to make the same type name declared in multiple packages, and used in the admin sherpa api, unique. with sherpadoc's new rename, we can make them unique when generating the api definition/docs, and the Go code can use nicer names.
2024-04-19 11:51:24 +03:00
&tlsrptdb.SuppressAddress{ReportingAddress: "tls-reports@xn--74h.example", Until: time.Now().Add(-time.Minute)}, // Expired, so ignored.
&tlsrptdb.SuppressAddress{ReportingAddress: "tls-reports1@mailhost.xn--74h.example", Until: time.Now().Add(time.Minute)}, // Still valid.
&tlsrptdb.SuppressAddress{ReportingAddress: "tls-reports3@mailhost.xn--74h.example", Until: time.Now().Add(31 * 24 * time.Hour)}, // Still valid.
add suppression list for outgoing dmarc and tls reports for reporting addresses that cause DSNs to be returned. that just adds noise. the admin can add/remove/extend addresses through the webadmin. in the future, we could send reports with a smtp mail from of "postmaster+<signed-encoded-recipient>@...", and add the reporting recipient on the suppression list automatically when a DSN comes in on that address, but for now this will probably do.
2023-11-13 15:48:52 +03:00
)
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
test(tlsResults, map[string][]tlsrpt.Report{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"tls-reports@xn--74h.example": {report1},
"tls-reports2@mailhost.xn--74h.example": {report2},
better handling of outgoing tls reports to recipient domains vs hosts based on discussion on uta mailing list. it seems the intention of the tlsrpt is to only send reports to recipient domains. but i was able to interpret the tlsrpt rfc as sending reports to mx hosts too ("policy domain", and because it makes sense given how DANE works per MX host, not recipient domain). this change makes the behaviour of outgoing reports to recipient domains work more in line with expectations most folks may have about tls reporting (i.e. also include per-mx host tlsa policies in the report). this also keeps reports to mx hosts working, and makes them more useful by including the recipient domains of affected deliveries.
2023-11-20 13:31:46 +03:00
})
// Make reports success-only, ensuring we don't get a report anymore.
for i := range tlsResults {
for j := range tlsResults[i].Results {
tlsResults[i].Results[j].Summary.TotalFailureSessionCount = 0
tlsResults[i].Results[j].FailureDetails = nil
}
}
test(tlsResults, map[string][]tlsrpt.Report{})
// But when we want to send report for all-successful connections, we get reports again.
mox.Conf.Static.OutgoingTLSReportsForAllSuccess = true
for _, report := range []*tlsrpt.Report{&report1, &report2} {
for i := range report.Policies {
report.Policies[i].Summary.TotalFailureSessionCount = 0
report.Policies[i].FailureDetails = nil
}
}
test(tlsResults, map[string][]tlsrpt.Report{
when sending tls reports, ensure we use ASCII A-labels, not U-labels in the policy-domain field
2024-01-24 12:36:20 +03:00
"tls-reports@xn--74h.example": {report1},
"tls-reports2@mailhost.xn--74h.example": {report2},
add suppression list for outgoing dmarc and tls reports for reporting addresses that cause DSNs to be returned. that just adds noise. the admin can add/remove/extend addresses through the webadmin. in the future, we could send reports with a smtp mail from of "postmaster+<signed-encoded-recipient>@...", and add the reporting recipient on the suppression list automatically when a DSN comes in on that address, but for now this will probably do.
2023-11-13 15:48:52 +03:00
})
implement outgoing tls reports we were already accepting, processing and displaying incoming tls reports. now we start tracking TLS connection and security-policy-related errors for outgoing message deliveries as well. we send reports once a day, to the reporting addresses specified in TLSRPT records (rua) of a policy domain. these reports are about MTA-STS policies and/or DANE policies, and about STARTTLS-related failures. sending reports is enabled by default, but can be disabled through setting NoOutgoingTLSReports in mox.conf. only at the end of the implementation process came the realization that the TLSRPT policy domain for DANE (MX) hosts are separate from the TLSRPT policy for the recipient domain, and that MTA-STS and DANE TLS/policy results are typically delivered in separate reports. so MX hosts need their own TLSRPT policies. config for the per-host TLSRPT policy should be added to mox.conf for existing installs, in field HostTLSRPT. it is automatically configured by quickstart for new installs. with a HostTLSRPT config, the "dns records" and "dns check" admin pages now suggest the per-host TLSRPT record. by creating that record, you're requesting TLS reports about your MX host. gathering all the TLS/policy results is somewhat tricky. the tentacles go throughout the code. the positive result is that the TLS/policy-related code had to be cleaned up a bit. for example, the smtpclient TLS modes now reflect reality better, with independent settings about whether PKIX and/or DANE verification has to be done, and/or whether verification errors have to be ignored (e.g. for tls-required: no header). also, cached mtasts policies of mode "none" are now cleaned up once the MTA-STS DNS record goes away.
2023-11-09 19:40:46 +03:00
}
Reference in a new issue Copy permalink