mox/message/compose.go
Mechiel Lukkien 09fcc49223
add a webapi and webhooks for a simple http/json-based api
for applications to compose/send messages, receive delivery feedback, and
maintain suppression lists.

this is an alternative to applications using a library to compose messages,
submitting those messages using smtp, and monitoring a mailbox with imap for
DSNs, which can be processed into the equivalent of suppression lists. but you
need to know about all these standards/protocols and find libraries. by using
the webapi & webhooks, you just need a http & json library.

unfortunately, there is no standard for these kinds of api, so mox has made up
yet another one...

matching incoming DSNs about deliveries to original outgoing messages requires
keeping history of "retired" messages (delivered from the queue, either
successfully or failed). this can be enabled per account. history is also
useful for debugging deliveries. we now also keep history of each delivery
attempt, accessible while still in the queue, and kept when a message is
retired. the queue webadmin pages now also have pagination, to show potentially
large history.

a queue of webhook calls is now managed too. failures are retried similar to
message deliveries. webhooks can also be saved to the retired list after
completing. also configurable per account.

messages can be sent with a "unique smtp mail from" address. this can only be
used if the domain is configured with a localpart catchall separator such as
"+". when enabled, a queued message gets assigned a random "fromid", which is
added after the separator when sending. when DSNs are returned, they can be
related to previously sent messages based on this fromid. in the future, we can
implement matching on the "envid" used in the smtp dsn extension, or on the
"message-id" of the message. using a fromid can be triggered by authenticating
with a login email address that is configured as enabling fromid.

suppression lists are automatically managed per account. if a delivery attempt
results in certain smtp errors, the destination address is added to the
suppression list. future messages queued for that recipient will immediately
fail without a delivery attempt. suppression lists protect your mail server
reputation.

submitted messages can carry "extra" data through the queue and webhooks for
outgoing deliveries. through webapi as a json object, through smtp submission
as message headers of the form "x-mox-extra-<key>: value".

to make it easy to test webapi/webhooks locally, the "localserve" mode actually
puts messages in the queue. when it's time to deliver, it still won't do a full
delivery attempt, but just delivers to the sender account. unless the recipient
address has a special form, simulating a failure to deliver.

admins now have more control over the queue. "hold rules" can be added to mark
newly queued messages as "on hold", pausing delivery. rules can be about
certain sender or recipient domains/addresses, or apply to all messages pausing
the entire queue. also useful for (local) testing.

new config options have been introduced. they are editable through the admin
and/or account web interfaces.

the webapi http endpoints are enabled for newly generated configs with the
quickstart, and in localserve. existing configurations must explicitly enable
the webapi in mox.conf.

gopherwatch.org was created to dogfood this code. it initially used just the
compose/smtpclient/imapclient mox packages to send messages and process
delivery feedback. it will get a config option to use the mox webapi/webhooks
instead. the gopherwatch code to use webapi/webhook is smaller and simpler, and
developing that shaped development of the mox webapi/webhooks.

for issue #31 by cuu508
2024-04-15 21:49:02 +02:00

176 lines
4.6 KiB
Go

package message
import (
"bufio"
"errors"
"fmt"
"io"
"mime"
"mime/quotedprintable"
"net/mail"
"strings"
"github.com/mjl-/mox/smtp"
)
var (
ErrMessageSize = errors.New("message too large")
ErrCompose = errors.New("compose")
)
// Composer helps compose a message. Operations that fail call panic, which should
// be caught with recover(), checking for ErrCompose and optionally ErrMessageSize.
// Writes are buffered.
type Composer struct {
Has8bit bool // Whether message contains 8bit data.
SMTPUTF8 bool // Whether message needs to be sent with SMTPUTF8 extension.
Size int64 // Total bytes written.
bw *bufio.Writer
maxSize int64 // If greater than zero, writes beyond maximum size raise ErrMessageSize.
}
// NewComposer initializes a new composer with a buffered writer around w, and
// with a maximum message size if maxSize is greater than zero.
//
// smtputf8 must be set when the message must be delivered with smtputf8: if any
// email address localpart has non-ascii (utf-8).
//
// Operations on a Composer do not return an error. Caller must use recover() to
// catch ErrCompose and optionally ErrMessageSize errors.
func NewComposer(w io.Writer, maxSize int64, smtputf8 bool) *Composer {
return &Composer{bw: bufio.NewWriter(w), maxSize: maxSize, SMTPUTF8: smtputf8, Has8bit: smtputf8}
}
// Write implements io.Writer, but calls panic (that is handled higher up) on
// i/o errors.
func (c *Composer) Write(buf []byte) (int, error) {
if c.maxSize > 0 && c.Size+int64(len(buf)) > c.maxSize {
c.Checkf(ErrMessageSize, "writing message")
}
n, err := c.bw.Write(buf)
if n > 0 {
c.Size += int64(n)
}
c.Checkf(err, "write")
return n, nil
}
// Checkf checks err, panicing with sentinel error value.
func (c *Composer) Checkf(err error, format string, args ...any) {
if err != nil {
// We expose the original error too, needed at least for ErrMessageSize.
panic(fmt.Errorf("%w: %w: %v", ErrCompose, err, fmt.Sprintf(format, args...)))
}
}
// Flush writes any buffered output.
func (c *Composer) Flush() {
err := c.bw.Flush()
c.Checkf(err, "flush")
}
// Header writes a message header.
func (c *Composer) Header(k, v string) {
fmt.Fprintf(c, "%s: %s\r\n", k, v)
}
// NameAddress holds both an address display name, and an SMTP path address.
type NameAddress struct {
DisplayName string
Address smtp.Address
}
// HeaderAddrs writes a message header with addresses.
func (c *Composer) HeaderAddrs(k string, l []NameAddress) {
if len(l) == 0 {
return
}
v := ""
linelen := len(k) + len(": ")
for _, a := range l {
if v != "" {
v += ","
linelen++
}
addr := mail.Address{Name: a.DisplayName, Address: a.Address.Pack(c.SMTPUTF8)}
s := addr.String()
if v != "" && linelen+1+len(s) > 77 {
v += "\r\n\t"
linelen = 1
} else if v != "" {
v += " "
linelen++
}
v += s
linelen += len(s)
}
fmt.Fprintf(c, "%s: %s\r\n", k, v)
}
// Subject writes a subject message header.
func (c *Composer) Subject(subject string) {
var subjectValue string
subjectLineLen := len("Subject: ")
subjectWord := false
for i, word := range strings.Split(subject, " ") {
if !c.SMTPUTF8 && !isASCII(word) {
word = mime.QEncoding.Encode("utf-8", word)
}
if i > 0 {
subjectValue += " "
subjectLineLen++
}
if subjectWord && subjectLineLen+len(word) > 77 {
subjectValue += "\r\n\t"
subjectLineLen = 1
}
subjectValue += word
subjectLineLen += len(word)
subjectWord = true
}
c.Header("Subject", subjectValue)
}
// Line writes an empty line.
func (c *Composer) Line() {
_, _ = c.Write([]byte("\r\n"))
}
// TextPart prepares a text part to be added. Text should contain lines terminated
// with newlines (lf), which are replaced with crlf. The returned text may be
// quotedprintable, if needed. The returned ct and cte headers are for use with
// Content-Type and Content-Transfer-Encoding headers.
func (c *Composer) TextPart(subtype, text string) (textBody []byte, ct, cte string) {
if !strings.HasSuffix(text, "\n") {
text += "\n"
}
text = strings.ReplaceAll(text, "\n", "\r\n")
charset := "us-ascii"
if !isASCII(text) {
charset = "utf-8"
}
if NeedsQuotedPrintable(text) {
var sb strings.Builder
_, err := io.Copy(quotedprintable.NewWriter(&sb), strings.NewReader(text))
c.Checkf(err, "converting text to quoted printable")
text = sb.String()
cte = "quoted-printable"
} else if c.Has8bit || charset == "utf-8" {
cte = "8bit"
} else {
cte = "7bit"
}
ct = mime.FormatMediaType("text/"+subtype, map[string]string{"charset": charset})
return []byte(text), ct, cte
}
func isASCII(s string) bool {
for _, c := range s {
if c >= 0x80 {
return false
}
}
return true
}