for applications to compose/send messages, receive delivery feedback, and
maintain suppression lists.
this is an alternative to applications using a library to compose messages,
submitting those messages using smtp, and monitoring a mailbox with imap for
DSNs, which can be processed into the equivalent of suppression lists. but you
need to know about all these standards/protocols and find libraries. by using
the webapi & webhooks, you just need a http & json library.
unfortunately, there is no standard for these kinds of api, so mox has made up
yet another one...
matching incoming DSNs about deliveries to original outgoing messages requires
keeping history of "retired" messages (delivered from the queue, either
successfully or failed). this can be enabled per account. history is also
useful for debugging deliveries. we now also keep history of each delivery
attempt, accessible while still in the queue, and kept when a message is
retired. the queue webadmin pages now also have pagination, to show potentially
large history.
a queue of webhook calls is now managed too. failures are retried similar to
message deliveries. webhooks can also be saved to the retired list after
completing. also configurable per account.
messages can be sent with a "unique smtp mail from" address. this can only be
used if the domain is configured with a localpart catchall separator such as
"+". when enabled, a queued message gets assigned a random "fromid", which is
added after the separator when sending. when DSNs are returned, they can be
related to previously sent messages based on this fromid. in the future, we can
implement matching on the "envid" used in the smtp dsn extension, or on the
"message-id" of the message. using a fromid can be triggered by authenticating
with a login email address that is configured as enabling fromid.
suppression lists are automatically managed per account. if a delivery attempt
results in certain smtp errors, the destination address is added to the
suppression list. future messages queued for that recipient will immediately
fail without a delivery attempt. suppression lists protect your mail server
reputation.
submitted messages can carry "extra" data through the queue and webhooks for
outgoing deliveries. through webapi as a json object, through smtp submission
as message headers of the form "x-mox-extra-<key>: value".
to make it easy to test webapi/webhooks locally, the "localserve" mode actually
puts messages in the queue. when it's time to deliver, it still won't do a full
delivery attempt, but just delivers to the sender account. unless the recipient
address has a special form, simulating a failure to deliver.
admins now have more control over the queue. "hold rules" can be added to mark
newly queued messages as "on hold", pausing delivery. rules can be about
certain sender or recipient domains/addresses, or apply to all messages pausing
the entire queue. also useful for (local) testing.
new config options have been introduced. they are editable through the admin
and/or account web interfaces.
the webapi http endpoints are enabled for newly generated configs with the
quickstart, and in localserve. existing configurations must explicitly enable
the webapi in mox.conf.
gopherwatch.org was created to dogfood this code. it initially used just the
compose/smtpclient/imapclient mox packages to send messages and process
delivery feedback. it will get a config option to use the mox webapi/webhooks
instead. the gopherwatch code to use webapi/webhook is smaller and simpler, and
developing that shaped development of the mox webapi/webhooks.
for issue #31 by cuu508
- add option to put messages in the queue "on hold", preventing delivery
attempts until taken off hold again.
- add "hold rules", to automatically mark some/all submitted messages as "on
hold", e.g. from a specific account or to a specific domain.
- add operation to "fail" a message, causing a DSN to be delivered to the
sender. previously we could only drop a message from the queue.
- update admin page & add new cli tools for these operations, with new
filtering rules for selecting the messages to operate on. in the admin
interface, add filtering and checkboxes to select a set of messages to operate
on.
by explaining (in the titles/hovers) what the concepts and requirements are, by
using selects/dropdowns or datalist suggestions where we have a known list, by
automatically suggesting a good account name, and putting the input fields in a
more sensible order.
based on issue #132 by ally9335
actually, this fix can reduce focus changes for more operations. withStatus is
often used to show an operation in progress in the status bar, only when the
operation isn't done within 1 second. we would restore focus to the element
before the operation started. that was done because we disable elements
sometimes (preventing duplicate form submission). for things like the
autocomplete, with the tab key, which also moves focus to the next element, we
don't want that focus switched back again.
the smtp extension, rfc 4865.
also implement in the webmail.
the queueing/delivery part hardly required changes: we just set the first
delivery time in the future instead of immediately.
still have to find the first client that implements it.
to start composing a message.
the help popup now has a button to register the "mailto:" links with the mox
webmail (typically only works over https, not all browsers support it).
the mailto links are specified in 6068. we support the to/cc/bcc/subject/body
parameters. other parameters should be seen as custom headers, but we don't
support messages with custom headers at all at the moment, so we ignore them.
we now also turn text of the form "mailto:user@host" into a clickable link
(will not be too common). we could be recognizing any "x@x.x" as email address
and make them clickable in the future.
thanks to Hans-Jörg for explaining this functionality.
before, we showed the xn-- ascii names, along with the unicode name. but users
of internationalized email don't want to see any xn-- names. we now put those
in an html title attribute for some cases, so you can still see them if you
really want to, by hovering.
after talking to arnt at fosdem.
if a browser is ahead just a few seconds, we would show "-<1min", not great.
just show "<1min" in that case. we'll still show negative age if drift is more
than 1 minute, which seems like a good hint to get time fixed on either client
or server.
the http basic auth we had was very simple to reason about, and to implement.
but it has a major downside:
there is no way to logout, browsers keep sending credentials. ideally, browsers
themselves would show a button to stop sending credentials.
a related downside: the http auth mechanism doesn't indicate for which server
paths the credentials are.
another downside: the original password is sent to the server with each
request. though sending original passwords to web servers seems to be
considered normal.
our new approach uses session cookies, along with csrf values when we can. the
sessions are server-side managed, automatically extended on each use. this
makes it easy to invalidate sessions and keeps the frontend simpler (than with
long- vs short-term sessions and refreshing). the cookies are httponly,
samesite=strict, scoped to the path of the web interface. cookies are set
"secure" when set over https. the cookie is set by a successful call to Login.
a call to Logout invalidates a session. changing a password invalidates all
sessions for a user, but keeps the session with which the password was changed
alive. the csrf value is also random, and associated with the session cookie.
the csrf must be sent as header for api calls, or as parameter for direct form
posts (where we cannot set a custom header). rest-like calls made directly by
the browser, e.g. for images, don't have a csrf protection. the csrf value is
returned by the Login api call and stored in localstorage.
api calls without credentials return code "user:noAuth", and with bad
credentials return "user:badAuth". the api client recognizes this and triggers
a login. after a login, all auth-failed api calls are automatically retried.
only for "user:badAuth" is an error message displayed in the login form (e.g.
session expired).
in an ideal world, browsers would take care of most session management. a
server would indicate authentication is needed (like http basic auth), and the
browsers uses trusted ui to request credentials for the server & path. the
browser could use safer mechanism than sending original passwords to the
server, such as scram, along with a standard way to create sessions. for now,
web developers have to do authentication themselves: from showing the login
prompt, ensuring the right session/csrf cookies/localstorage/headers/etc are
sent with each request.
webauthn is a newer way to do authentication, perhaps we'll implement it in the
future. though hardware tokens aren't an attractive option for many users, and
it may be overkill as long as we still do old-fashioned authentication in smtp
& imap where passwords can be sent to the server.
for issue #58
all ui frontend code is now in typescript. we no longer need jshint, and we
build the frontend code during "make build".
this also changes tlsrpt types for a Report, not encoding field names with
dashes, but to keep them valid identifiers in javascript. this makes it more
conveniently to work with in the frontend, and works around a sherpats
limitation.
i'm not sure this is good enough.
this is based on field MsgFromValidation, but it doesn't hold the full DMARC information.
we also don't know mailing list-status for all historic messages.
so the red underline can occur too often.
we were trying to offset the timezone, but that makes no sense: we already
created a date in the local timezone based on (milli)seconds passed. so we can
just use that date instead of calculating a wrong date.
with requiretls, the tls verification mode/rules for email deliveries can be
changed by the sender/submitter. in two ways:
1. "requiretls" smtp extension to always enforce verified tls (with mta-sts or
dnssec+dane), along the entire delivery path until delivery into the final
destination mailbox (so entire transport is verified-tls-protected).
2. "tls-required: no" message header, to ignore any tls and tls verification
errors even if the recipient domain has a policy that requires tls verification
(mta-sts and/or dnssec+dane), allowing delivery of non-sensitive messages in
case of misconfiguration/interoperability issues (at least useful for sending
tls reports).
we enable requiretls by default (only when tls is active), for smtp and
submission. it can be disabled through the config.
for each delivery attempt, we now store (per recipient domain, in the account
of the sender) whether the smtp server supports starttls and requiretls. this
support is shown (after having sent a first message) in the webmail when
sending a message (the previous 3 bars under the address input field are now 5
bars, the first for starttls support, the last for requiretls support). when
all recipient domains for a message are known to implement requiretls,
requiretls is automatically selected for sending (instead of "default" tls
behaviour). users can also select the "fallback to insecure" to add the
"tls-required: no" header.
new metrics are added for insight into requiretls errors and (some, not yet
all) cases where tls-required-no ignored a tls/verification error.
the admin can change the requiretls status for messages in the queue. so with
default delivery attempts, when verified tls is required by failing, an admin
could potentially change the field to "tls-required: no"-behaviour.
messages received (over smtp) with the requiretls option, get a comment added
to their Received header line, just before "id", after "with".
the bar is currently showing 3 properties:
1. mta-sts enforced;
2. mx lookup returned dnssec-signed response;
3. first delivery destination host has dane records
the colors are: red for not-implemented, green for implemented, gray for error,
nothing for unknown/irrelevant.
the plan is to implement "requiretls" soon and start caching per domain whether
delivery can be done with starttls and whether the domain supports requiretls.
and show that in two new parts of the bar.
thanks to damian poddebniak for pointing out that security indicators should
always be visible, not only for positive/negative result. otherwise users won't
notice their absence.
so full name/email address is visible.
using a hidden grid element that gets the same content as the input element.
from https://css-tricks.com/auto-growing-inputs-textareas/
a recent commit probably also make the compose window full-screen-width on
chrome, this restores to the intended behaviour of a less wide default size.
if you add multiple address fields, the compose window will still grow. not
great, in the future, we should make the compose window resizable by dragging.
the textarea is resizable (though it's not convenient to do in firefox which
only shows a dragcorner in the bottomright, usually located in the bottom
corner of the screen, so there is little space left to drag the corner; the
workaround is to move the window temporarily).
by using a String object as the textarea child. instead of a regular js string
that would be unicode-block-switch-highlighted, which would cause it to be
split into parts, with odd or even parts added as span elements, which the
textarea would then ignore.
top-posting causes "On $datetime, $sender wrote:" above the quoted text to be
added (unless there was no Date header or valid address in a From header).
in the near future we should create settings, and add a setting for adding the
"on ... wrote"-line, ideally including a template.
for issue #83 by mattfbacon, thanks!
removing an item from the selected list should be done regardless of focus,
i.e. the code snippet shouldn't have been behind the "if (focus...)" condition.
