Commit graph

674 commits

Author SHA1 Message Date
Mechiel Lukkien
5229d01601
webmail: for replies/forwards, add button "send and archive thread" next to the "send" button, and give it a control+shift+Enter shortcut
the regular send shortcut is control+Enter. the shift enables "archive thread".
there is no configuration option, you'll always get the button, but only for
reply/forward, not for new compose.

we may do "send and move thread to thrash", but let's wait until people want it.

for github issue #135 by mattfbacon
2024-04-19 21:17:42 +02:00
Mechiel Lukkien
b54e903f01
webmail: ctrl Backspace now removes an address input field if it is empty
instead of "ctrl -". i found ctrl backspace more intuitive.
2024-04-19 18:03:56 +02:00
Mechiel Lukkien
8a1d81c29a
webmail: show link to webaccount interface in top right
only if account web interface is enabled on the same listener and same http/https scheme.
2024-04-19 18:02:30 +02:00
Mechiel Lukkien
70adf353ee
webmail: add server-side stored settings, for signature, top/bottom reply and showing the security indications below address input fields
should solve #102
2024-04-19 18:02:24 +02:00
Mechiel Lukkien
3bbd7c7d9b
website: mention "mox localserve" as a good way to get a feeling for mox 2024-04-19 11:12:17 +02:00
Mechiel Lukkien
ec967ef321
use new sherpadoc rename mechanism to remove some typename stuttering
the stuttering was introduced to make the same type name declared in multiple
packages, and used in the admin sherpa api, unique. with sherpadoc's new
rename, we can make them unique when generating the api definition/docs, and
the Go code can use nicer names.
2024-04-19 10:51:24 +02:00
Mechiel Lukkien
962575f21b
mention webhook retry intervals in webhook docs
for github issue #31, feedback from cuu508
2024-04-19 10:33:28 +02:00
Mechiel Lukkien
e702f45d32
webadmin: make remaining domain settings configurable via admin web interface
for dmarc reporting address, tls reporting address, mtasts policy, dkim keys/selectors.

should make it easier for webadmin-using admins to discover these settings.

the webadmin interface is now on par with functionality you would set through
the configuration file, let's keep it that way.
2024-04-19 10:23:53 +02:00
Mechiel Lukkien
a69887bfab
webadmin: make routes configurable: globally, per domain, per account
this simplifies some of the code that makes modifications to the config file. a
few protected functions can make changes to the dynamic config, which webadmin
can use. instead of having separate functions in mox-/admin.go for each type of
change.

this also exports the parsed full dynamic config to webadmin, so we need fewer
functions for specific config fields too.
2024-04-18 11:14:24 +02:00
Mechiel Lukkien
baf4df55a6
make more account config fields configurable through web interface
so users can change it themselves, instead of requiring an admin to change the
settings.
2024-04-17 21:31:26 +02:00
Mechiel Lukkien
8bcce40c55
webmail: recognize multiple urls in List-Post addresses
there may be a http(s)-address, which we'll ignore. the mailto may come after
that. like in google groups.
2024-04-16 20:26:37 +02:00
Mechiel Lukkien
8654a1f901
with localserve, in queue, when "delivering" to the sender account, mark domain "localhost" as dkimverified
may be useful for testing, e.g. for rulesets to deliver messages to mailboxes other than Inbox.
2024-04-16 19:26:26 +02:00
Mechiel Lukkien
0a10283de0
show separate localpart and dropdown of domains instead of full email field when adding a new account (with initial email address) 2024-04-16 19:23:00 +02:00
Mechiel Lukkien
c9451d4d06
in webmail & webapisrv, store bcc header in sent messages
when sending a message with bcc's, prepend the bcc header to the message we
store in the sent folder. still not in the message we send to the recipients.
2024-04-16 17:57:46 +02:00
Mechiel Lukkien
abd098e8c0
in more tests, after closing accounts, check the last reference is indeed gone 2024-04-16 17:33:54 +02:00
Mechiel Lukkien
afc47c8108
if webauth login cookie is missing, and forwarding was configured, hint that reverse proxy may be stripping path
the cookies are set with a specific path, because the webadmin, webaccount and
webmail cookies can be on the same domain (this is the default). if the reverse
proxy strips the path while forwarding, the browser won't set the cookie and
the login attempt will fail.

based on github issue #151 from naturalethic
2024-04-16 16:06:31 +02:00
Mechiel Lukkien
daa88480cb
fix potential endless loop during queue msg/hook pagination when environment has TZ UTC, triggered by tests introduced in previous test
time.Now() returns a timestamp with timezone Local. if you marshal & unmarshal
it again, it'll get the Local timezone again. unless the local timezone is UTC.
then it will get the UTC timezone. the same time.Time but with explicit UTC
timezone vs explicit UTC-as-Local timezone are not the same when comparing with
==. so comparison should be done with time.Time.Equal, or comparison should be
done after having called .Local() on parsed timestamps (so the explicit UTC
timezone gets converted to the UTC-as-Local timezone). somewhat surprising that
time.Local isn't the same as time.UTC if TZ=/TZ=UTC. there are warnings
throughout the time package about handling of UTC.
2024-04-16 14:18:11 +02:00
Mechiel Lukkien
09fcc49223
add a webapi and webhooks for a simple http/json-based api
for applications to compose/send messages, receive delivery feedback, and
maintain suppression lists.

this is an alternative to applications using a library to compose messages,
submitting those messages using smtp, and monitoring a mailbox with imap for
DSNs, which can be processed into the equivalent of suppression lists. but you
need to know about all these standards/protocols and find libraries. by using
the webapi & webhooks, you just need a http & json library.

unfortunately, there is no standard for these kinds of api, so mox has made up
yet another one...

matching incoming DSNs about deliveries to original outgoing messages requires
keeping history of "retired" messages (delivered from the queue, either
successfully or failed). this can be enabled per account. history is also
useful for debugging deliveries. we now also keep history of each delivery
attempt, accessible while still in the queue, and kept when a message is
retired. the queue webadmin pages now also have pagination, to show potentially
large history.

a queue of webhook calls is now managed too. failures are retried similar to
message deliveries. webhooks can also be saved to the retired list after
completing. also configurable per account.

messages can be sent with a "unique smtp mail from" address. this can only be
used if the domain is configured with a localpart catchall separator such as
"+". when enabled, a queued message gets assigned a random "fromid", which is
added after the separator when sending. when DSNs are returned, they can be
related to previously sent messages based on this fromid. in the future, we can
implement matching on the "envid" used in the smtp dsn extension, or on the
"message-id" of the message. using a fromid can be triggered by authenticating
with a login email address that is configured as enabling fromid.

suppression lists are automatically managed per account. if a delivery attempt
results in certain smtp errors, the destination address is added to the
suppression list. future messages queued for that recipient will immediately
fail without a delivery attempt. suppression lists protect your mail server
reputation.

submitted messages can carry "extra" data through the queue and webhooks for
outgoing deliveries. through webapi as a json object, through smtp submission
as message headers of the form "x-mox-extra-<key>: value".

to make it easy to test webapi/webhooks locally, the "localserve" mode actually
puts messages in the queue. when it's time to deliver, it still won't do a full
delivery attempt, but just delivers to the sender account. unless the recipient
address has a special form, simulating a failure to deliver.

admins now have more control over the queue. "hold rules" can be added to mark
newly queued messages as "on hold", pausing delivery. rules can be about
certain sender or recipient domains/addresses, or apply to all messages pausing
the entire queue. also useful for (local) testing.

new config options have been introduced. they are editable through the admin
and/or account web interfaces.

the webapi http endpoints are enabled for newly generated configs with the
quickstart, and in localserve. existing configurations must explicitly enable
the webapi in mox.conf.

gopherwatch.org was created to dogfood this code. it initially used just the
compose/smtpclient/imapclient mox packages to send messages and process
delivery feedback. it will get a config option to use the mox webapi/webhooks
instead. the gopherwatch code to use webapi/webhook is smaller and simpler, and
developing that shaped development of the mox webapi/webhooks.

for issue #31 by cuu508
2024-04-15 21:49:02 +02:00
Mechiel Lukkien
8bec5ef7d4
also trigger use of smtputf8 for utf8 localpart in Reply-To header 2024-04-15 20:47:53 +02:00
Mechiel Lukkien
d014303617
use wlock when delivering message about new mox version 2024-04-15 20:40:16 +02:00
Mechiel Lukkien
b7ed035730
add godoc to metrics/ 2024-04-15 20:33:44 +02:00
Mechiel Lukkien
e1dbc07dba
fix harmless race where the same value is written to a tls config concurrently 2024-04-15 20:07:39 +02:00
Mechiel Lukkien
11eaa8cd1a
make imapserver faster like before again
in the precis password change before the previous release, the password used in
fuzzing wasn't correct, triggering sleeps due to botched protocols often, which
made the tests run much longer.
2024-04-14 17:41:36 +02:00
Mechiel Lukkien
12e6975aa7
return smtp response/error correctly in more cases 2024-04-14 17:28:00 +02:00
Mechiel Lukkien
4012b72d96
use type config.Account in sherpa api for better typing, and update to latest sherpa lib
typescript now knows the full types, not just "any" for account config.
inline structs previously in config.Account are given their own type definition
so sherpa can generate types.

also update to latest sherpa lib that knows about time.Duration, to be used soon.
2024-04-14 17:18:20 +02:00
Mechiel Lukkien
b7d6540d51
style nit: only take address of structs when passed on 2024-04-14 12:46:24 +02:00
Mechiel Lukkien
2a949f9f79
fix typo in smtp error code 2024-04-14 12:42:47 +02:00
Mechiel Lukkien
e585a4d180
don't fail to generate apidiff when packages are introduced 2024-04-14 12:38:58 +02:00
Mechiel Lukkien
4b459af4a8
add install as target, calling "go install"
convenient for local testing, i'm often running "mox localserve", often helpful
if it's the latest.
2024-04-14 12:37:52 +02:00
Mechiel Lukkien
1ea851bb53
Merge commit 'feb8e6c37947b21baaa7dcf724ade0f2435a8280'
github PR #152, also for issue #149
2024-04-13 13:36:11 +02:00
Mechiel Lukkien
34572d14d0
regenerate apidiff/next.txt after change to smtpclient
by calling "make genapidiff"
2024-04-13 13:31:32 +02:00
Mechiel Lukkien
73381d26ed
Merge commit 'be570d1c7d3de0ddacb011b6411a302d7f7e9f9e'
from github PR #153
2024-04-13 13:31:02 +02:00
Laurent Meunier
feb8e6c379 queue: retry with another IP when first attempt fails for dualstack remote servers
mox was already giving another try for received errors after the
`HELO`/`EHLO` command. Now mox do the same for received errors when
trying to deliver the message to the remote SMTP server.

This should help to deliver messages to SMTP server that rejects
incoming messages because of bad ipv4 or ipv6 configuration (for example
for servers checking reverse DNS records). mox will now try to deliver
messages on both ip family instead before considering the error as
permanent.

fix #149
2024-04-12 17:44:33 +02:00
Laurent Meunier
be570d1c7d add TransportDirect transport
The `TransportDirect` transport allows to tweak outgoing SMTP
connections to remote servers. Currently, it only allows to select
network IP family (ipv4, ipv6 or both).

For example, to disable ipv6 for all outgoing SMTP connections:
- add these lines in mox.conf to create a new transport named
"disableipv6":
```
Transports:
  disableipv6:
    Direct:
      DisableIpv6: true
```
- then add these lines in domains.conf to use this transport:
```
Routes:
  -
    Transport: disableipv6
```

fix #149
2024-04-12 17:27:39 +02:00
Mechiel Lukkien
f4b6e14cb9
quickstart: if initial address has non-ascii localpart, use "postmaster@" for registering with let's encrypt
because let's encrypt won't create an account for contact addresses with non-ascii characters.
we'll get an error message like:

	400 urn:ietf:params:acme:error:invalidContact: Error creating new account :: contact email [\"mailto:...\"] contains non-ASCII characters

found & reported by arnt, thanks!
2024-04-11 23:58:40 +02:00
Mechiel Lukkien
ad8c5616b1
do not use input type=email for email addresses
despite the name, it doesn't actually check for valid email addresses:
it doesn't allow non-ascii localparts, accepts various invalid localparts, and
rejects various valid localparts. no point in using it.
2024-04-11 23:45:47 +02:00
Mechiel Lukkien
606b915447
sync genapidiff 2024-04-11 23:28:52 +02:00
Mechiel Lukkien
00c8dacc56
fix previous commit, go fmt 2024-04-11 23:22:03 +02:00
Mechiel Lukkien
666f84edea
fix login for account names with non-ascii chars
we include the username in session cookie values. but cookie values must be ascii-only, go's net/http's drops bad values. the typical solution is to querystring-encode/decode the cookie values, which we'll now do.

problem found by arnt, thanks for reporting!
2024-04-11 23:11:31 +02:00
Mechiel Lukkien
d74610c345
bugfix: missing account close in queue direct send
found while writing new tests for upcoming functionality.
the test had an embarrassing workaround for the symptoms...
2024-04-08 20:22:52 +02:00
Mechiel Lukkien
89a9a8bc97
when we get a tls connection with an unrecognized sni hostname/ip, cause an alert "unrecognized name" rather than "internal error"
more helpful error for users trying to debug whats going on.

problem pointed out by arnt, thanks!
2024-04-08 14:22:52 +02:00
Mechiel Lukkien
ecf6163409
improve previous about using mtime from imported maildir message files
don't treat just any number from filename as timestamp. require it has 2 dots.
prevents filenames with just a number as being seen as a timestamp, like when
you import files from a mox accounts msgs directory.
2024-04-02 20:04:09 +02:00
Mechiel Lukkien
6d38a1e9a4
when reading maildirs for imports, use the file mtime as fallback for "received" time
more useful than the time.Time zero file in case the maildir filename isn't
properly formed with a timestamp. this is not too uncommon when people
reconstruct maildirs from other sources of message files to then import the
maildir.

based on message from abdul h
2024-04-02 19:43:45 +02:00
Mechiel Lukkien
96e3e5e33e
make staticcheck happy
i don't think it's actually better, but it is helpful to keep the code base
free of staticcheck findings.
2024-03-31 15:30:24 +02:00
Laurent Meunier
9c5d234162
do not require the SMTPUTF8 extension when not needed (#145)
Squashed commit of the following:

commit 11c25d727f
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:37:09 2024 +0200

    Fix style issue

commit c075a8cd8b
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:35:04 2024 +0200

    Also check smtputf8 for submitted messages or when in pedantic mode

commit c02328f881
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:33:20 2024 +0200

    Calls to `newParser` should use `c.smtputf8`

commit a0bbd13afc
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:32:12 2024 +0200

    Improve SMTPUTF8 tests

commit 08735690f3
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sat Mar 30 17:22:33 2024 +0100

    do earlier smtputf8-check

commit 3484651691
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Thu Mar 28 17:47:11 2024 +0100

    do not require the SMTPUTF8 extension when not needed

    fix #145
2024-03-31 15:23:53 +02:00
Mechiel Lukkien
d34dd8aae6
update to latest bstore, with a bugfix for queries with multiple orders that were partially handled by an index
causing returned order to be incorrect.
was triggered by new code i'm working on.
2024-03-30 09:39:18 +01:00
Mechiel Lukkien
54b24931c9
add faq entry about configuring mox to send through a smart host
suggested by arnt & friend, thanks for reporting!
2024-03-27 10:23:37 +01:00
Mechiel Lukkien
6516a27689
update to latest sconf, which now gives more helpful error messages about some invalid config lines, like one with only whitespace
from arnt & friend, thanks for reporting!
2024-03-27 10:08:15 +01:00
Mechiel Lukkien
0262f4621e
in quickstart, check outgoing smtp connectivity by dialing gmail.com mx host
if connection cannot be made, warn about it and point to configuring a
smarthost and the config options.

suggested by arnt & friend
2024-03-27 09:35:16 +01:00
Mechiel Lukkien
d4958732c8
add more of a "getting started with building" to develop.txt
based on #145 by lmeunier
2024-03-26 09:34:03 +01:00