Commit graph

644 commits

Author SHA1 Message Date
Mechiel Lukkien
34572d14d0
regenerate apidiff/next.txt after change to smtpclient
by calling "make genapidiff"
2024-04-13 13:31:32 +02:00
Mechiel Lukkien
73381d26ed
Merge commit 'be570d1c7d3de0ddacb011b6411a302d7f7e9f9e'
from github PR #153
2024-04-13 13:31:02 +02:00
Laurent Meunier
feb8e6c379 queue: retry with another IP when first attempt fails for dualstack remote servers
mox was already giving another try for received errors after the
`HELO`/`EHLO` command. Now mox do the same for received errors when
trying to deliver the message to the remote SMTP server.

This should help to deliver messages to SMTP server that rejects
incoming messages because of bad ipv4 or ipv6 configuration (for example
for servers checking reverse DNS records). mox will now try to deliver
messages on both ip family instead before considering the error as
permanent.

fix #149
2024-04-12 17:44:33 +02:00
Laurent Meunier
be570d1c7d add TransportDirect transport
The `TransportDirect` transport allows to tweak outgoing SMTP
connections to remote servers. Currently, it only allows to select
network IP family (ipv4, ipv6 or both).

For example, to disable ipv6 for all outgoing SMTP connections:
- add these lines in mox.conf to create a new transport named
"disableipv6":
```
Transports:
  disableipv6:
    Direct:
      DisableIpv6: true
```
- then add these lines in domains.conf to use this transport:
```
Routes:
  -
    Transport: disableipv6
```

fix #149
2024-04-12 17:27:39 +02:00
Mechiel Lukkien
f4b6e14cb9
quickstart: if initial address has non-ascii localpart, use "postmaster@" for registering with let's encrypt
because let's encrypt won't create an account for contact addresses with non-ascii characters.
we'll get an error message like:

	400 urn:ietf:params:acme:error:invalidContact: Error creating new account :: contact email [\"mailto:...\"] contains non-ASCII characters

found & reported by arnt, thanks!
2024-04-11 23:58:40 +02:00
Mechiel Lukkien
ad8c5616b1
do not use input type=email for email addresses
despite the name, it doesn't actually check for valid email addresses:
it doesn't allow non-ascii localparts, accepts various invalid localparts, and
rejects various valid localparts. no point in using it.
2024-04-11 23:45:47 +02:00
Mechiel Lukkien
606b915447
sync genapidiff 2024-04-11 23:28:52 +02:00
Mechiel Lukkien
00c8dacc56
fix previous commit, go fmt 2024-04-11 23:22:03 +02:00
Mechiel Lukkien
666f84edea
fix login for account names with non-ascii chars
we include the username in session cookie values. but cookie values must be ascii-only, go's net/http's drops bad values. the typical solution is to querystring-encode/decode the cookie values, which we'll now do.

problem found by arnt, thanks for reporting!
2024-04-11 23:11:31 +02:00
Mechiel Lukkien
d74610c345
bugfix: missing account close in queue direct send
found while writing new tests for upcoming functionality.
the test had an embarrassing workaround for the symptoms...
2024-04-08 20:22:52 +02:00
Mechiel Lukkien
89a9a8bc97
when we get a tls connection with an unrecognized sni hostname/ip, cause an alert "unrecognized name" rather than "internal error"
more helpful error for users trying to debug whats going on.

problem pointed out by arnt, thanks!
2024-04-08 14:22:52 +02:00
Mechiel Lukkien
ecf6163409
improve previous about using mtime from imported maildir message files
don't treat just any number from filename as timestamp. require it has 2 dots.
prevents filenames with just a number as being seen as a timestamp, like when
you import files from a mox accounts msgs directory.
2024-04-02 20:04:09 +02:00
Mechiel Lukkien
6d38a1e9a4
when reading maildirs for imports, use the file mtime as fallback for "received" time
more useful than the time.Time zero file in case the maildir filename isn't
properly formed with a timestamp. this is not too uncommon when people
reconstruct maildirs from other sources of message files to then import the
maildir.

based on message from abdul h
2024-04-02 19:43:45 +02:00
Mechiel Lukkien
96e3e5e33e
make staticcheck happy
i don't think it's actually better, but it is helpful to keep the code base
free of staticcheck findings.
2024-03-31 15:30:24 +02:00
Laurent Meunier
9c5d234162
do not require the SMTPUTF8 extension when not needed (#145)
Squashed commit of the following:

commit 11c25d727f
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:37:09 2024 +0200

    Fix style issue

commit c075a8cd8b
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:35:04 2024 +0200

    Also check smtputf8 for submitted messages or when in pedantic mode

commit c02328f881
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:33:20 2024 +0200

    Calls to `newParser` should use `c.smtputf8`

commit a0bbd13afc
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sun Mar 31 12:32:12 2024 +0200

    Improve SMTPUTF8 tests

commit 08735690f3
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Sat Mar 30 17:22:33 2024 +0100

    do earlier smtputf8-check

commit 3484651691
Author: Laurent Meunier <laurent@deltalima.net>
Date:   Thu Mar 28 17:47:11 2024 +0100

    do not require the SMTPUTF8 extension when not needed

    fix #145
2024-03-31 15:23:53 +02:00
Mechiel Lukkien
d34dd8aae6
update to latest bstore, with a bugfix for queries with multiple orders that were partially handled by an index
causing returned order to be incorrect.
was triggered by new code i'm working on.
2024-03-30 09:39:18 +01:00
Mechiel Lukkien
54b24931c9
add faq entry about configuring mox to send through a smart host
suggested by arnt & friend, thanks for reporting!
2024-03-27 10:23:37 +01:00
Mechiel Lukkien
6516a27689
update to latest sconf, which now gives more helpful error messages about some invalid config lines, like one with only whitespace
from arnt & friend, thanks for reporting!
2024-03-27 10:08:15 +01:00
Mechiel Lukkien
0262f4621e
in quickstart, check outgoing smtp connectivity by dialing gmail.com mx host
if connection cannot be made, warn about it and point to configuring a
smarthost and the config options.

suggested by arnt & friend
2024-03-27 09:35:16 +01:00
Mechiel Lukkien
d4958732c8
add more of a "getting started with building" to develop.txt
based on #145 by lmeunier
2024-03-26 09:34:03 +01:00
Mechiel Lukkien
40ade995a5
improve queue management
- add option to put messages in the queue "on hold", preventing delivery
  attempts until taken off hold again.
- add "hold rules", to automatically mark some/all submitted messages as "on
  hold", e.g. from a specific account or to a specific domain.
- add operation to "fail" a message, causing a DSN to be delivered to the
  sender. previously we could only drop a message from the queue.
- update admin page & add new cli tools for these operations, with new
  filtering rules for selecting the messages to operate on. in the admin
  interface, add filtering and checkboxes to select a set of messages to operate
  on.
2024-03-18 08:50:42 +01:00
Mechiel Lukkien
79f1054b64
factor common typescript api call code pattern into a function 2024-03-17 08:41:33 +01:00
Mechiel Lukkien
25b2ea164f
on build page, mention that changes can be tested easily with mox localserve 2024-03-17 07:58:02 +01:00
Mechiel Lukkien
79fb72f3cd
don't show default domain on admin account page
it is a remnant from the time domains didn't have to be specific in
"Destination" addresses. we still use it for as default selection for adding a
new address to an account. but there's not much point in showing it so
prominently. that raises more questions than it is helpful.

for issue #142 by tabatinga0xffff
2024-03-17 07:39:00 +01:00
Mechiel Lukkien
cef83341e5
make it harder to forget to set smtputf8 on message.Composer
we should do better: first gather all headers, and only write it when we start
on the body, and then calculate smtputf8 ourselves.
2024-03-16 20:59:19 +01:00
Mechiel Lukkien
8b2c97808d
add account option to skip the first-time sender delay
useful for accounts that automatically process messages and want to process quickly
2024-03-16 20:24:07 +01:00
Mechiel Lukkien
281411c297
add styling for sticky table headers, for scrolling with long tables 2024-03-16 19:27:29 +01:00
Mechiel Lukkien
fdee24f3bd
in web interfaces, put crumbs path in document title, for more useful browser history 2024-03-16 19:13:44 +01:00
Mechiel Lukkien
dfe587fdeb
prevent the help output of the reparse subcommand from appearing as a title in the generated documentation 2024-03-14 20:31:31 +01:00
Mechiel Lukkien
2c9cb5b847
add parser of Authentication-Results, and fix bugs it found in our generated headers
we weren't always quoting the values, like dkim's header.b=abc/def. the "/"
requires that the value be quoted.
2024-03-13 17:35:53 +01:00
Mechiel Lukkien
b91480b5af
add /b/ to website that explains how to compile mox, or gives a link to gobuild
the location.hash is used as the version to link to. this can be a tag
(release, e.g. v0.0.1), branch (e.g. main), or commit hash.
2024-03-12 09:41:09 +01:00
Mechiel Lukkien
411cb8fc78
for apidiff, generate apidiff/next.txt and rotate it on release
instead of already giving it a version name before the release. the released
version could be different.
2024-03-11 15:27:25 +01:00
Mechiel Lukkien
bcf737cbec
fix the Status command on imapclient.Conn
it needs at least 1 attribute.
also make types for those attributes, so its harder to get them wrong.
nothing was using this function.
2024-03-11 15:22:41 +01:00
Mechiel Lukkien
4dea2de343
implement imap quota extension (rfc 9208)
we only have a "storage" limit. for total disk usage. we don't have a limit on
messages (count) or mailboxes (count). also not on total annotation size, but
we don't have support annotations at all at the moment.

we don't implement setquota. with rfc 9208 that's allowed. with the previous
quota rfc 2087 it wasn't.

the status command can now return "DELETED-STORAGE". which should be the disk
space that can be reclaimed by removing messages with the \Deleted flags.
however, it's not very likely clients set the \Deleted flag without expunging
the message immediately. we don't want to go through all messages to calculate
the sum of message sizes with the deleted flag. we also don't currently track
that in MailboxCount. so we just respond with "0". not compliant, but let's
wait until someone complains.

when returning quota information, it is not possible to give the current usage
when no limit is configured. clients implementing rfc 9208 should probably
conclude from the presence of QUOTA=RES-* capabilities (only in rfc 9208, not
in 2087) and the absence of those limits in quota responses (or the absence of
an untagged quota response at all) that a resource type doesn't have a limit.
thunderbird will claim there is no quota information when no limit was
configured, so we can probably conclude that it implements rfc 2087, but not
rfc 9208.

we now also show the usage & limit on the account page.

for issue #115 by pmarini
2024-03-11 14:24:32 +01:00
Mechiel Lukkien
6c92949f13
in code/rfc cross-referenced side-by-side view, link to datatracker for rfc's 2024-03-11 09:14:26 +01:00
Mechiel Lukkien
4699504c9f
show goversion and goos/goarch on admin page 2024-03-11 08:58:40 +01:00
Mechiel Lukkien
b115c7b10d
detect whitespace issues in rfc/index.txt earlier
by checking with each fetch and update.
2024-03-11 08:46:40 +01:00
Mechiel Lukkien
5f1157060e
make video work on macos safari
by mentioning mp4 first.  it seems safari doesn't understand this webm
(resolution too high?). still doesn't seem to work on iphone/ipad safari.
2024-03-10 08:47:30 +01:00
Mechiel Lukkien
6984a2ae07
fix latest release on website, tweaks to release process 2024-03-09 20:45:23 +01:00
Mechiel Lukkien
f3501b4e06
fix spacing in rfc/index.txt
genwebsite fails on it.
will make tools that run more often on that file check more strictly too.
2024-03-09 19:55:37 +01:00
Mechiel Lukkien
c6eea5e1cf
add v0.0.10 to the website 2024-03-09 19:49:16 +01:00
Mechiel Lukkien
a601814c3d
fix build after previous commit 2024-03-09 15:52:28 +01:00
Mechiel Lukkien
0c800f3d7e
update to latest sherpats fixing typo in error message, handle absent dmarc "policy override" reason 2024-03-09 15:43:49 +01:00
Mechiel Lukkien
a96493946b
sync latest adns 2024-03-09 15:32:37 +01:00
Mechiel Lukkien
71981ebf43
ensure "make build" on macos generates the same documentation output
it has been i while since i used the old macos machine...
2024-03-09 15:06:42 +01:00
Mechiel Lukkien
a5163493e7
add release process note about updating website 2024-03-09 12:04:15 +01:00
Mechiel Lukkien
7969cf002a
allow zero configured addresses for an account
preventing writing out a domains.conf that is invalid and can't be parsed
again. this happens when the last address was removed from an account. just a
click in the admin web interface.

accounts without email address cannot log in.

for issue #133 by ally9335
2024-03-09 11:51:02 +01:00
Mechiel Lukkien
92e0d2a682
webadmin: be more helpful when adding domains/accounts/addresses
by explaining (in the titles/hovers) what the concepts and requirements are, by
using selects/dropdowns or datalist suggestions where we have a known list, by
automatically suggesting a good account name, and putting the input fields in a
more sensible order.

based on issue #132 by ally9335
2024-03-09 11:11:52 +01:00
Mechiel Lukkien
63cef8e3a5
webmail: fix for ignoring error about sending to invalid address
before, an error about an invalid address was not used, causing a delivery
attempt to an empty address (empty localpart/domain). delivery to that address
would fail, but we should've prevented that message from being queued at all.

additionally, an error in adding the message to the queue was ignored too.
2024-03-09 09:51:24 +01:00
Mechiel Lukkien
c57aeac7f0
prevent unicode-confusion in password by applying PRECIS, and username/email address by applying unicode NFC normalization
an é (e with accent) can also be written as e+\u0301. the first form is NFC,
the second NFD. when logging in, we transform usernames (email addresses) to
NFC. so both forms will be accepted. if a client is using NFD, they can log
in too.

for passwords, we apply the PRECIS "opaquestring", which (despite the name)
transforms the value too: unicode spaces are replaced with ascii spaces. the
string is also normalized to NFC. PRECIS may reject confusing passwords when
you set a password.
2024-03-09 09:20:29 +01:00