2023-01-30 16:27:06 +03:00
package imapserver
import (
"context"
"crypto/ed25519"
cryptorand "crypto/rand"
"crypto/tls"
"crypto/x509"
"fmt"
"math/big"
"net"
"os"
make mox compile on windows, without "mox serve" but with working "mox localserve"
getting mox to compile required changing code in only a few places where
package "syscall" was used: for accessing file access times and for umask
handling. an open problem is how to start a process as an unprivileged user on
windows. that's why "mox serve" isn't implemented yet. and just finding a way
to implement it now may not be good enough in the near future: we may want to
starting using a more complete privilege separation approach, with a process
handling sensitive tasks (handling private keys, authentication), where we may
want to pass file descriptors between processes. how would that work on
windows?
anyway, getting mox to compile for windows doesn't mean it works properly on
windows. the largest issue: mox would normally open a file, rename or remove
it, and finally close it. this happens during message delivery. that doesn't
work on windows, the rename/remove would fail because the file is still open.
so this commit swaps many "remove" and "close" calls. renames are a longer
story: message delivery had two ways to deliver: with "consuming" the
(temporary) message file (which would rename it to its final destination), and
without consuming (by hardlinking the file, falling back to copying). the last
delivery to a recipient of a message (and the only one in the common case of a
single recipient) would consume the message, and the earlier recipients would
not. during delivery, the already open message file was used, to parse the
message. we still want to use that open message file, and the caller now stays
responsible for closing it, but we no longer try to rename (consume) the file.
we always hardlink (or copy) during delivery (this works on windows), and the
caller is responsible for closing and removing (in that order) the original
temporary file. this does cost one syscall more. but it makes the delivery code
(responsibilities) a bit simpler.
there is one more obvious issue: the file system path separator. mox already
used the "filepath" package to join paths in many places, but not everywhere.
and it still used strings with slashes for local file access. with this commit,
the code now uses filepath.FromSlash for path strings with slashes, uses
"filepath" in a few more places where it previously didn't. also switches from
"filepath" to regular "path" package when handling mailbox names in a few
places, because those always use forward slashes, regardless of local file
system conventions. windows can handle forward slashes when opening files, so
test code that passes path strings with forward slashes straight to go stdlib
file i/o functions are left unchanged to reduce code churn. the regular
non-test code, or test code that uses path strings in places other than
standard i/o functions, does have the paths converted for consistent paths
(otherwise we would end up with paths with mixed forward/backward slashes in
log messages).
windows cannot dup a listening socket. for "mox localserve", it isn't
important, and we can work around the issue. the current approach for "mox
serve" (forking a process and passing file descriptors of listening sockets on
"privileged" ports) won't work on windows. perhaps it isn't needed on windows,
and any user can listen on "privileged" ports? that would be welcome.
on windows, os.Open cannot open a directory, so we cannot call Sync on it after
message delivery. a cursory internet search indicates that directories cannot
be synced on windows. the story is probably much more nuanced than that, with
long deep technical details/discussions/disagreement/confusion, like on unix.
for "mox localserve" we can get away with making syncdir a no-op.
2023-10-14 11:54:07 +03:00
"path/filepath"
2023-01-30 16:27:06 +03:00
"reflect"
"strings"
"testing"
"time"
"github.com/mjl-/mox/imapclient"
2023-12-05 15:35:58 +03:00
"github.com/mjl-/mox/mlog"
2023-01-30 16:27:06 +03:00
"github.com/mjl-/mox/mox-"
"github.com/mjl-/mox/moxvar"
"github.com/mjl-/mox/store"
)
2023-07-24 22:21:05 +03:00
var ctxbg = context . Background ( )
2023-12-05 15:35:58 +03:00
var pkglog = mlog . New ( "imapserver" , nil )
2023-07-24 22:21:05 +03:00
2023-01-30 16:27:06 +03:00
func init ( ) {
sanityChecks = true
2023-02-08 23:45:32 +03:00
// Don't slow down tests.
badClientDelay = 0
2023-03-10 12:23:43 +03:00
authFailDelay = 0
2023-01-30 16:27:06 +03:00
}
func tocrlf ( s string ) string {
return strings . ReplaceAll ( s , "\n" , "\r\n" )
}
// From ../rfc/3501:2589
var exampleMsg = tocrlf ( ` Date : Mon , 7 Feb 1994 21 : 52 : 25 - 0 800 ( PST )
From : Fred Foobar < foobar @ Blurdybloop . example >
Subject : afternoon meeting
To : mooch @ owatagu . siam . edu . example
Message - Id : < B27397 - 0100000 @ Blurdybloop . example >
MIME - Version : 1.0
Content - Type : TEXT / PLAIN ; CHARSET = US - ASCII
Hello Joe , do you think we can meet at 3 : 30 tomorrow ?
` )
/ *
From . . / rfc / 2049 : 801
Message structure :
Message - multipart / mixed
Part 1 - no content - type
Part 2 - text / plain
Part 3 - multipart / parallel
Part 3.1 - audio / basic ( base64 )
Part 3.2 - image / jpeg ( base64 , empty )
Part 4 - text / enriched
Part 5 - message / rfc822
Part 5.1 - text / plain ( quoted - printable )
* /
var nestedMessage = tocrlf ( ` MIME - Version : 1.0
From : Nathaniel Borenstein < nsb @ nsb . fv . com >
To : Ned Freed < ned @ innosoft . com >
Date : Fri , 07 Oct 1994 16 : 15 : 05 - 0700 ( PDT )
Subject : A multipart example
Content - Type : multipart / mixed ;
boundary = unique - boundary - 1
This is the preamble area of a multipart message .
Mail readers that understand multipart format
should ignore this preamble .
If you are reading this text , you might want to
consider changing to a mail reader that understands
how to properly display multipart messages .
-- unique - boundary - 1
... Some text appears here ...
[ Note that the blank between the boundary and the start
of the text in this part means no header fields were
given and this is text in the US - ASCII character set .
It could have been done with explicit typing as in the
next part . ]
-- unique - boundary - 1
Content - type : text / plain ; charset = US - ASCII
This could have been part of the previous part , but
illustrates explicit versus implicit typing of body
parts .
-- unique - boundary - 1
Content - Type : multipart / parallel ; boundary = unique - boundary - 2
-- unique - boundary - 2
Content - Type : audio / basic
Content - Transfer - Encoding : base64
aGVsbG8NCndvcmxkDQo =
-- unique - boundary - 2
Content - Type : image / jpeg
Content - Transfer - Encoding : base64
-- unique - boundary - 2 --
-- unique - boundary - 1
Content - type : text / enriched
This is < bold > < italic > enriched . < / italic > < / bold >
< smaller > as defined in RFC 1896 < / smaller >
Isn ' t it
< bigger > < bigger > cool ? < / bigger > < / bigger >
-- unique - boundary - 1
Content - Type : message / rfc822
From : info @ mox . example
To : mox < info @ mox . example >
Subject : ( subject in US - ASCII )
Content - Type : Text / plain ; charset = ISO - 8859 - 1
Content - Transfer - Encoding : Quoted - printable
... Additional text in ISO - 8859 - 1 goes here ...
-- unique - boundary - 1 --
` )
func tcheck ( t * testing . T , err error , msg string ) {
t . Helper ( )
if err != nil {
t . Fatalf ( "%s: %s" , msg , err )
}
}
func mockUIDValidity ( ) func ( ) {
orig := store . InitialUIDValidity
store . InitialUIDValidity = func ( ) uint32 {
return 1
}
return func ( ) {
store . InitialUIDValidity = orig
}
}
type testconn struct {
t * testing . T
conn net . Conn
client * imapclient . Conn
done chan struct { }
serverConn net . Conn
2023-07-24 22:21:05 +03:00
account * store . Account
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
switchStop func ( )
2023-01-30 16:27:06 +03:00
// Result of last command.
lastUntagged [ ] imapclient . Untagged
lastResult imapclient . Result
lastErr error
}
func ( tc * testconn ) check ( err error , msg string ) {
tc . t . Helper ( )
if err != nil {
tc . t . Fatalf ( "%s: %s" , msg , err )
}
}
func ( tc * testconn ) last ( l [ ] imapclient . Untagged , r imapclient . Result , err error ) {
tc . lastUntagged = l
tc . lastResult = r
tc . lastErr = err
}
func ( tc * testconn ) xcode ( s string ) {
tc . t . Helper ( )
if tc . lastResult . Code != s {
tc . t . Fatalf ( "got last code %q, expected %q" , tc . lastResult . Code , s )
}
}
func ( tc * testconn ) xcodeArg ( v any ) {
tc . t . Helper ( )
if ! reflect . DeepEqual ( tc . lastResult . CodeArg , v ) {
tc . t . Fatalf ( "got last code argument %v, expected %v" , tc . lastResult . CodeArg , v )
}
}
2023-07-24 22:21:05 +03:00
func ( tc * testconn ) xuntagged ( exps ... imapclient . Untagged ) {
2023-06-28 20:41:58 +03:00
tc . t . Helper ( )
2023-07-24 22:21:05 +03:00
tc . xuntaggedOpt ( true , exps ... )
2023-06-24 01:24:43 +03:00
}
2023-07-24 22:21:05 +03:00
func ( tc * testconn ) xuntaggedOpt ( all bool , exps ... imapclient . Untagged ) {
2023-01-30 16:27:06 +03:00
tc . t . Helper ( )
last := append ( [ ] imapclient . Untagged { } , tc . lastUntagged ... )
2023-07-24 22:21:05 +03:00
var mismatch any
2023-01-30 16:27:06 +03:00
next :
for ei , exp := range exps {
for i , l := range last {
if reflect . TypeOf ( l ) != reflect . TypeOf ( exp ) {
continue
}
if ! reflect . DeepEqual ( l , exp ) {
2023-07-24 22:21:05 +03:00
mismatch = l
continue
2023-01-30 16:27:06 +03:00
}
copy ( last [ i : ] , last [ i + 1 : ] )
last = last [ : len ( last ) - 1 ]
continue next
}
2023-07-24 22:21:05 +03:00
if mismatch != nil {
tc . t . Fatalf ( "untagged data mismatch, got:\n\t%T %#v\nexpected:\n\t%T %#v" , mismatch , mismatch , exp , exp )
}
2023-01-30 16:27:06 +03:00
var next string
if len ( tc . lastUntagged ) > 0 {
next = fmt . Sprintf ( ", next %#v" , tc . lastUntagged [ 0 ] )
}
tc . t . Fatalf ( "did not find untagged response %#v %T (%d) in %v%s" , exp , exp , ei , tc . lastUntagged , next )
}
2023-06-24 01:24:43 +03:00
if len ( last ) > 0 && all {
2023-01-30 16:27:06 +03:00
tc . t . Fatalf ( "leftover untagged responses %v" , last )
}
}
func tuntagged ( t * testing . T , got imapclient . Untagged , dst any ) {
t . Helper ( )
gotv := reflect . ValueOf ( got )
dstv := reflect . ValueOf ( dst )
if gotv . Type ( ) != dstv . Type ( ) . Elem ( ) {
t . Fatalf ( "got %v, expected %v" , gotv . Type ( ) , dstv . Type ( ) . Elem ( ) )
}
dstv . Elem ( ) . Set ( gotv )
}
func ( tc * testconn ) xnountagged ( ) {
tc . t . Helper ( )
if len ( tc . lastUntagged ) != 0 {
tc . t . Fatalf ( "got %v untagged, expected 0" , tc . lastUntagged )
}
}
func ( tc * testconn ) transactf ( status , format string , args ... any ) {
tc . t . Helper ( )
tc . cmdf ( "" , format , args ... )
tc . response ( status )
}
func ( tc * testconn ) response ( status string ) {
tc . t . Helper ( )
tc . lastUntagged , tc . lastResult , tc . lastErr = tc . client . Response ( )
tcheck ( tc . t , tc . lastErr , "read imap response" )
if strings . ToUpper ( status ) != string ( tc . lastResult . Status ) {
tc . t . Fatalf ( "got status %q, expected %q" , tc . lastResult . Status , status )
}
}
func ( tc * testconn ) cmdf ( tag , format string , args ... any ) {
tc . t . Helper ( )
err := tc . client . Commandf ( tag , format , args ... )
tcheck ( tc . t , err , "writing imap command" )
}
func ( tc * testconn ) readstatus ( status string ) {
tc . t . Helper ( )
tc . response ( status )
}
func ( tc * testconn ) readprefixline ( pre string ) {
tc . t . Helper ( )
line , err := tc . client . Readline ( )
tcheck ( tc . t , err , "read line" )
if ! strings . HasPrefix ( line , pre ) {
tc . t . Fatalf ( "expected prefix %q, got %q" , pre , line )
}
}
func ( tc * testconn ) writelinef ( format string , args ... any ) {
tc . t . Helper ( )
err := tc . client . Writelinef ( format , args ... )
tcheck ( tc . t , err , "write line" )
}
// wait at most 1 second for server to quit.
func ( tc * testconn ) waitDone ( ) {
tc . t . Helper ( )
t := time . NewTimer ( time . Second )
select {
case <- tc . done :
t . Stop ( )
case <- t . C :
tc . t . Fatalf ( "server not done within 1s" )
}
}
func ( tc * testconn ) close ( ) {
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
if tc . account == nil {
// Already closed, we are not strict about closing multiple times.
return
}
2023-07-24 22:21:05 +03:00
err := tc . account . Close ( )
tc . check ( err , "close account" )
2024-04-16 18:33:54 +03:00
// no account.CheckClosed(), the tests open accounts multiple times.
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
tc . account = nil
2023-01-30 16:27:06 +03:00
tc . client . Close ( )
tc . serverConn . Close ( )
tc . waitDone ( )
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
if tc . switchStop != nil {
tc . switchStop ( )
}
2023-01-30 16:27:06 +03:00
}
2023-07-24 22:21:05 +03:00
func xparseNumSet ( s string ) imapclient . NumSet {
ns , err := imapclient . ParseNumSet ( s )
if err != nil {
panic ( fmt . Sprintf ( "parsing numset %s: %s" , s , err ) )
}
return ns
}
2023-01-30 16:27:06 +03:00
var connCounter int64
func start ( t * testing . T ) * testconn {
2023-12-20 22:54:12 +03:00
return startArgs ( t , true , false , true , true , "mjl" )
2023-01-30 16:27:06 +03:00
}
func startNoSwitchboard ( t * testing . T ) * testconn {
2023-12-20 22:54:12 +03:00
return startArgs ( t , false , false , true , false , "mjl" )
2023-01-30 16:27:06 +03:00
}
2024-03-09 01:29:15 +03:00
const password0 = "te\u0301st \u00a0\u2002\u200a" // NFD and various unicode spaces.
const password1 = "tést " // PRECIS normalized, with NFC.
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
func startArgs ( t * testing . T , first , immediateTLS bool , allowLoginWithoutTLS , setPassword bool , accname string ) * testconn {
return startArgsMore ( t , first , immediateTLS , nil , nil , allowLoginWithoutTLS , false , setPassword , accname , nil )
}
// todo: the parameters and usage are too much now. change to scheme similar to smtpserver, with params in a struct, and a separate method for init and making a connection.
func startArgsMore ( t * testing . T , first , immediateTLS bool , serverConfig , clientConfig * tls . Config , allowLoginWithoutTLS , noCloseSwitchboard , setPassword bool , accname string , afterInit func ( ) error ) * testconn {
2023-02-08 00:56:03 +03:00
limitersInit ( ) // Reset rate limiters.
2023-07-24 22:21:05 +03:00
mox . Context = ctxbg
make mox compile on windows, without "mox serve" but with working "mox localserve"
getting mox to compile required changing code in only a few places where
package "syscall" was used: for accessing file access times and for umask
handling. an open problem is how to start a process as an unprivileged user on
windows. that's why "mox serve" isn't implemented yet. and just finding a way
to implement it now may not be good enough in the near future: we may want to
starting using a more complete privilege separation approach, with a process
handling sensitive tasks (handling private keys, authentication), where we may
want to pass file descriptors between processes. how would that work on
windows?
anyway, getting mox to compile for windows doesn't mean it works properly on
windows. the largest issue: mox would normally open a file, rename or remove
it, and finally close it. this happens during message delivery. that doesn't
work on windows, the rename/remove would fail because the file is still open.
so this commit swaps many "remove" and "close" calls. renames are a longer
story: message delivery had two ways to deliver: with "consuming" the
(temporary) message file (which would rename it to its final destination), and
without consuming (by hardlinking the file, falling back to copying). the last
delivery to a recipient of a message (and the only one in the common case of a
single recipient) would consume the message, and the earlier recipients would
not. during delivery, the already open message file was used, to parse the
message. we still want to use that open message file, and the caller now stays
responsible for closing it, but we no longer try to rename (consume) the file.
we always hardlink (or copy) during delivery (this works on windows), and the
caller is responsible for closing and removing (in that order) the original
temporary file. this does cost one syscall more. but it makes the delivery code
(responsibilities) a bit simpler.
there is one more obvious issue: the file system path separator. mox already
used the "filepath" package to join paths in many places, but not everywhere.
and it still used strings with slashes for local file access. with this commit,
the code now uses filepath.FromSlash for path strings with slashes, uses
"filepath" in a few more places where it previously didn't. also switches from
"filepath" to regular "path" package when handling mailbox names in a few
places, because those always use forward slashes, regardless of local file
system conventions. windows can handle forward slashes when opening files, so
test code that passes path strings with forward slashes straight to go stdlib
file i/o functions are left unchanged to reduce code churn. the regular
non-test code, or test code that uses path strings in places other than
standard i/o functions, does have the paths converted for consistent paths
(otherwise we would end up with paths with mixed forward/backward slashes in
log messages).
windows cannot dup a listening socket. for "mox localserve", it isn't
important, and we can work around the issue. the current approach for "mox
serve" (forking a process and passing file descriptors of listening sockets on
"privileged" ports) won't work on windows. perhaps it isn't needed on windows,
and any user can listen on "privileged" ports? that would be welcome.
on windows, os.Open cannot open a directory, so we cannot call Sync on it after
message delivery. a cursory internet search indicates that directories cannot
be synced on windows. the story is probably much more nuanced than that, with
long deep technical details/discussions/disagreement/confusion, like on unix.
for "mox localserve" we can get away with making syncdir a no-op.
2023-10-14 11:54:07 +03:00
mox . ConfigStaticPath = filepath . FromSlash ( "../testdata/imap/mox.conf" )
2023-06-16 14:27:27 +03:00
mox . MustLoadConfig ( true , false )
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
if first {
store . Close ( ) // May not be open, we ignore error.
os . RemoveAll ( "../testdata/imap/data" )
err := store . Init ( ctxbg )
tcheck ( t , err , "store init" )
}
2023-12-20 22:54:12 +03:00
acc , err := store . OpenAccount ( pkglog , accname )
2023-01-30 16:27:06 +03:00
tcheck ( t , err , "open account" )
2023-12-20 22:54:12 +03:00
if setPassword {
2024-03-09 01:29:15 +03:00
err = acc . SetPassword ( pkglog , password0 )
2023-01-30 16:27:06 +03:00
tcheck ( t , err , "set password" )
}
2023-08-08 00:14:31 +03:00
switchStop := func ( ) { }
2023-01-30 16:27:06 +03:00
if first {
2023-08-08 00:14:31 +03:00
switchStop = store . Switchboard ( )
2023-01-30 16:27:06 +03:00
}
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
if afterInit != nil {
err := afterInit ( )
tcheck ( t , err , "after init" )
}
2023-01-30 16:27:06 +03:00
serverConn , clientConn := net . Pipe ( )
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
if serverConfig == nil {
serverConfig = & tls . Config {
Certificates : [ ] tls . Certificate { fakeCert ( t , false ) } ,
}
2023-01-30 16:27:06 +03:00
}
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
if immediateTLS {
if clientConfig == nil {
clientConfig = & tls . Config { InsecureSkipVerify : true }
}
clientConn = tls . Client ( clientConn , clientConfig )
2023-01-30 16:27:06 +03:00
}
done := make ( chan struct { } )
connCounter ++
cid := connCounter
go func ( ) {
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
serve ( "test" , cid , serverConfig , serverConn , immediateTLS , allowLoginWithoutTLS )
if ! noCloseSwitchboard {
switchStop ( )
}
2023-01-30 16:27:06 +03:00
close ( done )
} ( )
client , err := imapclient . New ( clientConn , true )
tcheck ( t , err , "new client" )
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
tc := & testconn { t : t , conn : clientConn , client : client , done : done , serverConn : serverConn , account : acc }
if first && noCloseSwitchboard {
tc . switchStop = switchStop
}
return tc
2023-01-30 16:27:06 +03:00
}
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
func fakeCert ( t * testing . T , randomkey bool ) tls . Certificate {
seed := make ( [ ] byte , ed25519 . SeedSize )
if randomkey {
cryptorand . Read ( seed )
}
privKey := ed25519 . NewKeyFromSeed ( seed ) // Fake key, don't use this for real!
2023-01-30 16:27:06 +03:00
template := & x509 . Certificate {
SerialNumber : big . NewInt ( 1 ) , // Required field...
implement tls client certificate authentication
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
2024-12-06 00:41:49 +03:00
// Valid period is needed to get session resumption enabled.
NotBefore : time . Now ( ) . Add ( - time . Minute ) ,
NotAfter : time . Now ( ) . Add ( time . Hour ) ,
2023-01-30 16:27:06 +03:00
}
localCertBuf , err := x509 . CreateCertificate ( cryptorand . Reader , template , template , privKey . Public ( ) , privKey )
if err != nil {
t . Fatalf ( "making certificate: %s" , err )
}
cert , err := x509 . ParseCertificate ( localCertBuf )
if err != nil {
t . Fatalf ( "parsing generated certificate: %s" , err )
}
c := tls . Certificate {
Certificate : [ ] [ ] byte { localCertBuf } ,
PrivateKey : privKey ,
Leaf : cert ,
}
return c
}
func TestLogin ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
tc . transactf ( "bad" , "login too many args" )
tc . transactf ( "bad" , "login" ) // no args
tc . transactf ( "no" , "login mjl@mox.example badpass" )
2024-03-09 01:29:15 +03:00
tc . transactf ( "no" , ` login mjl "%s" ` , password0 ) // must use email, not account
2023-01-30 16:27:06 +03:00
tc . transactf ( "no" , "login mjl@mox.example test" )
tc . transactf ( "no" , "login mjl@mox.example testtesttest" )
tc . transactf ( "no" , ` login "mjl@mox.example" "testtesttest" ` )
tc . transactf ( "no" , "login \"m\xf8x@mox.example\" \"testtesttest\"" )
2024-03-09 01:29:15 +03:00
tc . transactf ( "ok" , ` login mjl@mox.example "%s" ` , password0 )
2023-01-30 16:27:06 +03:00
tc . close ( )
tc = start ( t )
2024-03-09 01:29:15 +03:00
tc . transactf ( "ok" , ` login "mjl@mox.example" "%s" ` , password0 )
2023-06-03 16:29:18 +03:00
tc . close ( )
tc = start ( t )
2024-03-09 01:29:15 +03:00
tc . transactf ( "ok" , ` login "\"\"@mox.example" "%s" ` , password0 )
2023-06-03 16:29:18 +03:00
defer tc . close ( )
2023-01-30 16:27:06 +03:00
tc . transactf ( "bad" , "logout badarg" )
tc . transactf ( "ok" , "logout" )
}
// Test that commands don't work in the states they are not supposed to.
func TestState ( t * testing . T ) {
tc := start ( t )
notAuthenticated := [ ] string { "starttls" , "authenticate" , "login" }
authenticatedOrSelected := [ ] string { "enable" , "select" , "examine" , "create" , "delete" , "rename" , "subscribe" , "unsubscribe" , "list" , "namespace" , "status" , "append" , "idle" , "lsub" }
selected := [ ] string { "close" , "unselect" , "expunge" , "search" , "fetch" , "store" , "copy" , "move" , "uid expunge" }
// Always allowed.
tc . transactf ( "ok" , "capability" )
tc . transactf ( "ok" , "noop" )
tc . transactf ( "ok" , "logout" )
tc . close ( )
tc = start ( t )
defer tc . close ( )
// Not authenticated, lots of commands not allowed.
for _ , cmd := range append ( append ( [ ] string { } , authenticatedOrSelected ... ) , selected ... ) {
tc . transactf ( "no" , "%s" , cmd )
}
// Some commands not allowed when authenticated.
2024-03-09 01:29:15 +03:00
tc . transactf ( "ok" , ` login mjl@mox.example "%s" ` , password0 )
2023-01-30 16:27:06 +03:00
for _ , cmd := range append ( append ( [ ] string { } , notAuthenticated ... ) , selected ... ) {
tc . transactf ( "no" , "%s" , cmd )
}
2023-03-10 12:23:43 +03:00
tc . transactf ( "bad" , "boguscommand" )
}
func TestNonIMAP ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
// imap greeting has already been read, we sidestep the imapclient.
_ , err := fmt . Fprintf ( tc . conn , "bogus\r\n" )
tc . check ( err , "write bogus command" )
tc . readprefixline ( "* BYE " )
if _ , err := tc . conn . Read ( make ( [ ] byte , 1 ) ) ; err == nil {
t . Fatalf ( "connection not closed after initial bad command" )
}
2023-01-30 16:27:06 +03:00
}
func TestLiterals ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc . client . Create ( "tmpbox" )
tc . transactf ( "ok" , "rename {6+}\r\ntmpbox {7+}\r\nntmpbox" )
from := "ntmpbox"
to := "tmpbox"
fmt . Fprint ( tc . client , "xtag rename " )
tc . client . WriteSyncLiteral ( from )
fmt . Fprint ( tc . client , " " )
tc . client . WriteSyncLiteral ( to )
fmt . Fprint ( tc . client , "\r\n" )
tc . client . LastTag = "xtag"
tc . last ( tc . client . Response ( ) )
if tc . lastResult . Status != "OK" {
tc . t . Fatalf ( ` got %q, expected "OK" ` , tc . lastResult . Status )
}
}
// Test longer scenario with login, lists, subscribes, status, selects, etc.
func TestScenario ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . transactf ( "ok" , ` login mjl@mox.example "%s" ` , password0 )
2023-01-30 16:27:06 +03:00
tc . transactf ( "bad" , " missingcommand" )
tc . transactf ( "ok" , "examine inbox" )
tc . transactf ( "ok" , "unselect" )
tc . transactf ( "ok" , "examine inbox" )
tc . transactf ( "ok" , "close" )
tc . transactf ( "ok" , "select inbox" )
tc . transactf ( "ok" , "close" )
tc . transactf ( "ok" , "select inbox" )
tc . transactf ( "ok" , "expunge" )
tc . transactf ( "ok" , "check" )
tc . transactf ( "ok" , "subscribe inbox" )
tc . transactf ( "ok" , "unsubscribe inbox" )
tc . transactf ( "ok" , "subscribe inbox" )
tc . transactf ( "ok" , ` lsub "" "*" ` )
tc . transactf ( "ok" , ` list "" "" ` )
tc . transactf ( "ok" , ` namespace ` )
tc . transactf ( "ok" , "enable utf8=accept" )
tc . transactf ( "ok" , "enable imap4rev2 utf8=accept" )
tc . transactf ( "no" , "create inbox" )
tc . transactf ( "ok" , "create tmpbox" )
tc . transactf ( "ok" , "rename tmpbox ntmpbox" )
tc . transactf ( "ok" , "delete ntmpbox" )
tc . transactf ( "ok" , "status inbox (uidnext messages uidvalidity deleted size unseen recent)" )
tc . transactf ( "ok" , "append inbox (\\seen) {%d+}\r\n%s" , len ( exampleMsg ) , exampleMsg )
tc . transactf ( "no" , "append bogus () {%d}" , len ( exampleMsg ) )
tc . cmdf ( "" , "append inbox () {%d}" , len ( exampleMsg ) )
2023-07-24 20:54:55 +03:00
tc . readprefixline ( "+ " )
2023-01-30 16:27:06 +03:00
_ , err := tc . conn . Write ( [ ] byte ( exampleMsg + "\r\n" ) )
tc . check ( err , "write message" )
tc . response ( "ok" )
tc . transactf ( "ok" , "fetch 1 all" )
tc . transactf ( "ok" , "fetch 1 body" )
tc . transactf ( "ok" , "fetch 1 binary[]" )
tc . transactf ( "ok" , ` store 1 flags (\seen \answered) ` )
tc . transactf ( "ok" , ` store 1 +flags ($junk) ` ) // should train as junk.
tc . transactf ( "ok" , ` store 1 -flags ($junk) ` ) // should retrain as non-junk.
tc . transactf ( "ok" , ` store 1 -flags (\seen) ` ) // should untrain completely.
tc . transactf ( "ok" , ` store 1 -flags (\answered) ` )
tc . transactf ( "ok" , ` store 1 +flags (\answered) ` )
tc . transactf ( "ok" , ` store 1 flags.silent (\seen \answered) ` )
tc . transactf ( "ok" , ` store 1 -flags.silent (\answered) ` )
tc . transactf ( "ok" , ` store 1 +flags.silent (\answered) ` )
2023-06-24 01:24:43 +03:00
tc . transactf ( "bad" , ` store 1 flags (\badflag) ` )
2023-01-30 16:27:06 +03:00
tc . transactf ( "ok" , "noop" )
tc . transactf ( "ok" , "copy 1 Trash" )
tc . transactf ( "ok" , "copy 1 Trash" )
tc . transactf ( "ok" , "move 1 Trash" )
tc . transactf ( "ok" , "close" )
tc . transactf ( "ok" , "select Trash" )
tc . transactf ( "ok" , ` store 1 flags (\deleted) ` )
tc . transactf ( "ok" , "expunge" )
tc . transactf ( "ok" , "noop" )
tc . transactf ( "ok" , ` store 1 flags (\deleted) ` )
tc . transactf ( "ok" , "close" )
tc . transactf ( "ok" , "delete Trash" )
}
func TestMailbox ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
invalid := [ ] string {
"e\u0301" , // é but as e + acute, not unicode-normalized
"/leadingslash" ,
"a//b" ,
"Inbox/" ,
"\x01" ,
" " ,
"\x7f" ,
"\x80" ,
"\u2028" ,
"\u2029" ,
}
for _ , bad := range invalid {
tc . transactf ( "no" , "select {%d+}\r\n%s" , len ( bad ) , bad )
}
}
func TestMailboxDeleted ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
tc2 := startNoSwitchboard ( t )
defer tc2 . close ( )
2024-12-25 18:50:23 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2024-03-09 01:29:15 +03:00
tc2 . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc . client . Create ( "testbox" )
tc2 . client . Select ( "testbox" )
tc . client . Delete ( "testbox" )
// Now try to operate on testbox while it has been removed.
tc2 . transactf ( "no" , "check" )
tc2 . transactf ( "no" , "expunge" )
tc2 . transactf ( "no" , "uid expunge 1" )
tc2 . transactf ( "no" , "search all" )
tc2 . transactf ( "no" , "uid search all" )
tc2 . transactf ( "no" , "fetch 1:* all" )
tc2 . transactf ( "no" , "uid fetch 1 all" )
tc2 . transactf ( "no" , "store 1 flags ()" )
tc2 . transactf ( "no" , "uid store 1 flags ()" )
tc2 . transactf ( "bad" , "copy 1 inbox" ) // msgseq 1 not available.
tc2 . transactf ( "no" , "uid copy 1 inbox" )
tc2 . transactf ( "bad" , "move 1 inbox" ) // msgseq 1 not available.
tc2 . transactf ( "no" , "uid move 1 inbox" )
tc2 . transactf ( "ok" , "unselect" )
tc . client . Create ( "testbox" )
tc2 . client . Select ( "testbox" )
tc . client . Delete ( "testbox" )
tc2 . transactf ( "ok" , "close" )
}
func TestID ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc . transactf ( "ok" , "id nil" )
tc . xuntagged ( imapclient . UntaggedID { "name" : "mox" , "version" : moxvar . Version } )
tc . transactf ( "ok" , ` id ("name" "mox" "version" "1.2.3" "other" "test" "test" nil) ` )
tc . xuntagged ( imapclient . UntaggedID { "name" : "mox" , "version" : moxvar . Version } )
tc . transactf ( "bad" , ` id ("name" "mox" "name" "mox") ` ) // Duplicate field.
}
func TestSequence ( t * testing . T ) {
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc . client . Select ( "inbox" )
tc . transactf ( "bad" , "fetch * all" ) // ../rfc/9051:7018
tc . transactf ( "bad" , "fetch 1 all" ) // ../rfc/9051:7018
tc . transactf ( "ok" , "uid fetch 1 all" ) // non-existing messages are OK for uids.
tc . transactf ( "ok" , "uid fetch * all" ) // * is like uidnext, a non-existing message.
tc . client . Append ( "inbox" , nil , nil , [ ] byte ( exampleMsg ) )
tc . client . Append ( "inbox" , nil , nil , [ ] byte ( exampleMsg ) )
tc . transactf ( "ok" , "fetch 2:1,1 uid" ) // We reorder 2:1 to 1:2, but we don't deduplicate numbers.
tc . xuntagged (
imapclient . UntaggedFetch { Seq : 1 , Attrs : [ ] imapclient . FetchAttr { imapclient . FetchUID ( 1 ) } } ,
imapclient . UntaggedFetch { Seq : 2 , Attrs : [ ] imapclient . FetchAttr { imapclient . FetchUID ( 2 ) } } ,
imapclient . UntaggedFetch { Seq : 1 , Attrs : [ ] imapclient . FetchAttr { imapclient . FetchUID ( 1 ) } } ,
)
tc . transactf ( "ok" , "uid fetch 3:* uid" ) // Because * is the last message, which is 2, the range becomes 3:2, which matches the last message.
tc . xuntagged ( imapclient . UntaggedFetch { Seq : 2 , Attrs : [ ] imapclient . FetchAttr { imapclient . FetchUID ( 2 ) } } )
}
// Test that a message that is expunged by another session can be read as long as a
// reference is held by a session. New sessions do not see the expunged message.
// todo: possibly implement the additional reference counting. so far it hasn't been worth the trouble.
2023-07-24 14:55:36 +03:00
func DisabledTestReference ( t * testing . T ) {
2023-01-30 16:27:06 +03:00
tc := start ( t )
defer tc . close ( )
2024-03-09 01:29:15 +03:00
tc . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc . client . Select ( "inbox" )
tc . client . Append ( "inbox" , nil , nil , [ ] byte ( exampleMsg ) )
tc2 := startNoSwitchboard ( t )
defer tc2 . close ( )
2024-03-09 01:29:15 +03:00
tc2 . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc2 . client . Select ( "inbox" )
tc . client . StoreFlagsSet ( "1" , true , ` \Deleted ` )
tc . client . Expunge ( )
tc3 := startNoSwitchboard ( t )
defer tc3 . close ( )
2024-03-09 01:29:15 +03:00
tc3 . client . Login ( "mjl@mox.example" , password0 )
2023-01-30 16:27:06 +03:00
tc3 . transactf ( "ok" , ` list "" "inbox" return (status (messages)) ` )
2024-03-11 17:22:41 +03:00
tc3 . xuntagged ( imapclient . UntaggedList { Separator : '/' , Mailbox : "Inbox" } , imapclient . UntaggedStatus { Mailbox : "Inbox" , Attrs : map [ imapclient . StatusAttr ] int64 { imapclient . StatusMessages : 0 } } )
2023-01-30 16:27:06 +03:00
tc2 . transactf ( "ok" , "fetch 1 rfc822.size" )
tc . xuntagged ( imapclient . UntaggedFetch { Seq : 1 , Attrs : [ ] imapclient . FetchAttr { imapclient . FetchRFC822Size ( len ( exampleMsg ) ) } } )
}