add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "Webmail" ,
"Docs" : "" ,
"Functions" : [
{
"Name" : "Token" ,
"Docs" : "Token returns a token to use for an SSE connection. A token can only be used for\na single SSE connection. Tokens are stored in memory for a maximum of 1 minute,\nwith at most 10 unused tokens (the most recently created) per account." ,
"Params" : [ ] ,
"Returns" : [
{
"Name" : "r0" ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "Request" ,
"Docs" : "Requests sends a new request for an open SSE connection. Any currently active\nrequest for the connection will be canceled, but this is done asynchrously, so\nthe SSE connection may still send results for the previous request. Callers\nshould take care to ignore such results. If req.Cancel is set, no new request is\nstarted." ,
"Params" : [
{
"Name" : "req" ,
"Typewords" : [
"Request"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "ParsedMessage" ,
"Docs" : "ParsedMessage returns enough to render the textual body of a message. It is\nassumed the client already has other fields through MessageItem." ,
"Params" : [
{
"Name" : "msgID" ,
"Typewords" : [
"int64"
]
}
] ,
"Returns" : [
{
"Name" : "pm" ,
"Typewords" : [
"ParsedMessage"
]
}
]
} ,
{
"Name" : "MessageSubmit" ,
"Docs" : "MessageSubmit sends a message by submitting it the outgoing email queue. The\nmessage is sent to all addresses listed in the To, Cc and Bcc addresses, without\nBcc message header.\n\nIf a Sent mailbox is configured, messages are added to it after submitting\nto the delivery queue." ,
"Params" : [
{
"Name" : "m" ,
"Typewords" : [
"SubmitMessage"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MessageMove" ,
"Docs" : "MessageMove moves messages to another mailbox. If the message is already in\nthe mailbox an error is returned." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "mailboxID" ,
"Typewords" : [
"int64"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MessageDelete" ,
"Docs" : "MessageDelete permanently deletes messages, without moving them to the Trash mailbox." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "FlagsAdd" ,
"Docs" : "FlagsAdd adds flags, either system flags like \\Seen or custom keywords. The\nflags should be lower-case, but will be converted and verified." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "flaglist" ,
"Typewords" : [
"[]" ,
"string"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "FlagsClear" ,
"Docs" : "FlagsClear clears flags, either system flags like \\Seen or custom keywords." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "flaglist" ,
"Typewords" : [
"[]" ,
"string"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MailboxCreate" ,
"Docs" : "MailboxCreate creates a new mailbox." ,
"Params" : [
{
"Name" : "name" ,
"Typewords" : [
"string"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MailboxDelete" ,
"Docs" : "MailboxDelete deletes a mailbox and all its messages." ,
"Params" : [
{
"Name" : "mailboxID" ,
"Typewords" : [
"int64"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MailboxEmpty" ,
"Docs" : "MailboxEmpty empties a mailbox, removing all messages from the mailbox, but not\nits child mailboxes." ,
"Params" : [
{
"Name" : "mailboxID" ,
"Typewords" : [
"int64"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "MailboxRename" ,
"Docs" : "MailboxRename renames a mailbox, possibly moving it to a new parent. The mailbox\nID and its messages are unchanged." ,
"Params" : [
{
"Name" : "mailboxID" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "newName" ,
"Typewords" : [
"string"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "CompleteRecipient" ,
"Docs" : "CompleteRecipient returns autocomplete matches for a recipient, returning the\nmatches, most recently used first, and whether this is the full list and further\nrequests for longer prefixes aren't necessary." ,
"Params" : [
{
"Name" : "search" ,
"Typewords" : [
"string"
]
}
] ,
"Returns" : [
{
"Name" : "r0" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "r1" ,
"Typewords" : [
"bool"
]
}
]
} ,
{
"Name" : "MailboxSetSpecialUse" ,
"Docs" : "MailboxSetSpecialUse sets the special use flags of a mailbox." ,
"Params" : [
{
"Name" : "mb" ,
"Typewords" : [
"Mailbox"
]
}
] ,
"Returns" : [ ]
} ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "ThreadCollapse" ,
"Docs" : "ThreadCollapse saves the ThreadCollapse field for the messages and its\nchildren. The messageIDs are typically thread roots. But not all roots\n(without parent) of a thread need to have the same collapsed state." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "collapse" ,
"Typewords" : [
"bool"
]
}
] ,
"Returns" : [ ]
} ,
{
"Name" : "ThreadMute" ,
"Docs" : "ThreadMute saves the ThreadMute field for the messages and their children.\nIf messages are muted, they are also marked collapsed." ,
"Params" : [
{
"Name" : "messageIDs" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "mute" ,
"Typewords" : [
"bool"
]
}
] ,
"Returns" : [ ]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "SSETypes" ,
"Docs" : "SSETypes exists to ensure the generated API contains the types, for use in SSE events." ,
"Params" : [ ] ,
"Returns" : [
{
"Name" : "start" ,
"Typewords" : [
"EventStart"
]
} ,
{
"Name" : "viewErr" ,
"Typewords" : [
"EventViewErr"
]
} ,
{
"Name" : "viewReset" ,
"Typewords" : [
"EventViewReset"
]
} ,
{
"Name" : "viewMsgs" ,
"Typewords" : [
"EventViewMsgs"
]
} ,
{
"Name" : "viewChanges" ,
"Typewords" : [
"EventViewChanges"
]
} ,
{
"Name" : "msgAdd" ,
"Typewords" : [
"ChangeMsgAdd"
]
} ,
{
"Name" : "msgRemove" ,
"Typewords" : [
"ChangeMsgRemove"
]
} ,
{
"Name" : "msgFlags" ,
"Typewords" : [
"ChangeMsgFlags"
]
} ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "msgThread" ,
"Typewords" : [
"ChangeMsgThread"
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "mailboxRemove" ,
"Typewords" : [
"ChangeMailboxRemove"
]
} ,
{
"Name" : "mailboxAdd" ,
"Typewords" : [
"ChangeMailboxAdd"
]
} ,
{
"Name" : "mailboxRename" ,
"Typewords" : [
"ChangeMailboxRename"
]
} ,
{
"Name" : "mailboxCounts" ,
"Typewords" : [
"ChangeMailboxCounts"
]
} ,
{
"Name" : "mailboxSpecialUse" ,
"Typewords" : [
"ChangeMailboxSpecialUse"
]
} ,
{
"Name" : "mailboxKeywords" ,
"Typewords" : [
"ChangeMailboxKeywords"
]
} ,
{
"Name" : "flags" ,
"Typewords" : [
"Flags"
]
}
]
}
] ,
"Sections" : [ ] ,
"Structs" : [
{
"Name" : "Request" ,
"Docs" : "Request is a request to an SSE connection to send messages, either for a new\nview, to continue with an existing view, or to a cancel an ongoing request." ,
"Fields" : [
{
"Name" : "ID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "SSEID" ,
"Docs" : "SSE connection." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "ViewID" ,
"Docs" : "To indicate a request is a continuation (more results) of the previous view. Echoed in events, client checks if it is getting results for the latest request." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Cancel" ,
"Docs" : "If set, this request and its view are canceled. A new view must be started." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Query" ,
"Docs" : "" ,
"Typewords" : [
"Query"
]
} ,
{
"Name" : "Page" ,
"Docs" : "" ,
"Typewords" : [
"Page"
]
}
]
} ,
{
"Name" : "Query" ,
"Docs" : "Query is a request for messages that match filters, in a given order." ,
"Fields" : [
{
"Name" : "OrderAsc" ,
"Docs" : "Order by received ascending or desending." ,
"Typewords" : [
"bool"
]
} ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "Threading" ,
"Docs" : "" ,
"Typewords" : [
"ThreadMode"
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "Filter" ,
"Docs" : "" ,
"Typewords" : [
"Filter"
]
} ,
{
"Name" : "NotFilter" ,
"Docs" : "" ,
"Typewords" : [
"NotFilter"
]
}
]
} ,
{
"Name" : "Filter" ,
"Docs" : "Filter selects the messages to return. Fields that are set must all match,\nfor slices each element by match (\"and\")." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "If -1, then all mailboxes except Trash/Junk/Rejects. Otherwise, only active if \u003e 0." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MailboxChildrenIncluded" ,
"Docs" : "If true, also submailboxes are included in the search." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "MailboxName" ,
"Docs" : "In case client doesn't know mailboxes and their IDs yet. Only used during sse connection setup, where it is turned into a MailboxID. Filtering only looks at MailboxID." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Words" ,
"Docs" : "Case insensitive substring match for each string." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "From" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "To" ,
"Docs" : "Including Cc and Bcc." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Oldest" ,
"Docs" : "" ,
"Typewords" : [
"nullable" ,
"timestamp"
]
} ,
{
"Name" : "Newest" ,
"Docs" : "" ,
"Typewords" : [
"nullable" ,
"timestamp"
]
} ,
{
"Name" : "Subject" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Attachments" ,
"Docs" : "" ,
"Typewords" : [
"AttachmentType"
]
} ,
{
"Name" : "Labels" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Headers" ,
"Docs" : "Header values can be empty, it's a check if the header is present, regardless of value." ,
"Typewords" : [
"[]" ,
"[]" ,
"string"
]
} ,
{
"Name" : "SizeMin" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "SizeMax" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
}
]
} ,
{
"Name" : "NotFilter" ,
"Docs" : "NotFilter matches messages that don't match these fields." ,
"Fields" : [
{
"Name" : "Words" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "From" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "To" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Subject" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Attachments" ,
"Docs" : "" ,
"Typewords" : [
"AttachmentType"
]
} ,
{
"Name" : "Labels" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
}
]
} ,
{
"Name" : "Page" ,
"Docs" : "Page holds pagination parameters for a request." ,
"Fields" : [
{
"Name" : "AnchorMessageID" ,
"Docs" : "Start returning messages after this ID, if \u003e 0. For pagination, fetching the next set of messages." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Count" ,
"Docs" : "Number of messages to return, must be \u003e= 1, we never return more than 10000 for one request." ,
"Typewords" : [
"int32"
]
} ,
{
"Name" : "DestMessageID" ,
"Docs" : "If \u003e 0, return messages until DestMessageID is found. More than Count messages can be returned. For long-running searches, it may take a while before this message if found." ,
"Typewords" : [
"int64"
]
}
]
} ,
{
"Name" : "ParsedMessage" ,
"Docs" : "ParsedMessage has more parsed/derived information about a message, intended\nfor rendering the (contents of the) message. Information from MessageItem is\nnot duplicated." ,
"Fields" : [
{
"Name" : "ID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Part" ,
"Docs" : "" ,
"Typewords" : [
"Part"
]
} ,
{
"Name" : "Headers" ,
"Docs" : "" ,
"Typewords" : [
"{}" ,
"[]" ,
"string"
]
} ,
{
"Name" : "Texts" ,
"Docs" : "Text parts, can be empty." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "HasHTML" ,
"Docs" : "Whether there is an HTML part. The webclient renders HTML message parts through an iframe and a separate request with strict CSP headers to prevent script execution and loading of external resources, which isn't possible when loading in iframe with inline HTML because not all browsers support the iframe csp attribute." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "ListReplyAddress" ,
"Docs" : "From List-Post." ,
"Typewords" : [
"nullable" ,
"MessageAddress"
]
}
]
} ,
{
"Name" : "Part" ,
"Docs" : "Part represents a whole mail message, or a part of a multipart message. It\nis designed to handle IMAP requirements efficiently." ,
"Fields" : [
{
"Name" : "BoundaryOffset" ,
"Docs" : "Offset in message where bound starts. -1 for top-level message." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "HeaderOffset" ,
"Docs" : "Offset in message file where header starts." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "BodyOffset" ,
"Docs" : "Offset in message file where body starts." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "EndOffset" ,
"Docs" : "Where body of part ends. Set when part is fully read." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "RawLineCount" ,
"Docs" : "Number of lines in raw, undecoded, body of part. Set when part is fully read." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "DecodedSize" ,
"Docs" : "Number of octets when decoded. If this is a text mediatype, lines ending only in LF are changed end in CRLF and DecodedSize reflects that." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MediaType" ,
"Docs" : "From Content-Type, upper case. E.g. \"TEXT\". Can be empty because content-type may be absent. In this case, the part may be treated as TEXT/PLAIN." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MediaSubType" ,
"Docs" : "From Content-Type, upper case. E.g. \"PLAIN\"." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "ContentTypeParams" ,
"Docs" : "E.g. holds \"boundary\" for multipart messages. Has lower-case keys, and original case values." ,
"Typewords" : [
"{}" ,
"string"
]
} ,
{
"Name" : "ContentID" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "ContentDescription" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "ContentTransferEncoding" ,
"Docs" : "In upper case." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Envelope" ,
"Docs" : "Email message headers. Not for non-message parts." ,
"Typewords" : [
"nullable" ,
"Envelope"
]
} ,
{
"Name" : "Parts" ,
"Docs" : "Parts if this is a multipart." ,
"Typewords" : [
"[]" ,
"Part"
]
} ,
{
"Name" : "Message" ,
"Docs" : "Only for message/rfc822 and message/global. This part may have a buffer as backing io.ReaderAt, because a message/global can have a non-identity content-transfer-encoding. This part has a nil parent." ,
"Typewords" : [
"nullable" ,
"Part"
]
}
]
} ,
{
"Name" : "Envelope" ,
"Docs" : "Envelope holds the basic/common message headers as used in IMAP4." ,
"Fields" : [
{
"Name" : "Date" ,
"Docs" : "" ,
"Typewords" : [
"timestamp"
]
} ,
{
"Name" : "Subject" ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Docs" : "Q/B-word-decoded." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "From" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "Sender" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "ReplyTo" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "To" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "CC" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "BCC" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Address"
]
} ,
{
"Name" : "InReplyTo" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MessageID" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "Address" ,
"Docs" : "Address as used in From and To headers." ,
"Fields" : [
{
"Name" : "Name" ,
"Docs" : "Free-form name for display in mail applications." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "User" ,
"Docs" : "Localpart." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Host" ,
"Docs" : "Domain in ASCII." ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "MessageAddress" ,
"Docs" : "MessageAddress is like message.Address, but with a dns.Domain, with unicode name\nincluded." ,
"Fields" : [
{
"Name" : "Name" ,
"Docs" : "Free-form name for display in mail applications." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "User" ,
"Docs" : "Localpart, encoded." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Domain" ,
"Docs" : "" ,
"Typewords" : [
"Domain"
]
}
]
} ,
{
"Name" : "Domain" ,
"Docs" : "Domain is a domain name, with one or more labels, with at least an ASCII\nrepresentation, and for IDNA non-ASCII domains a unicode representation.\nThe ASCII string must be used for DNS lookups." ,
"Fields" : [
{
"Name" : "ASCII" ,
"Docs" : "A non-unicode domain, e.g. with A-labels (xn--...) or NR-LDH (non-reserved letters/digits/hyphens) labels. Always in lower case." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Unicode" ,
"Docs" : "Name as U-labels. Empty if this is an ASCII-only domain." ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "SubmitMessage" ,
"Docs" : "SubmitMessage is an email message to be sent to one or more recipients.\nAddresses are formatted as just email address, or with a name like \"name\n\u003cuser@host\u003e\"." ,
"Fields" : [
{
"Name" : "From" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "To" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Cc" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Bcc" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Subject" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "TextBody" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Attachments" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"File"
]
} ,
{
"Name" : "ForwardAttachments" ,
"Docs" : "" ,
"Typewords" : [
"ForwardAttachments"
]
} ,
{
"Name" : "IsForward" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "ResponseMessageID" ,
"Docs" : "If set, this was a reply or forward, based on IsForward." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "ReplyTo" ,
"Docs" : "If non-empty, Reply-To header to add to message." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "UserAgent" ,
"Docs" : "User-Agent header added if not empty." ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "File" ,
"Docs" : "File is a new attachment (not from an existing message that is being\nforwarded) to send with a SubmitMessage." ,
"Fields" : [
{
"Name" : "Filename" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "DataURI" ,
"Docs" : "Full data of the attachment, with base64 encoding and including content-type." ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "ForwardAttachments" ,
"Docs" : "ForwardAttachments references attachments by a list of message.Part paths." ,
"Fields" : [
{
"Name" : "MessageID" ,
"Docs" : "Only relevant if MessageID is not 0." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Paths" ,
"Docs" : "List of attachments, each path is a list of indices into the top-level message.Part.Parts." ,
"Typewords" : [
"[]" ,
"[]" ,
"int32"
]
}
]
} ,
{
"Name" : "Mailbox" ,
"Docs" : "Mailbox is collection of messages, e.g. Inbox or Sent." ,
"Fields" : [
{
"Name" : "ID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Name" ,
"Docs" : "\"Inbox\" is the name for the special IMAP \"INBOX\". Slash separated for hierarchy." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "UIDValidity" ,
"Docs" : "If UIDs are invalidated, e.g. when renaming a mailbox to a previously existing name, UIDValidity must be changed. Used by IMAP for synchronization." ,
"Typewords" : [
"uint32"
]
} ,
{
"Name" : "UIDNext" ,
"Docs" : "UID likely to be assigned to next message. Used by IMAP to detect messages delivered to a mailbox." ,
"Typewords" : [
"UID"
]
} ,
{
"Name" : "Archive" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Draft" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Junk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Sent" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Trash" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Keywords" ,
"Docs" : "Keywords as used in messages. Storing a non-system keyword for a message automatically adds it to this list. Used in the IMAP FLAGS response. Only \"atoms\" are allowed (IMAP syntax), keywords are case-insensitive, only stored in lower case (for JMAP), sorted." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "HaveCounts" ,
"Docs" : "Whether MailboxCounts have been initialized." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Total" ,
"Docs" : "Total number of messages, excluding \\Deleted. For JMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Deleted" ,
"Docs" : "Number of messages with \\Deleted flag. Used for IMAP message count that includes messages with \\Deleted." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Unread" ,
"Docs" : "Messages without \\Seen, excluding those with \\Deleted, for JMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Unseen" ,
"Docs" : "Messages without \\Seen, including those with \\Deleted, for IMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Size" ,
"Docs" : "Number of bytes for all messages." ,
"Typewords" : [
"int64"
]
}
]
} ,
{
"Name" : "EventStart" ,
"Docs" : "EventStart is the first message sent on an SSE connection, giving the client\nbasic data to populate its UI. After this event, messages will follow quickly in\nan EventViewMsgs event." ,
"Fields" : [
{
"Name" : "SSEID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "LoginAddress" ,
"Docs" : "" ,
"Typewords" : [
"MessageAddress"
]
} ,
{
"Name" : "Addresses" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "DomainAddressConfigs" ,
"Docs" : "ASCII domain to address config." ,
"Typewords" : [
"{}" ,
"DomainAddressConfig"
]
} ,
{
"Name" : "MailboxName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Mailboxes" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Mailbox"
]
}
]
} ,
{
"Name" : "DomainAddressConfig" ,
"Docs" : "DomainAddressConfig has the address (localpart) configuration for a domain, so\nthe webmail client can decide if an address matches the addresses of the\naccount." ,
"Fields" : [
{
"Name" : "LocalpartCatchallSeparator" ,
"Docs" : "Can be empty." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "LocalpartCaseSensitive" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
}
]
} ,
{
"Name" : "EventViewErr" ,
"Docs" : "EventViewErr indicates an error during a query for messages. The request is\naborted, no more request-related messages will be sent until the next request." ,
"Fields" : [
{
"Name" : "ViewID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "RequestID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Err" ,
"Docs" : "To be displayed in client." ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "EventViewReset" ,
"Docs" : "EventViewReset indicates that a request for the next set of messages in a few\ncould not be fulfilled, e.g. because the anchor message does not exist anymore.\nThe client should clear its list of messages. This can happen before\nEventViewMsgs events are sent." ,
"Fields" : [
{
"Name" : "ViewID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "RequestID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
}
]
} ,
{
"Name" : "EventViewMsgs" ,
"Docs" : "EventViewMsgs contains messages for a view, possibly a continuation of an\nearlier list of messages." ,
"Fields" : [
{
"Name" : "ViewID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "RequestID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MessageItems" ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Docs" : "If empty, this was the last message for the request. If non-empty, a list of thread messages. Each with the first message being the reason this thread is included and can be used as AnchorID in followup requests. If the threading mode is \"off\" in the query, there will always be only a single message. If a thread is sent, all messages in the thread are sent, including those that don't match the query (e.g. from another mailbox). Threads can be displayed based on the ThreadParentIDs field, with possibly slightly different display based on field ThreadMissingLink." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"[]" ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"[]" ,
"MessageItem"
]
} ,
{
"Name" : "ParsedMessage" ,
"Docs" : "If set, will match the target page.DestMessageID from the request." ,
"Typewords" : [
"nullable" ,
"ParsedMessage"
]
} ,
{
"Name" : "ViewEnd" ,
"Docs" : "If set, there are no more messages in this view at this moment. Messages can be added, typically via Change messages, e.g. for new deliveries." ,
"Typewords" : [
"bool"
]
}
]
} ,
{
"Name" : "MessageItem" ,
"Docs" : "MessageItem is sent by queries, it has derived information analyzed from\nmessage.Part, made for the needs of the message items in the message list.\nmessages." ,
"Fields" : [
{
"Name" : "Message" ,
"Docs" : "Without ParsedBuf and MsgPrefix, for size." ,
"Typewords" : [
"Message"
]
} ,
{
"Name" : "Envelope" ,
"Docs" : "" ,
"Typewords" : [
"MessageEnvelope"
]
} ,
{
"Name" : "Attachments" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"Attachment"
]
} ,
{
"Name" : "IsSigned" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "IsEncrypted" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "FirstLine" ,
"Docs" : "Of message body, for showing as preview." ,
"Typewords" : [
"string"
]
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
} ,
{
"Name" : "MatchQuery" ,
"Docs" : "If message does not match query, it can still be included because of threading." ,
"Typewords" : [
"bool"
]
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
}
]
} ,
{
"Name" : "Message" ,
"Docs" : "Message stored in database and per-message file on disk.\n\nContents are always the combined data from MsgPrefix and the on-disk file named\nbased on ID.\n\nMessages always have a header section, even if empty. Incoming messages without\nheader section must get an empty header section added before inserting." ,
"Fields" : [
{
"Name" : "ID" ,
"Docs" : "ID, unchanged over lifetime, determines path to on-disk msg file. Set during deliver." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "UID" ,
"Docs" : "UID, for IMAP. Set during deliver." ,
"Typewords" : [
"UID"
]
} ,
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "ModSeq" ,
"Docs" : "Modification sequence, for faster syncing with IMAP QRESYNC and JMAP. ModSeq is the last modification. CreateSeq is the Seq the message was inserted, always \u003c= ModSeq. If Expunged is set, the message has been removed and should not be returned to the user. In this case, ModSeq is the Seq where the message is removed, and will never be changed again. We have an index on both ModSeq (for JMAP that synchronizes per account) and MailboxID+ModSeq (for IMAP that synchronizes per mailbox). The index on CreateSeq helps efficiently finding created messages for JMAP. The value of ModSeq is special for IMAP. Messages that existed before ModSeq was added have 0 as value. But modseq 0 in IMAP is special, so we return it as 1. If we get modseq 1 from a client, the IMAP server will translate it to 0. When we return modseq to clients, we turn 0 into 1." ,
"Typewords" : [
"ModSeq"
]
} ,
{
"Name" : "CreateSeq" ,
"Docs" : "" ,
"Typewords" : [
"ModSeq"
]
} ,
{
"Name" : "Expunged" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
2023-08-09 17:47:29 +03:00
{
"Name" : "IsReject" ,
"Docs" : "If set, this message was delivered to a Rejects mailbox. When it is moved to a different mailbox, its MailboxOrigID is set to the destination mailbox and this flag cleared." ,
"Typewords" : [
"bool"
]
} ,
2023-08-09 23:31:37 +03:00
{
"Name" : "IsForward" ,
"Docs" : "If set, this is a forwarded message (through a ruleset with IsForward). This causes fields used during junk analysis to be moved to their Orig variants, and masked IP fields cleared, so they aren't used in junk classifications for incoming messages. This ensures the forwarded messages don't cause negative reputation for the forwarding mail server, which may also be sending regular messages." ,
"Typewords" : [
"bool"
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "MailboxOrigID" ,
"Docs" : "MailboxOrigID is the mailbox the message was originally delivered to. Typically Inbox or Rejects, but can also be a mailbox configured in a Ruleset, or Postmaster, TLS/DMARC reporting addresses. MailboxOrigID is not changed when the message is moved to another mailbox, e.g. Archive/Trash/Junk. Used for per-mailbox reputation. MailboxDestinedID is normally 0, but when a message is delivered to the Rejects mailbox, it is set to the intended mailbox according to delivery rules, typically that of Inbox. When such a message is moved out of Rejects, the MailboxOrigID is corrected by setting it to MailboxDestinedID. This ensures the message is used for reputation calculation for future deliveries to that mailbox. These are not bstore references to prevent having to update all messages in a mailbox when the original mailbox is removed. Use of these fields requires checking if the mailbox still exists." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MailboxDestinedID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Received" ,
"Docs" : "" ,
"Typewords" : [
"timestamp"
]
} ,
{
"Name" : "RemoteIP" ,
2023-08-09 23:31:37 +03:00
"Docs" : "Full IP address of remote SMTP server. Empty if not delivered over SMTP. The masked IPs are used to classify incoming messages. They are left empty for messages matching a ruleset for forwarded messages." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "RemoteIPMasked1" ,
"Docs" : "For IPv4 /32, for IPv6 /64, for reputation." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "RemoteIPMasked2" ,
"Docs" : "For IPv4 /26, for IPv6 /48." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "RemoteIPMasked3" ,
"Docs" : "For IPv4 /21, for IPv6 /32." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "EHLODomain" ,
2023-08-09 23:31:37 +03:00
"Docs" : "Only set if present and not an IP address. Unicode string. Empty for forwarded messages." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "MailFrom" ,
"Docs" : "With localpart and domain. Can be empty." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MailFromLocalpart" ,
"Docs" : "SMTP \"MAIL FROM\", can be empty." ,
"Typewords" : [
"Localpart"
]
} ,
{
"Name" : "MailFromDomain" ,
2023-08-09 23:31:37 +03:00
"Docs" : "Only set if it is a domain, not an IP. Unicode string. Empty for forwarded messages, but see OrigMailFromDomain." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "RcptToLocalpart" ,
"Docs" : "SMTP \"RCPT TO\", can be empty." ,
"Typewords" : [
"Localpart"
]
} ,
{
"Name" : "RcptToDomain" ,
"Docs" : "Unicode string." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MsgFromLocalpart" ,
"Docs" : "Parsed \"From\" message header, used for reputation along with domain validation." ,
"Typewords" : [
"Localpart"
]
} ,
{
"Name" : "MsgFromDomain" ,
"Docs" : "Unicode string." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MsgFromOrgDomain" ,
"Docs" : "Unicode string." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "EHLOValidated" ,
"Docs" : "Simplified statements of the Validation fields below, used for incoming messages to check reputation." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "MailFromValidated" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "MsgFromValidated" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "EHLOValidation" ,
"Docs" : "Validation can also take reverse IP lookup into account, not only SPF." ,
"Typewords" : [
"Validation"
]
} ,
{
"Name" : "MailFromValidation" ,
"Docs" : "Can have SPF-specific validations like ValidationSoftfail." ,
"Typewords" : [
"Validation"
]
} ,
{
"Name" : "MsgFromValidation" ,
"Docs" : "Desirable validations: Strict, DMARC, Relaxed. Will not be just Pass." ,
"Typewords" : [
"Validation"
]
} ,
{
"Name" : "DKIMDomains" ,
2023-08-09 23:31:37 +03:00
"Docs" : "Domains with verified DKIM signatures. Unicode string. For forwarded messages, a DKIM domain that matched a ruleset's verified domain is left out, but included in OrigDKIMDomains." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "OrigEHLODomain" ,
"Docs" : "For forwarded messages," ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "OrigDKIMDomains" ,
"Docs" : "" ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "MessageID" ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Docs" : "Canonicalized Message-Id, always lower-case and normalized quoting, without \u003c\u003e's. Empty if missing. Used for matching message threads, and to prevent duplicate reject delivery." ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "SubjectBase" ,
"Docs" : "For matching threads in case there is no References/In-Reply-To header. It is lower-cased, white-space collapsed, mailing list tags and re/fwd tags removed." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "MessageHash" ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Docs" : "Hash of message. For rejects delivery in case there is no Message-ID, only set when delivered as reject." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"[]" ,
"uint8"
]
} ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "ThreadID" ,
"Docs" : "ID of message starting this thread." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "ThreadParentIDs" ,
"Docs" : "IDs of parent messages, from closest parent to the root message. Parent messages may be in a different mailbox, or may no longer exist. ThreadParentIDs must never contain the message id itself (a cycle), and parent messages must reference the same ancestors." ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "ThreadMissingLink" ,
"Docs" : "ThreadMissingLink is true if there is no match with a direct parent. E.g. first ID in ThreadParentIDs is not the direct ancestor (an intermediate message may have been deleted), or subject-based matching was done." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "ThreadMuted" ,
"Docs" : "If set, newly delivered child messages are automatically marked as read. This field is copied to new child messages. Changes are propagated to the webmail client." ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "ThreadCollapsed" ,
"Docs" : "If set, this (sub)thread is collapsed in the webmail client, for threading mode \"on\" (mode \"unread\" ignores it). This field is copied to new child message. Changes are propagated to the webmail client." ,
"Typewords" : [
"bool"
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "Seen" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Answered" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Flagged" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Forwarded" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Junk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Notjunk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Deleted" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Draft" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Phishing" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "MDNSent" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Keywords" ,
"Docs" : "For keywords other than system flags or the basic well-known $-flags. Only in \"atom\" syntax (IMAP), they are case-insensitive, always stored in lower-case (for JMAP), sorted." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
"Name" : "Size" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "TrainedJunk" ,
"Docs" : "If nil, no training done yet. Otherwise, true is trained as junk, false trained as nonjunk." ,
"Typewords" : [
"nullable" ,
"bool"
]
} ,
{
"Name" : "MsgPrefix" ,
"Docs" : "Typically holds received headers and/or header separator." ,
"Typewords" : [
"[]" ,
"uint8"
]
} ,
{
"Name" : "ParsedBuf" ,
"Docs" : "ParsedBuf message structure. Currently saved as JSON of message.Part because bstore cannot yet store recursive types. Created when first needed, and saved in the database. todo: once replaced with non-json storage, remove date fixup in ../message/part.go." ,
"Typewords" : [
"[]" ,
"uint8"
]
}
]
} ,
{
"Name" : "MessageEnvelope" ,
"Docs" : "MessageEnvelope is like message.Envelope, as used in message.Part, but including\nunicode host names for IDNA names." ,
"Fields" : [
{
"Name" : "Date" ,
"Docs" : "todo: should get sherpadoc to understand type embeds and embed the non-MessageAddress fields from message.Envelope." ,
"Typewords" : [
"timestamp"
]
} ,
{
"Name" : "Subject" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "From" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "Sender" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "ReplyTo" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "To" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "CC" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "BCC" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"MessageAddress"
]
} ,
{
"Name" : "InReplyTo" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "MessageID" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "Attachment" ,
"Docs" : "Attachment is a MIME part is an existing message that is not intended as\nviewable text or HTML part." ,
"Fields" : [
{
"Name" : "Path" ,
"Docs" : "Indices into top-level message.Part.Parts." ,
"Typewords" : [
"[]" ,
"int32"
]
} ,
{
"Name" : "Filename" ,
2023-09-21 15:51:47 +03:00
"Docs" : "File name based on \"name\" attribute of \"Content-Type\", or the \"filename\" attribute of \"Content-Disposition\". todo: decode non-ascii character sets" ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Typewords" : [
"string"
]
} ,
{
"Name" : "Part" ,
"Docs" : "" ,
"Typewords" : [
"Part"
]
}
]
} ,
{
"Name" : "EventViewChanges" ,
"Docs" : "EventViewChanges contain one or more changes relevant for the client, either\nwith new mailbox total/unseen message counts, or messages added/removed/modified\n(flags) for the current view." ,
"Fields" : [
{
"Name" : "ViewID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Changes" ,
"Docs" : "The first field of [2]any is a string, the second of the Change types below." ,
"Typewords" : [
"[]" ,
"[]" ,
"any"
]
}
]
} ,
{
"Name" : "ChangeMsgAdd" ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Docs" : "ChangeMsgAdd adds a new message and possibly its thread to the view." ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "UID" ,
"Docs" : "" ,
"Typewords" : [
"UID"
]
} ,
{
"Name" : "ModSeq" ,
"Docs" : "" ,
"Typewords" : [
"ModSeq"
]
} ,
{
"Name" : "Flags" ,
"Docs" : "System flags." ,
"Typewords" : [
"Flags"
]
} ,
{
"Name" : "Keywords" ,
"Docs" : "Other flags." ,
"Typewords" : [
"[]" ,
"string"
]
} ,
{
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Name" : "MessageItems" ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Docs" : "" ,
"Typewords" : [
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"[]" ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"MessageItem"
]
}
]
} ,
{
"Name" : "Flags" ,
"Docs" : "Flags for a mail message." ,
"Fields" : [
{
"Name" : "Seen" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Answered" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Flagged" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Forwarded" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Junk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Notjunk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Deleted" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Draft" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Phishing" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "MDNSent" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
}
]
} ,
{
"Name" : "ChangeMsgRemove" ,
"Docs" : "ChangeMsgRemove removes one or more messages from the view." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "UIDs" ,
"Docs" : "Must be in increasing UID order, for IMAP." ,
"Typewords" : [
"[]" ,
"UID"
]
} ,
{
"Name" : "ModSeq" ,
"Docs" : "" ,
"Typewords" : [
"ModSeq"
]
}
]
} ,
{
"Name" : "ChangeMsgFlags" ,
"Docs" : "ChangeMsgFlags updates flags for one message." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "UID" ,
"Docs" : "" ,
"Typewords" : [
"UID"
]
} ,
{
"Name" : "ModSeq" ,
"Docs" : "" ,
"Typewords" : [
"ModSeq"
]
} ,
{
"Name" : "Mask" ,
"Docs" : "Which flags are actually modified." ,
"Typewords" : [
"Flags"
]
} ,
{
"Name" : "Flags" ,
"Docs" : "New flag values. All are set, not just mask." ,
"Typewords" : [
"Flags"
]
} ,
{
"Name" : "Keywords" ,
"Docs" : "Non-system/well-known flags/keywords/labels." ,
"Typewords" : [
"[]" ,
"string"
]
}
]
} ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "ChangeMsgThread" ,
"Docs" : "ChangeMsgThread updates muted/collapsed fields for one message." ,
"Fields" : [
{
"Name" : "MessageIDs" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"int64"
]
} ,
{
"Name" : "Muted" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Collapsed" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
}
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "ChangeMailboxRemove" ,
"Docs" : "ChangeMailboxRemove indicates a mailbox was removed, including all its messages." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Name" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
}
]
} ,
{
"Name" : "ChangeMailboxAdd" ,
"Docs" : "ChangeMailboxAdd indicates a new mailbox was added, initially without any messages." ,
"Fields" : [
{
"Name" : "Mailbox" ,
"Docs" : "" ,
"Typewords" : [
"Mailbox"
]
}
]
} ,
{
"Name" : "ChangeMailboxRename" ,
"Docs" : "ChangeMailboxRename indicates a mailbox was renamed. Its ID stays the same.\nIt could be under a new parent." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "OldName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "NewName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Flags" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
}
]
} ,
{
"Name" : "ChangeMailboxCounts" ,
"Docs" : "ChangeMailboxCounts set new total and unseen message counts for a mailbox." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MailboxName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Total" ,
"Docs" : "Total number of messages, excluding \\Deleted. For JMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Deleted" ,
"Docs" : "Number of messages with \\Deleted flag. Used for IMAP message count that includes messages with \\Deleted." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Unread" ,
"Docs" : "Messages without \\Seen, excluding those with \\Deleted, for JMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Unseen" ,
"Docs" : "Messages without \\Seen, including those with \\Deleted, for IMAP." ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "Size" ,
"Docs" : "Number of bytes for all messages." ,
"Typewords" : [
"int64"
]
}
]
} ,
{
"Name" : "ChangeMailboxSpecialUse" ,
"Docs" : "ChangeMailboxSpecialUse has updated special-use flags for a mailbox." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MailboxName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "SpecialUse" ,
"Docs" : "" ,
"Typewords" : [
"SpecialUse"
]
}
]
} ,
{
"Name" : "SpecialUse" ,
"Docs" : "SpecialUse identifies a specific role for a mailbox, used by clients to\nunderstand where messages should go." ,
"Fields" : [
{
"Name" : "Archive" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Draft" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Junk" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Sent" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
} ,
{
"Name" : "Trash" ,
"Docs" : "" ,
"Typewords" : [
"bool"
]
}
]
} ,
{
"Name" : "ChangeMailboxKeywords" ,
"Docs" : "ChangeMailboxKeywords has an updated list of keywords for a mailbox, e.g. after\na message was added with a keyword that wasn't in the mailbox yet." ,
"Fields" : [
{
"Name" : "MailboxID" ,
"Docs" : "" ,
"Typewords" : [
"int64"
]
} ,
{
"Name" : "MailboxName" ,
"Docs" : "" ,
"Typewords" : [
"string"
]
} ,
{
"Name" : "Keywords" ,
"Docs" : "" ,
"Typewords" : [
"[]" ,
"string"
]
}
]
}
] ,
"Ints" : [
{
"Name" : "UID" ,
"Docs" : "IMAP UID." ,
"Values" : null
} ,
{
"Name" : "ModSeq" ,
"Docs" : "ModSeq represents a modseq as stored in the database. ModSeq 0 in the\ndatabase is sent to the client as 1, because modseq 0 is special in IMAP.\nModSeq coming from the client are of type int64." ,
"Values" : null
} ,
{
"Name" : "Validation" ,
"Docs" : "Validation of \"message From\" domain." ,
"Values" : [
{
"Name" : "ValidationUnknown" ,
"Value" : 0 ,
"Docs" : ""
} ,
{
"Name" : "ValidationStrict" ,
"Value" : 1 ,
"Docs" : "Like DMARC, with strict policies."
} ,
{
"Name" : "ValidationDMARC" ,
"Value" : 2 ,
"Docs" : "Actual DMARC policy."
} ,
{
"Name" : "ValidationRelaxed" ,
"Value" : 3 ,
"Docs" : "Like DMARC, with relaxed policies."
} ,
{
"Name" : "ValidationPass" ,
"Value" : 4 ,
"Docs" : "For SPF."
} ,
{
"Name" : "ValidationNeutral" ,
"Value" : 5 ,
"Docs" : "For SPF."
} ,
{
"Name" : "ValidationTemperror" ,
"Value" : 6 ,
"Docs" : ""
} ,
{
"Name" : "ValidationPermerror" ,
"Value" : 7 ,
"Docs" : ""
} ,
{
"Name" : "ValidationFail" ,
"Value" : 8 ,
"Docs" : ""
} ,
{
"Name" : "ValidationSoftfail" ,
"Value" : 9 ,
"Docs" : "For SPF."
} ,
{
"Name" : "ValidationNone" ,
"Value" : 10 ,
"Docs" : "E.g. No records."
}
]
}
] ,
"Strings" : [
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
{
"Name" : "ThreadMode" ,
"Docs" : "" ,
"Values" : [
{
"Name" : "ThreadOff" ,
"Value" : "off" ,
"Docs" : ""
} ,
{
"Name" : "ThreadOn" ,
"Value" : "on" ,
"Docs" : ""
} ,
{
"Name" : "ThreadUnread" ,
"Value" : "unread" ,
"Docs" : ""
}
]
} ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
{
"Name" : "AttachmentType" ,
"Docs" : "AttachmentType is for filtering by attachment type." ,
"Values" : [
{
"Name" : "AttachmentIndifferent" ,
"Value" : "" ,
"Docs" : ""
} ,
{
"Name" : "AttachmentNone" ,
"Value" : "none" ,
"Docs" : ""
} ,
{
"Name" : "AttachmentAny" ,
"Value" : "any" ,
"Docs" : ""
} ,
{
"Name" : "AttachmentImage" ,
"Value" : "image" ,
"Docs" : "png, jpg, gif, ..."
} ,
{
"Name" : "AttachmentPDF" ,
"Value" : "pdf" ,
"Docs" : ""
} ,
{
"Name" : "AttachmentArchive" ,
"Value" : "archive" ,
"Docs" : "zip files, tgz, ..."
} ,
{
"Name" : "AttachmentSpreadsheet" ,
"Value" : "spreadsheet" ,
"Docs" : "ods, xlsx, ..."
} ,
{
"Name" : "AttachmentDocument" ,
"Value" : "document" ,
"Docs" : "odt, docx, ..."
} ,
{
"Name" : "AttachmentPresentation" ,
"Value" : "presentation" ,
"Docs" : "odp, pptx, ..."
}
]
} ,
{
"Name" : "Localpart" ,
"Docs" : "Localpart is a decoded local part of an email address, before the \"@\".\nFor quoted strings, values do not hold the double quote or escaping backslashes.\nAn empty string can be a valid localpart." ,
"Values" : null
}
] ,
"SherpaVersion" : 0 ,
"SherpadocVersion" : 1
}