add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"use strict" ;
2023-12-31 13:55:22 +03:00
// Javascript is generated from typescript, do not modify generated javascript because changes will be overwritten.
const [ dom , style , attr , prop ] = ( function ( ) {
// Start of unicode block (rough approximation of script), from https://www.unicode.org/Public/UNIDATA/Blocks.txt
const scriptblocks = [ 0x0000 , 0x0080 , 0x0100 , 0x0180 , 0x0250 , 0x02B0 , 0x0300 , 0x0370 , 0x0400 , 0x0500 , 0x0530 , 0x0590 , 0x0600 , 0x0700 , 0x0750 , 0x0780 , 0x07C0 , 0x0800 , 0x0840 , 0x0860 , 0x0870 , 0x08A0 , 0x0900 , 0x0980 , 0x0A00 , 0x0A80 , 0x0B00 , 0x0B80 , 0x0C00 , 0x0C80 , 0x0D00 , 0x0D80 , 0x0E00 , 0x0E80 , 0x0F00 , 0x1000 , 0x10A0 , 0x1100 , 0x1200 , 0x1380 , 0x13A0 , 0x1400 , 0x1680 , 0x16A0 , 0x1700 , 0x1720 , 0x1740 , 0x1760 , 0x1780 , 0x1800 , 0x18B0 , 0x1900 , 0x1950 , 0x1980 , 0x19E0 , 0x1A00 , 0x1A20 , 0x1AB0 , 0x1B00 , 0x1B80 , 0x1BC0 , 0x1C00 , 0x1C50 , 0x1C80 , 0x1C90 , 0x1CC0 , 0x1CD0 , 0x1D00 , 0x1D80 , 0x1DC0 , 0x1E00 , 0x1F00 , 0x2000 , 0x2070 , 0x20A0 , 0x20D0 , 0x2100 , 0x2150 , 0x2190 , 0x2200 , 0x2300 , 0x2400 , 0x2440 , 0x2460 , 0x2500 , 0x2580 , 0x25A0 , 0x2600 , 0x2700 , 0x27C0 , 0x27F0 , 0x2800 , 0x2900 , 0x2980 , 0x2A00 , 0x2B00 , 0x2C00 , 0x2C60 , 0x2C80 , 0x2D00 , 0x2D30 , 0x2D80 , 0x2DE0 , 0x2E00 , 0x2E80 , 0x2F00 , 0x2FF0 , 0x3000 , 0x3040 , 0x30A0 , 0x3100 , 0x3130 , 0x3190 , 0x31A0 , 0x31C0 , 0x31F0 , 0x3200 , 0x3300 , 0x3400 , 0x4DC0 , 0x4E00 , 0xA000 , 0xA490 , 0xA4D0 , 0xA500 , 0xA640 , 0xA6A0 , 0xA700 , 0xA720 , 0xA800 , 0xA830 , 0xA840 , 0xA880 , 0xA8E0 , 0xA900 , 0xA930 , 0xA960 , 0xA980 , 0xA9E0 , 0xAA00 , 0xAA60 , 0xAA80 , 0xAAE0 , 0xAB00 , 0xAB30 , 0xAB70 , 0xABC0 , 0xAC00 , 0xD7B0 , 0xD800 , 0xDB80 , 0xDC00 , 0xE000 , 0xF900 , 0xFB00 , 0xFB50 , 0xFE00 , 0xFE10 , 0xFE20 , 0xFE30 , 0xFE50 , 0xFE70 , 0xFF00 , 0xFFF0 , 0x10000 , 0x10080 , 0x10100 , 0x10140 , 0x10190 , 0x101D0 , 0x10280 , 0x102A0 , 0x102E0 , 0x10300 , 0x10330 , 0x10350 , 0x10380 , 0x103A0 , 0x10400 , 0x10450 , 0x10480 , 0x104B0 , 0x10500 , 0x10530 , 0x10570 , 0x10600 , 0x10780 , 0x10800 , 0x10840 , 0x10860 , 0x10880 , 0x108E0 , 0x10900 , 0x10920 , 0x10980 , 0x109A0 , 0x10A00 , 0x10A60 , 0x10A80 , 0x10AC0 , 0x10B00 , 0x10B40 , 0x10B60 , 0x10B80 , 0x10C00 , 0x10C80 , 0x10D00 , 0x10E60 , 0x10E80 , 0x10EC0 , 0x10F00 , 0x10F30 , 0x10F70 , 0x10FB0 , 0x10FE0 , 0x11000 , 0x11080 , 0x110D0 , 0x11100 , 0x11150 , 0x11180 , 0x111E0 , 0x11200 , 0x11280 , 0x112B0 , 0x11300 , 0x11400 , 0x11480 , 0x11580 , 0x11600 , 0x11660 , 0x11680 , 0x11700 , 0x11800 , 0x118A0 , 0x11900 , 0x119A0 , 0x11A00 , 0x11A50 , 0x11AB0 , 0x11AC0 , 0x11B00 , 0x11C00 , 0x11C70 , 0x11D00 , 0x11D60 , 0x11EE0 , 0x11F00 , 0x11FB0 , 0x11FC0 , 0x12000 , 0x12400 , 0x12480 , 0x12F90 , 0x13000 , 0x13430 , 0x14400 , 0x16800 , 0x16A40 , 0x16A70 , 0x16AD0 , 0x16B00 , 0x16E40 , 0x16F00 , 0x16FE0 , 0x17000 , 0x18800 , 0x18B00 , 0x18D00 , 0x1AFF0 , 0x1B000 , 0x1B100 , 0x1B130 , 0x1B170 , 0x1BC00 , 0x1BCA0 , 0x1CF00 , 0x1D000 , 0x1D100 , 0x1D200 , 0x1D2C0 , 0x1D2E0 , 0x1D300 , 0x1D360 , 0x1D400 , 0x1D800 , 0x1DF00 , 0x1E000 , 0x1E030 , 0x1E100 , 0x1E290 , 0x1E2C0 , 0x1E4D0 , 0x1E7E0 , 0x1E800 , 0x1E900 , 0x1EC70 , 0x1ED00 , 0x1EE00 , 0x1F000 , 0x1F030 , 0x1F0A0 , 0x1F100 , 0x1F200 , 0x1F300 , 0x1F600 , 0x1F650 , 0x1F680 , 0x1F700 , 0x1F780 , 0x1F800 , 0x1F900 , 0x1FA00 , 0x1FA70 , 0x1FB00 , 0x20000 , 0x2A700 , 0x2B740 , 0x2B820 , 0x2CEB0 , 0x2F800 , 0x30000 , 0x31350 , 0xE0000 , 0xE0100 , 0xF0000 , 0x100000 ] ;
// Find block code belongs in.
const findBlock = ( code ) => {
let s = 0 ;
let e = scriptblocks . length ;
while ( s < e - 1 ) {
let i = Math . floor ( ( s + e ) / 2 ) ;
if ( code < scriptblocks [ i ] ) {
e = i ;
}
else {
s = i ;
}
}
return s ;
} ;
// formatText adds s to element e, in a way that makes switching unicode scripts
// clear, with alternating DOM TextNode and span elements with a "switchscript"
// class. Useful for highlighting look alikes, e.g. a (ascii 0x61) and а (cyrillic
// 0x430).
//
// This is only called one string at a time, so the UI can still display strings
// without highlighting switching scripts, by calling formatText on the parts.
const formatText = ( e , s ) => {
// Handle some common cases quickly.
if ( ! s ) {
return ;
}
let ascii = true ;
for ( const c of s ) {
const cp = c . codePointAt ( 0 ) ; // For typescript, to check for undefined.
if ( cp !== undefined && cp >= 0x0080 ) {
ascii = false ;
break ;
}
}
if ( ascii ) {
e . appendChild ( document . createTextNode ( s ) ) ;
return ;
}
// todo: handle grapheme clusters? wait for Intl.Segmenter?
let n = 0 ; // Number of text/span parts added.
let str = '' ; // Collected so far.
let block = - 1 ; // Previous block/script.
let mod = 1 ;
const put = ( nextblock ) => {
if ( n === 0 && nextblock === 0 ) {
// Start was non-ascii, second block is ascii, we'll start marked as switched.
mod = 0 ;
}
if ( n % 2 === mod ) {
const x = document . createElement ( 'span' ) ;
x . classList . add ( 'scriptswitch' ) ;
x . appendChild ( document . createTextNode ( str ) ) ;
e . appendChild ( x ) ;
}
else {
e . appendChild ( document . createTextNode ( str ) ) ;
}
n ++ ;
str = '' ;
} ;
for ( const c of s ) {
// Basic whitespace does not switch blocks. Will probably need to extend with more
// punctuation in the future. Possibly for digits too. But perhaps not in all
// scripts.
if ( c === ' ' || c === '\t' || c === '\r' || c === '\n' ) {
str += c ;
continue ;
}
const code = c . codePointAt ( 0 ) ;
if ( block < 0 || ! ( code >= scriptblocks [ block ] && ( code < scriptblocks [ block + 1 ] || block === scriptblocks . length - 1 ) ) ) {
const nextblock = code < 0x0080 ? 0 : findBlock ( code ) ;
if ( block >= 0 ) {
put ( nextblock ) ;
}
block = nextblock ;
}
str += c ;
}
put ( - 1 ) ;
} ;
const _domKids = ( e , l ) => {
l . forEach ( ( c ) => {
const xc = c ;
if ( typeof c === 'string' ) {
formatText ( e , c ) ;
}
else if ( c instanceof String ) {
// String is an escape-hatch for text that should not be formatted with
// unicode-block-change-highlighting, e.g. for textarea values.
e . appendChild ( document . createTextNode ( '' + c ) ) ;
}
else if ( c instanceof Element ) {
e . appendChild ( c ) ;
}
else if ( c instanceof Function ) {
if ( ! c . name ) {
throw new Error ( 'function without name' ) ;
}
e . addEventListener ( c . name , c ) ;
}
else if ( Array . isArray ( xc ) ) {
_domKids ( e , c ) ;
}
else if ( xc . _class ) {
for ( const s of xc . _class ) {
e . classList . toggle ( s , true ) ;
}
}
else if ( xc . _attrs ) {
for ( const k in xc . _attrs ) {
e . setAttribute ( k , xc . _attrs [ k ] ) ;
}
}
else if ( xc . _styles ) {
for ( const k in xc . _styles ) {
const estyle = e . style ;
estyle [ k ] = xc . _styles [ k ] ;
}
}
else if ( xc . _props ) {
for ( const k in xc . _props ) {
const eprops = e ;
eprops [ k ] = xc . _props [ k ] ;
}
}
else if ( xc . root ) {
e . appendChild ( xc . root ) ;
}
else {
console . log ( 'bad kid' , c ) ;
throw new Error ( 'bad kid' ) ;
}
} ) ;
return e ;
} ;
const dom = {
_kids : function ( e , ... kl ) {
while ( e . firstChild ) {
e . removeChild ( e . firstChild ) ;
}
_domKids ( e , kl ) ;
} ,
_attrs : ( x ) => { return { _attrs : x } ; } ,
_class : ( ... x ) => { return { _class : x } ; } ,
// The createElement calls are spelled out so typescript can derive function
// signatures with a specific HTML*Element return type.
div : ( ... l ) => _domKids ( document . createElement ( 'div' ) , l ) ,
span : ( ... l ) => _domKids ( document . createElement ( 'span' ) , l ) ,
a : ( ... l ) => _domKids ( document . createElement ( 'a' ) , l ) ,
input : ( ... l ) => _domKids ( document . createElement ( 'input' ) , l ) ,
textarea : ( ... l ) => _domKids ( document . createElement ( 'textarea' ) , l ) ,
select : ( ... l ) => _domKids ( document . createElement ( 'select' ) , l ) ,
option : ( ... l ) => _domKids ( document . createElement ( 'option' ) , l ) ,
clickbutton : ( ... l ) => _domKids ( document . createElement ( 'button' ) , [ attr . type ( 'button' ) , ... l ] ) ,
submitbutton : ( ... l ) => _domKids ( document . createElement ( 'button' ) , [ attr . type ( 'submit' ) , ... l ] ) ,
form : ( ... l ) => _domKids ( document . createElement ( 'form' ) , l ) ,
fieldset : ( ... l ) => _domKids ( document . createElement ( 'fieldset' ) , l ) ,
table : ( ... l ) => _domKids ( document . createElement ( 'table' ) , l ) ,
thead : ( ... l ) => _domKids ( document . createElement ( 'thead' ) , l ) ,
tbody : ( ... l ) => _domKids ( document . createElement ( 'tbody' ) , l ) ,
tfoot : ( ... l ) => _domKids ( document . createElement ( 'tfoot' ) , l ) ,
tr : ( ... l ) => _domKids ( document . createElement ( 'tr' ) , l ) ,
td : ( ... l ) => _domKids ( document . createElement ( 'td' ) , l ) ,
th : ( ... l ) => _domKids ( document . createElement ( 'th' ) , l ) ,
datalist : ( ... l ) => _domKids ( document . createElement ( 'datalist' ) , l ) ,
h1 : ( ... l ) => _domKids ( document . createElement ( 'h1' ) , l ) ,
h2 : ( ... l ) => _domKids ( document . createElement ( 'h2' ) , l ) ,
h3 : ( ... l ) => _domKids ( document . createElement ( 'h3' ) , l ) ,
br : ( ... l ) => _domKids ( document . createElement ( 'br' ) , l ) ,
hr : ( ... l ) => _domKids ( document . createElement ( 'hr' ) , l ) ,
pre : ( ... l ) => _domKids ( document . createElement ( 'pre' ) , l ) ,
label : ( ... l ) => _domKids ( document . createElement ( 'label' ) , l ) ,
ul : ( ... l ) => _domKids ( document . createElement ( 'ul' ) , l ) ,
li : ( ... l ) => _domKids ( document . createElement ( 'li' ) , l ) ,
iframe : ( ... l ) => _domKids ( document . createElement ( 'iframe' ) , l ) ,
b : ( ... l ) => _domKids ( document . createElement ( 'b' ) , l ) ,
img : ( ... l ) => _domKids ( document . createElement ( 'img' ) , l ) ,
style : ( ... l ) => _domKids ( document . createElement ( 'style' ) , l ) ,
search : ( ... l ) => _domKids ( document . createElement ( 'search' ) , l ) ,
p : ( ... l ) => _domKids ( document . createElement ( 'p' ) , l ) ,
} ;
const _attr = ( k , v ) => { const o = { } ; o [ k ] = v ; return { _attrs : o } ; } ;
const attr = {
title : ( s ) => _attr ( 'title' , s ) ,
value : ( s ) => _attr ( 'value' , s ) ,
type : ( s ) => _attr ( 'type' , s ) ,
tabindex : ( s ) => _attr ( 'tabindex' , s ) ,
src : ( s ) => _attr ( 'src' , s ) ,
placeholder : ( s ) => _attr ( 'placeholder' , s ) ,
href : ( s ) => _attr ( 'href' , s ) ,
checked : ( s ) => _attr ( 'checked' , s ) ,
selected : ( s ) => _attr ( 'selected' , s ) ,
id : ( s ) => _attr ( 'id' , s ) ,
datalist : ( s ) => _attr ( 'datalist' , s ) ,
rows : ( s ) => _attr ( 'rows' , s ) ,
target : ( s ) => _attr ( 'target' , s ) ,
rel : ( s ) => _attr ( 'rel' , s ) ,
required : ( s ) => _attr ( 'required' , s ) ,
multiple : ( s ) => _attr ( 'multiple' , s ) ,
download : ( s ) => _attr ( 'download' , s ) ,
disabled : ( s ) => _attr ( 'disabled' , s ) ,
draggable : ( s ) => _attr ( 'draggable' , s ) ,
rowspan : ( s ) => _attr ( 'rowspan' , s ) ,
colspan : ( s ) => _attr ( 'colspan' , s ) ,
for : ( s ) => _attr ( 'for' , s ) ,
role : ( s ) => _attr ( 'role' , s ) ,
arialabel : ( s ) => _attr ( 'aria-label' , s ) ,
arialive : ( s ) => _attr ( 'aria-live' , s ) ,
name : ( s ) => _attr ( 'name' , s ) ,
min : ( s ) => _attr ( 'min' , s ) ,
max : ( s ) => _attr ( 'max' , s ) ,
} ;
const style = ( x ) => { return { _styles : x } ; } ;
const prop = ( x ) => { return { _props : x } ; } ;
return [ dom , style , attr , prop ] ;
} ) ( ) ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
// NOTE: GENERATED by github.com/mjl-/sherpats, DO NOT MODIFY
var api ;
( function ( api ) {
// Validation of "message From" domain.
let Validation ;
( function ( Validation ) {
Validation [ Validation [ "ValidationUnknown" ] = 0 ] = "ValidationUnknown" ;
Validation [ Validation [ "ValidationStrict" ] = 1 ] = "ValidationStrict" ;
Validation [ Validation [ "ValidationDMARC" ] = 2 ] = "ValidationDMARC" ;
Validation [ Validation [ "ValidationRelaxed" ] = 3 ] = "ValidationRelaxed" ;
Validation [ Validation [ "ValidationPass" ] = 4 ] = "ValidationPass" ;
Validation [ Validation [ "ValidationNeutral" ] = 5 ] = "ValidationNeutral" ;
Validation [ Validation [ "ValidationTemperror" ] = 6 ] = "ValidationTemperror" ;
Validation [ Validation [ "ValidationPermerror" ] = 7 ] = "ValidationPermerror" ;
Validation [ Validation [ "ValidationFail" ] = 8 ] = "ValidationFail" ;
Validation [ Validation [ "ValidationSoftfail" ] = 9 ] = "ValidationSoftfail" ;
Validation [ Validation [ "ValidationNone" ] = 10 ] = "ValidationNone" ;
} ) ( Validation = api . Validation || ( api . Validation = { } ) ) ;
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
let ThreadMode ;
( function ( ThreadMode ) {
ThreadMode [ "ThreadOff" ] = "off" ;
ThreadMode [ "ThreadOn" ] = "on" ;
ThreadMode [ "ThreadUnread" ] = "unread" ;
} ) ( ThreadMode = api . ThreadMode || ( api . ThreadMode = { } ) ) ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
// AttachmentType is for filtering by attachment type.
let AttachmentType ;
( function ( AttachmentType ) {
AttachmentType [ "AttachmentIndifferent" ] = "" ;
AttachmentType [ "AttachmentNone" ] = "none" ;
AttachmentType [ "AttachmentAny" ] = "any" ;
AttachmentType [ "AttachmentImage" ] = "image" ;
AttachmentType [ "AttachmentPDF" ] = "pdf" ;
AttachmentType [ "AttachmentArchive" ] = "archive" ;
AttachmentType [ "AttachmentSpreadsheet" ] = "spreadsheet" ;
AttachmentType [ "AttachmentDocument" ] = "document" ;
AttachmentType [ "AttachmentPresentation" ] = "presentation" ;
} ) ( AttachmentType = api . AttachmentType || ( api . AttachmentType = { } ) ) ;
2023-10-15 16:05:20 +03:00
// SecurityResult indicates whether a security feature is supported.
let SecurityResult ;
( function ( SecurityResult ) {
SecurityResult [ "SecurityResultError" ] = "error" ;
SecurityResult [ "SecurityResultNo" ] = "no" ;
SecurityResult [ "SecurityResultYes" ] = "yes" ;
// Unknown whether supported. Finding out may only be (reasonably) possible when
// trying (e.g. SMTP STARTTLS). Once tried, the result may be cached for future
// lookups.
SecurityResult [ "SecurityResultUnknown" ] = "unknown" ;
} ) ( SecurityResult = api . SecurityResult || ( api . SecurityResult = { } ) ) ;
api . structTypes = { "Address" : true , "Attachment" : true , "ChangeMailboxAdd" : true , "ChangeMailboxCounts" : true , "ChangeMailboxKeywords" : true , "ChangeMailboxRemove" : true , "ChangeMailboxRename" : true , "ChangeMailboxSpecialUse" : true , "ChangeMsgAdd" : true , "ChangeMsgFlags" : true , "ChangeMsgRemove" : true , "ChangeMsgThread" : true , "Domain" : true , "DomainAddressConfig" : true , "Envelope" : true , "EventStart" : true , "EventViewChanges" : true , "EventViewErr" : true , "EventViewMsgs" : true , "EventViewReset" : true , "File" : true , "Filter" : true , "Flags" : true , "ForwardAttachments" : true , "Mailbox" : true , "Message" : true , "MessageAddress" : true , "MessageEnvelope" : true , "MessageItem" : true , "NotFilter" : true , "Page" : true , "ParsedMessage" : true , "Part" : true , "Query" : true , "RecipientSecurity" : true , "Request" : true , "SpecialUse" : true , "SubmitMessage" : true } ;
api . stringsTypes = { "AttachmentType" : true , "Localpart" : true , "SecurityResult" : true , "ThreadMode" : true } ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
api . intsTypes = { "ModSeq" : true , "UID" : true , "Validation" : true } ;
api . types = {
"Request" : { "Name" : "Request" , "Docs" : "" , "Fields" : [ { "Name" : "ID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "SSEID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "ViewID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Cancel" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Query" , "Docs" : "" , "Typewords" : [ "Query" ] } , { "Name" : "Page" , "Docs" : "" , "Typewords" : [ "Page" ] } ] } ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"Query" : { "Name" : "Query" , "Docs" : "" , "Fields" : [ { "Name" : "OrderAsc" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Threading" , "Docs" : "" , "Typewords" : [ "ThreadMode" ] } , { "Name" : "Filter" , "Docs" : "" , "Typewords" : [ "Filter" ] } , { "Name" : "NotFilter" , "Docs" : "" , "Typewords" : [ "NotFilter" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Filter" : { "Name" : "Filter" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MailboxChildrenIncluded" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MailboxName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Words" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "From" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "To" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Oldest" , "Docs" : "" , "Typewords" : [ "nullable" , "timestamp" ] } , { "Name" : "Newest" , "Docs" : "" , "Typewords" : [ "nullable" , "timestamp" ] } , { "Name" : "Subject" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Attachments" , "Docs" : "" , "Typewords" : [ "AttachmentType" ] } , { "Name" : "Labels" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Headers" , "Docs" : "" , "Typewords" : [ "[]" , "[]" , "string" ] } , { "Name" : "SizeMin" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "SizeMax" , "Docs" : "" , "Typewords" : [ "int64" ] } ] } ,
"NotFilter" : { "Name" : "NotFilter" , "Docs" : "" , "Fields" : [ { "Name" : "Words" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "From" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "To" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Subject" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Attachments" , "Docs" : "" , "Typewords" : [ "AttachmentType" ] } , { "Name" : "Labels" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } ] } ,
"Page" : { "Name" : "Page" , "Docs" : "" , "Fields" : [ { "Name" : "AnchorMessageID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Count" , "Docs" : "" , "Typewords" : [ "int32" ] } , { "Name" : "DestMessageID" , "Docs" : "" , "Typewords" : [ "int64" ] } ] } ,
"ParsedMessage" : { "Name" : "ParsedMessage" , "Docs" : "" , "Fields" : [ { "Name" : "ID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Part" , "Docs" : "" , "Typewords" : [ "Part" ] } , { "Name" : "Headers" , "Docs" : "" , "Typewords" : [ "{}" , "[]" , "string" ] } , { "Name" : "Texts" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "HasHTML" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "ListReplyAddress" , "Docs" : "" , "Typewords" : [ "nullable" , "MessageAddress" ] } ] } ,
"Part" : { "Name" : "Part" , "Docs" : "" , "Fields" : [ { "Name" : "BoundaryOffset" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "HeaderOffset" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "BodyOffset" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "EndOffset" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "RawLineCount" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "DecodedSize" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MediaType" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MediaSubType" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "ContentTypeParams" , "Docs" : "" , "Typewords" : [ "{}" , "string" ] } , { "Name" : "ContentID" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "ContentDescription" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "ContentTransferEncoding" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Envelope" , "Docs" : "" , "Typewords" : [ "nullable" , "Envelope" ] } , { "Name" : "Parts" , "Docs" : "" , "Typewords" : [ "[]" , "Part" ] } , { "Name" : "Message" , "Docs" : "" , "Typewords" : [ "nullable" , "Part" ] } ] } ,
"Envelope" : { "Name" : "Envelope" , "Docs" : "" , "Fields" : [ { "Name" : "Date" , "Docs" : "" , "Typewords" : [ "timestamp" ] } , { "Name" : "Subject" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "From" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "Sender" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "ReplyTo" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "To" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "CC" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "BCC" , "Docs" : "" , "Typewords" : [ "[]" , "Address" ] } , { "Name" : "InReplyTo" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MessageID" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"Address" : { "Name" : "Address" , "Docs" : "" , "Fields" : [ { "Name" : "Name" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "User" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Host" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"MessageAddress" : { "Name" : "MessageAddress" , "Docs" : "" , "Fields" : [ { "Name" : "Name" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "User" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Domain" , "Docs" : "" , "Typewords" : [ "Domain" ] } ] } ,
"Domain" : { "Name" : "Domain" , "Docs" : "" , "Fields" : [ { "Name" : "ASCII" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Unicode" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
implement "requiretls", rfc 8689
with requiretls, the tls verification mode/rules for email deliveries can be
changed by the sender/submitter. in two ways:
1. "requiretls" smtp extension to always enforce verified tls (with mta-sts or
dnssec+dane), along the entire delivery path until delivery into the final
destination mailbox (so entire transport is verified-tls-protected).
2. "tls-required: no" message header, to ignore any tls and tls verification
errors even if the recipient domain has a policy that requires tls verification
(mta-sts and/or dnssec+dane), allowing delivery of non-sensitive messages in
case of misconfiguration/interoperability issues (at least useful for sending
tls reports).
we enable requiretls by default (only when tls is active), for smtp and
submission. it can be disabled through the config.
for each delivery attempt, we now store (per recipient domain, in the account
of the sender) whether the smtp server supports starttls and requiretls. this
support is shown (after having sent a first message) in the webmail when
sending a message (the previous 3 bars under the address input field are now 5
bars, the first for starttls support, the last for requiretls support). when
all recipient domains for a message are known to implement requiretls,
requiretls is automatically selected for sending (instead of "default" tls
behaviour). users can also select the "fallback to insecure" to add the
"tls-required: no" header.
new metrics are added for insight into requiretls errors and (some, not yet
all) cases where tls-required-no ignored a tls/verification error.
the admin can change the requiretls status for messages in the queue. so with
default delivery attempts, when verified tls is required by failing, an admin
could potentially change the field to "tls-required: no"-behaviour.
messages received (over smtp) with the requiretls option, get a comment added
to their Received header line, just before "id", after "with".
2023-10-24 11:06:16 +03:00
"SubmitMessage" : { "Name" : "SubmitMessage" , "Docs" : "" , "Fields" : [ { "Name" : "From" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "To" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Cc" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Bcc" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Subject" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "TextBody" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Attachments" , "Docs" : "" , "Typewords" : [ "[]" , "File" ] } , { "Name" : "ForwardAttachments" , "Docs" : "" , "Typewords" : [ "ForwardAttachments" ] } , { "Name" : "IsForward" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "ResponseMessageID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "ReplyTo" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "UserAgent" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "RequireTLS" , "Docs" : "" , "Typewords" : [ "nullable" , "bool" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"File" : { "Name" : "File" , "Docs" : "" , "Fields" : [ { "Name" : "Filename" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "DataURI" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"ForwardAttachments" : { "Name" : "ForwardAttachments" , "Docs" : "" , "Fields" : [ { "Name" : "MessageID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Paths" , "Docs" : "" , "Typewords" : [ "[]" , "[]" , "int32" ] } ] } ,
"Mailbox" : { "Name" : "Mailbox" , "Docs" : "" , "Fields" : [ { "Name" : "ID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Name" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "UIDValidity" , "Docs" : "" , "Typewords" : [ "uint32" ] } , { "Name" : "UIDNext" , "Docs" : "" , "Typewords" : [ "UID" ] } , { "Name" : "Archive" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Draft" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Junk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Sent" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Trash" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Keywords" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "HaveCounts" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Total" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Deleted" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Unread" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Unseen" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Size" , "Docs" : "" , "Typewords" : [ "int64" ] } ] } ,
implement "requiretls", rfc 8689
with requiretls, the tls verification mode/rules for email deliveries can be
changed by the sender/submitter. in two ways:
1. "requiretls" smtp extension to always enforce verified tls (with mta-sts or
dnssec+dane), along the entire delivery path until delivery into the final
destination mailbox (so entire transport is verified-tls-protected).
2. "tls-required: no" message header, to ignore any tls and tls verification
errors even if the recipient domain has a policy that requires tls verification
(mta-sts and/or dnssec+dane), allowing delivery of non-sensitive messages in
case of misconfiguration/interoperability issues (at least useful for sending
tls reports).
we enable requiretls by default (only when tls is active), for smtp and
submission. it can be disabled through the config.
for each delivery attempt, we now store (per recipient domain, in the account
of the sender) whether the smtp server supports starttls and requiretls. this
support is shown (after having sent a first message) in the webmail when
sending a message (the previous 3 bars under the address input field are now 5
bars, the first for starttls support, the last for requiretls support). when
all recipient domains for a message are known to implement requiretls,
requiretls is automatically selected for sending (instead of "default" tls
behaviour). users can also select the "fallback to insecure" to add the
"tls-required: no" header.
new metrics are added for insight into requiretls errors and (some, not yet
all) cases where tls-required-no ignored a tls/verification error.
the admin can change the requiretls status for messages in the queue. so with
default delivery attempts, when verified tls is required by failing, an admin
could potentially change the field to "tls-required: no"-behaviour.
messages received (over smtp) with the requiretls option, get a comment added
to their Received header line, just before "id", after "with".
2023-10-24 11:06:16 +03:00
"RecipientSecurity" : { "Name" : "RecipientSecurity" , "Docs" : "" , "Fields" : [ { "Name" : "STARTTLS" , "Docs" : "" , "Typewords" : [ "SecurityResult" ] } , { "Name" : "MTASTS" , "Docs" : "" , "Typewords" : [ "SecurityResult" ] } , { "Name" : "DNSSEC" , "Docs" : "" , "Typewords" : [ "SecurityResult" ] } , { "Name" : "DANE" , "Docs" : "" , "Typewords" : [ "SecurityResult" ] } , { "Name" : "RequireTLS" , "Docs" : "" , "Typewords" : [ "SecurityResult" ] } ] } ,
2023-11-27 10:02:01 +03:00
"EventStart" : { "Name" : "EventStart" , "Docs" : "" , "Fields" : [ { "Name" : "SSEID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "LoginAddress" , "Docs" : "" , "Typewords" : [ "MessageAddress" ] } , { "Name" : "Addresses" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "DomainAddressConfigs" , "Docs" : "" , "Typewords" : [ "{}" , "DomainAddressConfig" ] } , { "Name" : "MailboxName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Mailboxes" , "Docs" : "" , "Typewords" : [ "[]" , "Mailbox" ] } , { "Name" : "RejectsMailbox" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Version" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"DomainAddressConfig" : { "Name" : "DomainAddressConfig" , "Docs" : "" , "Fields" : [ { "Name" : "LocalpartCatchallSeparator" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "LocalpartCaseSensitive" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
"EventViewErr" : { "Name" : "EventViewErr" , "Docs" : "" , "Fields" : [ { "Name" : "ViewID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "RequestID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Err" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"EventViewReset" : { "Name" : "EventViewReset" , "Docs" : "" , "Fields" : [ { "Name" : "ViewID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "RequestID" , "Docs" : "" , "Typewords" : [ "int64" ] } ] } ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"EventViewMsgs" : { "Name" : "EventViewMsgs" , "Docs" : "" , "Fields" : [ { "Name" : "ViewID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "RequestID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MessageItems" , "Docs" : "" , "Typewords" : [ "[]" , "[]" , "MessageItem" ] } , { "Name" : "ParsedMessage" , "Docs" : "" , "Typewords" : [ "nullable" , "ParsedMessage" ] } , { "Name" : "ViewEnd" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
"MessageItem" : { "Name" : "MessageItem" , "Docs" : "" , "Fields" : [ { "Name" : "Message" , "Docs" : "" , "Typewords" : [ "Message" ] } , { "Name" : "Envelope" , "Docs" : "" , "Typewords" : [ "MessageEnvelope" ] } , { "Name" : "Attachments" , "Docs" : "" , "Typewords" : [ "[]" , "Attachment" ] } , { "Name" : "IsSigned" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "IsEncrypted" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "FirstLine" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MatchQuery" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
2023-11-02 22:03:47 +03:00
"Message" : { "Name" : "Message" , "Docs" : "" , "Fields" : [ { "Name" : "ID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "UID" , "Docs" : "" , "Typewords" : [ "UID" ] } , { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "ModSeq" , "Docs" : "" , "Typewords" : [ "ModSeq" ] } , { "Name" : "CreateSeq" , "Docs" : "" , "Typewords" : [ "ModSeq" ] } , { "Name" : "Expunged" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "IsReject" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "IsForward" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MailboxOrigID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MailboxDestinedID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Received" , "Docs" : "" , "Typewords" : [ "timestamp" ] } , { "Name" : "RemoteIP" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "RemoteIPMasked1" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "RemoteIPMasked2" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "RemoteIPMasked3" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "EHLODomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MailFrom" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MailFromLocalpart" , "Docs" : "" , "Typewords" : [ "Localpart" ] } , { "Name" : "MailFromDomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "RcptToLocalpart" , "Docs" : "" , "Typewords" : [ "Localpart" ] } , { "Name" : "RcptToDomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MsgFromLocalpart" , "Docs" : "" , "Typewords" : [ "Localpart" ] } , { "Name" : "MsgFromDomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MsgFromOrgDomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "EHLOValidated" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MailFromValidated" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MsgFromValidated" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "EHLOValidation" , "Docs" : "" , "Typewords" : [ "Validation" ] } , { "Name" : "MailFromValidation" , "Docs" : "" , "Typewords" : [ "Validation" ] } , { "Name" : "MsgFromValidation" , "Docs" : "" , "Typewords" : [ "Validation" ] } , { "Name" : "DKIMDomains" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "OrigEHLODomain" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "OrigDKIMDomains" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "MessageID" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "SubjectBase" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MessageHash" , "Docs" : "" , "Typewords" : [ "nullable" , "string" ] } , { "Name" : "ThreadID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "ThreadParentIDs" , "Docs" : "" , "Typewords" : [ "[]" , "int64" ] } , { "Name" : "ThreadMissingLink" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "ThreadMuted" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "ThreadCollapsed" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "IsMailingList" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "ReceivedTLSVersion" , "Docs" : "" , "Typewords" : [ "uint16" ] } , { "Name" : "ReceivedTLSCipherSuite" , "Docs" : "" , "Typewords" : [ "uint16" ] } , { "Name" : "ReceivedRequireTLS" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Seen" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Answered" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Flagged" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Forwarded" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Junk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Notjunk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Deleted" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Draft" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Phishing" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MDNSent" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Keywords" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "Size" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "TrainedJunk" , "Docs" : "" , "Typewords" : [ "nullable" , "bool" ] } , { "Name" : "MsgPrefix" , "Docs" : "" , "Typewords" : [ "nullable" , "string" ] } , { "Name" : "ParsedBuf" , "Docs" : "" , "Typewords" : [ "nullable" , "string" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"MessageEnvelope" : { "Name" : "MessageEnvelope" , "Docs" : "" , "Fields" : [ { "Name" : "Date" , "Docs" : "" , "Typewords" : [ "timestamp" ] } , { "Name" : "Subject" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "From" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "Sender" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "ReplyTo" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "To" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "CC" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "BCC" , "Docs" : "" , "Typewords" : [ "[]" , "MessageAddress" ] } , { "Name" : "InReplyTo" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "MessageID" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"Attachment" : { "Name" : "Attachment" , "Docs" : "" , "Fields" : [ { "Name" : "Path" , "Docs" : "" , "Typewords" : [ "[]" , "int32" ] } , { "Name" : "Filename" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Part" , "Docs" : "" , "Typewords" : [ "Part" ] } ] } ,
"EventViewChanges" : { "Name" : "EventViewChanges" , "Docs" : "" , "Fields" : [ { "Name" : "ViewID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Changes" , "Docs" : "" , "Typewords" : [ "[]" , "[]" , "any" ] } ] } ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"ChangeMsgAdd" : { "Name" : "ChangeMsgAdd" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "UID" , "Docs" : "" , "Typewords" : [ "UID" ] } , { "Name" : "ModSeq" , "Docs" : "" , "Typewords" : [ "ModSeq" ] } , { "Name" : "Flags" , "Docs" : "" , "Typewords" : [ "Flags" ] } , { "Name" : "Keywords" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } , { "Name" : "MessageItems" , "Docs" : "" , "Typewords" : [ "[]" , "MessageItem" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Flags" : { "Name" : "Flags" , "Docs" : "" , "Fields" : [ { "Name" : "Seen" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Answered" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Flagged" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Forwarded" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Junk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Notjunk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Deleted" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Draft" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Phishing" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "MDNSent" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
"ChangeMsgRemove" : { "Name" : "ChangeMsgRemove" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "UIDs" , "Docs" : "" , "Typewords" : [ "[]" , "UID" ] } , { "Name" : "ModSeq" , "Docs" : "" , "Typewords" : [ "ModSeq" ] } ] } ,
"ChangeMsgFlags" : { "Name" : "ChangeMsgFlags" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "UID" , "Docs" : "" , "Typewords" : [ "UID" ] } , { "Name" : "ModSeq" , "Docs" : "" , "Typewords" : [ "ModSeq" ] } , { "Name" : "Mask" , "Docs" : "" , "Typewords" : [ "Flags" ] } , { "Name" : "Flags" , "Docs" : "" , "Typewords" : [ "Flags" ] } , { "Name" : "Keywords" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } ] } ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"ChangeMsgThread" : { "Name" : "ChangeMsgThread" , "Docs" : "" , "Fields" : [ { "Name" : "MessageIDs" , "Docs" : "" , "Typewords" : [ "[]" , "int64" ] } , { "Name" : "Muted" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Collapsed" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"ChangeMailboxRemove" : { "Name" : "ChangeMailboxRemove" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Name" , "Docs" : "" , "Typewords" : [ "string" ] } ] } ,
"ChangeMailboxAdd" : { "Name" : "ChangeMailboxAdd" , "Docs" : "" , "Fields" : [ { "Name" : "Mailbox" , "Docs" : "" , "Typewords" : [ "Mailbox" ] } ] } ,
"ChangeMailboxRename" : { "Name" : "ChangeMailboxRename" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "OldName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "NewName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Flags" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } ] } ,
"ChangeMailboxCounts" : { "Name" : "ChangeMailboxCounts" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MailboxName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Total" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Deleted" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Unread" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Unseen" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "Size" , "Docs" : "" , "Typewords" : [ "int64" ] } ] } ,
"ChangeMailboxSpecialUse" : { "Name" : "ChangeMailboxSpecialUse" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MailboxName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "SpecialUse" , "Docs" : "" , "Typewords" : [ "SpecialUse" ] } ] } ,
"SpecialUse" : { "Name" : "SpecialUse" , "Docs" : "" , "Fields" : [ { "Name" : "Archive" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Draft" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Junk" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Sent" , "Docs" : "" , "Typewords" : [ "bool" ] } , { "Name" : "Trash" , "Docs" : "" , "Typewords" : [ "bool" ] } ] } ,
"ChangeMailboxKeywords" : { "Name" : "ChangeMailboxKeywords" , "Docs" : "" , "Fields" : [ { "Name" : "MailboxID" , "Docs" : "" , "Typewords" : [ "int64" ] } , { "Name" : "MailboxName" , "Docs" : "" , "Typewords" : [ "string" ] } , { "Name" : "Keywords" , "Docs" : "" , "Typewords" : [ "[]" , "string" ] } ] } ,
"UID" : { "Name" : "UID" , "Docs" : "" , "Values" : null } ,
"ModSeq" : { "Name" : "ModSeq" , "Docs" : "" , "Values" : null } ,
"Validation" : { "Name" : "Validation" , "Docs" : "" , "Values" : [ { "Name" : "ValidationUnknown" , "Value" : 0 , "Docs" : "" } , { "Name" : "ValidationStrict" , "Value" : 1 , "Docs" : "" } , { "Name" : "ValidationDMARC" , "Value" : 2 , "Docs" : "" } , { "Name" : "ValidationRelaxed" , "Value" : 3 , "Docs" : "" } , { "Name" : "ValidationPass" , "Value" : 4 , "Docs" : "" } , { "Name" : "ValidationNeutral" , "Value" : 5 , "Docs" : "" } , { "Name" : "ValidationTemperror" , "Value" : 6 , "Docs" : "" } , { "Name" : "ValidationPermerror" , "Value" : 7 , "Docs" : "" } , { "Name" : "ValidationFail" , "Value" : 8 , "Docs" : "" } , { "Name" : "ValidationSoftfail" , "Value" : 9 , "Docs" : "" } , { "Name" : "ValidationNone" , "Value" : 10 , "Docs" : "" } ] } ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
"ThreadMode" : { "Name" : "ThreadMode" , "Docs" : "" , "Values" : [ { "Name" : "ThreadOff" , "Value" : "off" , "Docs" : "" } , { "Name" : "ThreadOn" , "Value" : "on" , "Docs" : "" } , { "Name" : "ThreadUnread" , "Value" : "unread" , "Docs" : "" } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"AttachmentType" : { "Name" : "AttachmentType" , "Docs" : "" , "Values" : [ { "Name" : "AttachmentIndifferent" , "Value" : "" , "Docs" : "" } , { "Name" : "AttachmentNone" , "Value" : "none" , "Docs" : "" } , { "Name" : "AttachmentAny" , "Value" : "any" , "Docs" : "" } , { "Name" : "AttachmentImage" , "Value" : "image" , "Docs" : "" } , { "Name" : "AttachmentPDF" , "Value" : "pdf" , "Docs" : "" } , { "Name" : "AttachmentArchive" , "Value" : "archive" , "Docs" : "" } , { "Name" : "AttachmentSpreadsheet" , "Value" : "spreadsheet" , "Docs" : "" } , { "Name" : "AttachmentDocument" , "Value" : "document" , "Docs" : "" } , { "Name" : "AttachmentPresentation" , "Value" : "presentation" , "Docs" : "" } ] } ,
2023-10-15 16:05:20 +03:00
"SecurityResult" : { "Name" : "SecurityResult" , "Docs" : "" , "Values" : [ { "Name" : "SecurityResultError" , "Value" : "error" , "Docs" : "" } , { "Name" : "SecurityResultNo" , "Value" : "no" , "Docs" : "" } , { "Name" : "SecurityResultYes" , "Value" : "yes" , "Docs" : "" } , { "Name" : "SecurityResultUnknown" , "Value" : "unknown" , "Docs" : "" } ] } ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
"Localpart" : { "Name" : "Localpart" , "Docs" : "" , "Values" : null } ,
} ;
api . parser = {
Request : ( v ) => api . parse ( "Request" , v ) ,
Query : ( v ) => api . parse ( "Query" , v ) ,
Filter : ( v ) => api . parse ( "Filter" , v ) ,
NotFilter : ( v ) => api . parse ( "NotFilter" , v ) ,
Page : ( v ) => api . parse ( "Page" , v ) ,
ParsedMessage : ( v ) => api . parse ( "ParsedMessage" , v ) ,
Part : ( v ) => api . parse ( "Part" , v ) ,
Envelope : ( v ) => api . parse ( "Envelope" , v ) ,
Address : ( v ) => api . parse ( "Address" , v ) ,
MessageAddress : ( v ) => api . parse ( "MessageAddress" , v ) ,
Domain : ( v ) => api . parse ( "Domain" , v ) ,
SubmitMessage : ( v ) => api . parse ( "SubmitMessage" , v ) ,
File : ( v ) => api . parse ( "File" , v ) ,
ForwardAttachments : ( v ) => api . parse ( "ForwardAttachments" , v ) ,
Mailbox : ( v ) => api . parse ( "Mailbox" , v ) ,
2023-10-15 16:05:20 +03:00
RecipientSecurity : ( v ) => api . parse ( "RecipientSecurity" , v ) ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
EventStart : ( v ) => api . parse ( "EventStart" , v ) ,
DomainAddressConfig : ( v ) => api . parse ( "DomainAddressConfig" , v ) ,
EventViewErr : ( v ) => api . parse ( "EventViewErr" , v ) ,
EventViewReset : ( v ) => api . parse ( "EventViewReset" , v ) ,
EventViewMsgs : ( v ) => api . parse ( "EventViewMsgs" , v ) ,
MessageItem : ( v ) => api . parse ( "MessageItem" , v ) ,
Message : ( v ) => api . parse ( "Message" , v ) ,
MessageEnvelope : ( v ) => api . parse ( "MessageEnvelope" , v ) ,
Attachment : ( v ) => api . parse ( "Attachment" , v ) ,
EventViewChanges : ( v ) => api . parse ( "EventViewChanges" , v ) ,
ChangeMsgAdd : ( v ) => api . parse ( "ChangeMsgAdd" , v ) ,
Flags : ( v ) => api . parse ( "Flags" , v ) ,
ChangeMsgRemove : ( v ) => api . parse ( "ChangeMsgRemove" , v ) ,
ChangeMsgFlags : ( v ) => api . parse ( "ChangeMsgFlags" , v ) ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
ChangeMsgThread : ( v ) => api . parse ( "ChangeMsgThread" , v ) ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
ChangeMailboxRemove : ( v ) => api . parse ( "ChangeMailboxRemove" , v ) ,
ChangeMailboxAdd : ( v ) => api . parse ( "ChangeMailboxAdd" , v ) ,
ChangeMailboxRename : ( v ) => api . parse ( "ChangeMailboxRename" , v ) ,
ChangeMailboxCounts : ( v ) => api . parse ( "ChangeMailboxCounts" , v ) ,
ChangeMailboxSpecialUse : ( v ) => api . parse ( "ChangeMailboxSpecialUse" , v ) ,
SpecialUse : ( v ) => api . parse ( "SpecialUse" , v ) ,
ChangeMailboxKeywords : ( v ) => api . parse ( "ChangeMailboxKeywords" , v ) ,
UID : ( v ) => api . parse ( "UID" , v ) ,
ModSeq : ( v ) => api . parse ( "ModSeq" , v ) ,
Validation : ( v ) => api . parse ( "Validation" , v ) ,
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
ThreadMode : ( v ) => api . parse ( "ThreadMode" , v ) ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
AttachmentType : ( v ) => api . parse ( "AttachmentType" , v ) ,
2023-10-15 16:05:20 +03:00
SecurityResult : ( v ) => api . parse ( "SecurityResult" , v ) ,
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
Localpart : ( v ) => api . parse ( "Localpart" , v ) ,
} ;
let defaultOptions = { slicesNullable : true , mapsNullable : true , nullableOptional : true } ;
class Client {
2024-01-01 16:13:05 +03:00
baseURL ;
options ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
constructor ( baseURL = api . defaultBaseURL , options ) {
this . baseURL = baseURL ;
this . options = options ;
if ( ! options ) {
this . options = defaultOptions ;
}
}
withOptions ( options ) {
return new Client ( this . baseURL , { ... this . options , ... options } ) ;
}
// Token returns a token to use for an SSE connection. A token can only be used for
// a single SSE connection. Tokens are stored in memory for a maximum of 1 minute,
// with at most 10 unused tokens (the most recently created) per account.
async Token ( ) {
const fn = "Token" ;
const paramTypes = [ ] ;
const returnTypes = [ [ "string" ] ] ;
const params = [ ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// Requests sends a new request for an open SSE connection. Any currently active
// request for the connection will be canceled, but this is done asynchrously, so
// the SSE connection may still send results for the previous request. Callers
// should take care to ignore such results. If req.Cancel is set, no new request is
// started.
async Request ( req ) {
const fn = "Request" ;
const paramTypes = [ [ "Request" ] ] ;
const returnTypes = [ ] ;
const params = [ req ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// ParsedMessage returns enough to render the textual body of a message. It is
// assumed the client already has other fields through MessageItem.
async ParsedMessage ( msgID ) {
const fn = "ParsedMessage" ;
const paramTypes = [ [ "int64" ] ] ;
const returnTypes = [ [ "ParsedMessage" ] ] ;
const params = [ msgID ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MessageSubmit sends a message by submitting it the outgoing email queue. The
// message is sent to all addresses listed in the To, Cc and Bcc addresses, without
// Bcc message header.
//
// If a Sent mailbox is configured, messages are added to it after submitting
// to the delivery queue.
async MessageSubmit ( m ) {
const fn = "MessageSubmit" ;
const paramTypes = [ [ "SubmitMessage" ] ] ;
const returnTypes = [ ] ;
const params = [ m ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MessageMove moves messages to another mailbox. If the message is already in
// the mailbox an error is returned.
async MessageMove ( messageIDs , mailboxID ) {
const fn = "MessageMove" ;
const paramTypes = [ [ "[]" , "int64" ] , [ "int64" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs , mailboxID ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MessageDelete permanently deletes messages, without moving them to the Trash mailbox.
async MessageDelete ( messageIDs ) {
const fn = "MessageDelete" ;
const paramTypes = [ [ "[]" , "int64" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// FlagsAdd adds flags, either system flags like \Seen or custom keywords. The
// flags should be lower-case, but will be converted and verified.
async FlagsAdd ( messageIDs , flaglist ) {
const fn = "FlagsAdd" ;
const paramTypes = [ [ "[]" , "int64" ] , [ "[]" , "string" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs , flaglist ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// FlagsClear clears flags, either system flags like \Seen or custom keywords.
async FlagsClear ( messageIDs , flaglist ) {
const fn = "FlagsClear" ;
const paramTypes = [ [ "[]" , "int64" ] , [ "[]" , "string" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs , flaglist ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MailboxCreate creates a new mailbox.
async MailboxCreate ( name ) {
const fn = "MailboxCreate" ;
const paramTypes = [ [ "string" ] ] ;
const returnTypes = [ ] ;
const params = [ name ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MailboxDelete deletes a mailbox and all its messages.
async MailboxDelete ( mailboxID ) {
const fn = "MailboxDelete" ;
const paramTypes = [ [ "int64" ] ] ;
const returnTypes = [ ] ;
const params = [ mailboxID ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MailboxEmpty empties a mailbox, removing all messages from the mailbox, but not
// its child mailboxes.
async MailboxEmpty ( mailboxID ) {
const fn = "MailboxEmpty" ;
const paramTypes = [ [ "int64" ] ] ;
const returnTypes = [ ] ;
const params = [ mailboxID ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MailboxRename renames a mailbox, possibly moving it to a new parent. The mailbox
// ID and its messages are unchanged.
async MailboxRename ( mailboxID , newName ) {
const fn = "MailboxRename" ;
const paramTypes = [ [ "int64" ] , [ "string" ] ] ;
const returnTypes = [ ] ;
const params = [ mailboxID , newName ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// CompleteRecipient returns autocomplete matches for a recipient, returning the
// matches, most recently used first, and whether this is the full list and further
// requests for longer prefixes aren't necessary.
async CompleteRecipient ( search ) {
const fn = "CompleteRecipient" ;
const paramTypes = [ [ "string" ] ] ;
const returnTypes = [ [ "[]" , "string" ] , [ "bool" ] ] ;
const params = [ search ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// MailboxSetSpecialUse sets the special use flags of a mailbox.
async MailboxSetSpecialUse ( mb ) {
const fn = "MailboxSetSpecialUse" ;
const paramTypes = [ [ "Mailbox" ] ] ;
const returnTypes = [ ] ;
const params = [ mb ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
// ThreadCollapse saves the ThreadCollapse field for the messages and its
// children. The messageIDs are typically thread roots. But not all roots
// (without parent) of a thread need to have the same collapsed state.
async ThreadCollapse ( messageIDs , collapse ) {
const fn = "ThreadCollapse" ;
const paramTypes = [ [ "[]" , "int64" ] , [ "bool" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs , collapse ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
// ThreadMute saves the ThreadMute field for the messages and their children.
// If messages are muted, they are also marked collapsed.
async ThreadMute ( messageIDs , mute ) {
const fn = "ThreadMute" ;
const paramTypes = [ [ "[]" , "int64" ] , [ "bool" ] ] ;
const returnTypes = [ ] ;
const params = [ messageIDs , mute ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
2023-10-15 16:05:20 +03:00
// RecipientSecurity looks up security properties of the address in the
// single-address message addressee (as it appears in a To/Cc/Bcc/etc header).
async RecipientSecurity ( messageAddressee ) {
const fn = "RecipientSecurity" ;
const paramTypes = [ [ "string" ] ] ;
const returnTypes = [ [ "RecipientSecurity" ] ] ;
const params = [ messageAddressee ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
// SSETypes exists to ensure the generated API contains the types, for use in SSE events.
async SSETypes ( ) {
const fn = "SSETypes" ;
const paramTypes = [ ] ;
implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).
we store a threadid with messages. all messages in a thread have the same
threadid. messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root. then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".
threads can be muted: newly delivered messages are automatically marked as
read/seen. threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads). the muted and collapsed fields are copied from their parent on
message delivery.
the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.
the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).
the threading is not yet exposed over imap.
2023-09-13 09:51:50 +03:00
const returnTypes = [ [ "EventStart" ] , [ "EventViewErr" ] , [ "EventViewReset" ] , [ "EventViewMsgs" ] , [ "EventViewChanges" ] , [ "ChangeMsgAdd" ] , [ "ChangeMsgRemove" ] , [ "ChangeMsgFlags" ] , [ "ChangeMsgThread" ] , [ "ChangeMailboxRemove" ] , [ "ChangeMailboxAdd" ] , [ "ChangeMailboxRename" ] , [ "ChangeMailboxCounts" ] , [ "ChangeMailboxSpecialUse" ] , [ "ChangeMailboxKeywords" ] , [ "Flags" ] ] ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
const params = [ ] ;
return await _sherpaCall ( this . baseURL , { ... this . options } , paramTypes , returnTypes , fn , params ) ;
}
}
api . Client = Client ;
api . defaultBaseURL = ( function ( ) {
let p = location . pathname ;
if ( p && p [ p . length - 1 ] !== '/' ) {
let l = location . pathname . split ( '/' ) ;
l = l . slice ( 0 , l . length - 1 ) ;
p = '/' + l . join ( '/' ) + '/' ;
}
return location . protocol + '//' + location . host + p + 'api/' ;
} ) ( ) ;
// NOTE: code below is shared between github.com/mjl-/sherpaweb and github.com/mjl-/sherpats.
// KEEP IN SYNC.
api . supportedSherpaVersion = 1 ;
// verifyArg typechecks "v" against "typewords", returning a new (possibly modified) value for JSON-encoding.
// toJS indicate if the data is coming into JS. If so, timestamps are turned into JS Dates. Otherwise, JS Dates are turned into strings.
// allowUnknownKeys configures whether unknown keys in structs are allowed.
// types are the named types of the API.
api . verifyArg = ( path , v , typewords , toJS , allowUnknownKeys , types , opts ) => {
return new verifier ( types , toJS , allowUnknownKeys , opts ) . verify ( path , v , typewords ) ;
} ;
api . parse = ( name , v ) => api . verifyArg ( name , v , [ name ] , true , false , api . types , defaultOptions ) ;
class verifier {
2024-01-01 16:13:05 +03:00
types ;
toJS ;
allowUnknownKeys ;
opts ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
constructor ( types , toJS , allowUnknownKeys , opts ) {
this . types = types ;
this . toJS = toJS ;
this . allowUnknownKeys = allowUnknownKeys ;
this . opts = opts ;
}
verify ( path , v , typewords ) {
typewords = typewords . slice ( 0 ) ;
const ww = typewords . shift ( ) ;
const error = ( msg ) => {
if ( path != '' ) {
msg = path + ': ' + msg ;
}
throw new Error ( msg ) ;
} ;
if ( typeof ww !== 'string' ) {
error ( 'bad typewords' ) ;
return ; // should not be necessary, typescript doesn't see error always throws an exception?
}
const w = ww ;
const ensure = ( ok , expect ) => {
if ( ! ok ) {
error ( 'got ' + JSON . stringify ( v ) + ', expected ' + expect ) ;
}
return v ;
} ;
switch ( w ) {
case 'nullable' :
if ( v === null || v === undefined && this . opts . nullableOptional ) {
return v ;
}
return this . verify ( path , v , typewords ) ;
case '[]' :
if ( v === null && this . opts . slicesNullable || v === undefined && this . opts . slicesNullable && this . opts . nullableOptional ) {
return v ;
}
ensure ( Array . isArray ( v ) , "array" ) ;
return v . map ( ( e , i ) => this . verify ( path + '[' + i + ']' , e , typewords ) ) ;
case '{}' :
if ( v === null && this . opts . mapsNullable || v === undefined && this . opts . mapsNullable && this . opts . nullableOptional ) {
return v ;
}
ensure ( v !== null || typeof v === 'object' , "object" ) ;
const r = { } ;
for ( const k in v ) {
r [ k ] = this . verify ( path + '.' + k , v [ k ] , typewords ) ;
}
return r ;
}
ensure ( typewords . length == 0 , "empty typewords" ) ;
const t = typeof v ;
switch ( w ) {
case 'any' :
return v ;
case 'bool' :
ensure ( t === 'boolean' , 'bool' ) ;
return v ;
case 'int8' :
case 'uint8' :
case 'int16' :
case 'uint16' :
case 'int32' :
case 'uint32' :
case 'int64' :
case 'uint64' :
ensure ( t === 'number' && Number . isInteger ( v ) , 'integer' ) ;
return v ;
case 'float32' :
case 'float64' :
ensure ( t === 'number' , 'float' ) ;
return v ;
case 'int64s' :
case 'uint64s' :
ensure ( t === 'number' && Number . isInteger ( v ) || t === 'string' , 'integer fitting in float without precision loss, or string' ) ;
return '' + v ;
case 'string' :
ensure ( t === 'string' , 'string' ) ;
return v ;
case 'timestamp' :
if ( this . toJS ) {
ensure ( t === 'string' , 'string, with timestamp' ) ;
const d = new Date ( v ) ;
if ( d instanceof Date && ! isNaN ( d . getTime ( ) ) ) {
return d ;
}
error ( 'invalid date ' + v ) ;
}
else {
ensure ( t === 'object' && v !== null , 'non-null object' ) ;
ensure ( v . _ _proto _ _ === Date . prototype , 'Date' ) ;
return v . toISOString ( ) ;
}
}
// We're left with named types.
const nt = this . types [ w ] ;
if ( ! nt ) {
error ( 'unknown type ' + w ) ;
}
if ( v === null ) {
error ( 'bad value ' + v + ' for named type ' + w ) ;
}
if ( api . structTypes [ nt . Name ] ) {
const t = nt ;
if ( typeof v !== 'object' ) {
error ( 'bad value ' + v + ' for struct ' + w ) ;
}
const r = { } ;
for ( const f of t . Fields ) {
r [ f . Name ] = this . verify ( path + '.' + f . Name , v [ f . Name ] , f . Typewords ) ;
}
// If going to JSON also verify no unknown fields are present.
if ( ! this . allowUnknownKeys ) {
const known = { } ;
for ( const f of t . Fields ) {
known [ f . Name ] = true ;
}
Object . keys ( v ) . forEach ( ( k ) => {
if ( ! known [ k ] ) {
error ( 'unknown key ' + k + ' for struct ' + w ) ;
}
} ) ;
}
return r ;
}
else if ( api . stringsTypes [ nt . Name ] ) {
const t = nt ;
if ( typeof v !== 'string' ) {
error ( 'mistyped value ' + v + ' for named strings ' + t . Name ) ;
}
if ( ! t . Values || t . Values . length === 0 ) {
return v ;
}
for ( const sv of t . Values ) {
if ( sv . Value === v ) {
return v ;
}
}
error ( 'unknkown value ' + v + ' for named strings ' + t . Name ) ;
}
else if ( api . intsTypes [ nt . Name ] ) {
const t = nt ;
if ( typeof v !== 'number' || ! Number . isInteger ( v ) ) {
error ( 'mistyped value ' + v + ' for named ints ' + t . Name ) ;
}
if ( ! t . Values || t . Values . length === 0 ) {
return v ;
}
for ( const sv of t . Values ) {
if ( sv . Value === v ) {
return v ;
}
}
error ( 'unknkown value ' + v + ' for named ints ' + t . Name ) ;
}
else {
throw new Error ( 'unexpected named type ' + nt ) ;
}
}
}
const _sherpaCall = async ( baseURL , options , paramTypes , returnTypes , name , params ) => {
if ( ! options . skipParamCheck ) {
if ( params . length !== paramTypes . length ) {
return Promise . reject ( { message : 'wrong number of parameters in sherpa call, saw ' + params . length + ' != expected ' + paramTypes . length } ) ;
}
params = params . map ( ( v , index ) => api . verifyArg ( 'params[' + index + ']' , v , paramTypes [ index ] , false , false , api . types , options ) ) ;
}
const simulate = async ( json ) => {
const config = JSON . parse ( json || 'null' ) || { } ;
const waitMinMsec = config . waitMinMsec || 0 ;
const waitMaxMsec = config . waitMaxMsec || 0 ;
const wait = Math . random ( ) * ( waitMaxMsec - waitMinMsec ) ;
const failRate = config . failRate || 0 ;
return new Promise ( ( resolve , reject ) => {
if ( options . aborter ) {
options . aborter . abort = ( ) => {
reject ( { message : 'call to ' + name + ' aborted by user' , code : 'sherpa:aborted' } ) ;
reject = resolve = ( ) => { } ;
} ;
}
setTimeout ( ( ) => {
const r = Math . random ( ) ;
if ( r < failRate ) {
reject ( { message : 'injected failure on ' + name , code : 'server:injected' } ) ;
}
else {
resolve ( ) ;
}
reject = resolve = ( ) => { } ;
} , waitMinMsec + wait ) ;
} ) ;
} ;
// Only simulate when there is a debug string. Otherwise it would always interfere
// with setting options.aborter.
let json = '' ;
try {
json = window . localStorage . getItem ( 'sherpats-debug' ) || '' ;
}
catch ( err ) { }
if ( json ) {
await simulate ( json ) ;
}
// Immediately create promise, so options.aborter is changed before returning.
const promise = new Promise ( ( resolve , reject ) => {
let resolve1 = ( v ) => {
resolve ( v ) ;
resolve1 = ( ) => { } ;
reject1 = ( ) => { } ;
} ;
let reject1 = ( v ) => {
reject ( v ) ;
resolve1 = ( ) => { } ;
reject1 = ( ) => { } ;
} ;
const url = baseURL + name ;
const req = new window . XMLHttpRequest ( ) ;
if ( options . aborter ) {
options . aborter . abort = ( ) => {
req . abort ( ) ;
reject1 ( { code : 'sherpa:aborted' , message : 'request aborted' } ) ;
} ;
}
req . open ( 'POST' , url , true ) ;
if ( options . timeoutMsec ) {
req . timeout = options . timeoutMsec ;
}
req . onload = ( ) => {
if ( req . status !== 200 ) {
if ( req . status === 404 ) {
reject1 ( { code : 'sherpa:badFunction' , message : 'function does not exist' } ) ;
}
else {
reject1 ( { code : 'sherpa:http' , message : 'error calling function, HTTP status: ' + req . status } ) ;
}
return ;
}
let resp ;
try {
resp = JSON . parse ( req . responseText ) ;
}
catch ( err ) {
reject1 ( { code : 'sherpa:badResponse' , message : 'bad JSON from server' } ) ;
return ;
}
if ( resp && resp . error ) {
const err = resp . error ;
reject1 ( { code : err . code , message : err . message } ) ;
return ;
}
else if ( ! resp || ! resp . hasOwnProperty ( 'result' ) ) {
reject1 ( { code : 'sherpa:badResponse' , message : "invalid sherpa response object, missing 'result'" } ) ;
return ;
}
if ( options . skipReturnCheck ) {
resolve1 ( resp . result ) ;
return ;
}
let result = resp . result ;
try {
if ( returnTypes . length === 0 ) {
if ( result ) {
throw new Error ( 'function ' + name + ' returned a value while prototype says it returns "void"' ) ;
}
}
else if ( returnTypes . length === 1 ) {
result = api . verifyArg ( 'result' , result , returnTypes [ 0 ] , true , true , api . types , options ) ;
}
else {
if ( result . length != returnTypes . length ) {
throw new Error ( 'wrong number of values returned by ' + name + ', saw ' + result . length + ' != expected ' + returnTypes . length ) ;
}
result = result . map ( ( v , index ) => api . verifyArg ( 'result[' + index + ']' , v , returnTypes [ index ] , true , true , api . types , options ) ) ;
}
}
catch ( err ) {
let errmsg = 'bad types' ;
if ( err instanceof Error ) {
errmsg = err . message ;
}
reject1 ( { code : 'sherpa:badTypes' , message : errmsg } ) ;
}
resolve1 ( result ) ;
} ;
req . onerror = ( ) => {
reject1 ( { code : 'sherpa:connection' , message : 'connection failed' } ) ;
} ;
req . ontimeout = ( ) => {
reject1 ( { code : 'sherpa:timeout' , message : 'request timeout' } ) ;
} ;
req . setRequestHeader ( 'Content-Type' , 'application/json' ) ;
try {
req . send ( JSON . stringify ( { params : params } ) ) ;
}
catch ( err ) {
reject1 ( { code : 'sherpa:badData' , message : 'cannot marshal to JSON' } ) ;
}
} ) ;
return await promise ;
} ;
} ) ( api || ( api = { } ) ) ;
// Javascript is generated from typescript, do not modify generated javascript because changes will be overwritten.
2023-11-27 14:11:05 +03:00
// For authentication/security results.
const underlineGreen = '#50c40f' ;
const underlineRed = '#e15d1c' ;
const underlineBlue = '#09f' ;
const underlineGrey = '#aaa' ;
const underlineYellow = 'yellow' ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
// join elements in l with the results of calls to efn. efn can return
// HTMLElements, which cannot be inserted into the dom multiple times, hence the
// function.
const join = ( l , efn ) => {
const r = [ ] ;
const n = l . length ;
for ( let i = 0 ; i < n ; i ++ ) {
r . push ( l [ i ] ) ;
if ( i < n - 1 ) {
r . push ( efn ( ) ) ;
}
}
return r ;
} ;
// addLinks turns a line of text into alternating strings and links. Links that
// would end with interpunction followed by whitespace are returned with that
// interpunction moved to the next string instead.
const addLinks = ( text ) => {
// todo: look at ../rfc/3986 and fix up regexp. we should probably accept utf-8.
const re = RegExp ( '(http|https):\/\/([:%0-9a-zA-Z._~!$&\'/()*+,;=-]+@)?([\\[\\]0-9a-zA-Z.-]+)(:[0-9]+)?([:@%0-9a-zA-Z._~!$&\'/()*+,;=-]*)(\\?[:@%0-9a-zA-Z._~!$&\'/()*+,;=?-]*)?(#[:@%0-9a-zA-Z._~!$&\'/()*+,;=?-]*)?' ) ;
const r = [ ] ;
while ( text . length > 0 ) {
const l = re . exec ( text ) ;
if ( ! l ) {
r . push ( text ) ;
break ;
}
let s = text . substring ( 0 , l . index ) ;
let url = l [ 0 ] ;
text = text . substring ( l . index + url . length ) ;
r . push ( s ) ;
// If URL ends with interpunction, and next character is whitespace or end, don't
// include the interpunction in the URL.
2023-09-21 12:09:27 +03:00
if ( ! text || /^[ \t\r\n]/ . test ( text ) ) {
if ( /[)>][!,.:;?]$/ . test ( url ) ) {
text = url . substring ( url . length - 2 ) + text ;
url = url . substring ( 0 , url . length - 2 ) ;
}
else if ( /[)>!,.:;?]$/ . test ( url ) ) {
text = url . substring ( url . length - 1 ) + text ;
url = url . substring ( 0 , url . length - 1 ) ;
}
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
}
r . push ( dom . a ( url , attr . href ( url ) , attr . target ( '_blank' ) , attr . rel ( 'noopener noreferrer' ) ) ) ;
}
return r ;
} ;
// renderText turns text into a renderable element with ">" interpreted as quoted
// text (with different levels), and URLs replaced by links.
const renderText = ( text ) => {
return dom . div ( text . split ( '\n' ) . map ( line => {
let q = 0 ;
for ( const c of line ) {
if ( c == '>' ) {
q ++ ;
}
else if ( c !== ' ' ) {
break ;
}
}
if ( q == 0 ) {
return [ addLinks ( line ) , '\n' ] ;
}
q = ( q - 1 ) % 3 + 1 ;
return dom . div ( dom . _class ( 'quoted' + q ) , addLinks ( line ) ) ;
} ) ) ;
} ;
const displayName = ( s ) => {
// ../rfc/5322:1216
// ../rfc/5322:1270
// todo: need support for group addresses (eg "undisclosed recipients").
// ../rfc/5322:697
const specials = /[()<>\[\]:;@\\,."]/ ;
if ( specials . test ( s ) ) {
return '"' + s . replace ( '\\' , '\\\\' ) . replace ( '"' , '\\"' ) + '"' ;
}
return s ;
} ;
// format an address with both name and email address.
const formatAddress = ( a ) => {
let s = '<' + a . User + '@' + a . Domain . ASCII + '>' ;
if ( a . Name ) {
s = displayName ( a . Name ) + ' ' + s ;
}
return s ;
} ;
// returns an address with all available details, including unicode version if
// available.
const formatAddressFull = ( a ) => {
let s = '' ;
if ( a . Name ) {
s = a . Name + ' ' ;
}
s += '<' + a . User + '@' + a . Domain . ASCII + '>' ;
if ( a . Domain . Unicode ) {
s += ' (' + a . User + '@' + a . Domain . Unicode + ')' ;
}
return s ;
} ;
2023-11-27 14:11:05 +03:00
// like formatAddressFull, but underline domain with dmarc-like validation if appropriate.
const formatAddressFullValidated = ( a , m , use ) => {
const domainText = ( s ) => {
if ( ! use ) {
return s ;
}
// We want to show how "approved" this message is given the message From's domain.
// We have MsgFromValidation available. It's not the greatest, being a mix of
// potential strict validations, actual DMARC policy validation, potential relaxed
// validation, but no explicit fail or (temporary) errors. We also don't know if
// historic messages were from a mailing list. We could add a heuristic based on
// List-Id headers, but it would be unreliable...
// todo: add field to Message with the exact results.
let color = '' ;
let title = '' ;
switch ( m . MsgFromValidation ) {
case api . Validation . ValidationStrict :
color = underlineGreen ;
title = 'Message would have matched a strict DMARC policy.' ;
break ;
case api . Validation . ValidationDMARC :
color = underlineGreen ;
title = 'Message matched DMARC policy of domain.' ;
break ;
case api . Validation . ValidationRelaxed :
color = underlineGreen ;
title = 'Domain did not have a DMARC policy, but message would match a relaxed policy if it had existed.' ;
break ;
case api . Validation . ValidationNone :
if ( m . IsForward || m . IsMailingList ) {
color = underlineBlue ;
title = 'Message would not pass DMARC policy, but came in through a configured mailing list or forwarding address.' ;
}
else {
color = underlineRed ;
title = 'Either domain did not have a DMARC policy, or message did not adhere to it.' ;
}
break ;
default :
// Also for zero value, when unknown. E.g. for sent messages added with IMAP.
return dom . span ( attr . title ( 'Unknown DMARC verification result.' ) , s ) ;
}
return dom . span ( attr . title ( title ) , style ( { borderBottom : '1.5px solid ' + color , textDecoration : 'none' } ) , s ) ;
} ;
let l = [ ] ;
if ( a . Name ) {
l . push ( a . Name + ' ' ) ;
}
l . push ( '<' + a . User + '@' ) ;
l . push ( domainText ( a . Domain . ASCII ) ) ;
l . push ( '>' ) ;
if ( a . Domain . Unicode ) {
// Not underlining because unicode domain may already cause underlining.
l . push ( ' (' + a . User + '@' + a . Domain . Unicode + ')' ) ;
}
return l ;
} ;
2023-11-01 21:12:52 +03:00
// format just the name if present and it doesn't look like an address, or otherwise just the email address.
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
const formatAddressShort = ( a ) => {
2023-11-01 21:12:52 +03:00
const n = a . Name ;
if ( n && ! n . includes ( '<' ) && ! n . includes ( '@' ) && ! n . includes ( '>' ) ) {
return n ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
}
return '<' + a . User + '@' + a . Domain . ASCII + '>' ;
} ;
// return just the email address.
const formatEmailASCII = ( a ) => {
return a . User + '@' + a . Domain . ASCII ;
} ;
const equalAddress = ( a , b ) => {
return ( ! a . User || ! b . User || a . User === b . User ) && a . Domain . ASCII === b . Domain . ASCII ;
} ;
2023-11-20 23:36:40 +03:00
const addressList = ( allAddrs , l ) => {
if ( l . length <= 5 || allAddrs ) {
return dom . span ( join ( l . map ( a => formatAddressFull ( a ) ) , ( ) => ', ' ) ) ;
}
let elem = dom . span ( join ( l . slice ( 0 , 4 ) . map ( a => formatAddressFull ( a ) ) , ( ) => ', ' ) , ' ' , dom . clickbutton ( 'More...' , attr . title ( 'More addresses:\n' + l . slice ( 4 ) . map ( a => formatAddressFull ( a ) ) . join ( ',\n' ) ) , function click ( ) {
const nelem = dom . span ( join ( l . map ( a => formatAddressFull ( a ) ) , ( ) => ', ' ) , ' ' , dom . clickbutton ( 'Less...' , function click ( ) {
elem . replaceWith ( addressList ( allAddrs , l ) ) ;
} ) ) ;
elem . replaceWith ( nelem ) ;
elem = nelem ;
} ) ) ;
return elem ;
} ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
// loadMsgheaderView loads the common message headers into msgheaderelem.
// if refineKeyword is set, labels are shown and a click causes a call to
// refineKeyword.
2023-11-20 23:36:40 +03:00
const loadMsgheaderView = ( msgheaderelem , mi , moreHeaders , refineKeyword , allAddrs ) => {
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
const msgenv = mi . Envelope ;
const received = mi . Message . Received ;
2023-11-05 01:35:44 +03:00
const receivedlocal = new Date ( received . getTime ( ) ) ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
dom . _kids ( msgheaderelem ,
// todo: make addresses clickable, start search (keep current mailbox if any)
2023-11-27 14:11:05 +03:00
dom . tr ( dom . td ( 'From:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( style ( { width : '100%' } ) , dom . div ( style ( { display : 'flex' , justifyContent : 'space-between' } ) , dom . div ( join ( ( msgenv . From || [ ] ) . map ( a => formatAddressFullValidated ( a , mi . Message , ! ! msgenv . From && msgenv . From . length === 1 ) ) , ( ) => ', ' ) ) , dom . div ( attr . title ( 'Received: ' + received . toString ( ) + ';\nDate header in message: ' + ( msgenv . Date ? msgenv . Date . toString ( ) : '(missing/invalid)' ) ) , receivedlocal . toDateString ( ) + ' ' + receivedlocal . toTimeString ( ) . split ( ' ' ) [ 0 ] ) ) ) ) , ( msgenv . ReplyTo || [ ] ) . length === 0 ? [ ] : dom . tr ( dom . td ( 'Reply-To:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( join ( ( msgenv . ReplyTo || [ ] ) . map ( a => formatAddressFull ( a ) ) , ( ) => ', ' ) ) ) , dom . tr ( dom . td ( 'To:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( addressList ( allAddrs , msgenv . To || [ ] ) ) ) , ( msgenv . CC || [ ] ) . length === 0 ? [ ] : dom . tr ( dom . td ( 'Cc:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( addressList ( allAddrs , msgenv . CC || [ ] ) ) ) , ( msgenv . BCC || [ ] ) . length === 0 ? [ ] : dom . tr ( dom . td ( 'Bcc:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( addressList ( allAddrs , msgenv . BCC || [ ] ) ) ) , dom . tr ( dom . td ( 'Subject:' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( dom . div ( style ( { display : 'flex' , justifyContent : 'space-between' } ) , dom . div ( msgenv . Subject || '' ) , dom . div ( mi . Message . IsForward ? dom . span ( style ( { padding : '0px 0.15em' , fontSize : '.9em' } ) , 'Forwarded' , attr . title ( 'Message came in from a forwarded address. Some message authentication policies, like DMARC, were not evaluated.' ) ) : [ ] , mi . Message . IsMailingList ? dom . span ( style ( { padding : '0px 0.15em' , fontSize : '.9em' } ) , 'Mailing list' , attr . title ( 'Message was received from a mailing list. Some message authentication policies, like DMARC, were not evaluated.' ) ) : [ ] , mi . Message . ReceivedTLSVersion === 1 ? dom . span ( style ( { padding : '0px 0.15em' , fontSize : '.9em' , borderBottom : '1.5px solid #e15d1c' } ) , 'Without TLS' , attr . title ( 'Message received (last hop) without TLS.' ) ) : [ ] , mi . Message . ReceivedTLSVersion > 1 && ! mi . Message . ReceivedRequireTLS ? dom . span ( style ( { padding : '0px 0.15em' , fontSize : '.9em' , borderBottom : '1.5px solid #50c40f' } ) , 'With TLS' , attr . title ( 'Message received (last hop) with TLS.' ) ) : [ ] , mi . Message . ReceivedRequireTLS ? dom . span ( style ( { padding : '.1em .3em' , fontSize : '.9em' , backgroundColor : '#d2f791' , border : '1px solid #ccc' , borderRadius : '3px' } ) , 'With RequireTLS' , attr . title ( 'Transported with RequireTLS, ensuring TLS along the entire delivery path from sender to recipient, with TLS certificate verification through MTA-STS and/or DANE.' ) ) : [ ] , mi . IsSigned ? dom . span ( style ( { backgroundColor : '#666' , padding : '0px 0.15em' , fontSize : '.9em' , color : 'white' , borderRadius : '.15em' } ) , 'Message has a signature' ) : [ ] , mi . IsEncrypted ? dom . span ( style ( { backgroundColor : '#666' , padding : '0px 0.15em' , fontSize : '.9em' , color : 'white' , borderRadius : '.15em' } ) , 'Message is encrypted' ) : [ ] , refineKeyword ? ( mi . Message . Keywords || [ ] ) . map ( kw => dom . clickbutton ( dom . _class ( 'keyword' ) , kw , async function click ( ) {
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
await refineKeyword ( kw ) ;
2023-08-11 09:38:57 +03:00
} ) ) : [ ] ) ) ) ) , moreHeaders . map ( k => dom . tr ( dom . td ( k + ':' , style ( { textAlign : 'right' , color : '#555' , whiteSpace : 'nowrap' } ) ) , dom . td ( ) ) ) ) ;
add webmail
it was far down on the roadmap, but implemented earlier, because it's
interesting, and to help prepare for a jmap implementation. for jmap we need to
implement more client-like functionality than with just imap. internal data
structures need to change. jmap has lots of other requirements, so it's already
a big project. by implementing a webmail now, some of the required data
structure changes become clear and can be made now, so the later jmap
implementation can do things similarly to the webmail code. the webmail
frontend and webmail are written together, making their interface/api much
smaller and simpler than jmap.
one of the internal changes is that we now keep track of per-mailbox
total/unread/unseen/deleted message counts and mailbox sizes. keeping this
data consistent after any change to the stored messages (through the code base)
is tricky, so mox now has a consistency check that verifies the counts are
correct, which runs only during tests, each time an internal account reference
is closed. we have a few more internal "changes" that are propagated for the
webmail frontend (that imap doesn't have a way to propagate on a connection),
like changes to the special-use flags on mailboxes, and used keywords in a
mailbox. more changes that will be required have revealed themselves while
implementing the webmail, and will be implemented next.
the webmail user interface is modeled after the mail clients i use or have
used: thunderbird, macos mail, mutt; and webmails i normally only use for
testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed,
but still the goal is to make this webmail client easy to use for everyone. the
user interface looks like most other mail clients: a list of mailboxes, a
search bar, a message list view, and message details. there is a top/bottom and
a left/right layout for the list/message view, default is automatic based on
screen size. the panes can be resized by the user. buttons for actions are just
text, not icons. clicking a button briefly shows the shortcut for the action in
the bottom right, helping with learning to operate quickly. any text that is
underdotted has a title attribute that causes more information to be displayed,
e.g. what a button does or a field is about. to highlight potential phishing
attempts, any text (anywhere in the webclient) that switches unicode "blocks"
(a rough approximation to (language) scripts) within a word is underlined
orange. multiple messages can be selected with familiar ui interaction:
clicking while holding control and/or shift keys. keyboard navigation works
with arrows/page up/down and home/end keys, and also with a few basic vi-like
keys for list/message navigation. we prefer showing the text instead of
html (with inlined images only) version of a message. html messages are shown
in an iframe served from an endpoint with CSP headers to prevent dangerous
resources (scripts, external images) from being loaded. the html is also
sanitized, with javascript removed. a user can choose to load external
resources (e.g. images for tracking purposes).
the frontend is just (strict) typescript, no external frameworks. all
incoming/outgoing data is typechecked, both the api request parameters and
response types, and the data coming in over SSE. the types and checking code
are generated with sherpats, which uses the api definitions generated by
sherpadoc based on the Go code. so types from the backend are automatically
propagated to the frontend. since there is no framework to automatically
propagate properties and rerender components, changes coming in over the SSE
connection are propagated explicitly with regular function calls. the ui is
separated into "views", each with a "root" dom element that is added to the
visible document. these views have additional functions for getting changes
propagated, often resulting in the view updating its (internal) ui state (dom).
we keep the frontend compilation simple, it's just a few typescript files that
get compiled (combined and types stripped) into a single js file, no additional
runtime code needed or complicated build processes used. the webmail is served
is served from a compressed, cachable html file that includes style and the
javascript, currently just over 225kb uncompressed, under 60kb compressed (not
minified, including comments). we include the generated js files in the
repository, to keep Go's easily buildable self-contained binaries.
authentication is basic http, as with the account and admin pages. most data
comes in over one long-term SSE connection to the backend. api requests signal
which mailbox/search/messages are requested over the SSE connection. fetching
individual messages, and making changes, are done through api calls. the
operations are similar to imap, so some code has been moved from package
imapserver to package store. the future jmap implementation will benefit from
these changes too. more functionality will probably be moved to the store
package in the future.
the quickstart enables webmail on the internal listener by default (for new
installs). users can enable it on the public listener if they want to. mox
localserve enables it too. to enable webmail on existing installs, add settings
like the following to the listeners in mox.conf, similar to AccountHTTP(S):
WebmailHTTP:
Enabled: true
WebmailHTTPS:
Enabled: true
special thanks to liesbeth, gerben, andrii for early user feedback.
there is plenty still to do, see the list at the top of webmail/webmail.ts.
feedback welcome as always.
2023-08-07 22:57:03 +03:00
} ;
// Javascript is generated from typescript, do not modify generated javascript because changes will be overwritten.
const init = async ( ) => {
const pm = api . parser . ParsedMessage ( parsedMessage ) ;
dom . _kids ( document . body , dom . div ( dom . _class ( 'pad' , 'mono' ) , style ( { whiteSpace : 'pre-wrap' } ) , join ( ( pm . Texts || [ ] ) . map ( t => renderText ( t ) ) , ( ) => dom . hr ( style ( { margin : '2ex 0' } ) ) ) ) ) ;
} ;
init ( )
. catch ( ( err ) => {
window . alert ( 'Error: ' + ( err . message || '(no message)' ) ) ;
} ) ;