serv/mox
1
0
Fork 0
mirror of https://github.com/mjl-/mox.git synced 2024-12-25 16:03:48 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

make mox compile on windows, without "mox serve" but with working "mox localserve"

Browse source 
getting mox to compile required changing code in only a few places where
package "syscall" was used: for accessing file access times and for umask
handling. an open problem is how to start a process as an unprivileged user on
windows.  that's why "mox serve" isn't implemented yet. and just finding a way
to implement it now may not be good enough in the near future: we may want to
starting using a more complete privilege separation approach, with a process
handling sensitive tasks (handling private keys, authentication), where we may
want to pass file descriptors between processes. how would that work on
windows?

anyway, getting mox to compile for windows doesn't mean it works properly on
windows. the largest issue: mox would normally open a file, rename or remove
it, and finally close it. this happens during message delivery. that doesn't
work on windows, the rename/remove would fail because the file is still open.
so this commit swaps many "remove" and "close" calls. renames are a longer
story: message delivery had two ways to deliver: with "consuming" the
(temporary) message file (which would rename it to its final destination), and
without consuming (by hardlinking the file, falling back to copying). the last
delivery to a recipient of a message (and the only one in the common case of a
single recipient) would consume the message, and the earlier recipients would
not.  during delivery, the already open message file was used, to parse the
message.  we still want to use that open message file, and the caller now stays
responsible for closing it, but we no longer try to rename (consume) the file.
we always hardlink (or copy) during delivery (this works on windows), and the
caller is responsible for closing and removing (in that order) the original
temporary file. this does cost one syscall more. but it makes the delivery code
(responsibilities) a bit simpler.

there is one more obvious issue: the file system path separator. mox already
used the "filepath" package to join paths in many places, but not everywhere.
and it still used strings with slashes for local file access. with this commit,
the code now uses filepath.FromSlash for path strings with slashes, uses
"filepath" in a few more places where it previously didn't. also switches from
"filepath" to regular "path" package when handling mailbox names in a few
places, because those always use forward slashes, regardless of local file
system conventions.  windows can handle forward slashes when opening files, so
test code that passes path strings with forward slashes straight to go stdlib
file i/o functions are left unchanged to reduce code churn. the regular
non-test code, or test code that uses path strings in places other than
standard i/o functions, does have the paths converted for consistent paths
(otherwise we would end up with paths with mixed forward/backward slashes in
log messages).

windows cannot dup a listening socket. for "mox localserve", it isn't
important, and we can work around the issue. the current approach for "mox
serve" (forking a process and passing file descriptors of listening sockets on
"privileged" ports) won't work on windows. perhaps it isn't needed on windows,
and any user can listen on "privileged" ports? that would be welcome.

on windows, os.Open cannot open a directory, so we cannot call Sync on it after
message delivery. a cursory internet search indicates that directories cannot
be synced on windows. the story is probably much more nuanced than that, with
long deep technical details/discussions/disagreement/confusion, like on unix.
for "mox localserve" we can get away with making syncdir a no-op.
This commit is contained in:
Mechiel Lukkien 2023-10-14 10:54:07 +02:00
parent 96774de8d6
commit 28fae96a9b
No known key found for this signature in database
78 changed files with 1155 additions and 938 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
Makefile
View file

@ -107,3 +107,19 @@ docker:
docker-release:
./docker-release.sh
buildall:
GOOS=linux GOARCH=arm go build
GOOS=linux GOARCH=arm64 go build
GOOS=linux GOARCH=amd64 go build
GOOS=linux GOARCH=386 go build
GOOS=openbsd GOARCH=amd64 go build
GOOS=freebsd GOARCH=amd64 go build
GOOS=netbsd GOARCH=amd64 go build
GOOS=darwin GOARCH=amd64 go build
GOOS=dragonfly GOARCH=amd64 go build
GOOS=illumos GOARCH=amd64 go build
GOOS=solaris GOARCH=amd64 go build
GOOS=aix GOARCH=ppc64 go build
GOOS=windows GOARCH=amd64 go build
# no plan9 for now

5
README.md
View file

@ -91,7 +91,10 @@ Verify you have a working mox binary:
./mox version
Mox only compiles for/works on unix systems, not on Plan 9 or Windows.
Mox only compiles for and fully works on unix systems. Mox also compiles for
Windows, but "mox serve" does not yet work, though "mox localserve" (for a
local test instance) and most other subcommands do. Mox does not compile for
Plan 9.
You can also run mox with docker image `r.xmox.nl/mox`, with tags like `v0.0.1`
and `v0.0.1-go1.20.1-alpine3.17.2`, see https://r.xmox.nl/r/mox/. Though new

4
autotls/autotls.go
View file

@ -83,7 +83,7 @@ func Load(name, acmeDir, contactEmail, directoryURL string, getPrivateKey func(h
}
// Load identity key if it exists. Otherwise, create a new key.
p := filepath.Join(acmeDir + "/" + name + ".key")
p := filepath.Join(acmeDir, name+".key")
var key crypto.Signer
f, err := os.Open(p)
if f != nil {
@ -135,7 +135,7 @@ func Load(name, acmeDir, contactEmail, directoryURL string, getPrivateKey func(h
}
m := &autocert.Manager{
Cache: dirCache(acmeDir + "/keycerts/" + name),
Cache: dirCache(filepath.Join(acmeDir, "keycerts", name)),
Prompt: autocert.AcceptTOS,
Email: contactEmail,
Client: &acme.Client{

4
backup.go
View file

@ -372,7 +372,7 @@ func backupctl(ctx context.Context, ctl *ctl) {
xvlog("queue backed finished", mlog.Field("duration", time.Since(tmQueue)))
}
backupQueue("queue/index.db")
backupQueue(filepath.FromSlash("queue/index.db"))
backupAccount := func(acc *store.Account) {
defer acc.Close()
@ -469,7 +469,7 @@ func backupctl(ctx context.Context, ctl *ctl) {
return nil
}
ap := filepath.Join("accounts", acc.Name, p)
if strings.HasPrefix(p, "msg/") {
if strings.HasPrefix(p, "msg"+string(filepath.Separator)) {
xwarnx("backing up unrecognized file in account message directory (should be moved away)", nil, mlog.Field("path", ap))
} else {
xwarnx("backing up unrecognized file in account directory", nil, mlog.Field("path", ap))

16
ctl.go
View file

@ -320,12 +320,11 @@ func servectlcmd(ctx context.Context, ctl *ctl, shutdown func()) {
msgFile, err := store.CreateMessageTemp("ctl-deliver")
ctl.xcheck(err, "creating temporary message file")
defer func() {
if msgFile != nil {
err := os.Remove(msgFile.Name())
log.Check(err, "removing temporary message file", mlog.Field("path", msgFile.Name()))
err = msgFile.Close()
log.Check(err, "closing temporary message file")
}
name := msgFile.Name()
err := msgFile.Close()
log.Check(err, "closing temporary message file")
err = os.Remove(name)
log.Check(err, "removing temporary message file", mlog.Field("path", name))
}()
mw := message.NewWriter(msgFile)
ctl.xwriteok()
@ -340,14 +339,11 @@ func servectlcmd(ctx context.Context, ctl *ctl, shutdown func()) {
}
a.WithWLock(func() {
err := a.DeliverDestination(log, addr, m, msgFile, true)
err := a.DeliverDestination(log, addr, m, msgFile)
ctl.xcheck(err, "delivering message")
log.Info("message delivered through ctl", mlog.Field("to", to))
})
err = msgFile.Close()
log.Check(err, "closing delivered message file")
msgFile = nil
err = a.Close()
ctl.xcheck(err, "closing account")
ctl.xwriteok()

23
ctl_test.go
View file

@ -7,6 +7,7 @@ import (
"flag"
"net"
"os"
"path/filepath"
"testing"
"github.com/mjl-/mox/dmarcdb"
@ -33,8 +34,8 @@ func tcheck(t *testing.T, err error, errmsg string) {
// unhandled errors would cause a panic.
func TestCtl(t *testing.T) {
os.RemoveAll("testdata/ctl/data")
mox.ConfigStaticPath = "testdata/ctl/mox.conf"
mox.ConfigDynamicPath = "testdata/ctl/domains.conf"
mox.ConfigStaticPath = filepath.FromSlash("testdata/ctl/mox.conf")
mox.ConfigDynamicPath = filepath.FromSlash("testdata/ctl/domains.conf")
if errs := mox.LoadConfig(ctxbg, true, false); len(errs) > 0 {
t.Fatalf("loading mox config: %v", errs)
}
@ -147,13 +148,13 @@ func TestCtl(t *testing.T) {
})
// Export data, import it again
xcmdExport(true, []string{"testdata/ctl/data/tmp/export/mbox/", "testdata/ctl/data/accounts/mjl"}, nil)
xcmdExport(false, []string{"testdata/ctl/data/tmp/export/maildir/", "testdata/ctl/data/accounts/mjl"}, nil)
xcmdExport(true, []string{filepath.FromSlash("testdata/ctl/data/tmp/export/mbox/"), filepath.FromSlash("testdata/ctl/data/accounts/mjl")}, nil)
xcmdExport(false, []string{filepath.FromSlash("testdata/ctl/data/tmp/export/maildir/"), filepath.FromSlash("testdata/ctl/data/accounts/mjl")}, nil)
testctl(func(ctl *ctl) {
ctlcmdImport(ctl, true, "mjl", "inbox", "testdata/ctl/data/tmp/export/mbox/Inbox.mbox")
ctlcmdImport(ctl, true, "mjl", "inbox", filepath.FromSlash("testdata/ctl/data/tmp/export/mbox/Inbox.mbox"))
})
testctl(func(ctl *ctl) {
ctlcmdImport(ctl, false, "mjl", "inbox", "testdata/ctl/data/tmp/export/maildir/Inbox")
ctlcmdImport(ctl, false, "mjl", "inbox", filepath.FromSlash("testdata/ctl/data/tmp/export/maildir/Inbox"))
})
// "recalculatemailboxcounts"
@ -177,12 +178,12 @@ func TestCtl(t *testing.T) {
m.Size = int64(len(content))
msgf, err := store.CreateMessageTemp("ctltest")
tcheck(t, err, "create temp file")
defer os.Remove(msgf.Name())
defer msgf.Close()
_, err = msgf.Write(content)
tcheck(t, err, "write message file")
err = acc.DeliverMailbox(xlog, "Inbox", m, msgf, true)
err = acc.DeliverMailbox(xlog, "Inbox", m, msgf)
tcheck(t, err, "deliver message")
err = msgf.Close()
tcheck(t, err, "close message file")
}
var msgBadSize store.Message
@ -236,13 +237,13 @@ func TestCtl(t *testing.T) {
os.RemoveAll("testdata/ctl/data/tmp/backup-data")
err := os.WriteFile("testdata/ctl/data/receivedid.key", make([]byte, 16), 0600)
tcheck(t, err, "writing receivedid.key")
ctlcmdBackup(ctl, "testdata/ctl/data/tmp/backup-data", false)
ctlcmdBackup(ctl, filepath.FromSlash("testdata/ctl/data/tmp/backup-data"), false)
})
// Verify the backup.
xcmd := cmd{
flag: flag.NewFlagSet("", flag.ExitOnError),
flagArgs: []string{"testdata/ctl/data/tmp/backup-data"},
flagArgs: []string{filepath.FromSlash("testdata/ctl/data/tmp/backup-data")},
}
cmdVerifydata(&xcmd)
}

10
develop.txt
View file

@ -1,5 +1,15 @@
This file has notes useful for mox developers.
# Code style & guidelines
- Keep the same style as existing code.
- For Windows: use package "path/filepath" when dealing with files/directories.
Test code can pass forward-slashed paths directly to standard library functions,
but use proper filepath functions when parameters are passed and in non-test
code. Mailbox names always use forward slash, so use package "path" for mailbox
name/path manipulation. Do not remove/rename files that are still open.
# TLS certificates
https://github.com/cloudflare/cfssl is useful for testing with TLS

5
dmarcdb/db_test.go
View file

@ -17,16 +17,17 @@ var ctxbg = context.Background()
func TestDMARCDB(t *testing.T) {
mox.Shutdown = ctxbg
mox.ConfigStaticPath = "../testdata/dmarcdb/fake.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/dmarcdb/fake.conf")
mox.Conf.Static.DataDir = "."
dbpath := mox.DataDirPath("dmarcrpt.db")
os.MkdirAll(filepath.Dir(dbpath), 0770)
defer os.Remove(dbpath)
if err := Init(); err != nil {
t.Fatalf("init database: %s", err)
}
defer os.Remove(dbpath)
defer DB.Close()
feedback := &dmarcrpt.Feedback{
ReportMetadata: dmarcrpt.ReportMetadata{

5
dmarcrpt/parse_test.go
View file

@ -2,6 +2,7 @@ package dmarcrpt
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@ -122,14 +123,14 @@ func TestParseReport(t *testing.T) {
}
func TestParseMessageReport(t *testing.T) {
const dir = "../testdata/dmarc-reports"
dir := filepath.FromSlash("../testdata/dmarc-reports")
files, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("listing dmarc report emails: %s", err)
}
for _, file := range files {
p := dir + "/" + file.Name()
p := filepath.Join(dir, file.Name())
f, err := os.Open(p)
if err != nil {
t.Fatalf("open %q: %s", p, err)

2
doc.go
View file

@ -89,6 +89,8 @@ IMAP. HTTP listeners are started for the admin/account web interfaces, and for
automated TLS configuration. Missing essential TLS certificates are immediately
requested, other TLS certificates are requested on demand.
Only implemented on unix systems, not Windows.
usage: mox serve
# mox quickstart

3
dsn/dsn_test.go
View file

@ -6,6 +6,7 @@ import (
"fmt"
"io"
"net"
"path/filepath"
"reflect"
"strings"
"testing"
@ -131,7 +132,7 @@ func TestDSN(t *testing.T) {
// Test for valid DKIM signature.
mox.Context = context.Background()
mox.ConfigStaticPath = "../testdata/dsn/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/dsn/mox.conf")
mox.MustLoadConfig(true, false)
msgbuf, err = m.Compose(log, false)
if err != nil {

25
gentestdata.go
View file

@ -85,8 +85,8 @@ Accounts:
IgnoreWords: 0.1
`
mox.ConfigStaticPath = "/tmp/mox-bogus/mox.conf"
mox.ConfigDynamicPath = "/tmp/mox-bogus/domains.conf"
mox.ConfigStaticPath = filepath.FromSlash("/tmp/mox-bogus/mox.conf")
mox.ConfigDynamicPath = filepath.FromSlash("/tmp/mox-bogus/domains.conf")
mox.Conf.DynamicLastCheck = time.Now() // Should prevent warning.
mox.Conf.Static = config.Static{
DataDir: destDataDir,
@ -228,11 +228,12 @@ Accounts:
prefix := []byte{}
mf := tempfile()
xcheckf(err, "temp file for queue message")
defer os.Remove(mf.Name())
defer mf.Close()
const qmsg = "From: <test0@mox.example>\r\nTo: <other@remote.example>\r\nSubject: test\r\n\r\nthe message...\r\n"
_, err = fmt.Fprint(mf, qmsg)
xcheckf(err, "writing message")
_, err = queue.Add(ctxbg, log, "test0", mailfrom, rcptto, false, false, int64(len(qmsg)), "<test@localhost>", prefix, mf, nil, true)
_, err = queue.Add(ctxbg, log, "test0", mailfrom, rcptto, false, false, int64(len(qmsg)), "<test@localhost>", prefix, mf, nil)
xcheckf(err, "enqueue message")
// Create three accounts.
@ -283,10 +284,14 @@ Accounts:
xcheckf(err, "creating temp file for delivery")
_, err = fmt.Fprint(mf, msg)
xcheckf(err, "writing deliver message to file")
err = accTest1.DeliverMessage(log, tx, &m, mf, true, false, true, false)
err = accTest1.DeliverMessage(log, tx, &m, mf, false, true, false)
mfname := mf.Name()
xcheckf(err, "add message to account test1")
err = mf.Close()
xcheckf(err, "closing file")
err = os.Remove(mfname)
xcheckf(err, "removing temp message file")
err = tx.Get(&inbox)
xcheckf(err, "get inbox")
@ -339,10 +344,14 @@ Accounts:
xcheckf(err, "creating temp file for delivery")
_, err = fmt.Fprint(mf0, msg0)
xcheckf(err, "writing deliver message to file")
err = accTest2.DeliverMessage(log, tx, &m0, mf0, true, false, false, false)
err = accTest2.DeliverMessage(log, tx, &m0, mf0, false, false, false)
xcheckf(err, "add message to account test2")
mf0name := mf0.Name()
err = mf0.Close()
xcheckf(err, "closing file")
err = os.Remove(mf0name)
xcheckf(err, "removing temp message file")
err = tx.Get(&inbox)
xcheckf(err, "get inbox")
@ -366,10 +375,14 @@ Accounts:
xcheckf(err, "creating temp file for delivery")
_, err = fmt.Fprint(mf1, msg1)
xcheckf(err, "writing deliver message to file")
err = accTest2.DeliverMessage(log, tx, &m1, mf1, true, false, false, false)
err = accTest2.DeliverMessage(log, tx, &m1, mf1, false, false, false)
xcheckf(err, "add message to account test2")
mf1name := mf1.Name()
err = mf1.Close()
xcheckf(err, "closing file")
err = os.Remove(mf1name)
xcheckf(err, "removing temp message file")
err = tx.Get(&sent)
xcheckf(err, "get sent")

2
go.mod
View file

@ -3,7 +3,7 @@ module github.com/mjl-/mox
go 1.20
require (
github.com/mjl-/adns v0.0.0-20231009145311-e3834995f16c
github.com/mjl-/adns v0.0.0-20231013194548-ea0378d616ab
github.com/mjl-/autocert v0.0.0-20231013072455-c361ae2e20a6
github.com/mjl-/bstore v0.0.2
github.com/mjl-/sconf v0.0.5

6
go.sum
View file

@ -145,10 +145,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mjl-/adns v0.0.0-20231009145311-e3834995f16c h1:ZOr9KnCxfAwJWSeZn8Qs6cSF7TrmBa8hVIpLcEvx/Ec=
github.com/mjl-/adns v0.0.0-20231009145311-e3834995f16c/go.mod h1:JWhGACVviyVUEra9Zv1M8JMkDVXArVt+AIXjTXtuwb4=
github.com/mjl-/autocert v0.0.0-20231009155929-d0d48f2f0290 h1:0hCRSu8+XCZ2cSRW+ZtP/7L5wMYjOKFSQthoyj+4cN8=
github.com/mjl-/autocert v0.0.0-20231009155929-d0d48f2f0290/go.mod h1:taMFU86abMxKLPV4Bynhv8enbYmS67b8LG80qZv2Qus=
github.com/mjl-/adns v0.0.0-20231013194548-ea0378d616ab h1:fL+dZP+IxX08+ugLq42bkvOfV42muXET+T+Ei1K16bI=
github.com/mjl-/adns v0.0.0-20231013194548-ea0378d616ab/go.mod h1:v47qUMJnipnmDTRGaHwpCwzE6oypa5K33mUvBfzZBn8=
github.com/mjl-/autocert v0.0.0-20231013072455-c361ae2e20a6 h1:TEXyTghAN9pmV2ffzdnhmzkML08e1Z/oGywJ9eunbRI=
github.com/mjl-/autocert v0.0.0-20231013072455-c361ae2e20a6/go.mod h1:taMFU86abMxKLPV4Bynhv8enbYmS67b8LG80qZv2Qus=
github.com/mjl-/bstore v0.0.2 h1:4fdpIOY/+Dv1dBHyzdqa4PD90p8Mz86FeyRpI4qcehw=

15
http/atime.go
View file

@ -1,9 +1,16 @@
//go:build !netbsd && !freebsd && !darwin
//go:build !netbsd && !freebsd && !darwin && !windows
package http
import "syscall"
import (
"fmt"
"syscall"
)
func statAtime(sys *syscall.Stat_t) int64 {
return int64(sys.Atim.Sec)*1000*1000*1000 + int64(sys.Atim.Nsec)
func statAtime(sys any) (int64, error) {
x, ok := sys.(*syscall.Stat_t)
if !ok {
return 0, fmt.Errorf("sys is a %T, expected *syscall.Stat_t", sys)
}
return int64(x.Atim.Sec)*1000*1000*1000 + int64(x.Atim.Nsec), nil
}

13
http/atime_bsd.go
View file

@ -2,8 +2,15 @@
package http
import "syscall"
import (
"fmt"
"syscall"
)
func statAtime(sys *syscall.Stat_t) int64 {
return int64(sys.Atimespec.Sec)*1000*1000*1000 + int64(sys.Atimespec.Nsec)
func statAtime(sys any) (int64, error) {
x, ok := sys.(*syscall.Stat_t)
if !ok {
return 0, fmt.Errorf("stat sys is a %T, expected *syscall.Stat_t", sys)
}
return int64(x.Atimespec.Sec)*1000*1000*1000 + int64(x.Atimespec.Nsec), nil
}

16
http/atime_windows.go Normal file
View file

@ -0,0 +1,16 @@
//go:build windows
package http
import (
"fmt"
"syscall"
)
func statAtime(sys any) (int64, error) {
x, ok := sys.(*syscall.Win32FileAttributeData)
if !ok {
return 0, fmt.Errorf("sys is a %T, expected *syscall.Win32FileAttributeData", sys)
}
return x.LastAccessTime.Nanoseconds(), nil
}

7
http/gzcache.go
View file

@ -14,7 +14,6 @@ import (
"strconv"
"strings"
"sync"
"syscall"
"time"
"github.com/mjl-/mox/mlog"
@ -109,11 +108,7 @@ func loadStaticGzipCache(dir string, maxSize int64) {
}
var atime int64
if err == nil {
if sys, sysok := fi.Sys().(*syscall.Stat_t); !sysok {
err = fmt.Errorf("FileInfo.Sys not a *syscall.Stat_t but %T", fi.Sys())
} else {
atime = statAtime(sys)
}
atime, err = statAtime(fi.Sys())
}
if err != nil {
xlog.Infox("removing unusable/unrecognized file in static gzip cache dir", err)

2
http/web_test.go
View file

@ -15,7 +15,7 @@ import (
func TestServeHTTP(t *testing.T) {
os.RemoveAll("../testdata/web/data")
mox.ConfigStaticPath = "../testdata/web/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/web/mox.conf")
mox.ConfigDynamicPath = filepath.Join(filepath.Dir(mox.ConfigStaticPath), "domains.conf")
mox.MustLoadConfig(true, false)

4
http/webserver_test.go
View file

@ -27,7 +27,7 @@ func tcheck(t *testing.T, err error, msg string) {
func TestWebserver(t *testing.T) {
os.RemoveAll("../testdata/webserver/data")
mox.ConfigStaticPath = "../testdata/webserver/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/webserver/mox.conf")
mox.ConfigDynamicPath = filepath.Join(filepath.Dir(mox.ConfigStaticPath), "domains.conf")
mox.MustLoadConfig(true, false)
@ -158,7 +158,7 @@ func TestWebserver(t *testing.T) {
func TestWebsocket(t *testing.T) {
os.RemoveAll("../testdata/websocket/data")
mox.ConfigStaticPath = "../testdata/websocket/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/websocket/mox.conf")
mox.ConfigDynamicPath = filepath.Join(filepath.Dir(mox.ConfigStaticPath), "domains.conf")
mox.MustLoadConfig(true, false)

3
imapserver/fuzz_test.go
View file

@ -7,6 +7,7 @@ import (
"io"
"net"
"os"
"path/filepath"
"testing"
"time"
@ -58,7 +59,7 @@ func FuzzServer(f *testing.F) {
}
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/imapserverfuzz/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/imapserverfuzz/mox.conf")
mox.MustLoadConfig(true, false)
dataDir := mox.ConfigDirPath(mox.Conf.Static.DataDir)
os.RemoveAll(dataDir)

6
imapserver/list.go
View file

@ -2,7 +2,7 @@ package imapserver
import (
"fmt"
"path/filepath"
"path"
"sort"
"strings"
@ -133,7 +133,7 @@ func (c *conn) cmdList(tag, cmd string, p *parser) {
err := q.ForEach(func(mb store.Mailbox) error {
names[mb.Name] = info{mailbox: &mb}
nameList = append(nameList, mb.Name)
for p := filepath.Dir(mb.Name); p != "."; p = filepath.Dir(p) {
for p := path.Dir(mb.Name); p != "."; p = path.Dir(p) {
hasChild[p] = true
}
return nil
@ -148,7 +148,7 @@ func (c *conn) cmdList(tag, cmd string, p *parser) {
if !ok {
nameList = append(nameList, sub.Name)
}
for p := filepath.Dir(sub.Name); p != "."; p = filepath.Dir(p) {
for p := path.Dir(sub.Name); p != "."; p = path.Dir(p) {
hasSubscribedChild[p] = true
}
return nil

22
imapserver/server.go
View file

@ -50,6 +50,7 @@ import (
"math"
"net"
"os"
"path"
"path/filepath"
"regexp"
"runtime/debug"
@ -895,7 +896,7 @@ func xmailboxPatternMatcher(ref string, patterns []string) matchStringer {
s := pat
if ref != "" {
s = filepath.Join(ref, pat)
s = path.Join(ref, pat)
}
// Fix casing for all Inbox paths.
@ -2481,7 +2482,7 @@ func (c *conn) cmdLsub(tag, cmd string, p *parser) {
for _, sub := range subscriptions {
name := sub.Name
if ispercent {
for p := filepath.Dir(name); p != "."; p = filepath.Dir(p) {
for p := path.Dir(name); p != "."; p = path.Dir(p) {
subscribedKids[p] = true
}
}
@ -2675,12 +2676,11 @@ func (c *conn) cmdAppend(tag, cmd string, p *parser) {
msgFile, err := store.CreateMessageTemp("imap-append")
xcheckf(err, "creating temp file for message")
defer func() {
if msgFile != nil {
err := os.Remove(msgFile.Name())
c.xsanity(err, "removing APPEND temporary file")
err = msgFile.Close()
c.xsanity(err, "closing APPEND temporary file")
}
p := msgFile.Name()
err := msgFile.Close()
c.xsanity(err, "closing APPEND temporary file")
err = os.Remove(p)
c.xsanity(err, "removing APPEND temporary file")
}()
defer c.xtrace(mlog.LevelTracedata)()
mw := message.NewWriter(msgFile)
@ -2740,7 +2740,7 @@ func (c *conn) cmdAppend(tag, cmd string, p *parser) {
err = tx.Update(&mb)
xcheckf(err, "updating mailbox counts")
err := c.account.DeliverMessage(c.log, tx, &m, msgFile, true, true, false, false)
err := c.account.DeliverMessage(c.log, tx, &m, msgFile, true, false, false)
xcheckf(err, "delivering message")
})
@ -2754,10 +2754,6 @@ func (c *conn) cmdAppend(tag, cmd string, p *parser) {
c.broadcast(changes)
})
err = msgFile.Close()
c.log.Check(err, "closing appended file")
msgFile = nil
if c.mailboxID == mb.ID {
c.applyChanges(pendingChanges, false)
c.uidAppend(m.UID)

3
imapserver/server_test.go
View file

@ -10,6 +10,7 @@ import (
"math/big"
"net"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@ -338,7 +339,7 @@ func startArgs(t *testing.T, first, isTLS, allowLoginWithoutTLS bool) *testconn
os.RemoveAll("../testdata/imap/data")
}
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/imap/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/imap/mox.conf")
mox.MustLoadConfig(true, false)
acc, err := store.OpenAccount("mjl")
tcheck(t, err, "open account")

16
import.go
View file

@ -276,11 +276,10 @@ func importctl(ctx context.Context, ctl *ctl, mbox bool) {
xdeliver := func(m *store.Message, mf *os.File) {
// todo: possibly set dmarcdomain to the domain of the from address? at least for non-spams that have been seen. otherwise user would start without any reputations. the assumption would be that the user has accepted email and deemed it legit, coming from the indicated sender.
const consumeFile = true
const sync = false
const notrain = true
const nothreads = true
err := a.DeliverMessage(ctl.log, tx, m, mf, consumeFile, sync, notrain, nothreads)
err := a.DeliverMessage(ctl.log, tx, m, mf, sync, notrain, nothreads)
ctl.xcheck(err, "delivering message")
deliveredIDs = append(deliveredIDs, m.ID)
ctl.log.Debug("delivered message", mlog.Field("id", m.ID))
@ -313,13 +312,11 @@ func importctl(ctx context.Context, ctl *ctl, mbox bool) {
process := func(m *store.Message, msgf *os.File, origPath string) {
defer func() {
if msgf == nil {
return
}
err := os.Remove(msgf.Name())
ctl.log.Check(err, "removing temporary message after failing to import")
err = msgf.Close()
name := msgf.Name()
err := msgf.Close()
ctl.log.Check(err, "closing temporary message after failing to import")
err = os.Remove(name)
ctl.log.Check(err, "removing temporary message after failing to import", mlog.Field("path", name))
}()
for _, kw := range m.Keywords {
@ -373,9 +370,6 @@ func importctl(ctx context.Context, ctl *ctl, mbox bool) {
m.CreateSeq = modseq
m.ModSeq = modseq
xdeliver(m, msgf)
err = msgf.Close()
ctl.log.Check(err, "closing message after delivery")
msgf = nil
n++
if n%1000 == 0 {

9
junk.go
View file

@ -20,6 +20,7 @@ import (
"log"
mathrand "math/rand"
"os"
"path/filepath"
"sort"
"time"
@ -157,7 +158,7 @@ func cmdJunkTest(c *cmd) {
files, err := os.ReadDir(dir)
xcheckf(err, "readdir %q", dir)
for _, fi := range files {
path := dir + "/" + fi.Name()
path := filepath.Join(dir, fi.Name())
prob, _, _, _, err := f.ClassifyMessagePath(context.Background(), path)
if err != nil {
log.Printf("classify message %q: %s", path, err)
@ -249,7 +250,7 @@ messages are shuffled, with optional random seed.`
testDir := func(dir string, files []string, ham bool) (ok, bad, malformed int) {
for _, name := range files {
path := dir + "/" + name
path := filepath.Join(dir, name)
prob, _, _, _, err := f.ClassifyMessagePath(context.Background(), path)
if err != nil {
// log.Infof("%s: %s", path, err)
@ -313,7 +314,7 @@ func cmdJunkPlay(c *cmd) {
scanDir := func(dir string, ham, sent bool) {
for _, name := range listDir(dir) {
path := dir + "/" + name
path := filepath.Join(dir, name)
mf, err := os.Open(path)
xcheckf(err, "open %q", path)
fi, err := mf.Stat()
@ -366,7 +367,7 @@ func cmdJunkPlay(c *cmd) {
play := func(msg msg) {
var words map[string]struct{}
path := msg.dir + "/" + msg.filename
path := filepath.Join(msg.dir, msg.filename)
if !msg.sent {
var prob float64
var err error

3
junk/filter.go
View file

@ -17,6 +17,7 @@ import (
"io"
"math"
"os"
"path/filepath"
"sort"
"time"
@ -644,7 +645,7 @@ func (f *Filter) TrainDir(dir string, files []string, ham bool) (n, malformed ui
}
for _, name := range files {
p := fmt.Sprintf("%s/%s", dir, name)
p := filepath.Join(dir, name)
valid, words, err := f.tokenizeMail(p)
if err != nil {
// f.log.Infox("tokenizing mail", err, mlog.Field("path", p))

8
junk/filter_test.go
View file

@ -42,8 +42,8 @@ func TestFilter(t *testing.T) {
IgnoreWords: 0.1,
RareWords: 1,
}
dbPath := "../testdata/junk/filter.db"
bloomPath := "../testdata/junk/filter.bloom"
dbPath := filepath.FromSlash("../testdata/junk/filter.db")
bloomPath := filepath.FromSlash("../testdata/junk/filter.bloom")
os.Remove(dbPath)
os.Remove(bloomPath)
f, err := NewFilter(ctxbg, log, params, dbPath, bloomPath)
@ -59,8 +59,8 @@ func TestFilter(t *testing.T) {
os.MkdirAll("../testdata/train/ham", 0770)
os.MkdirAll("../testdata/train/spam", 0770)
hamdir := "../testdata/train/ham"
spamdir := "../testdata/train/spam"
hamdir := filepath.FromSlash("../testdata/train/ham")
spamdir := filepath.FromSlash("../testdata/train/spam")
hamfiles := tlistdir(t, hamdir)
if len(hamfiles) > 100 {
hamfiles = hamfiles[:100]

11
junk/parse_test.go
View file

@ -2,6 +2,7 @@ package junk
import (
"os"
"path/filepath"
"testing"
)
@ -14,12 +15,12 @@ func FuzzParseMessage(f *testing.F) {
}
f.Add(string(buf))
}
add("../testdata/junk/parse.eml")
add("../testdata/junk/parse2.eml")
add("../testdata/junk/parse3.eml")
add(filepath.FromSlash("../testdata/junk/parse.eml"))
add(filepath.FromSlash("../testdata/junk/parse2.eml"))
add(filepath.FromSlash("../testdata/junk/parse3.eml"))
dbPath := "../testdata/junk/parse.db"
bloomPath := "../testdata/junk/parse.bloom"
dbPath := filepath.FromSlash("../testdata/junk/parse.db")
bloomPath := filepath.FromSlash("../testdata/junk/parse.bloom")
os.Remove(dbPath)
os.Remove(bloomPath)
params := Params{Twograms: true}

6
localserve.go
View file

@ -321,11 +321,15 @@ func writeLocalConfig(log *mlog.Log, dir, ip string) (rerr error) {
local.WebserverHTTPS.Enabled = true
local.WebserverHTTPS.Port = 1443
uid := os.Getuid()
if uid < 0 {
uid = 1 // For windows.
}
static := config.Static{
DataDir: ".",
LogLevel: "traceauth",
Hostname: "localhost",
User: fmt.Sprintf("%d", os.Getuid()),
User: fmt.Sprintf("%d", uid),
AdminPasswordFile: "adminpasswd",
Pedantic: true,
Listeners: map[string]config.Listener{

4
main.go
View file

@ -428,7 +428,7 @@ func main() {
return
}
flag.StringVar(&mox.ConfigStaticPath, "config", envString("MOXCONF", "config/mox.conf"), "configuration file, other config files are looked up in the same directory, defaults to $MOXCONF with a fallback to mox.conf")
flag.StringVar(&mox.ConfigStaticPath, "config", envString("MOXCONF", filepath.FromSlash("config/mox.conf")), "configuration file, other config files are looked up in the same directory, defaults to $MOXCONF with a fallback to mox.conf")
flag.StringVar(&loglevel, "loglevel", "", "if non-empty, this log level is set early in startup")
flag.BoolVar(&pedantic, "pedantic", false, "protocol violations result in errors instead of accepting/working around them")
flag.BoolVar(&store.CheckConsistencyOnClose, "checkconsistency", false, "dangerous option for testing only, enables data checks that abort/panic when inconsistencies are found")
@ -1050,7 +1050,7 @@ for a domain and create the TLSA DNS records it suggests to enable DANE.
p := mox.DataDirPath(filepath.Join("acme", "keycerts", l.TLS.ACME, filename))
privKey := xtryLoadPrivateKey(kt, p)
relPath := fmt.Sprintf("hostkeys/%s.%s.%s.privatekey.pkcs8.pem", host.Name(), timestamp, kind)
relPath := filepath.Join("hostkeys", fmt.Sprintf("%s.%s.%s.privatekey.pkcs8.pem", host.Name(), timestamp, kind))
destPath := mox.ConfigDirPath(relPath)
err := writeHostPrivateKey(privKey, destPath)
xcheckf(err, "writing host private key file to %s: %v", destPath, err)

6
mox-/admin.go
View file

@ -178,10 +178,10 @@ func MakeDomainConfig(ctx context.Context, domain, hostname dns.Domain, accountN
}
defer func() {
if f != nil {
err := os.Remove(path)
log.Check(err, "removing file after error")
err = f.Close()
err := f.Close()
log.Check(err, "closing file after error")
err = os.Remove(path)
log.Check(err, "removing file after error", mlog.Field("path", path))
}
}()
if _, err := f.Write(data); err != nil {

5
mox-/config.go
View file

@ -471,8 +471,7 @@ func PrepareStaticConfig(ctx context.Context, configFile string, conf *Config, c
c.User = "mox"
}
u, err := user.Lookup(c.User)
var userErr user.UnknownUserError
if err != nil && errors.As(err, &userErr) {
if err != nil {
uid, err := strconv.ParseUint(c.User, 10, 32)
if err != nil {
addErrorf("parsing unknown user %s as uid: %v (hint: add user mox with \"useradd -d $PWD mox\" or specify a different username on the quickstart command-line)", c.User, err)
@ -481,8 +480,6 @@ func PrepareStaticConfig(ctx context.Context, configFile string, conf *Config, c
c.UID = uint32(uid)
c.GID = uint32(uid)
}
} else if err != nil {
addErrorf("looking up user: %v", err)
} else {
if uid, err := strconv.ParseUint(u.Uid, 10, 32); err != nil {
addErrorf("parsing uid %s: %v", u.Uid, err)

74
mox-/forkexec_unix.go Normal file
View file

@ -0,0 +1,74 @@
//go:build unix
package mox
import (
"os"
"os/signal"
"strings"
"syscall"
"github.com/mjl-/mox/mlog"
)
// Fork and exec as unprivileged user.
//
// We don't use just setuid because it is hard to guarantee that no other
// privileged go worker processes have been started before we get here. E.g. init
// functions in packages can start goroutines.
func ForkExecUnprivileged() {
prog, err := os.Executable()
if err != nil {
xlog.Fatalx("finding executable for exec", err)
}
files := []*os.File{os.Stdin, os.Stdout, os.Stderr}
var addrs []string
for addr, f := range passedListeners {
files = append(files, f)
addrs = append(addrs, addr)
}
var paths []string
for path, fl := range passedFiles {
for _, f := range fl {
files = append(files, f)
paths = append(paths, path)
}
}
env := os.Environ()
env = append(env, "MOX_SOCKETS="+strings.Join(addrs, ","), "MOX_FILES="+strings.Join(paths, ","))
p, err := os.StartProcess(prog, os.Args, &os.ProcAttr{
Env: env,
Files: files,
Sys: &syscall.SysProcAttr{
Credential: &syscall.Credential{
Uid: Conf.Static.UID,
Gid: Conf.Static.GID,
},
},
})
if err != nil {
xlog.Fatalx("fork and exec", err)
}
CleanupPassedFiles()
// If we get a interrupt/terminate signal, pass it on to the child. For interrupt,
// the child probably already got it.
// todo: see if we tie up child and root process so a kill -9 of the root process
// kills the child process too.
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, syscall.SIGTERM)
go func() {
sig := <-sigc
p.Signal(sig)
}()
st, err := p.Wait()
if err != nil {
xlog.Fatalx("wait", err)
}
code := st.ExitCode()
xlog.Print("stopping after child exit", mlog.Field("exitcode", code))
os.Exit(code)
}

9
mox-/forkexec_windows.go Normal file
View file

@ -0,0 +1,9 @@
package mox
// Fork and exec as unprivileged user.
//
// Not implemented yet on windows. Would need to understand its security model
// first.
func ForkExecUnprivileged() {
xlog.Fatal("fork and exec to unprivileged user not yet implemented on windows")
}

86
mox-/lifecycle.go
View file

@ -5,18 +5,14 @@ import (
"fmt"
"net"
"os"
"os/signal"
"runtime"
"runtime/debug"
"strings"
"sync"
"syscall"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/mjl-/mox/mlog"
)
// We start up as root, bind to sockets, open private key/cert files and fork and
@ -56,68 +52,6 @@ func RestorePassedFiles() {
}
}
// Fork and exec as unprivileged user.
//
// We don't use just setuid because it is hard to guarantee that no other
// privileged go worker processes have been started before we get here. E.g. init
// functions in packages can start goroutines.
func ForkExecUnprivileged() {
prog, err := os.Executable()
if err != nil {
xlog.Fatalx("finding executable for exec", err)
}
files := []*os.File{os.Stdin, os.Stdout, os.Stderr}
var addrs []string
for addr, f := range passedListeners {
files = append(files, f)
addrs = append(addrs, addr)
}
var paths []string
for path, fl := range passedFiles {
for _, f := range fl {
files = append(files, f)
paths = append(paths, path)
}
}
env := os.Environ()
env = append(env, "MOX_SOCKETS="+strings.Join(addrs, ","), "MOX_FILES="+strings.Join(paths, ","))
p, err := os.StartProcess(prog, os.Args, &os.ProcAttr{
Env: env,
Files: files,
Sys: &syscall.SysProcAttr{
Credential: &syscall.Credential{
Uid: Conf.Static.UID,
Gid: Conf.Static.GID,
},
},
})
if err != nil {
xlog.Fatalx("fork and exec", err)
}
CleanupPassedFiles()
// If we get a interrupt/terminate signal, pass it on to the child. For interrupt,
// the child probably already got it.
// todo: see if we tie up child and root process so a kill -9 of the root process
// kills the child process too.
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, syscall.SIGTERM)
go func() {
sig := <-sigc
p.Signal(sig)
}()
st, err := p.Wait()
if err != nil {
xlog.Fatalx("wait", err)
}
code := st.ExitCode()
xlog.Print("stopping after child exit", mlog.Field("exitcode", code))
os.Exit(code)
}
// CleanupPassedFiles closes the listening socket file descriptors and files passed
// in by the parent process. To be called by the unprivileged child after listeners
// have been recreated (they dup the file descriptor), and by the privileged
@ -164,15 +98,19 @@ func Listen(network, addr string) (net.Listener, error) {
if err != nil {
return nil, err
}
tcpln, ok := ln.(*net.TCPListener)
if !ok {
return nil, fmt.Errorf("listener not a tcp listener, but %T, for network %s, address %s", ln, network, addr)
// On windows, we cannot duplicate a socket. We don't need to for mox localserve
// with FilesImmediate.
if !FilesImmediate {
tcpln, ok := ln.(*net.TCPListener)
if !ok {
return nil, fmt.Errorf("listener not a tcp listener, but %T, for network %s, address %s", ln, network, addr)
}
f, err := tcpln.File()
if err != nil {
return nil, fmt.Errorf("dup listener: %v", err)
}
passedListeners[addr] = f
}
f, err := tcpln.File()
if err != nil {
return nil, fmt.Errorf("dup listener: %v", err)
}
passedListeners[addr] = f
return ln, err
}

8
moxio/linkcopy.go
View file

@ -45,10 +45,10 @@ func LinkOrCopy(log *mlog.Log, dst, src string, srcReaderOpt io.Reader, sync boo
}
defer func() {
if df != nil {
err = os.Remove(dst)
log.Check(err, "removing partial destination file")
err = df.Close()
err := df.Close()
log.Check(err, "closing partial destination file")
err = os.Remove(dst)
log.Check(err, "removing partial destination file", mlog.Field("path", dst))
}
}()
@ -64,7 +64,7 @@ func LinkOrCopy(log *mlog.Log, dst, src string, srcReaderOpt io.Reader, sync boo
df = nil
if err != nil {
err := os.Remove(dst)
log.Check(err, "removing partial destination file")
log.Check(err, "removing partial destination file", mlog.Field("path", dst))
return err
}
return nil

1
moxio/linkcopy_test.go
View file

@ -26,6 +26,7 @@ func TestLinkOrCopy(t *testing.T) {
f, err := os.Create(src)
tcheckf(t, err, "creating test file")
defer os.Remove(src)
defer f.Close()
err = LinkOrCopy(log, "linkorcopytest-dst.txt", src, nil, false)
tcheckf(t, err, "linking file")
err = os.Remove("linkorcopytest-dst.txt")

2
moxio/syncdir.go
View file

@ -1,3 +1,5 @@
//go:build !windows
package moxio
import (

8
moxio/syncdir_windows.go Normal file
View file

@ -0,0 +1,8 @@
package moxio
// SyncDir opens a directory and syncs its contents to disk.
// SyncDir is a no-op on Windows.
func SyncDir(dir string) error {
// todo: how to sync a directory on windows?
return nil
}

2
mtastsdb/db_test.go
View file

@ -24,7 +24,7 @@ func tcheckf(t *testing.T, err error, format string, args ...any) {
func TestDB(t *testing.T) {
mox.Shutdown = ctxbg
mox.ConfigStaticPath = "../testdata/mtasts/fake.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/mtasts/fake.conf")
mox.Conf.Static.DataDir = "."
dbpath := mox.DataDirPath("mtasts.db")

2
mtastsdb/refresh_test.go
View file

@ -30,7 +30,7 @@ var ctxbg = context.Background()
func TestRefresh(t *testing.T) {
mox.Shutdown = ctxbg
mox.ConfigStaticPath = "../testdata/mtasts/fake.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/mtasts/fake.conf")
mox.Conf.Static.DataDir = "."
dbpath := mox.DataDirPath("mtasts.db")

16
queue/dsn.go
View file

@ -154,12 +154,11 @@ func queueDSN(log *mlog.Log, m Msg, remoteMTA dsn.NameIP, secodeOpt, errmsg stri
return
}
defer func() {
if msgFile != nil {
err := os.Remove(msgFile.Name())
log.Check(err, "removing message file", mlog.Field("path", msgFile.Name()))
err = msgFile.Close()
log.Check(err, "closing message file")
}
name := msgFile.Name()
err := msgFile.Close()
log.Check(err, "closing message file")
err = os.Remove(name)
log.Check(err, "removing message file", mlog.Field("path", name))
}()
msgWriter := message.NewWriter(msgFile)
@ -174,12 +173,9 @@ func queueDSN(log *mlog.Log, m Msg, remoteMTA dsn.NameIP, secodeOpt, errmsg stri
MsgPrefix: []byte{},
}
acc.WithWLock(func() {
if err := acc.DeliverMailbox(log, mailbox, msg, msgFile, true); err != nil {
if err := acc.DeliverMailbox(log, mailbox, msg, msgFile); err != nil {
qlog("delivering dsn to mailbox", err)
return
}
})
err = msgFile.Close()
log.Check(err, "closing dsn file")
msgFile = nil
}

16
queue/queue.go
View file

@ -118,7 +118,7 @@ func (m Msg) MessagePath() string {
// Init opens the queue database without starting delivery.
func Init() error {
qpath := mox.DataDirPath("queue/index.db")
qpath := mox.DataDirPath(filepath.FromSlash("queue/index.db"))
os.MkdirAll(filepath.Dir(qpath), 0770)
isNew := false
if _, err := os.Stat(qpath); err != nil && os.IsNotExist(err) {
@ -176,14 +176,11 @@ func Count(ctx context.Context) (int, error) {
// Add a new message to the queue. The queue is kicked immediately to start a
// first delivery attempt.
//
// If consumeFile is true, it is removed as part of delivery (by rename or copy
// and remove). msgFile is never closed by Add.
//
// dnsutf8Opt is a utf8-version of the message, to be used only for DNSs. If set,
// this data is used as the message when delivering the DSN and the remote SMTP
// server supports SMTPUTF8. If the remote SMTP server does not support SMTPUTF8,
// the regular non-utf8 message is delivered.
func Add(ctx context.Context, log *mlog.Log, senderAccount string, mailFrom, rcptTo smtp.Path, has8bit, smtputf8 bool, size int64, messageID string, msgPrefix []byte, msgFile *os.File, dsnutf8Opt []byte, consumeFile bool) (int64, error) {
func Add(ctx context.Context, log *mlog.Log, senderAccount string, mailFrom, rcptTo smtp.Path, has8bit, smtputf8 bool, size int64, messageID string, msgPrefix []byte, msgFile *os.File, dsnutf8Opt []byte) (int64, error) {
// todo: Add should accept multiple rcptTo if they are for the same domain. so we can queue them for delivery in one (or just a few) session(s), transferring the data only once. ../rfc/5321:3759
if Localserve {
@ -202,7 +199,7 @@ func Add(ctx context.Context, log *mlog.Log, senderAccount string, mailFrom, rcp
conf, _ := acc.Conf()
dest := conf.Destinations[mailFrom.String()]
acc.WithWLock(func() {
err = acc.DeliverDestination(log, dest, &m, msgFile, consumeFile)
err = acc.DeliverDestination(log, dest, &m, msgFile)
})
if err != nil {
return 0, fmt.Errorf("delivering message: %v", err)
@ -239,12 +236,7 @@ func Add(ctx context.Context, log *mlog.Log, senderAccount string, mailFrom, rcp
}()
dstDir := filepath.Dir(dst)
os.MkdirAll(dstDir, 0770)
if consumeFile {
if err := os.Rename(msgFile.Name(), dst); err != nil {
// Could be due to cross-filesystem rename. Users shouldn't configure their systems that way.
return 0, fmt.Errorf("move message into queue dir: %w", err)
}
} else if err := moxio.LinkOrCopy(log, dst, msgFile.Name(), nil, true); err != nil {
if err := moxio.LinkOrCopy(log, dst, msgFile.Name(), nil, true); err != nil {
return 0, fmt.Errorf("linking/copying message to new file: %s", err)
} else if err := moxio.SyncDir(dstDir); err != nil {
return 0, fmt.Errorf("sync directory: %v", err)

36
queue/queue_test.go
View file

@ -13,6 +13,7 @@ import (
"math/big"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
@ -40,7 +41,7 @@ func setup(t *testing.T) (*store.Account, func()) {
// Prepare config so email can be delivered to mjl@mox.example.
os.RemoveAll("../testdata/queue/data")
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/queue/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/queue/mox.conf")
mox.MustLoadConfig(true, false)
acc, err := store.OpenAccount("mjl")
tcheck(t, err, "open account")
@ -86,13 +87,15 @@ func TestQueue(t *testing.T) {
}
path := smtp.Path{Localpart: "mjl", IPDomain: dns.IPDomain{Domain: dns.Domain{ASCII: "mox.example"}}}
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, prepareFile(t), nil, true)
mf := prepareFile(t)
defer os.Remove(mf.Name())
defer mf.Close()
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
mf2 := prepareFile(t)
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf2, nil, false)
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
os.Remove(mf2.Name())
msgs, err = List(ctxbg)
tcheck(t, err, "listing queue")
@ -385,7 +388,7 @@ func TestQueue(t *testing.T) {
// Add a message to be delivered with submit because of its route.
topath := smtp.Path{Localpart: "mjl", IPDomain: dns.IPDomain{Domain: dns.Domain{ASCII: "submit.example"}}}
_, err = Add(ctxbg, xlog, "mjl", path, topath, false, false, int64(len(testmsg)), "<test@localhost>", nil, prepareFile(t), nil, true)
_, err = Add(ctxbg, xlog, "mjl", path, topath, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
wasNetDialer = testDeliver(fakeSubmitServer)
if !wasNetDialer {
@ -393,7 +396,7 @@ func TestQueue(t *testing.T) {
}
// Add a message to be delivered with submit because of explicitly configured transport, that uses TLS.
msgID, err := Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, prepareFile(t), nil, true)
msgID, err := Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
transportSubmitTLS := "submittls"
n, err = Kick(ctxbg, msgID, "", "", &transportSubmitTLS)
@ -417,7 +420,7 @@ func TestQueue(t *testing.T) {
}
// Add a message to be delivered with socks.
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<socks@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<socks@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
transportSocks := "socks"
n, err = Kick(ctxbg, msgID, "", "", &transportSocks)
@ -431,7 +434,7 @@ func TestQueue(t *testing.T) {
}
// Add message to be delivered with opportunistic TLS verification.
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<opportunistictls@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<opportunistictls@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
n, err = Kick(ctxbg, msgID, "", "", nil)
tcheck(t, err, "kick queue")
@ -441,7 +444,7 @@ func TestQueue(t *testing.T) {
testDeliver(fakeSMTPSTARTTLSServer)
// Test fallback to plain text with TLS handshake fails.
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<badtls@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<badtls@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
n, err = Kick(ctxbg, msgID, "", "", nil)
tcheck(t, err, "kick queue")
@ -457,7 +460,7 @@ func TestQueue(t *testing.T) {
{Usage: adns.TLSAUsageDANEEE, Selector: adns.TLSASelectorSPKI, MatchType: adns.TLSAMatchTypeFull, CertAssoc: moxCert.Leaf.RawSubjectPublicKeyInfo},
},
}
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<dane@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<dane@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
n, err = Kick(ctxbg, msgID, "", "", nil)
tcheck(t, err, "kick queue")
@ -472,7 +475,7 @@ func TestQueue(t *testing.T) {
{},
},
}
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<daneunusable@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<daneunusable@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
n, err = Kick(ctxbg, msgID, "", "", nil)
tcheck(t, err, "kick queue")
@ -489,7 +492,7 @@ func TestQueue(t *testing.T) {
{Usage: adns.TLSAUsageDANEEE, Selector: adns.TLSASelectorSPKI, MatchType: adns.TLSAMatchTypeFull, CertAssoc: make([]byte, sha256.Size)},
},
}
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<daneinsecure@localhost>", nil, prepareFile(t), nil, true)
msgID, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<daneinsecure@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
n, err = Kick(ctxbg, msgID, "", "", nil)
tcheck(t, err, "kick queue")
@ -504,7 +507,7 @@ func TestQueue(t *testing.T) {
resolver.TLSA = nil
// Add another message that we'll fail to deliver entirely.
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, prepareFile(t), nil, true)
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
msgs, err = List(ctxbg)
@ -660,7 +663,10 @@ func TestQueueStart(t *testing.T) {
}
path := smtp.Path{Localpart: "mjl", IPDomain: dns.IPDomain{Domain: dns.Domain{ASCII: "mox.example"}}}
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, prepareFile(t), nil, true)
mf := prepareFile(t)
defer os.Remove(mf.Name())
defer mf.Close()
_, err = Add(ctxbg, xlog, "mjl", path, path, false, false, int64(len(testmsg)), "<test@localhost>", nil, mf, nil)
tcheck(t, err, "add message to queue for delivery")
checkDialed(true)

22
quickstart.go
View file

@ -592,7 +592,7 @@ many authentication failures).
dc := config.Dynamic{}
sc := config.Static{
DataDir: "../data",
DataDir: filepath.FromSlash("../data"),
User: user,
LogLevel: "debug", // Help new users, they'll bring it back to info when it all works.
Hostname: dnshostname.Name(),
@ -625,9 +625,9 @@ many authentication failures).
public.IMAPS.Enabled = true
if existingWebserver {
hostbase := fmt.Sprintf("path/to/%s", dnshostname.Name())
mtastsbase := fmt.Sprintf("path/to/mta-sts.%s", domain.Name())
autoconfigbase := fmt.Sprintf("path/to/autoconfig.%s", domain.Name())
hostbase := filepath.FromSlash("path/to/" + dnshostname.Name())
mtastsbase := filepath.FromSlash("path/to/mta-sts." + domain.Name())
autoconfigbase := filepath.FromSlash("path/to/autoconfig." + domain.Name())
public.TLS = &config.TLS{
KeyCerts: []config.KeyCert{
{CertFile: hostbase + "-chain.crt.pem", KeyFile: hostbase + ".key.pem"},
@ -657,8 +657,8 @@ and check the admin page for the needed DNS records.`)
}
now := time.Now()
timestamp := now.Format("20060102T150405")
hostRSAPrivateKeyFile := fmt.Sprintf("hostkeys/%s.%s.%s.privatekey.pkcs8.pem", dnshostname.Name(), timestamp, "rsa2048")
hostECDSAPrivateKeyFile := fmt.Sprintf("hostkeys/%s.%s.%s.privatekey.pkcs8.pem", dnshostname.Name(), timestamp, "ecdsap256")
hostRSAPrivateKeyFile := filepath.Join("hostkeys", fmt.Sprintf("%s.%s.%s.privatekey.pkcs8.pem", dnshostname.Name(), timestamp, "rsa2048"))
hostECDSAPrivateKeyFile := filepath.Join("hostkeys", fmt.Sprintf("%s.%s.%s.privatekey.pkcs8.pem", dnshostname.Name(), timestamp, "ecdsap256"))
xwritehostkeyfile := func(path string, key crypto.Signer) {
buf, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
@ -727,8 +727,8 @@ and check the admin page for the needed DNS records.`)
sc.Postmaster.Account = accountName
sc.Postmaster.Mailbox = "Postmaster"
mox.ConfigStaticPath = "config/mox.conf"
mox.ConfigDynamicPath = "config/domains.conf"
mox.ConfigStaticPath = filepath.FromSlash("config/mox.conf")
mox.ConfigDynamicPath = filepath.FromSlash("config/domains.conf")
mox.Conf.DynamicLastCheck = time.Now() // Prevent error logging by Make calls below.
@ -759,7 +759,7 @@ and check the admin page for the needed DNS records.`)
for _, bl := range public.SMTP.DNSBLs {
confstr = strings.ReplaceAll(confstr, "- "+bl+"\n", "#- "+bl+"\n")
}
xwritefile("config/mox.conf", []byte(confstr), 0660)
xwritefile(filepath.FromSlash("config/mox.conf"), []byte(confstr), 0660)
// Generate domains config, and add a commented out example for delivery to a mailing list.
var db bytes.Buffer
@ -798,11 +798,11 @@ and check the admin page for the needed DNS records.`)
ndests += "#\t\t" + line + "\n"
}
dconfstr := strings.ReplaceAll(db.String(), odests, ndests)
xwritefile("config/domains.conf", []byte(dconfstr), 0660)
xwritefile(filepath.FromSlash("config/domains.conf"), []byte(dconfstr), 0660)
// Verify config.
loadTLSKeyCerts := !existingWebserver
mc, errs := mox.ParseConfig(context.Background(), "config/mox.conf", true, loadTLSKeyCerts, false)
mc, errs := mox.ParseConfig(context.Background(), filepath.FromSlash("config/mox.conf"), true, loadTLSKeyCerts, false)
if len(errs) > 0 {
if len(errs) > 1 {
log.Printf("checking generated config, multiple errors:")

8
sendmail.go
View file

@ -220,13 +220,7 @@ binary should be setgid that group:
os.Mkdir(maildir, 0700)
f, err := os.CreateTemp(maildir, "newmsg.")
xcheckf(err, "creating temp file for storing message after failed delivery")
defer func() {
if f != nil {
if err := os.Remove(f.Name()); err != nil {
log.Printf("removing temp file after failure storing failed delivery: %v", err)
}
}
}()
// note: not removing the partial file if writing/closing below fails.
_, err = f.Write([]byte(msg))
xcheckf(err, "writing message to temp file after failed delivery")
name := f.Name()

532
serve.go
View file

@ -1,393 +1,23 @@
package main
import (
"context"
cryptorand "crypto/rand"
"fmt"
"io/fs"
"net"
"os"
"os/signal"
"path/filepath"
"runtime"
"runtime/debug"
"strings"
"sync"
"syscall"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/mjl-/mox/dmarcdb"
"github.com/mjl-/mox/dns"
"github.com/mjl-/mox/dnsbl"
"github.com/mjl-/mox/http"
"github.com/mjl-/mox/imapserver"
"github.com/mjl-/mox/message"
"github.com/mjl-/mox/metrics"
"github.com/mjl-/mox/mlog"
"github.com/mjl-/mox/mox-"
"github.com/mjl-/mox/moxvar"
"github.com/mjl-/mox/mtastsdb"
"github.com/mjl-/mox/queue"
"github.com/mjl-/mox/smtpserver"
"github.com/mjl-/mox/store"
"github.com/mjl-/mox/tlsrptdb"
"github.com/mjl-/mox/updates"
)
func monitorDNSBL(log *mlog.Log) {
defer func() {
// On error, don't bring down the entire server.
x := recover()
if x != nil {
log.Error("monitordnsbl panic", mlog.Field("panic", x))
debug.PrintStack()
metrics.PanicInc(metrics.Serve)
}
}()
l, ok := mox.Conf.Static.Listeners["public"]
if !ok {
log.Info("no listener named public, not monitoring our ips at dnsbls")
return
}
var zones []dns.Domain
for _, zone := range l.SMTP.DNSBLs {
d, err := dns.ParseDomain(zone)
if err != nil {
log.Fatalx("parsing dnsbls zone", err, mlog.Field("zone", zone))
}
zones = append(zones, d)
}
if len(zones) == 0 {
return
}
type key struct {
zone dns.Domain
ip string
}
metrics := map[key]prometheus.GaugeFunc{}
var statusMutex sync.Mutex
statuses := map[key]bool{}
resolver := dns.StrictResolver{Pkg: "dnsblmonitor"}
var sleep time.Duration // No sleep on first iteration.
for {
time.Sleep(sleep)
sleep = 3 * time.Hour
ips, err := mox.IPs(mox.Context, false)
if err != nil {
log.Errorx("listing ips for dnsbl monitor", err)
continue
}
for _, ip := range ips {
if ip.IsLoopback() || ip.IsPrivate() {
continue
}
for _, zone := range zones {
status, expl, err := dnsbl.Lookup(mox.Context, resolver, zone, ip)
if err != nil {
log.Errorx("dnsbl monitor lookup", err, mlog.Field("ip", ip), mlog.Field("zone", zone), mlog.Field("expl", expl), mlog.Field("status", status))
}
k := key{zone, ip.String()}
statusMutex.Lock()
statuses[k] = status == dnsbl.StatusPass
statusMutex.Unlock()
if _, ok := metrics[k]; !ok {
metrics[k] = promauto.NewGaugeFunc(
prometheus.GaugeOpts{
Name: "mox_dnsbl_ips_success",
Help: "DNSBL lookups to configured DNSBLs of our IPs.",
ConstLabels: prometheus.Labels{
"zone": zone.LogString(),
"ip": k.ip,
},
},
func() float64 {
statusMutex.Lock()
defer statusMutex.Unlock()
if statuses[k] {
return 1
}
return 0
},
)
}
time.Sleep(time.Second)
}
}
}
}
// also see localserve.go, code is similar or even shared.
func cmdServe(c *cmd) {
c.help = `Start mox, serving SMTP/IMAP/HTTPS.
Incoming email is accepted over SMTP. Email can be retrieved by users using
IMAP. HTTP listeners are started for the admin/account web interfaces, and for
automated TLS configuration. Missing essential TLS certificates are immediately
requested, other TLS certificates are requested on demand.
`
args := c.Parse()
if len(args) != 0 {
c.Usage()
}
// Set debug logging until config is fully loaded.
mlog.Logfmt = true
mox.Conf.Log[""] = mlog.LevelDebug
mlog.SetConfig(mox.Conf.Log)
checkACMEHosts := os.Getuid() != 0
log := mlog.New("serve")
if os.Getuid() == 0 {
mox.MustLoadConfig(true, checkACMEHosts)
// No need to potentially start and keep multiple processes. As root, we just need
// to start the child process.
runtime.GOMAXPROCS(1)
moxconf, err := filepath.Abs(mox.ConfigStaticPath)
log.Check(err, "finding absolute mox.conf path")
domainsconf, err := filepath.Abs(mox.ConfigDynamicPath)
log.Check(err, "finding absolute domains.conf path")
log.Print("starting as root, initializing network listeners", mlog.Field("version", moxvar.Version), mlog.Field("pid", os.Getpid()), mlog.Field("moxconf", moxconf), mlog.Field("domainsconf", domainsconf))
if os.Getenv("MOX_SOCKETS") != "" {
log.Fatal("refusing to start as root with $MOX_SOCKETS set")
}
if os.Getenv("MOX_FILES") != "" {
log.Fatal("refusing to start as root with $MOX_FILES set")
}
if !mox.Conf.Static.NoFixPermissions {
// Fix permissions now that we have privilege to do so. Useful for update of v0.0.1
// that was running directly as mox-user.
workdir, err := os.Getwd()
if err != nil {
log.Printx("get working dir, continuing without potentially fixing up permissions", err)
} else {
configdir := filepath.Dir(mox.ConfigStaticPath)
datadir := mox.DataDirPath(".")
err := fixperms(log, workdir, configdir, datadir, mox.Conf.Static.UID, mox.Conf.Static.GID)
if err != nil {
log.Fatalx("fixing permissions", err)
}
}
}
} else {
mox.RestorePassedFiles()
mox.MustLoadConfig(true, checkACMEHosts)
log.Print("starting as unprivileged user", mlog.Field("user", mox.Conf.Static.User), mlog.Field("uid", mox.Conf.Static.UID), mlog.Field("gid", mox.Conf.Static.GID), mlog.Field("pid", os.Getpid()))
}
syscall.Umask(syscall.Umask(007) | 007)
// Initialize key and random buffer for creating opaque SMTP
// transaction IDs based on "cid"s.
recvidpath := mox.DataDirPath("receivedid.key")
recvidbuf, err := os.ReadFile(recvidpath)
if err != nil || len(recvidbuf) != 16+8 {
recvidbuf = make([]byte, 16+8)
if _, err := cryptorand.Read(recvidbuf); err != nil {
log.Fatalx("reading random recvid data", err)
}
if err := os.WriteFile(recvidpath, recvidbuf, 0660); err != nil {
log.Fatalx("writing recvidpath", err, mlog.Field("path", recvidpath))
}
err := os.Chown(recvidpath, int(mox.Conf.Static.UID), 0)
log.Check(err, "chown receveidid.key", mlog.Field("path", recvidpath), mlog.Field("uid", mox.Conf.Static.UID), mlog.Field("gid", 0))
err = os.Chmod(recvidpath, 0640)
log.Check(err, "chmod receveidid.key to 0640", mlog.Field("path", recvidpath))
}
if err := mox.ReceivedIDInit(recvidbuf[:16], recvidbuf[16:]); err != nil {
log.Fatalx("init receivedid", err)
}
// Start mox. If running as root, this will bind/listen on network sockets, and
// fork and exec itself as unprivileged user, then waits for the child to stop and
// exit. When running as root, this function never returns. But the new
// unprivileged user will get here again, with network sockets prepared.
//
// We listen to the unix domain ctl socket afterwards, which we always remove
// before listening. We need to do that because we may not have cleaned up our
// control socket during unexpected shutdown. We don't want to remove and listen on
// the unix domain socket first. If we would, we would make the existing instance
// unreachable over its ctl socket, and then fail because the network addresses are
// taken.
const mtastsdbRefresher = true
const skipForkExec = false
if err := start(mtastsdbRefresher, skipForkExec); err != nil {
log.Fatalx("start", err)
}
log.Print("ready to serve")
if mox.Conf.Static.CheckUpdates {
checkUpdates := func() time.Duration {
next := 24 * time.Hour
current, lastknown, mtime, err := mox.LastKnown()
if err != nil {
log.Infox("determining own version before checking for updates, trying again in 24h", err)
return next
}
// We don't want to check for updates at every startup. So we sleep based on file
// mtime. But file won't exist initially.
if !mtime.IsZero() && time.Since(mtime) < 24*time.Hour {
d := 24*time.Hour - time.Since(mtime)
log.Debug("sleeping for next check for updates", mlog.Field("sleep", d))
time.Sleep(d)
next = 0
}
now := time.Now()
if err := os.Chtimes(mox.DataDirPath("lastknownversion"), now, now); err != nil {
if !os.IsNotExist(err) {
log.Infox("setting mtime on lastknownversion file, continuing", err)
}
}
log.Debug("checking for updates", mlog.Field("lastknown", lastknown))
updatesctx, updatescancel := context.WithTimeout(mox.Context, time.Minute)
latest, _, changelog, err := updates.Check(updatesctx, dns.StrictResolver{}, dns.Domain{ASCII: changelogDomain}, lastknown, changelogURL, changelogPubKey)
updatescancel()
if err != nil {
log.Infox("checking for updates", err, mlog.Field("latest", latest))
return next
}
if !latest.After(lastknown) {
log.Debug("no new version available")
return next
}
if len(changelog.Changes) == 0 {
log.Info("new version available, but changelog is empty, ignoring", mlog.Field("latest", latest))
return next
}
var cl string
for _, c := range changelog.Changes {
cl += "----\n\n" + strings.TrimSpace(c.Text) + "\n\n"
}
cl += "----"
a, err := store.OpenAccount(mox.Conf.Static.Postmaster.Account)
if err != nil {
log.Infox("open account for postmaster changelog delivery", err)
return next
}
defer func() {
err := a.Close()
log.Check(err, "closing account")
}()
f, err := store.CreateMessageTemp("changelog")
if err != nil {
log.Infox("making temporary message file for changelog delivery", err)
return next
}
defer func() {
if f != nil {
err := os.Remove(f.Name())
log.Check(err, "removing temp changelog file")
err = f.Close()
log.Check(err, "closing temp changelog file")
}
}()
m := &store.Message{
Received: time.Now(),
Flags: store.Flags{Flagged: true},
}
n, err := fmt.Fprintf(f, "Date: %s\r\nSubject: mox %s available\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Transfer-Encoding: 8-bit\r\n\r\nHi!\r\n\r\nVersion %s of mox is available, this install is at %s.\r\n\r\nChanges:\r\n\r\n%s\r\n\r\nRemember to make a backup with \"mox backup\" before upgrading.\r\nPlease report any issues at https://github.com/mjl-/mox, thanks!\r\n\r\nCheers,\r\nmox\r\n", time.Now().Format(message.RFC5322Z), latest, latest, current, strings.ReplaceAll(cl, "\n", "\r\n"))
if err != nil {
log.Infox("writing temporary message file for changelog delivery", err)
return next
}
m.Size = int64(n)
if err := a.DeliverMailbox(log, mox.Conf.Static.Postmaster.Mailbox, m, f, true); err != nil {
log.Errorx("changelog delivery", err)
return next
}
f = nil
log.Info("delivered changelog", mlog.Field("current", current), mlog.Field("lastknown", lastknown), mlog.Field("latest", latest))
if err := mox.StoreLastKnown(latest); err != nil {
// This will be awkward, we'll keep notifying the postmaster once every 24h...
log.Infox("updating last known version", err)
}
return next
}
go func() {
for {
next := checkUpdates()
time.Sleep(next)
}
}()
}
go monitorDNSBL(log)
ctlpath := mox.DataDirPath("ctl")
_ = os.Remove(ctlpath)
ctl, err := net.Listen("unix", ctlpath)
if err != nil {
log.Fatalx("listen on ctl unix domain socket", err)
}
go func() {
for {
conn, err := ctl.Accept()
if err != nil {
log.Printx("accept for ctl", err)
continue
}
cid := mox.Cid()
ctx := context.WithValue(mox.Context, mlog.CidKey, cid)
go servectl(ctx, log.WithCid(cid), conn, func() { shutdown(log) })
}
}()
// Remove old temporary files that somehow haven't been cleaned up.
tmpdir := mox.DataDirPath("tmp")
os.MkdirAll(tmpdir, 0770)
tmps, err := os.ReadDir(tmpdir)
if err != nil {
log.Errorx("listing files in tmpdir", err)
} else {
now := time.Now()
for _, e := range tmps {
if fi, err := e.Info(); err != nil {
log.Errorx("stat tmp file", err, mlog.Field("filename", e.Name()))
} else if now.Sub(fi.ModTime()) > 7*24*time.Hour && !fi.IsDir() {
p := filepath.Join(tmpdir, e.Name())
if err := os.Remove(p); err != nil {
log.Errorx("removing stale temporary file", err, mlog.Field("path", p))
} else {
log.Info("removed stale temporary file", mlog.Field("path", p))
}
}
}
}
// Graceful shutdown.
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, syscall.SIGTERM)
sig := <-sigc
log.Print("shutting down, waiting max 3s for existing connections", mlog.Field("signal", sig))
shutdown(log)
if num, ok := sig.(syscall.Signal); ok {
os.Exit(int(num))
} else {
os.Exit(1)
}
}
func shutdown(log *mlog.Log) {
// We indicate we are shutting down. Causes new connections and new SMTP commands
// to be rejected. Should stop active connections pretty quickly.
@ -420,168 +50,6 @@ func shutdown(log *mlog.Log) {
log.Check(err, "removing ctl unix domain socket during shutdown")
}
// Set correct permissions for mox working directory, binary, config and data and service file.
//
// We require being able to stat the basic non-optional paths. Then we'll try to
// fix up permissions. If an error occurs when fixing permissions, we log and
// continue (could not be an actual problem).
func fixperms(log *mlog.Log, workdir, configdir, datadir string, moxuid, moxgid uint32) (rerr error) {
type fserr struct{ Err error }
defer func() {
x := recover()
if x == nil {
return
}
e, ok := x.(fserr)
if ok {
rerr = e.Err
} else {
panic(x)
}
}()
checkf := func(err error, format string, args ...any) {
if err != nil {
panic(fserr{fmt.Errorf(format, args...)})
}
}
// Changes we have to make. We collect them first, then apply.
type change struct {
path string
uid, gid *uint32
olduid, oldgid uint32
mode *fs.FileMode
oldmode fs.FileMode
}
var changes []change
ensure := func(p string, uid, gid uint32, perm fs.FileMode) bool {
fi, err := os.Stat(p)
checkf(err, "stat %s", p)
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
checkf(fmt.Errorf("got %T", st), "stat sys, expected syscall.Stat_t")
}
var ch change
if st.Uid != uid || st.Gid != gid {
ch.uid = &uid
ch.gid = &gid
ch.olduid = st.Uid
ch.oldgid = st.Gid
}
if perm != fi.Mode()&(fs.ModeSetgid|0777) {
ch.mode = &perm
ch.oldmode = fi.Mode() & (fs.ModeSetgid | 0777)
}
var zerochange change
if ch == zerochange {
return false
}
ch.path = p
changes = append(changes, ch)
return true
}
xexists := func(p string) bool {
_, err := os.Stat(p)
if err != nil && !os.IsNotExist(err) {
checkf(err, "stat %s", p)
}
return err == nil
}
// We ensure these permissions:
//
// $workdir root:mox 0751
// $configdir mox:root 0750 + setgid, and recursively (but files 0640)
// $datadir mox:root 0750 + setgid, and recursively (but files 0640)
// $workdir/mox (binary, optional) root:mox 0750
// $workdir/mox.service (systemd service file, optional) root:root 0644
const root = 0
ensure(workdir, root, moxgid, 0751)
fixconfig := ensure(configdir, moxuid, 0, fs.ModeSetgid|0750)
fixdata := ensure(datadir, moxuid, 0, fs.ModeSetgid|0750)
// Binary and systemd service file do not exist (there) when running under docker.
binary := filepath.Join(workdir, "mox")
if xexists(binary) {
ensure(binary, root, moxgid, 0750)
}
svc := filepath.Join(workdir, "mox.service")
if xexists(svc) {
ensure(svc, root, root, 0644)
}
if len(changes) == 0 {
return
}
// Apply changes.
log.Print("fixing up permissions, will continue on errors")
for _, ch := range changes {
if ch.uid != nil {
err := os.Chown(ch.path, int(*ch.uid), int(*ch.gid))
log.Printx("chown, fixing uid/gid", err, mlog.Field("path", ch.path), mlog.Field("olduid", ch.olduid), mlog.Field("oldgid", ch.oldgid), mlog.Field("newuid", *ch.uid), mlog.Field("newgid", *ch.gid))
}
if ch.mode != nil {
err := os.Chmod(ch.path, *ch.mode)
log.Printx("chmod, fixing permissions", err, mlog.Field("path", ch.path), mlog.Field("oldmode", fmt.Sprintf("%03o", ch.oldmode)), mlog.Field("newmode", fmt.Sprintf("%03o", *ch.mode)))
}
}
walkchange := func(dir string) {
err := filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
log.Printx("walk error, continuing", err, mlog.Field("path", path))
return nil
}
fi, err := d.Info()
if err != nil {
log.Printx("stat during walk, continuing", err, mlog.Field("path", path))
return nil
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
log.Printx("syscall stat during walk, continuing", err, mlog.Field("path", path))
return nil
}
if st.Uid != moxuid || st.Gid != root {
err := os.Chown(path, int(moxuid), root)
log.Printx("walk chown, fixing uid/gid", err, mlog.Field("path", path), mlog.Field("olduid", st.Uid), mlog.Field("oldgid", st.Gid), mlog.Field("newuid", moxuid), mlog.Field("newgid", root))
}
omode := fi.Mode() & (fs.ModeSetgid | 0777)
var nmode fs.FileMode
if fi.IsDir() {
nmode = fs.ModeSetgid | 0750
} else {
nmode = 0640
}
if omode != nmode {
err := os.Chmod(path, nmode)
log.Printx("walk chmod, fixing permissions", err, mlog.Field("path", path), mlog.Field("oldmode", fmt.Sprintf("%03o", omode)), mlog.Field("newmode", fmt.Sprintf("%03o", nmode)))
}
return nil
})
log.Check(err, "walking dir to fix permissions", mlog.Field("dir", dir))
}
// If config or data dir needed fixing, also set uid/gid and mode and files/dirs
// inside, recursively. We don't always recurse, data probably contains many files.
if fixconfig {
log.Print("fixing permissions in config dir", mlog.Field("configdir", configdir))
walkchange(configdir)
}
if fixdata {
log.Print("fixing permissions in data dir", mlog.Field("configdir", configdir))
walkchange(datadir)
}
return nil
}
// start initializes all packages, starts all listeners and the switchboard
// goroutine, then returns.
func start(mtastsdbRefresher, skipForkExec bool) error {

546
serve_unix.go Normal file
View file

@ -0,0 +1,546 @@
//go:build !windows
package main
import (
"context"
cryptorand "crypto/rand"
"fmt"
"io/fs"
"net"
"os"
"os/signal"
"path/filepath"
"runtime"
"runtime/debug"
"strings"
"sync"
"syscall"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/mjl-/mox/dns"
"github.com/mjl-/mox/dnsbl"
"github.com/mjl-/mox/message"
"github.com/mjl-/mox/metrics"
"github.com/mjl-/mox/mlog"
"github.com/mjl-/mox/mox-"
"github.com/mjl-/mox/moxvar"
"github.com/mjl-/mox/store"
"github.com/mjl-/mox/updates"
)
func monitorDNSBL(log *mlog.Log) {
defer func() {
// On error, don't bring down the entire server.
x := recover()
if x != nil {
log.Error("monitordnsbl panic", mlog.Field("panic", x))
debug.PrintStack()
metrics.PanicInc(metrics.Serve)
}
}()
l, ok := mox.Conf.Static.Listeners["public"]
if !ok {
log.Info("no listener named public, not monitoring our ips at dnsbls")
return
}
var zones []dns.Domain
for _, zone := range l.SMTP.DNSBLs {
d, err := dns.ParseDomain(zone)
if err != nil {
log.Fatalx("parsing dnsbls zone", err, mlog.Field("zone", zone))
}
zones = append(zones, d)
}
if len(zones) == 0 {
return
}
type key struct {
zone dns.Domain
ip string
}
metrics := map[key]prometheus.GaugeFunc{}
var statusMutex sync.Mutex
statuses := map[key]bool{}
resolver := dns.StrictResolver{Pkg: "dnsblmonitor"}
var sleep time.Duration // No sleep on first iteration.
for {
time.Sleep(sleep)
sleep = 3 * time.Hour
ips, err := mox.IPs(mox.Context, false)
if err != nil {
log.Errorx("listing ips for dnsbl monitor", err)
continue
}
for _, ip := range ips {
if ip.IsLoopback() || ip.IsPrivate() {
continue
}
for _, zone := range zones {
status, expl, err := dnsbl.Lookup(mox.Context, resolver, zone, ip)
if err != nil {
log.Errorx("dnsbl monitor lookup", err, mlog.Field("ip", ip), mlog.Field("zone", zone), mlog.Field("expl", expl), mlog.Field("status", status))
}
k := key{zone, ip.String()}
statusMutex.Lock()
statuses[k] = status == dnsbl.StatusPass
statusMutex.Unlock()
if _, ok := metrics[k]; !ok {
metrics[k] = promauto.NewGaugeFunc(
prometheus.GaugeOpts{
Name: "mox_dnsbl_ips_success",
Help: "DNSBL lookups to configured DNSBLs of our IPs.",
ConstLabels: prometheus.Labels{
"zone": zone.LogString(),
"ip": k.ip,
},
},
func() float64 {
statusMutex.Lock()
defer statusMutex.Unlock()
if statuses[k] {
return 1
}
return 0
},
)
}
time.Sleep(time.Second)
}
}
}
}
// also see localserve.go, code is similar or even shared.
func cmdServe(c *cmd) {
c.help = `Start mox, serving SMTP/IMAP/HTTPS.
Incoming email is accepted over SMTP. Email can be retrieved by users using
IMAP. HTTP listeners are started for the admin/account web interfaces, and for
automated TLS configuration. Missing essential TLS certificates are immediately
requested, other TLS certificates are requested on demand.
Only implemented on unix systems, not Windows.
`
args := c.Parse()
if len(args) != 0 {
c.Usage()
}
// Set debug logging until config is fully loaded.
mlog.Logfmt = true
mox.Conf.Log[""] = mlog.LevelDebug
mlog.SetConfig(mox.Conf.Log)
checkACMEHosts := os.Getuid() != 0
log := mlog.New("serve")
if os.Getuid() == 0 {
mox.MustLoadConfig(true, checkACMEHosts)
// No need to potentially start and keep multiple processes. As root, we just need
// to start the child process.
runtime.GOMAXPROCS(1)
moxconf, err := filepath.Abs(mox.ConfigStaticPath)
log.Check(err, "finding absolute mox.conf path")
domainsconf, err := filepath.Abs(mox.ConfigDynamicPath)
log.Check(err, "finding absolute domains.conf path")
log.Print("starting as root, initializing network listeners", mlog.Field("version", moxvar.Version), mlog.Field("pid", os.Getpid()), mlog.Field("moxconf", moxconf), mlog.Field("domainsconf", domainsconf))
if os.Getenv("MOX_SOCKETS") != "" {
log.Fatal("refusing to start as root with $MOX_SOCKETS set")
}
if os.Getenv("MOX_FILES") != "" {
log.Fatal("refusing to start as root with $MOX_FILES set")
}
if !mox.Conf.Static.NoFixPermissions {
// Fix permissions now that we have privilege to do so. Useful for update of v0.0.1
// that was running directly as mox-user.
workdir, err := os.Getwd()
if err != nil {
log.Printx("get working dir, continuing without potentially fixing up permissions", err)
} else {
configdir := filepath.Dir(mox.ConfigStaticPath)
datadir := mox.DataDirPath(".")
err := fixperms(log, workdir, configdir, datadir, mox.Conf.Static.UID, mox.Conf.Static.GID)
if err != nil {
log.Fatalx("fixing permissions", err)
}
}
}
} else {
mox.RestorePassedFiles()
mox.MustLoadConfig(true, checkACMEHosts)
log.Print("starting as unprivileged user", mlog.Field("user", mox.Conf.Static.User), mlog.Field("uid", mox.Conf.Static.UID), mlog.Field("gid", mox.Conf.Static.GID), mlog.Field("pid", os.Getpid()))
}
syscall.Umask(syscall.Umask(007) | 007)
// Initialize key and random buffer for creating opaque SMTP
// transaction IDs based on "cid"s.
recvidpath := mox.DataDirPath("receivedid.key")
recvidbuf, err := os.ReadFile(recvidpath)
if err != nil || len(recvidbuf) != 16+8 {
recvidbuf = make([]byte, 16+8)
if _, err := cryptorand.Read(recvidbuf); err != nil {
log.Fatalx("reading random recvid data", err)
}
if err := os.WriteFile(recvidpath, recvidbuf, 0660); err != nil {
log.Fatalx("writing recvidpath", err, mlog.Field("path", recvidpath))
}
err := os.Chown(recvidpath, int(mox.Conf.Static.UID), 0)
log.Check(err, "chown receveidid.key", mlog.Field("path", recvidpath), mlog.Field("uid", mox.Conf.Static.UID), mlog.Field("gid", 0))
err = os.Chmod(recvidpath, 0640)
log.Check(err, "chmod receveidid.key to 0640", mlog.Field("path", recvidpath))
}
if err := mox.ReceivedIDInit(recvidbuf[:16], recvidbuf[16:]); err != nil {
log.Fatalx("init receivedid", err)
}
// Start mox. If running as root, this will bind/listen on network sockets, and
// fork and exec itself as unprivileged user, then waits for the child to stop and
// exit. When running as root, this function never returns. But the new
// unprivileged user will get here again, with network sockets prepared.
//
// We listen to the unix domain ctl socket afterwards, which we always remove
// before listening. We need to do that because we may not have cleaned up our
// control socket during unexpected shutdown. We don't want to remove and listen on
// the unix domain socket first. If we would, we would make the existing instance
// unreachable over its ctl socket, and then fail because the network addresses are
// taken.
const mtastsdbRefresher = true
const skipForkExec = false
if err := start(mtastsdbRefresher, skipForkExec); err != nil {
log.Fatalx("start", err)
}
log.Print("ready to serve")
if mox.Conf.Static.CheckUpdates {
checkUpdates := func() time.Duration {
next := 24 * time.Hour
current, lastknown, mtime, err := mox.LastKnown()
if err != nil {
log.Infox("determining own version before checking for updates, trying again in 24h", err)
return next
}
// We don't want to check for updates at every startup. So we sleep based on file
// mtime. But file won't exist initially.
if !mtime.IsZero() && time.Since(mtime) < 24*time.Hour {
d := 24*time.Hour - time.Since(mtime)
log.Debug("sleeping for next check for updates", mlog.Field("sleep", d))
time.Sleep(d)
next = 0
}
now := time.Now()
if err := os.Chtimes(mox.DataDirPath("lastknownversion"), now, now); err != nil {
if !os.IsNotExist(err) {
log.Infox("setting mtime on lastknownversion file, continuing", err)
}
}
log.Debug("checking for updates", mlog.Field("lastknown", lastknown))
updatesctx, updatescancel := context.WithTimeout(mox.Context, time.Minute)
latest, _, changelog, err := updates.Check(updatesctx, dns.StrictResolver{}, dns.Domain{ASCII: changelogDomain}, lastknown, changelogURL, changelogPubKey)
updatescancel()
if err != nil {
log.Infox("checking for updates", err, mlog.Field("latest", latest))
return next
}
if !latest.After(lastknown) {
log.Debug("no new version available")
return next
}
if len(changelog.Changes) == 0 {
log.Info("new version available, but changelog is empty, ignoring", mlog.Field("latest", latest))
return next
}
var cl string
for _, c := range changelog.Changes {
cl += "----\n\n" + strings.TrimSpace(c.Text) + "\n\n"
}
cl += "----"
a, err := store.OpenAccount(mox.Conf.Static.Postmaster.Account)
if err != nil {
log.Infox("open account for postmaster changelog delivery", err)
return next
}
defer func() {
err := a.Close()
log.Check(err, "closing account")
}()
f, err := store.CreateMessageTemp("changelog")
if err != nil {
log.Infox("making temporary message file for changelog delivery", err)
return next
}
defer func() {
name := f.Name()
err = f.Close()
log.Check(err, "closing temp changelog file")
err := os.Remove(name)
log.Check(err, "removing temp changelog file", mlog.Field("path", name))
}()
m := &store.Message{
Received: time.Now(),
Flags: store.Flags{Flagged: true},
}
n, err := fmt.Fprintf(f, "Date: %s\r\nSubject: mox %s available\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Transfer-Encoding: 8-bit\r\n\r\nHi!\r\n\r\nVersion %s of mox is available, this install is at %s.\r\n\r\nChanges:\r\n\r\n%s\r\n\r\nRemember to make a backup with \"mox backup\" before upgrading.\r\nPlease report any issues at https://github.com/mjl-/mox, thanks!\r\n\r\nCheers,\r\nmox\r\n", time.Now().Format(message.RFC5322Z), latest, latest, current, strings.ReplaceAll(cl, "\n", "\r\n"))
if err != nil {
log.Infox("writing temporary message file for changelog delivery", err)
return next
}
m.Size = int64(n)
if err := a.DeliverMailbox(log, mox.Conf.Static.Postmaster.Mailbox, m, f); err != nil {
log.Errorx("changelog delivery", err)
return next
}
log.Info("delivered changelog", mlog.Field("current", current), mlog.Field("lastknown", lastknown), mlog.Field("latest", latest))
if err := mox.StoreLastKnown(latest); err != nil {
// This will be awkward, we'll keep notifying the postmaster once every 24h...
log.Infox("updating last known version", err)
}
return next
}
go func() {
for {
next := checkUpdates()
time.Sleep(next)
}
}()
}
go monitorDNSBL(log)
ctlpath := mox.DataDirPath("ctl")
_ = os.Remove(ctlpath)
ctl, err := net.Listen("unix", ctlpath)
if err != nil {
log.Fatalx("listen on ctl unix domain socket", err)
}
go func() {
for {
conn, err := ctl.Accept()
if err != nil {
log.Printx("accept for ctl", err)
continue
}
cid := mox.Cid()
ctx := context.WithValue(mox.Context, mlog.CidKey, cid)
go servectl(ctx, log.WithCid(cid), conn, func() { shutdown(log) })
}
}()
// Remove old temporary files that somehow haven't been cleaned up.
tmpdir := mox.DataDirPath("tmp")
os.MkdirAll(tmpdir, 0770)
tmps, err := os.ReadDir(tmpdir)
if err != nil {
log.Errorx("listing files in tmpdir", err)
} else {
now := time.Now()
for _, e := range tmps {
if fi, err := e.Info(); err != nil {
log.Errorx("stat tmp file", err, mlog.Field("filename", e.Name()))
} else if now.Sub(fi.ModTime()) > 7*24*time.Hour && !fi.IsDir() {
p := filepath.Join(tmpdir, e.Name())
if err := os.Remove(p); err != nil {
log.Errorx("removing stale temporary file", err, mlog.Field("path", p))
} else {
log.Info("removed stale temporary file", mlog.Field("path", p))
}
}
}
}
// Graceful shutdown.
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, syscall.SIGTERM)
sig := <-sigc
log.Print("shutting down, waiting max 3s for existing connections", mlog.Field("signal", sig))
shutdown(log)
if num, ok := sig.(syscall.Signal); ok {
os.Exit(int(num))
} else {
os.Exit(1)
}
}
// Set correct permissions for mox working directory, binary, config and data and service file.
//
// We require being able to stat the basic non-optional paths. Then we'll try to
// fix up permissions. If an error occurs when fixing permissions, we log and
// continue (could not be an actual problem).
func fixperms(log *mlog.Log, workdir, configdir, datadir string, moxuid, moxgid uint32) (rerr error) {
type fserr struct{ Err error }
defer func() {
x := recover()
if x == nil {
return
}
e, ok := x.(fserr)
if ok {
rerr = e.Err
} else {
panic(x)
}
}()
checkf := func(err error, format string, args ...any) {
if err != nil {
panic(fserr{fmt.Errorf(format, args...)})
}
}
// Changes we have to make. We collect them first, then apply.
type change struct {
path string
uid, gid *uint32
olduid, oldgid uint32
mode *fs.FileMode
oldmode fs.FileMode
}
var changes []change
ensure := func(p string, uid, gid uint32, perm fs.FileMode) bool {
fi, err := os.Stat(p)
checkf(err, "stat %s", p)
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
checkf(fmt.Errorf("got %T", st), "stat sys, expected syscall.Stat_t")
}
var ch change
if st.Uid != uid || st.Gid != gid {
ch.uid = &uid
ch.gid = &gid
ch.olduid = st.Uid
ch.oldgid = st.Gid
}
if perm != fi.Mode()&(fs.ModeSetgid|0777) {
ch.mode = &perm
ch.oldmode = fi.Mode() & (fs.ModeSetgid | 0777)
}
var zerochange change
if ch == zerochange {
return false
}
ch.path = p
changes = append(changes, ch)
return true
}
xexists := func(p string) bool {
_, err := os.Stat(p)
if err != nil && !os.IsNotExist(err) {
checkf(err, "stat %s", p)
}
return err == nil
}
// We ensure these permissions:
//
// $workdir root:mox 0751
// $configdir mox:root 0750 + setgid, and recursively (but files 0640)
// $datadir mox:root 0750 + setgid, and recursively (but files 0640)
// $workdir/mox (binary, optional) root:mox 0750
// $workdir/mox.service (systemd service file, optional) root:root 0644
const root = 0
ensure(workdir, root, moxgid, 0751)
fixconfig := ensure(configdir, moxuid, 0, fs.ModeSetgid|0750)
fixdata := ensure(datadir, moxuid, 0, fs.ModeSetgid|0750)
// Binary and systemd service file do not exist (there) when running under docker.
binary := filepath.Join(workdir, "mox")
if xexists(binary) {
ensure(binary, root, moxgid, 0750)
}
svc := filepath.Join(workdir, "mox.service")
if xexists(svc) {
ensure(svc, root, root, 0644)
}
if len(changes) == 0 {
return
}
// Apply changes.
log.Print("fixing up permissions, will continue on errors")
for _, ch := range changes {
if ch.uid != nil {
err := os.Chown(ch.path, int(*ch.uid), int(*ch.gid))
log.Printx("chown, fixing uid/gid", err, mlog.Field("path", ch.path), mlog.Field("olduid", ch.olduid), mlog.Field("oldgid", ch.oldgid), mlog.Field("newuid", *ch.uid), mlog.Field("newgid", *ch.gid))
}
if ch.mode != nil {
err := os.Chmod(ch.path, *ch.mode)
log.Printx("chmod, fixing permissions", err, mlog.Field("path", ch.path), mlog.Field("oldmode", fmt.Sprintf("%03o", ch.oldmode)), mlog.Field("newmode", fmt.Sprintf("%03o", *ch.mode)))
}
}
walkchange := func(dir string) {
err := filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
log.Printx("walk error, continuing", err, mlog.Field("path", path))
return nil
}
fi, err := d.Info()
if err != nil {
log.Printx("stat during walk, continuing", err, mlog.Field("path", path))
return nil
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
log.Printx("syscall stat during walk, continuing", err, mlog.Field("path", path))
return nil
}
if st.Uid != moxuid || st.Gid != root {
err := os.Chown(path, int(moxuid), root)
log.Printx("walk chown, fixing uid/gid", err, mlog.Field("path", path), mlog.Field("olduid", st.Uid), mlog.Field("oldgid", st.Gid), mlog.Field("newuid", moxuid), mlog.Field("newgid", root))
}
omode := fi.Mode() & (fs.ModeSetgid | 0777)
var nmode fs.FileMode
if fi.IsDir() {
nmode = fs.ModeSetgid | 0750
} else {
nmode = 0640
}
if omode != nmode {
err := os.Chmod(path, nmode)
log.Printx("walk chmod, fixing permissions", err, mlog.Field("path", path), mlog.Field("oldmode", fmt.Sprintf("%03o", omode)), mlog.Field("newmode", fmt.Sprintf("%03o", nmode)))
}
return nil
})
log.Check(err, "walking dir to fix permissions", mlog.Field("dir", dir))
}
// If config or data dir needed fixing, also set uid/gid and mode and files/dirs
// inside, recursively. We don't always recurse, data probably contains many files.
if fixconfig {
log.Print("fixing permissions in config dir", mlog.Field("configdir", configdir))
walkchange(configdir)
}
if fixdata {
log.Print("fixing permissions in data dir", mlog.Field("configdir", configdir))
walkchange(datadir)
}
return nil
}

16
serve_windows.go Normal file
View file

@ -0,0 +1,16 @@
package main
import (
"log"
)
// also see localserve.go, code is similar or even shared.
func cmdServe(c *cmd) {
c.help = `Start mox, serving SMTP/IMAP/HTTPS. Not implemented on windows.
`
args := c.Parse()
if len(args) != 0 {
c.Usage()
}
log.Fatalln("mox serve not implemented on windows yet due to unfamiliarity with the windows security model, other commands including localserve do work")
}

17
smtpserver/dsn.go
View file

@ -6,6 +6,7 @@ import (
"os"
"github.com/mjl-/mox/dsn"
"github.com/mjl-/mox/mlog"
"github.com/mjl-/mox/queue"
"github.com/mjl-/mox/smtp"
"github.com/mjl-/mox/store"
@ -30,12 +31,11 @@ func queueDSN(ctx context.Context, c *conn, rcptTo smtp.Path, m dsn.Message) err
return fmt.Errorf("creating temp file: %w", err)
}
defer func() {
if f != nil {
err := os.Remove(f.Name())
c.log.Check(err, "removing temporary dsn message file")
err = f.Close()
c.log.Check(err, "closing temporary dsn message file")
}
name := f.Name()
err = f.Close()
c.log.Check(err, "closing temporary dsn message file")
err := os.Remove(name)
c.log.Check(err, "removing temporary dsn message file", mlog.Field("path", name))
}()
if _, err := f.Write([]byte(buf)); err != nil {
return fmt.Errorf("writing dsn file: %w", err)
@ -46,11 +46,8 @@ func queueDSN(ctx context.Context, c *conn, rcptTo smtp.Path, m dsn.Message) err
// ../rfc/3464:433
const has8bit = false
const smtputf8 = false
if _, err := queue.Add(ctx, c.log, "", smtp.Path{}, rcptTo, has8bit, smtputf8, int64(len(buf)), m.MessageID, nil, f, bufUTF8, true); err != nil {
if _, err := queue.Add(ctx, c.log, "", smtp.Path{}, rcptTo, has8bit, smtputf8, int64(len(buf)), m.MessageID, nil, f, bufUTF8); err != nil {
return err
}
err = f.Close()
c.log.Check(err, "closing dsn file")
f = nil
return nil
}

3
smtpserver/fuzz_test.go
View file

@ -4,6 +4,7 @@ import (
"fmt"
"net"
"os"
"path/filepath"
"testing"
"time"
@ -30,7 +31,7 @@ func FuzzServer(f *testing.F) {
f.Add("QUIT")
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/smtpserverfuzz/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/smtpserverfuzz/mox.conf")
mox.MustLoadConfig(true, false)
dataDir := mox.ConfigDirPath(mox.Conf.Static.DataDir)
os.RemoveAll(dataDir)

3
smtpserver/reputation_test.go
View file

@ -4,6 +4,7 @@ import (
"fmt"
"net"
"os"
"path/filepath"
"testing"
"time"
@ -99,7 +100,7 @@ func TestReputation(t *testing.T) {
check := func(m store.Message, history []store.Message, expJunk *bool, expConclusive bool, expMethod reputationMethod) {
t.Helper()
p := "../testdata/smtpserver-reputation.db"
p := filepath.FromSlash("../testdata/smtpserver-reputation.db")
defer os.Remove(p)
db, err := bstore.Open(ctxbg, p, &bstore.Options{Timeout: 5 * time.Second}, store.DBTypes...)

41
smtpserver/server.go
View file

@ -1514,12 +1514,11 @@ func (c *conn) cmdData(p *parser) {
xsmtpServerErrorf(errCodes(smtp.C451LocalErr, smtp.SeSys3Other0, err), "creating temporary file for message: %s", err)
}
defer func() {
if dataFile != nil {
err := os.Remove(dataFile.Name())
c.log.Check(err, "removing temporary message file", mlog.Field("path", dataFile.Name()))
err = dataFile.Close()
c.log.Check(err, "removing temporary message file")
}
name := dataFile.Name()
err := dataFile.Close()
c.log.Check(err, "removing temporary message file")
err = os.Remove(name)
c.log.Check(err, "removing temporary message file", mlog.Field("path", name))
}()
msgWriter := message.NewWriter(dataFile)
dr := smtp.NewDataReader(c.r)
@ -1660,18 +1659,16 @@ func (c *conn) cmdData(p *parser) {
// handle it first, and leave the rest of the function for handling wild west
// internet traffic.
if c.submission {
c.submit(cmdctx, recvHdrFor, msgWriter, &dataFile)
c.submit(cmdctx, recvHdrFor, msgWriter, dataFile)
} else {
c.deliver(cmdctx, recvHdrFor, msgWriter, iprevStatus, iprevAuthentic, &dataFile)
c.deliver(cmdctx, recvHdrFor, msgWriter, iprevStatus, iprevAuthentic, dataFile)
}
}
// submit is used for mail from authenticated users that we will try to deliver.
func (c *conn) submit(ctx context.Context, recvHdrFor func(string) string, msgWriter *message.Writer, pdataFile **os.File) {
func (c *conn) submit(ctx context.Context, recvHdrFor func(string) string, msgWriter *message.Writer, dataFile *os.File) {
// Similar between ../smtpserver/server.go:/submit\( and ../webmail/webmail.go:/MessageSubmit\(
dataFile := *pdataFile
var msgPrefix []byte
// Check that user is only sending email as one of its configured identities. Not
@ -1774,7 +1771,7 @@ func (c *conn) submit(ctx context.Context, recvHdrFor func(string) string, msgWr
// We always deliver through the queue. It would be more efficient to deliver
// directly, but we don't want to circumvent all the anti-spam measures. Accounts
// on a single mox instance should be allowed to block each other.
for i, rcptAcc := range c.recipients {
for _, rcptAcc := range c.recipients {
if Localserve {
code, timeout := localserveNeedsError(rcptAcc.rcptTo.Localpart)
if timeout {
@ -1790,7 +1787,7 @@ func (c *conn) submit(ctx context.Context, recvHdrFor func(string) string, msgWr
xmsgPrefix := append([]byte(recvHdrFor(rcptAcc.rcptTo.String())), msgPrefix...)
msgSize := int64(len(xmsgPrefix)) + msgWriter.Size
if _, err := queue.Add(ctx, c.log, c.account.Name, *c.mailFrom, rcptAcc.rcptTo, msgWriter.Has8bit, c.smtputf8, msgSize, messageID, xmsgPrefix, dataFile, nil, i == len(c.recipients)-1); err != nil {
if _, err := queue.Add(ctx, c.log, c.account.Name, *c.mailFrom, rcptAcc.rcptTo, msgWriter.Has8bit, c.smtputf8, msgSize, messageID, xmsgPrefix, dataFile, nil); err != nil {
// Aborting the transaction is not great. But continuing and generating DSNs will
// probably result in errors as well...
metricSubmission.WithLabelValues("queueerror").Inc()
@ -1804,10 +1801,6 @@ func (c *conn) submit(ctx context.Context, recvHdrFor func(string) string, msgWr
xcheckf(err, "adding outgoing message")
}
err = dataFile.Close()
c.log.Check(err, "closing file after submission")
*pdataFile = nil
c.transactionGood++
c.transactionBad-- // Compensate for early earlier pessimistic increase.
@ -1866,9 +1859,7 @@ func (c *conn) xlocalserveError(lp smtp.Localpart) {
// deliver is called for incoming messages from external, typically untrusted
// sources. i.e. not submitted by authenticated users.
func (c *conn) deliver(ctx context.Context, recvHdrFor func(string) string, msgWriter *message.Writer, iprevStatus iprev.Status, iprevAuthentic bool, pdataFile **os.File) {
dataFile := *pdataFile
func (c *conn) deliver(ctx context.Context, recvHdrFor func(string) string, msgWriter *message.Writer, iprevStatus iprev.Status, iprevAuthentic bool, dataFile *os.File) {
// todo: in decision making process, if we run into (some) temporary errors, attempt to continue. if we decide to accept, all good. if we decide to reject, we'll make it a temporary reject.
msgFrom, headers, err := message.From(c.log, false, dataFile)
@ -2381,7 +2372,7 @@ func (c *conn) deliver(ctx context.Context, recvHdrFor func(string) string, msgW
if err != nil {
log.Errorx("tidying rejects mailbox", err)
} else if hasSpace {
if err := acc.DeliverMailbox(log, conf.RejectsMailbox, m, dataFile, false); err != nil {
if err := acc.DeliverMailbox(log, conf.RejectsMailbox, m, dataFile); err != nil {
log.Errorx("delivering spammy mail to rejects mailbox", err)
} else {
log.Info("delivered spammy mail to rejects mailbox")
@ -2456,7 +2447,7 @@ func (c *conn) deliver(ctx context.Context, recvHdrFor func(string) string, msgW
}
}
acc.WithWLock(func() {
if err := acc.DeliverMailbox(log, a.mailbox, m, dataFile, false); err != nil {
if err := acc.DeliverMailbox(log, a.mailbox, m, dataFile); err != nil {
log.Errorx("delivering", err)
metricDelivery.WithLabelValues("delivererror", a.reason).Inc()
addError(rcptAcc, smtp.C451LocalErr, smtp.SeSys3Other0, false, "error processing")
@ -2562,12 +2553,6 @@ func (c *conn) deliver(ctx context.Context, recvHdrFor func(string) string, msgW
}
}
err = os.Remove(dataFile.Name())
c.log.Check(err, "removing file after delivery")
err = dataFile.Close()
c.log.Check(err, "closing data file after delivery")
*pdataFile = nil
c.transactionGood++
c.transactionBad-- // Compensate for early earlier pessimistic increase.
c.rset()

38
smtpserver/server_test.go
View file

@ -193,7 +193,7 @@ func fakeCert(t *testing.T) tls.Certificate {
// Test submission from authenticated user.
func TestSubmission(t *testing.T) {
ts := newTestServer(t, "../testdata/smtp/mox.conf", dns.MockResolver{})
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), dns.MockResolver{})
defer ts.close()
// Set DKIM signing config.
@ -254,7 +254,7 @@ func TestDelivery(t *testing.T) {
},
PTR: map[string][]string{},
}
ts := newTestServer(t, "../testdata/smtp/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), resolver)
defer ts.close()
ts.run(func(err error, client *smtpclient.Client) {
@ -333,9 +333,11 @@ func TestDelivery(t *testing.T) {
func tinsertmsg(t *testing.T, acc *store.Account, mailbox string, m *store.Message, msg string) {
mf, err := store.CreateMessageTemp("queue-dsn")
tcheck(t, err, "temp message")
defer os.Remove(mf.Name())
defer mf.Close()
_, err = mf.Write([]byte(msg))
tcheck(t, err, "write message")
err = acc.DeliverMailbox(xlog, mailbox, m, mf, true)
err = acc.DeliverMailbox(xlog, mailbox, m, mf)
tcheck(t, err, "deliver message")
err = mf.Close()
tcheck(t, err, "close message")
@ -392,7 +394,7 @@ func TestSpam(t *testing.T) {
"_dmarc.example.org.": {"v=DMARC1;p=reject"},
},
}
ts := newTestServer(t, "../testdata/smtp/junk/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/junk/mox.conf"), resolver)
defer ts.close()
// Insert spammy messages. No junkfilter training yet.
@ -538,7 +540,7 @@ func TestForward(t *testing.T) {
rcptTo = "mjl@mox.example" // Without IsForward rule.
}
ts := newTestServer(t, "../testdata/smtp/junk/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/junk/mox.conf"), resolver)
defer ts.close()
var msgBad = strings.ReplaceAll(`From: <remote@bad.example>
@ -645,7 +647,7 @@ func TestDMARCSent(t *testing.T) {
"_dmarc.example.org.": {"v=DMARC1;p=reject"},
},
}
ts := newTestServer(t, "../testdata/smtp/junk/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/junk/mox.conf"), resolver)
defer ts.close()
// Insert spammy messages not related to the test message.
@ -710,7 +712,7 @@ func TestBlocklistedSubjectpass(t *testing.T) {
"127.0.0.10": {"example.org."}, // For iprev check.
},
}
ts := newTestServer(t, "../testdata/smtp/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), resolver)
ts.dnsbls = []dns.Domain{{ASCII: "dnsbl.example"}}
defer ts.close()
@ -776,7 +778,7 @@ func TestDMARCReport(t *testing.T) {
"127.0.0.10": {"example.org."}, // For iprev check.
},
}
ts := newTestServer(t, "../testdata/smtp/dmarcreport/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/dmarcreport/mox.conf"), resolver)
defer ts.close()
run := func(report string, n int) {
@ -899,7 +901,7 @@ func TestTLSReport(t *testing.T) {
"127.0.0.10": {"example.org."}, // For iprev check.
},
}
ts := newTestServer(t, "../testdata/smtp/tlsrpt/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/tlsrpt/mox.conf"), resolver)
defer ts.close()
run := func(tlsrpt string, n int) {
@ -939,7 +941,7 @@ func TestTLSReport(t *testing.T) {
}
func TestRatelimitConnectionrate(t *testing.T) {
ts := newTestServer(t, "../testdata/smtp/mox.conf", dns.MockResolver{})
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), dns.MockResolver{})
defer ts.close()
// We'll be creating 300 connections, no TLS and reduce noise.
@ -965,7 +967,7 @@ func TestRatelimitConnectionrate(t *testing.T) {
}
func TestRatelimitAuth(t *testing.T) {
ts := newTestServer(t, "../testdata/smtp/mox.conf", dns.MockResolver{})
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), dns.MockResolver{})
defer ts.close()
ts.submission = true
@ -1006,7 +1008,7 @@ func TestRatelimitDelivery(t *testing.T) {
"127.0.0.10": {"example.org."},
},
}
ts := newTestServer(t, "../testdata/smtp/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), resolver)
defer ts.close()
orig := limitIPMasked1MessagesPerMinute
@ -1061,7 +1063,7 @@ func TestRatelimitDelivery(t *testing.T) {
}
func TestNonSMTP(t *testing.T) {
ts := newTestServer(t, "../testdata/smtp/mox.conf", dns.MockResolver{})
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), dns.MockResolver{})
defer ts.close()
ts.cid += 2
@ -1105,7 +1107,7 @@ func TestNonSMTP(t *testing.T) {
// Test limits on outgoing messages.
func TestLimitOutgoing(t *testing.T) {
ts := newTestServer(t, "../testdata/smtp/sendlimit/mox.conf", dns.MockResolver{})
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/sendlimit/mox.conf"), dns.MockResolver{})
defer ts.close()
ts.user = "mjl@mox.example"
@ -1149,7 +1151,7 @@ func TestCatchall(t *testing.T) {
"127.0.0.10": {"other.example."},
},
}
ts := newTestServer(t, "../testdata/smtp/catchall/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/catchall/mox.conf"), resolver)
defer ts.close()
testDeliver := func(rcptTo string, expErr *smtpclient.Error) {
@ -1195,7 +1197,7 @@ func TestDKIMSign(t *testing.T) {
},
}
ts := newTestServer(t, "../testdata/smtp/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), resolver)
defer ts.close()
// Set DKIM signing config.
@ -1287,7 +1289,7 @@ func TestPostmaster(t *testing.T) {
"127.0.0.10": {"other.example."},
},
}
ts := newTestServer(t, "../testdata/smtp/postmaster/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/postmaster/mox.conf"), resolver)
defer ts.close()
testDeliver := func(rcptTo string, expErr *smtpclient.Error) {
@ -1321,7 +1323,7 @@ func TestEmptylocalpart(t *testing.T) {
"127.0.0.10": {"other.example."},
},
}
ts := newTestServer(t, "../testdata/smtp/mox.conf", resolver)
ts := newTestServer(t, filepath.FromSlash("../testdata/smtp/mox.conf"), resolver)
defer ts.close()
testDeliver := func(rcptTo string, expErr *smtpclient.Error) {

27
store/account.go
View file

@ -1189,9 +1189,6 @@ func (a *Account) WithRLock(fn func()) {
// DeliverMessage delivers a mail message to the account.
//
// If consumeFile is set, the original msgFile is moved/renamed or copied and
// removed as part of delivery.
//
// The message, with msg.MsgPrefix and msgFile combined, must have a header
// section. The caller is responsible for adding a header separator to
// msg.MsgPrefix if missing from an incoming message.
@ -1210,7 +1207,7 @@ func (a *Account) WithRLock(fn func()) {
// Caller must broadcast new message.
//
// Caller must update mailbox counts.
func (a *Account) DeliverMessage(log *mlog.Log, tx *bstore.Tx, m *Message, msgFile *os.File, consumeFile, sync, notrain, nothreads bool) error {
func (a *Account) DeliverMessage(log *mlog.Log, tx *bstore.Tx, m *Message, msgFile *os.File, sync, notrain, nothreads bool) error {
if m.Expunged {
return fmt.Errorf("cannot deliver expunged message")
}
@ -1346,12 +1343,7 @@ func (a *Account) DeliverMessage(log *mlog.Log, tx *bstore.Tx, m *Message, msgFi
}
}
if consumeFile {
if err := os.Rename(msgFile.Name(), msgPath); err != nil {
// Could be due to cross-filesystem rename. Users shouldn't configure their systems that way.
return fmt.Errorf("moving msg file to destination directory: %w", err)
}
} else if err := moxio.LinkOrCopy(log, msgPath, msgFile.Name(), &moxio.AtReader{R: msgFile}, true); err != nil {
if err := moxio.LinkOrCopy(log, msgPath, msgFile.Name(), &moxio.AtReader{R: msgFile}, true); err != nil {
return fmt.Errorf("linking/copying message to new file: %w", err)
}
@ -1647,7 +1639,7 @@ ruleset:
// MessagePath returns the file system path of a message.
func (a *Account) MessagePath(messageID int64) string {
return strings.Join(append([]string{a.Dir, "msg"}, messagePathElems(messageID)...), "/")
return strings.Join(append([]string{a.Dir, "msg"}, messagePathElems(messageID)...), string(filepath.Separator))
}
// MessageReader opens a message for reading, transparently combining the
@ -1661,7 +1653,7 @@ func (a *Account) MessageReader(m Message) *MsgReader {
// Caller must hold account wlock (mailbox may be created).
// Message delivery, possible mailbox creation, and updated mailbox counts are
// broadcasted.
func (a *Account) DeliverDestination(log *mlog.Log, dest config.Destination, m *Message, msgFile *os.File, consumeFile bool) error {
func (a *Account) DeliverDestination(log *mlog.Log, dest config.Destination, m *Message, msgFile *os.File) error {
var mailbox string
rs := MessageRuleset(log, dest, m, m.MsgPrefix, msgFile)
if rs != nil {
@ -1671,7 +1663,7 @@ func (a *Account) DeliverDestination(log *mlog.Log, dest config.Destination, m *
} else {
mailbox = dest.Mailbox
}
return a.DeliverMailbox(log, mailbox, m, msgFile, consumeFile)
return a.DeliverMailbox(log, mailbox, m, msgFile)
}
// DeliverMailbox delivers an email to the specified mailbox.
@ -1679,7 +1671,7 @@ func (a *Account) DeliverDestination(log *mlog.Log, dest config.Destination, m *
// Caller must hold account wlock (mailbox may be created).
// Message delivery, possible mailbox creation, and updated mailbox counts are
// broadcasted.
func (a *Account) DeliverMailbox(log *mlog.Log, mailbox string, m *Message, msgFile *os.File, consumeFile bool) error {
func (a *Account) DeliverMailbox(log *mlog.Log, mailbox string, m *Message, msgFile *os.File) error {
var changes []Change
err := a.DB.Write(context.TODO(), func(tx *bstore.Tx) error {
mb, chl, err := a.MailboxEnsure(tx, mailbox, true)
@ -1696,7 +1688,7 @@ func (a *Account) DeliverMailbox(log *mlog.Log, mailbox string, m *Message, msgF
return fmt.Errorf("updating mailbox for delivery: %w", err)
}
if err := a.DeliverMessage(log, tx, m, msgFile, consumeFile, true, false, false); err != nil {
if err := a.DeliverMessage(log, tx, m, msgFile, true, false, false); err != nil {
return err
}
@ -1977,10 +1969,11 @@ func OpenEmail(email string) (*Account, config.Destination, error) {
// 64 characters, must be power of 2 for MessagePath
const msgDirChars = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ-_"
// MessagePath returns the filename of the on-disk filename, relative to the containing directory such as <account>/msg or queue.
// MessagePath returns the filename of the on-disk filename, relative to the
// containing directory such as <account>/msg or queue.
// Returns names like "AB/1".
func MessagePath(messageID int64) string {
return strings.Join(messagePathElems(messageID), "/")
return strings.Join(messagePathElems(messageID), string(filepath.Separator))
}
// messagePathElems returns the elems, for a single join without intermediate

18
store/account_test.go
View file

@ -3,6 +3,7 @@ package store
import (
"context"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
@ -28,7 +29,7 @@ func tcheck(t *testing.T, err error, msg string) {
func TestMailbox(t *testing.T) {
os.RemoveAll("../testdata/store/data")
mox.ConfigStaticPath = "../testdata/store/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/store/mox.conf")
mox.MustLoadConfig(true, false)
acc, err := OpenAccount("mjl")
tcheck(t, err, "open account")
@ -44,6 +45,7 @@ func TestMailbox(t *testing.T) {
if err != nil {
t.Fatalf("creating temp msg file: %s", err)
}
defer os.Remove(msgFile.Name())
defer msgFile.Close()
msgWriter := message.NewWriter(msgFile)
if _, err := msgWriter.Write([]byte(" message")); err != nil {
@ -70,7 +72,7 @@ func TestMailbox(t *testing.T) {
}
acc.WithWLock(func() {
conf, _ := acc.Conf()
err := acc.DeliverDestination(xlog, conf.Destinations["mjl"], &m, msgFile, false)
err := acc.DeliverDestination(xlog, conf.Destinations["mjl"], &m, msgFile)
tcheck(t, err, "deliver without consume")
err = acc.DB.Write(ctxbg, func(tx *bstore.Tx) error {
@ -79,7 +81,7 @@ func TestMailbox(t *testing.T) {
tcheck(t, err, "sent mailbox")
msent.MailboxID = mbsent.ID
msent.MailboxOrigID = mbsent.ID
err = acc.DeliverMessage(xlog, tx, &msent, msgFile, false, true, false, false)
err = acc.DeliverMessage(xlog, tx, &msent, msgFile, true, false, false)
tcheck(t, err, "deliver message")
if !msent.ThreadMuted || !msent.ThreadCollapsed {
t.Fatalf("thread muted & collapsed should have been copied from parent (duplicate message-id) m")
@ -95,7 +97,7 @@ func TestMailbox(t *testing.T) {
tcheck(t, err, "insert rejects mailbox")
mreject.MailboxID = mbrejects.ID
mreject.MailboxOrigID = mbrejects.ID
err = acc.DeliverMessage(xlog, tx, &mreject, msgFile, false, true, false, false)
err = acc.DeliverMessage(xlog, tx, &mreject, msgFile, true, false, false)
tcheck(t, err, "deliver message")
err = tx.Get(&mbrejects)
@ -108,7 +110,7 @@ func TestMailbox(t *testing.T) {
})
tcheck(t, err, "deliver as sent and rejects")
err = acc.DeliverDestination(xlog, conf.Destinations["mjl"], &mconsumed, msgFile, true)
err = acc.DeliverDestination(xlog, conf.Destinations["mjl"], &mconsumed, msgFile)
tcheck(t, err, "deliver with consume")
err = acc.DB.Write(ctxbg, func(tx *bstore.Tx) error {
@ -251,9 +253,11 @@ func TestMailbox(t *testing.T) {
}
func TestMessageRuleset(t *testing.T) {
f, err := os.Open("/dev/null")
tcheck(t, err, "open")
f, err := CreateMessageTemp("msgruleset")
tcheck(t, err, "creating temp msg file")
defer os.Remove(f.Name())
defer f.Close()
msgBuf := []byte(strings.ReplaceAll(`List-ID: <test.mox.example>
test

13
store/export.go
View file

@ -82,10 +82,11 @@ type DirArchiver struct {
}
// Create create name in the file system, in dir.
// name must always use forwarded slashes.
func (a DirArchiver) Create(name string, size int64, mtime time.Time) (io.WriteCloser, error) {
isdir := strings.HasSuffix(name, "/")
name = strings.TrimSuffix(name, "/")
p := filepath.Join(a.Dir, name)
p := filepath.Join(a.Dir, filepath.FromSlash(name))
os.MkdirAll(filepath.Dir(p), 0770)
if isdir {
return nil, os.Mkdir(p, 0770)
@ -213,8 +214,11 @@ func ExportMessages(ctx context.Context, log *mlog.Log, db *bstore.DB, accountDi
var mboxwriter *bufio.Writer
defer func() {
if mboxtmp != nil {
name := mboxtmp.Name()
err := mboxtmp.Close()
log.Check(err, "closing mbox temp file")
err = os.Remove(name)
log.Check(err, "removing mbox temp file", mlog.Field("name", name))
}
}()
@ -287,8 +291,11 @@ func ExportMessages(ctx context.Context, log *mlog.Log, db *bstore.DB, accountDi
if err := w.Close(); err != nil {
return fmt.Errorf("closing message file: %v", err)
}
name := mboxtmp.Name()
err = mboxtmp.Close()
log.Check(err, "closing temporary mbox file")
err = os.Remove(name)
log.Check(err, "removing temporary mbox file", mlog.Field("path", name))
mboxwriter = nil
mboxtmp = nil
return nil
@ -524,10 +531,6 @@ func ExportMessages(ctx context.Context, log *mlog.Log, db *bstore.DB, accountDi
if err != nil {
return fmt.Errorf("creating temp mbox file: %v", err)
}
// Remove file immediately, so we are sure we don't leave it around.
if err := os.Remove(mboxtmp.Name()); err != nil {
return fmt.Errorf("removing temp file just created: %v", err)
}
mboxwriter = bufio.NewWriter(mboxtmp)
}
}

15
store/export_test.go
View file

@ -20,7 +20,7 @@ func TestExport(t *testing.T) {
// and maildir/mbox. check there are 2 files in the repo, no errors.txt.
os.RemoveAll("../testdata/store/data")
mox.ConfigStaticPath = "../testdata/store/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/store/mox.conf")
mox.MustLoadConfig(true, false)
acc, err := OpenAccount("mjl")
tcheck(t, err, "open account")
@ -32,16 +32,17 @@ func TestExport(t *testing.T) {
msgFile, err := CreateMessageTemp("mox-test-export")
tcheck(t, err, "create temp")
defer os.Remove(msgFile.Name()) // To be sure.
defer msgFile.Close()
const msg = "test: test\r\n\r\ntest\r\n"
_, err = msgFile.Write([]byte(msg))
tcheck(t, err, "write message")
m := Message{Received: time.Now(), Size: int64(len(msg))}
err = acc.DeliverMailbox(xlog, "Inbox", &m, msgFile, false)
err = acc.DeliverMailbox(xlog, "Inbox", &m, msgFile)
tcheck(t, err, "deliver")
m = Message{Received: time.Now(), Size: int64(len(msg))}
err = acc.DeliverMailbox(xlog, "Trash", &m, msgFile, true)
err = acc.DeliverMailbox(xlog, "Trash", &m, msgFile)
tcheck(t, err, "deliver")
var maildirZip, maildirTar, mboxZip, mboxTar bytes.Buffer
@ -61,8 +62,8 @@ func TestExport(t *testing.T) {
archive(ZipArchiver{zip.NewWriter(&mboxZip)}, false)
archive(TarArchiver{tar.NewWriter(&maildirTar)}, true)
archive(TarArchiver{tar.NewWriter(&mboxTar)}, false)
archive(DirArchiver{"../testdata/exportmaildir"}, true)
archive(DirArchiver{"../testdata/exportmbox"}, false)
archive(DirArchiver{filepath.FromSlash("../testdata/exportmaildir")}, true)
archive(DirArchiver{filepath.FromSlash("../testdata/exportmbox")}, false)
if r, err := zip.NewReader(bytes.NewReader(maildirZip.Bytes()), int64(maildirZip.Len())); err != nil {
t.Fatalf("reading maildir zip: %v", err)
@ -115,6 +116,6 @@ func TestExport(t *testing.T) {
}
}
checkDirFiles("../testdata/exportmaildir", 2)
checkDirFiles("../testdata/exportmbox", 2)
checkDirFiles(filepath.FromSlash("../testdata/exportmaildir"), 2)
checkDirFiles(filepath.FromSlash("../testdata/exportmbox"), 2)
}

14
store/import.go
View file

@ -84,10 +84,11 @@ func (mr *MboxReader) Next() (*Message, *os.File, string, error) {
}
defer func() {
if f != nil {
err := os.Remove(f.Name())
mr.log.Check(err, "removing temporary message file after mbox read error", mlog.Field("path", f.Name()))
err = f.Close()
name := f.Name()
err := f.Close()
mr.log.Check(err, "closing temporary message file after mbox read error")
err = os.Remove(name)
mr.log.Check(err, "removing temporary message file after mbox read error", mlog.Field("path", name))
}
}()
@ -272,10 +273,11 @@ func (mr *MaildirReader) Next() (*Message, *os.File, string, error) {
}
defer func() {
if f != nil {
err := os.Remove(f.Name())
mr.log.Check(err, "removing temporary message file after maildir read error", mlog.Field("path", f.Name()))
err = f.Close()
name := f.Name()
err := f.Close()
mr.log.Check(err, "closing temporary message file after maildir read error")
err = os.Remove(name)
mr.log.Check(err, "removing temporary message file after maildir read error", mlog.Field("path", name))
}
}()

8
store/import_test.go
View file

@ -24,15 +24,15 @@ func TestMboxReader(t *testing.T) {
if err != nil {
t.Fatalf("next mbox message: %v", err)
}
defer mf0.Close()
defer os.Remove(mf0.Name())
defer mf0.Close()
_, mf1, _, err := mr.Next()
if err != nil {
t.Fatalf("next mbox message: %v", err)
}
defer mf1.Close()
defer os.Remove(mf1.Name())
defer mf1.Close()
_, _, _, err = mr.Next()
if err != io.EOF {
@ -62,15 +62,15 @@ func TestMaildirReader(t *testing.T) {
if err != nil {
t.Fatalf("next maildir message: %v", err)
}
defer mf0.Close()
defer os.Remove(mf0.Name())
defer mf0.Close()
_, mf1, _, err := mr.Next()
if err != nil {
t.Fatalf("next maildir message: %v", err)
}
defer mf1.Close()
defer os.Remove(mf1.Name())
defer mf1.Close()
_, _, _, err = mr.Next()
if err != io.EOF {

11
store/msgreader_test.go
View file

@ -12,7 +12,13 @@ func TestMsgreader(t *testing.T) {
t.Fatalf("expected error for non-existing file, got %s", err)
}
if buf, err := io.ReadAll(&MsgReader{prefix: []byte("hello"), path: "/dev/null", size: int64(len("hello"))}); err != nil {
if err := os.WriteFile("emptyfile_test.txt", []byte{}, 0660); err != nil {
t.Fatalf("writing emptyfile_test.txt: %s", err)
}
defer os.Remove("emptyfile_test.txt")
mr := &MsgReader{prefix: []byte("hello"), path: "emptyfile_test.txt", size: int64(len("hello"))}
defer mr.Close()
if buf, err := io.ReadAll(mr); err != nil {
t.Fatalf("readall: %s", err)
} else if string(buf) != "hello" {
t.Fatalf("got %q, expected %q", buf, "hello")
@ -22,7 +28,8 @@ func TestMsgreader(t *testing.T) {
t.Fatalf("writing msgreader_test.txt: %s", err)
}
defer os.Remove("msgreader_test.txt")
mr := &MsgReader{prefix: []byte("hello"), path: "msgreader_test.txt", size: int64(len("hello world"))}
mr = &MsgReader{prefix: []byte("hello"), path: "msgreader_test.txt", size: int64(len("hello world"))}
defer mr.Close()
if buf, err := io.ReadAll(mr); err != nil {
t.Fatalf("readall: %s", err)
} else if string(buf) != "hello world" {

6
store/threads_test.go
View file

@ -2,6 +2,7 @@ package store
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@ -15,7 +16,7 @@ import (
func TestThreadingUpgrade(t *testing.T) {
os.RemoveAll("../testdata/store/data")
mox.ConfigStaticPath = "../testdata/store/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/store/mox.conf")
mox.MustLoadConfig(true, false)
acc, err := OpenAccount("mjl")
tcheck(t, err, "open account")
@ -32,6 +33,7 @@ func TestThreadingUpgrade(t *testing.T) {
t.Helper()
f, err := CreateMessageTemp("account-test")
tcheck(t, err, "temp file")
defer os.Remove(f.Name())
defer f.Close()
s = strings.ReplaceAll(s, "\n", "\r\n")
@ -40,7 +42,7 @@ func TestThreadingUpgrade(t *testing.T) {
MsgPrefix: []byte(s),
Received: recv,
}
err = acc.DeliverMailbox(log, "Inbox", &m, f, true)
err = acc.DeliverMailbox(log, "Inbox", &m, f)
tcheck(t, err, "deliver")
if expThreadID == 0 {
expThreadID = m.ID

2
tlsrptdb/db_test.go
View file

@ -63,7 +63,7 @@ const reportJSON = `{
func TestReport(t *testing.T) {
mox.Context = ctxbg
mox.Shutdown, mox.ShutdownCancel = context.WithCancel(ctxbg)
mox.ConfigStaticPath = "../testdata/tlsrpt/fake.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/tlsrpt/fake.conf")
mox.Conf.Static.DataDir = "."
// Recognize as configured domain.
mox.Conf.Dynamic.Domains = map[string]config.Domain{

3
vendor/github.com/mjl-/adns/Makefile generated vendored
View file

@ -27,7 +27,8 @@ buildall:
GOOS=illumos GOARCH=amd64 go build
GOOS=solaris GOARCH=amd64 go build
GOOS=aix GOARCH=ppc64 go build
# no windows or plan9 for now
GOOS=windows GOARCH=amd64 go build
# no plan9 for now
fmt:
gofmt -w -s *.go */*/*.go

65
vendor/github.com/mjl-/adns/dnsconfig_windows.go generated vendored Normal file
View file

@ -0,0 +1,65 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package adns
import (
"net"
"syscall"
"time"
"golang.org/x/sys/windows"
)
func dnsReadConfig(ignoredFilename string) (conf *dnsConfig) {
conf = &dnsConfig{
ndots: 1,
timeout: 5 * time.Second,
attempts: 2,
}
defer func() {
if len(conf.servers) == 0 {
conf.servers = defaultNS
}
}()
aas, err := adapterAddresses()
if err != nil {
return
}
// TODO(bradfitz): this just collects all the DNS servers on all
// the interfaces in some random order. It should order it by
// default route, or only use the default route(s) instead.
// In practice, however, it mostly works.
for _, aa := range aas {
for dns := aa.FirstDnsServerAddress; dns != nil; dns = dns.Next {
// Only take interfaces whose OperStatus is IfOperStatusUp(0x01) into DNS configs.
if aa.OperStatus != windows.IfOperStatusUp {
continue
}
sa, err := dns.Address.Sockaddr.Sockaddr()
if err != nil {
continue
}
var ip net.IP
switch sa := sa.(type) {
case *syscall.SockaddrInet4:
ip = net.IPv4(sa.Addr[0], sa.Addr[1], sa.Addr[2], sa.Addr[3])
case *syscall.SockaddrInet6:
ip = make(net.IP, net.IPv6len)
copy(ip, sa.Addr[:])
if ip[0] == 0xfe && ip[1] == 0xc0 {
// Ignore these fec0/10 ones. Windows seems to
// populate them as defaults on its misc rando
// interfaces.
continue
}
default:
// Unexpected type.
continue
}
conf.servers = append(conf.servers, JoinHostPort(ip.String(), "53"))
}
}
return conf
}

43
vendor/github.com/mjl-/adns/interface_windows.go generated vendored Normal file
View file

@ -0,0 +1,43 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package adns
import (
"os"
"syscall"
"unsafe"
"golang.org/x/sys/windows"
)
// adapterAddresses returns a list of IP adapter and address
// structures. The structure contains an IP adapter and flattened
// multiple IP addresses including unicast, anycast and multicast
// addresses.
func adapterAddresses() ([]*windows.IpAdapterAddresses, error) {
var b []byte
l := uint32(15000) // recommended initial size
for {
b = make([]byte, l)
err := windows.GetAdaptersAddresses(syscall.AF_UNSPEC, windows.GAA_FLAG_INCLUDE_PREFIX, 0, (*windows.IpAdapterAddresses)(unsafe.Pointer(&b[0])), &l)
if err == nil {
if l == 0 {
return nil, nil
}
break
}
if err.(syscall.Errno) != syscall.ERROR_BUFFER_OVERFLOW {
return nil, os.NewSyscallError("getadaptersaddresses", err)
}
if l <= uint32(len(b)) {
return nil, os.NewSyscallError("getadaptersaddresses", err)
}
}
var aas []*windows.IpAdapterAddresses
for aa := (*windows.IpAdapterAddresses)(unsafe.Pointer(&b[0])); aa != nil; aa = aa.Next {
aas = append(aas, aa)
}
return aas, nil
}

2
vendor/github.com/mjl-/adns/lookup_unix.go generated vendored
View file

@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build unix || wasip1
//go:build unix || wasip1 || windows
package adns

2
vendor/github.com/mjl-/adns/port_unix.go generated vendored
View file

@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build unix || (js && wasm) || wasip1
//go:build unix || (js && wasm) || wasip1 || windows
// Read system port mappings from /etc/services

2
vendor/modules.txt vendored
View file

@ -11,7 +11,7 @@ github.com/golang/protobuf/ptypes/timestamp
# github.com/matttproud/golang_protobuf_extensions v1.0.1
## explicit
github.com/matttproud/golang_protobuf_extensions/pbutil
# github.com/mjl-/adns v0.0.0-20231009145311-e3834995f16c
# github.com/mjl-/adns v0.0.0-20231013194548-ea0378d616ab
## explicit; go 1.20
github.com/mjl-/adns
github.com/mjl-/adns/internal/bytealg

10
webaccount/account.go
View file

@ -299,15 +299,13 @@ func Handle(w http.ResponseWriter, r *http.Request) {
}
defer func() {
if tmpf != nil {
name := tmpf.Name()
err := tmpf.Close()
log.Check(err, "closing uploaded file")
err = os.Remove(name)
log.Check(err, "removing temporary file", mlog.Field("path", name))
}
}()
if err := os.Remove(tmpf.Name()); err != nil {
log.Errorx("removing temporary file", err)
http.Error(w, "500 - internal server error - "+err.Error(), http.StatusInternalServerError)
return
}
if _, err := io.Copy(tmpf, f); err != nil {
log.Errorx("copying import to temporary file", err)
http.Error(w, "500 - internal server error - "+err.Error(), http.StatusInternalServerError)
@ -319,7 +317,7 @@ func Handle(w http.ResponseWriter, r *http.Request) {
http.Error(w, "500 - internal server error - "+err.Error(), http.StatusInternalServerError)
return
}
tmpf = nil // importStart is now responsible for closing.
tmpf = nil // importStart is now responsible for cleanup.
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]string{"ImportToken": token})

6
webaccount/account_test.go
View file

@ -38,7 +38,7 @@ func tcheck(t *testing.T, err error, msg string) {
func TestAccount(t *testing.T) {
os.RemoveAll("../testdata/httpaccount/data")
mox.ConfigStaticPath = "../testdata/httpaccount/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/httpaccount/mox.conf")
mox.ConfigDynamicPath = filepath.Join(filepath.Dir(mox.ConfigStaticPath), "domains.conf")
mox.MustLoadConfig(true, false)
acc, err := store.OpenAccount("mjl")
@ -144,8 +144,8 @@ func TestAccount(t *testing.T) {
t.Fatalf("imported %d messages, expected %d", count, expect)
}
}
testImport("../testdata/importtest.mbox.zip", 2)
testImport("../testdata/importtest.maildir.tgz", 2)
testImport(filepath.FromSlash("../testdata/importtest.mbox.zip"), 2)
testImport(filepath.FromSlash("../testdata/importtest.maildir.tgz"), 2)
// Check there are messages, with the right flags.
acc.DB.Read(ctxbg, func(tx *bstore.Tx) error {

30
webaccount/import.go
View file

@ -198,12 +198,15 @@ type importStep struct {
}
// importStart prepare the import and launches the goroutine to actually import.
// importStart is responsible for closing f.
// importStart is responsible for closing f and removing f.
func importStart(log *mlog.Log, accName string, f *os.File, skipMailboxPrefix string) (string, error) {
defer func() {
if f != nil {
name := f.Name()
err := f.Close()
log.Check(err, "closing uploaded file")
err = os.Remove(name)
log.Check(err, "removing uploaded file", mlog.Field("name", name))
}
}()
@ -270,7 +273,7 @@ func importStart(log *mlog.Log, accName string, f *os.File, skipMailboxPrefix st
log.Info("starting import")
go importMessages(ctx, log.WithCid(mox.Cid()), token, acc, tx, zr, tr, f, skipMailboxPrefix)
f = nil // importMessages is now responsible for closing.
f = nil // importMessages is now responsible for closing and removing.
return token, nil
}
@ -313,8 +316,11 @@ func importMessages(ctx context.Context, log *mlog.Log, token string, acc *store
}
defer func() {
name := f.Name()
err := f.Close()
log.Check(err, "closing uploaded messages file")
err = os.Remove(name)
log.Check(err, "removing uploaded messages file", mlog.Field("path", name))
for _, id := range deliveredIDs {
p := acc.MessagePath(id)
@ -482,12 +488,11 @@ func importMessages(ctx context.Context, log *mlog.Log, token string, acc *store
xdeliver := func(mb store.Mailbox, m *store.Message, f *os.File, pos string) {
defer func() {
if f != nil {
err := os.Remove(f.Name())
log.Check(err, "removing temporary message file for delivery")
err = f.Close()
log.Check(err, "closing temporary message file for delivery")
}
name := f.Name()
err = f.Close()
log.Check(err, "closing temporary message file for delivery")
err := os.Remove(name)
log.Check(err, "removing temporary message file for delivery")
}()
m.MailboxID = mb.ID
m.MailboxOrigID = mb.ID
@ -542,11 +547,10 @@ func importMessages(ctx context.Context, log *mlog.Log, token string, acc *store
trainMessage(m, p, pos)
}
const consumeFile = true
const sync = false
const notrain = true
const nothreads = true
if err := acc.DeliverMessage(log, tx, m, f, consumeFile, sync, notrain, nothreads); err != nil {
if err := acc.DeliverMessage(log, tx, m, f, sync, notrain, nothreads); err != nil {
problemf("delivering message %s: %s (continuing)", pos, err)
return
}
@ -557,7 +561,6 @@ func importMessages(ctx context.Context, log *mlog.Log, token string, acc *store
prevMailbox = mb.Name
sendEvent("count", importCount{mb.Name, messages[mb.Name]})
}
f = nil
}
ximportMbox := func(mailbox, filename string, r io.Reader) {
@ -591,10 +594,11 @@ func importMessages(ctx context.Context, log *mlog.Log, token string, acc *store
ximportcheckf(err, "creating temp message")
defer func() {
if f != nil {
err := os.Remove(f.Name())
log.Check(err, "removing temporary file for delivery")
name := f.Name()
err = f.Close()
log.Check(err, "closing temporary file for delivery")
err := os.Remove(name)
log.Check(err, "removing temporary file for delivery", mlog.Field("path", name))
}
}()

24
webmail/api.go
View file

@ -352,12 +352,11 @@ func (w Webmail) MessageSubmit(ctx context.Context, m SubmitMessage) {
dataFile, err := store.CreateMessageTemp("webmail-submit")
xcheckf(ctx, err, "creating temporary file for message")
defer func() {
if dataFile != nil {
err := dataFile.Close()
log.Check(err, "closing submit message file")
err = os.Remove(dataFile.Name())
log.Check(err, "removing temporary submit message file")
}
name := dataFile.Name()
err := dataFile.Close()
log.Check(err, "closing submit message file")
err = os.Remove(name)
log.Check(err, "removing temporary submit message file", mlog.Field("name", name))
}()
// If writing to the message file fails, we abort immediately.
@ -669,7 +668,7 @@ func (w Webmail) MessageSubmit(ctx context.Context, m SubmitMessage) {
Localpart: rcpt.Localpart,
IPDomain: dns.IPDomain{Domain: rcpt.Domain},
}
_, err := queue.Add(ctx, log, reqInfo.AccountName, fromPath, toPath, has8bit, smtputf8, msgSize, messageID, []byte(rcptMsgPrefix), dataFile, nil, false)
_, err := queue.Add(ctx, log, reqInfo.AccountName, fromPath, toPath, has8bit, smtputf8, msgSize, messageID, []byte(rcptMsgPrefix), dataFile, nil)
if err != nil {
metricSubmission.WithLabelValues("queueerror").Inc()
}
@ -720,11 +719,6 @@ func (w Webmail) MessageSubmit(ctx context.Context, m SubmitMessage) {
sentmb, err := bstore.QueryDB[store.Mailbox](ctx, acc.DB).FilterEqual("Sent", true).Get()
if err == bstore.ErrAbsent {
// There is no mailbox designated as Sent mailbox, so we're done.
err := os.Remove(dataFile.Name())
log.Check(err, "removing submitmessage file")
err = dataFile.Close()
log.Check(err, "closing submitmessage file")
dataFile = nil
return
}
xcheckf(ctx, err, "message submitted to queue, adding to Sent mailbox")
@ -749,7 +743,7 @@ func (w Webmail) MessageSubmit(ctx context.Context, m SubmitMessage) {
err = tx.Update(&sentmb)
xcheckf(ctx, err, "updating sent mailbox for counts")
err = acc.DeliverMessage(log, tx, &sentm, dataFile, true, true, false, false)
err = acc.DeliverMessage(log, tx, &sentm, dataFile, true, false, false)
if err != nil {
metricSubmission.WithLabelValues("storesenterror").Inc()
metricked = true
@ -757,10 +751,6 @@ func (w Webmail) MessageSubmit(ctx context.Context, m SubmitMessage) {
xcheckf(ctx, err, "message submitted to queue, appending message to Sent mailbox")
changes = append(changes, sentm.ChangeAddUID(), sentmb.ChangeCounts())
err = dataFile.Close()
log.Check(err, "closing submit message file")
dataFile = nil
})
store.BroadcastChanges(acc, changes)

3
webmail/api_test.go
View file

@ -5,6 +5,7 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"runtime/debug"
"testing"
@ -43,7 +44,7 @@ func TestAPI(t *testing.T) {
mox.LimitersInit()
os.RemoveAll("../testdata/webmail/data")
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/webmail/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/webmail/mox.conf")
mox.MustLoadConfig(true, false)
defer store.Switchboard()()

3
webmail/view_test.go
View file

@ -12,6 +12,7 @@ import (
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"reflect"
"testing"
"time"
@ -23,7 +24,7 @@ import (
func TestView(t *testing.T) {
os.RemoveAll("../testdata/webmail/data")
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/webmail/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/webmail/mox.conf")
mox.MustLoadConfig(true, false)
defer store.Switchboard()()

8
webmail/webmail.go
View file

@ -160,8 +160,8 @@ type merged struct {
var webmail = &merged{
fallbackHTML: webmailHTML,
fallbackJS: webmailJS,
htmlPath: "webmail/webmail.html",
jsPath: "webmail/webmail.js",
htmlPath: filepath.FromSlash("webmail/webmail.html"),
jsPath: filepath.FromSlash("webmail/webmail.js"),
}
// fallbackMtime returns a time to use for the Last-Modified header in case we
@ -733,7 +733,7 @@ func handle(apiHandler http.Handler, w http.ResponseWriter, r *http.Request) {
h.Set("Content-Type", "text/html; charset=utf-8")
h.Set("Cache-Control", "no-cache, max-age=0")
path := "webmail/msg.html"
path := filepath.FromSlash("webmail/msg.html")
fallback := webmailmsgHTML
serveContentFallback(log, w, r, path, fallback)
@ -800,7 +800,7 @@ func handle(apiHandler http.Handler, w http.ResponseWriter, r *http.Request) {
// We typically return the embedded file, but during development it's handy to load
// from disk.
path := "webmail/text.html"
path := filepath.FromSlash("webmail/text.html")
fallback := webmailtextHTML
serveContentFallback(log, w, r, path, fallback)

7
webmail/webmail_test.go
View file

@ -12,6 +12,7 @@ import (
"net/http/httptest"
"net/textproto"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@ -253,10 +254,12 @@ type testmsg struct {
func tdeliver(t *testing.T, acc *store.Account, tm *testmsg) {
msgFile, err := store.CreateMessageTemp("webmail-test")
tcheck(t, err, "create message temp")
defer os.Remove(msgFile.Name())
defer msgFile.Close()
size, err := msgFile.Write(tm.msg.Marshal(t))
tcheck(t, err, "write message temp")
m := store.Message{Flags: tm.Flags, Keywords: tm.Keywords, Size: int64(size)}
err = acc.DeliverMailbox(xlog, tm.Mailbox, &m, msgFile, true)
err = acc.DeliverMailbox(xlog, tm.Mailbox, &m, msgFile)
tcheck(t, err, "deliver test message")
err = msgFile.Close()
tcheck(t, err, "closing test message")
@ -272,7 +275,7 @@ func TestWebmail(t *testing.T) {
mox.LimitersInit()
os.RemoveAll("../testdata/webmail/data")
mox.Context = ctxbg
mox.ConfigStaticPath = "../testdata/webmail/mox.conf"
mox.ConfigStaticPath = filepath.FromSlash("../testdata/webmail/mox.conf")
mox.MustLoadConfig(true, false)
defer store.Switchboard()()