* log: make `sink` encodable
* deduplicate logger fields
* extract common fields into `BaseLog` and embed it into `SinkLog`
* amend godoc on `BaseLog` and `SinkLog`
* minor style change
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* caddyhttp: Determine real client IP if trusted proxies configured
* Support customizing client IP header
* Implement client_ip matcher, deprecate remote_ip's forwarded option
This adds:
- `{file.*}` -> `{http.request.uri.path.file.*}`
- `{file_match.*}` -> `{http.matchers.file.*}`
This is a follow-up to #4993 which introduces the new URI file placeholders, and a shortcut for using `file` matcher output.
For example, where the `try_files` directive is a shortcut for this:
```
@try_files file <files...>
rewrite @try_files {http.matchers.file.relative}
```
It could instead be:
```
@try_files file <files...>
rewrite @try_files {file_match.relative}
```
Previously, our "duplicate key in server block" logic was flawed because
it did not account for the site's bind address. We defer this check to
when the listener addresses have been assigned, but before we commit
a server block to its listener.
Also refined how network address parsing and joining works, which was
necessary for a less convoluted fix.
* httpcaddyfile: Add `{vars.*}` placeholder shortcut
I'm yoinking this from my https://github.com/caddyserver/caddy/pull/4657 PR because I think we should get this in ASAP for v2.5.0 along with the new `vars` directive.
* Sort vars by matchers in reverse
Guh, this is complicated.
Fixes#4640
This also follows up on #4398 (reverting it) which made a change that technically worked, but was incorrect. It changed the condition in `hostsFromKeysNotHTTP` from `&&` to `||`, but then the function no longer did what its name said it would do, and it would return hosts even if they were marked with `http://`, if they used a non-HTTP port. That wasn't the intent of it. The test added in there was kept though, because it is a valid usecase.
The actual fix is to check _earlier_ whether all the addresses explicitly have `http://`, and if so we can short circuit and skip considering the rest.
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.
* caddytls: GetCertificate modules; Tailscale
* Caddyfile support for get_certificate
Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).
And implement start of HTTP cert getter, still WIP.
* Update modules/caddytls/automation.go
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Use tsclient package, check status for name
* Implement HTTP cert getter
And use reuse CertMagic's PEM functions for private keys.
* Remove cache option from Tailscale getter
Tailscale does its own caching and we don't need the added complexity...
for now, at least.
* Several updates
- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging
* Run go mod tidy
* Try to get certificates from Tailscale implicitly
Only for domains ending in .ts.net.
I think this is really cool!
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Some new users mistakenly try to define two sites without braces around each. Doing this can yield a confusing error message saying that their site address is an "unknown directive".
We can do better by keeping track of whether the current site block was parsed with or without a brace, then changing the error message later based on that.
For example, now this invalid config:
```
foo.example.com
respond "foo"
bar.example.com
respond "bar"
```
Will yield this error message:
```
$ caddy adapt
2021/08/22 19:21:31.028 INFO using adjacent Caddyfile
adapt: Caddyfile:4: unrecognized directive: bar.example.com
Did you mean to define a second site? If so, you must use curly braces around each site to separate their configurations.
```
* httpcaddyfile: Add shortcut for proxy hostport placeholder
I've noticed that it's a pretty common pattern to write a proxy like this, when needing to proxy over HTTPS:
```
reverse_proxy https://example.com {
header_up Host {http.reverse_proxy.upstream.hostport}
}
```
I find it pretty hard to remember the exact placeholder to use for this, and I continually need to refer to the docs when I need it. I think a simple fix for this is to add another Caddyfile placeholder for this one to shorten it:
```
reverse_proxy https://example.com {
header_up Host {proxy_hostport}
}
```
* Switch the shortcut name
* httpcaddyfile: ensure hosts to skip can always be collected
Previously, some hosts that should be skipped in logging would
be missed as the current logic would only collect them after
encountering the first server that would log. This change makes sure
the ServerLogConfig is initialized before iterating over the server
blocks.
* httpcaddyfile: add test case for skip hosts behavior
This change is aimed at enhancing the logging module within the
Caddyfile directive to allow users to configure logs other than the HTTP
access log stream, which is the current capability of the Caddyfile [1].
The intent here is to leverage the same syntax as the server log
directive at a global level, so that similar customizations can be added
without needing to resort to a JSON-based configuration.
Discussion for this approach happened in the referenced issue.
Closes https://github.com/caddyserver/caddy/issues/3958
[1] https://caddyserver.com/docs/caddyfile/directives/log
The HTTP Caddyfile adapter can now configure the PKI app, and the acme_server directive can now be used to specify a custom CA used for issuing certificates. More customization options can follow later as needed.
This is probably an invasive change, but existing tests continue to pass.
It seems to make sense this way. There is likely an edge case I haven't
considered.
This changes the signature of UnmarshalGlobalFunc but this is probably OK since it's only used by this repo as far as we know.
We need this change in order to "remember" the previous value in case a global option appears more than once, which is now a possibility with the cert_issuer option since Caddy now supports multiple issuers in the order defined by the user.
Bonus: the issuer subdirective of tls now supports one-liner for "acme" when all you need to set is the directory:
issuer acme <dir>
* httpcaddyfile: First pass at implementing server options
* httpcaddyfile: Add listener wrapper support
* httpcaddyfile: Sort sbaddrs to make adapt output more deterministic
* httpcaddyfile: Add server options adapt tests
* httpcaddyfile: Windows line endings lol
* caddytest: More windows line endings lol (sorry Matt)
* Update caddyconfig/httpcaddyfile/serveroptions.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* httpcaddyfile: Reword listener address "matcher"
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* httpcaddyfile: Deprecate experimental_http3 option (moved to servers)
* httpcaddyfile: Remove validation step, no longer needed
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* httpcaddyfile: Ensure handle_path is sorted as equal to handle
* httpcaddyfile: Make mutual exclusivity grouping deterministic (I hope)
* httpcaddyfile: Add comment linking to the issue being fixed
* httpcaddyfile: Typo fix, comment clarity
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update caddyconfig/httpcaddyfile/httptype.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
We recently introduced `if !cp.SettingsEmpty()` which conditionally
adds the connection policy to the list. If the condition evaluates to
false, the policy wouldn't actually be added, even if
hasCatchAllTLSConnPolicy was set to true on the previous line.
Now we set that variable in accordance with whether we actually add
the policy.
While debugging this I noticed that catch-all policies added early in
that loop (i.e. not at the end if we later determine we need one) are
not always at the end of the list. They should be, though, since they
are selected by which one matches first, and having a catch-all first
would nullify any more specific ones later in the list. So I added a
sort in consolidateConnPolicies to take care of that.
Should fix#3670 and
https://caddy.community/t/combining-on-demand-tls-with-custom-ssl-certs-doesnt-seem-to-work-in-2-1-1/9719
but I won't know for sure until somebody verifies it, since at least in
the GitHub issue there is not yet enough information (the configs are
redacted).
* Bring `ensure_origin` and `origins` to caddyfile admin config
* Add unit test for caddyfile admin config update
* Add caddyfile adapt test for typical admin setup
* httpcaddyfile: Replace admin config error message when there's more arguments than needed
Replace d.Err() to d.ArgErr() since the latter provides similarly informative error message
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Catch-alls should always go last. Normally this is the case, but we have
a special case for comparing one wildcard-host site block to another
non-wildcard host site block; and a catch-all site block is also a
non-wildcard host site block, so now we have to special-case the
catch-all site block. Sigh.
This could be reproduced with a Caddyfile that has two site blocks:
":80" and "*.example.com", in that order.
* httpcaddyfile: Add shorthands for parameterized placeholders
httpcaddyfile: Now with regexp instead
httpcaddyfile: Allow dashes, gofmt
httpcaddyfile: Compile regexp only once
httpcaddyfile: Cleanup struct
httpcaddyfile: Optimize the replacers, pull out of the loop
httpcaddyfile: Add `{port}` shorthand
* httpcaddyfile: Switch `r.` to `re.`
