Commit graph

47 commits

Author SHA1 Message Date
Matthew Holt
e2f913bb7f
reverseproxy: Minor fixes and cleanup
Now use context cancellation to stop active health checker, which is
simpler than and just as effective as using a separate stop channel.
2020-08-07 18:02:24 -06:00
Kevin Lin
904f149e5b
reverse_proxy: fix bidirectional streams with encodings (fix #3606) (#3620)
* reverse_proxy: fix bi-h2stream breaking gzip encode handle(#3606).

* reverse_proxy: check http version of both sides to avoid affecting non-h2 upstream.

* Minor cleanup; apply review suggestions

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-08-03 20:50:38 -06:00
Kevin Lin
e9b1d7dcb4
reverse_proxy: flush HTTP/2 response when ContentLength is unknown (#3561)
* reverse proxy: Support more h2 stream scenarios (#3556)

* reverse proxy: add integration test for better h2 stream (#3556)

* reverse proxy: adjust comments as francislavoie suggests

* link to issue #3556 in the comments
2020-07-20 12:14:46 -06:00
Matthew Holt
246a31aacd
reverseproxy: Restore request's original host and header (fix #3509)
We already restore them within the retry loop, but after successful
proxy we didn't reset them, so as handlers bubble back up, they would
see the values used for proxying.

Thanks to @ziddey for identifying the cause.
2020-07-17 17:54:58 -06:00
Matthew Holt
7a99835dab
reverseproxy: Enable changing only the status code (close #2920) 2020-06-04 12:06:38 -06:00
Matthew Holt
881b826fb5
reverseproxy: Pool copy buffers (minor optimization) 2020-05-27 11:42:19 -06:00
Matthew Holt
538ddb8587 reverseproxy: Enable response interception (#1447, #2920)
It's a raw, low-level implementation for now, but it's very flexible.
More sugar-coating can be added after error handling is more developed.
2020-05-27 10:17:45 -06:00
Matthew Holt
2a8a198568
reverseproxy: Don't overwrite existing X-Forwarded-Proto header
Correct behavior is not well defined because this is a non-standard
header field. This could be a "hop-by-hop" field much like
X-Forwarded-For is, but even our X-Forwarded-For implementation
preserves prior entries. Or, it could be best to preserve the original
value from the first hop, representing the protocol as facing the
client.

Let's try it the other way for a bit and see how it goes.

See https://caddy.community/t/caddy2-w-wordpress-behind-nginx-reverse-proxy/8174/3?u=matt
2020-05-20 11:33:17 -06:00
Matthew Holt
9ee01dceac
reverseproxy: Make debug log safe if error occurs 2020-05-18 14:08:11 -06:00
Matthew Holt
812278acd8 reverseproxy: Emit debug log before checking error (#3425)
This way the upstream request will always be available even if it failed
2020-05-18 13:50:46 -06:00
Matt Holt
90c7b4b0a1
reverseproxy: Apply response header ops before copying it (fix #3382) (#3401) 2020-05-13 09:52:20 -06:00
Matthew Holt
1b061815b2
reverseproxy: Don't forget to provision embedded headers handler
https://caddy.community/t/set-cookie-manipulation-in-reverse-proxy/7666?u=matt
2020-04-22 19:57:06 -06:00
westwin
da8686c4b9
reverseproxy: always set req.URL.Host with upstream (#3297) 2020-04-21 20:34:00 -06:00
Matt Holt
76bbb473a5
reverseproxy: Set X-Forwarded-Proto (closes #3275) (#3276) 2020-04-17 09:53:06 -06:00
Matthew Holt
2c1b663156
reverseproxy: Remove NTLM transport; refactor and improve docs 2020-04-07 11:39:14 -06:00
Matthew Holt
105acfa086
Keep type information with placeholders until replacements happen 2020-03-30 11:49:53 -06:00
Matthew Holt
e207240f9a
reverse_proxy: Upstream.String() method returns either LookupSRV or Dial
Either Dial or LookupSRV will be set, but if we rely on Dial always
being set, we could run into bugs.

Note: Health checks don't support SRV upstreams.
2020-03-27 14:29:01 -06:00
Matt Holt
e02117cb8a
reverse_proxy: Add support for SRV backends (#3180)
* reverse_proxy: Begin SRV lookup support (WIP)

* reverse_proxy: Finish adding support for SRV-based backends (#3179)
2020-03-24 10:53:53 -06:00
Paolo Barbolini
42a6628935
reverseproxy: Add Alt-Svc to Hop-by-hop headers list (#3159)
Adds `Alt-Svc` to the list of headers that get removed when proxying
to a backend.

This fixes the issue of having the contents of the Alt-Svc header
duplicated when proxying to another Caddy server.
2020-03-20 06:54:28 -06:00
Matthew Holt
7cca291d62 reverse_proxy: Health checks: Don't cross the streams
Fixes https://caddy.community/t/v2-health-checks-are-going-to-the-wrong-upstream/7084?u=matt

... I think
2020-02-23 14:31:05 -07:00
Matt Holt
a5ebec0041
http: Change routes to sequential matcher evaluation (#2967)
Previously, all matchers in a route would be evaluated before any
handlers were executed, and a composite route of the matching routes
would be created. This made rewrites especially tricky, since the only
way to defer later matchers' evaluation was to wrap them in a subroute,
or to invoke a "rehandle" which often caused bugs.

Instead, this new sequential design evaluates each route's matchers then
its handlers in lock-step; matcher-handlers-matcher-handlers...

If the first matching route consists of a rewrite, then the second route
will be evaluated against the rewritten request, rather than the original
one, and so on.

This should do away with any need for rehandling.

I've also taken this opportunity to avoid adding new values to the
request context in the handler chain, as this creates a copy of the
Request struct, which may possibly lead to bugs like it has in the past
(see PR #1542, PR #1481, and maybe issue #2463). We now add all the
expected context values in the top-level handler at the server, then
any new values can be added to the variable table via the VarsCtxKey
context key, or just the GetVar/SetVar functions. In particular, we are
using this facility to convey dial information in the reverse proxy.

Had to be careful in one place as the middleware compilation logic has
changed, and moved a bit. We no longer compile a middleware chain per-
request; instead, we can compile it at provision-time, and defer only the
evaluation of matchers to request-time, which should slightly improve
performance. Doing this, however, we take advantage of multiple function
closures, and we also changed the use of HandlerFunc (function pointer)
to Handler (interface)... this led to a situation where, if we aren't
careful, allows one request routed a certain way to permanently change
the "next" handler for all/most other requests! We avoid this by making
a copy of the interface value (which is a lightweight pointer copy) and
using exclusively that within our wrapped handlers. This way, the
original stack frame is preserved in a "read-only" fashion. The comments
in the code describe this phenomenon.

This may very well be a breaking change for some configurations, however
I do not expect it to impact many people. I will make it clear in the
release notes that this change has occurred.
2020-01-09 10:00:13 -07:00
Matthew Holt
fdabac51a8
Improve docs, especially w.r.t. placeholders and template actions 2019-12-29 13:16:34 -07:00
Matthew Holt
95d944613b
Export Replacer and use concrete type instead of interface
The interface was only making things difficult; a concrete pointer is
probably best.
2019-12-29 13:12:52 -07:00
Matthew Holt
95ed603de7
Improve godocs all around
These will be used in the new automated documentation system
2019-12-23 12:45:35 -07:00
Matthew Holt
87b6cf470b
Minor improvements; comments and shorter placeholders & module IDs 2019-12-12 14:31:20 -07:00
Matt Holt
3c90e370a4
v2: Module documentation; refactor LoadModule(); new caddy struct tags (#2924)
This commit goes a long way toward making automated documentation of
Caddy config and Caddy modules possible. It's a broad, sweeping change,
but mostly internal. It allows us to automatically generate docs for all
Caddy modules (including future third-party ones) and make them viewable
on a web page; it also doubles as godoc comments.

As such, this commit makes significant progress in migrating the docs
from our temporary wiki page toward our new website which is still under
construction.

With this change, all host modules will use ctx.LoadModule() and pass in
both the struct pointer and the field name as a string. This allows the
reflect package to read the struct tag from that field so that it can
get the necessary information like the module namespace and the inline
key.

This has the nice side-effect of unifying the code and documentation. It
also simplifies module loading, and handles several variations on field
types for raw module fields (i.e. variations on json.RawMessage, such as
arrays and maps).

I also renamed ModuleInfo.Name -> ModuleInfo.ID, to make it clear that
the ID is the "full name" which includes both the module namespace and
the name. This clarity is helpful when describing module hierarchy.

As of this change, Caddy modules are no longer an experimental design.
I think the architecture is good enough to go forward.
2019-12-10 13:36:46 -07:00
lu4p
68adfdc559 Fix misspellings (#2908) 2019-12-04 16:28:13 -07:00
Matthew Holt
1228dd7d93
reverse_proxy: Allow buffering of client requests
This is a bad idea, but some backends apparently require it. See
discussion in #176.
2019-11-15 17:15:33 -07:00
Matthew Holt
8e515289cb
reverse_proxy: Add support for NTLM 2019-11-05 16:29:10 -07:00
Matthew Holt
97d918df3e
reverse_proxy: Make HTTP versions configurable, don't set NextProtos 2019-11-05 16:27:51 -07:00
Matthew Holt
dccba71276 reverse_proxy: Structured logs 2019-10-29 16:02:58 -06:00
Matt Holt
b00dfd3965
v2: Logging! (#2831)
* logging: Initial implementation

* logging: More encoder formats, better defaults

* logging: Fix repetition bug with FilterEncoder; add more presets

* logging: DiscardWriter; delete or no-op logs that discard their output

* logging: Add http.handlers.log module; enhance Replacer methods

The Replacer interface has new methods to customize how to handle empty
or unrecognized placeholders. Closes #2815.

* logging: Overhaul HTTP logging, fix bugs, improve filtering, etc.

* logging: General cleanup, begin transitioning to using new loggers

* Fixes after merge conflict
2019-10-28 14:39:37 -06:00
yzongyue
fcd8869f51 reverse_proxy: optimize MaxIdleConnsPerHost default (#2809) 2019-10-11 23:57:11 -06:00
Matthew Holt
1e31be8de0
reverse_proxy: Allow dynamic backends (closes #990 and #1539)
This PR enables the use of placeholders in an upstream's Dial address.

A Dial address must represent precisely one socket after replacements.

See also #998 and #1639.
2019-10-11 14:25:39 -06:00
Matthew Holt
9c0bf311f9
Miscellaneous cleanups / comments 2019-10-10 15:38:30 -06:00
Matthew Holt
be7abda7d4
reverse_proxy: Implement retry_match; by default only retry GET requests
See https://caddy.community/t/http-proxy-and-non-get-retries/6304
2019-10-05 16:22:05 -06:00
Matthew Holt
f15f0d5839
Eliminate some TODOs 2019-09-14 18:05:45 -06:00
Matthew Holt
e73b117332
reverse_proxy: Ability to mutate headers; set upstream placeholders 2019-09-14 13:25:26 -06:00
Matthew Holt
758269124e
reverseproxy: Fix host and port on requests; fix Caddyfile parser 2019-09-11 18:53:44 -06:00
Matthew Holt
0830fbad03
Reconcile upstream dial addresses and request host/URL information
My goodness that was complicated

Blessed be request.Context

Sort of
2019-09-05 13:14:39 -06:00
Matthew Holt
a60d54dbfd
reverse_proxy: Ignore context.Canceled errors
These happen when downstream clients cancel the request, but that's not
our problem nor a failure in our end
2019-09-03 19:10:09 -06:00
Matthew Holt
acb8f0e0c2
Integrate circuit breaker modules with reverse proxy 2019-09-03 19:06:54 -06:00
Matthew Holt
652460e03e
Some cleanup and godoc 2019-09-03 16:56:09 -06:00
Matthew Holt
ccfb12347b
reverse_proxy: Implement active health checks 2019-09-03 12:10:11 -06:00
Matthew Holt
026df7c5cb
reverse_proxy: WIP refactor and support for FastCGI 2019-09-02 22:01:02 -06:00
Matthew Holt
3177ee8010 Add license 2019-06-30 16:07:58 -06:00
Matthew Holt
1136e2cfee Add reverse proxy 2019-05-04 10:49:50 -06:00