- Go 100%
42db5690de* fix(proxy_protocol): size go-proxyproto's bufio to MaxBufLen
When caddy-l4's routes matching loop has prefetched bytes into cx.buf
before the proxy_protocol matcher matches, the handler constructs a
proxyproto.Conn whose internal bufio.Reader defaults to 256 bytes
(go-proxyproto protocol.go: const readBufferSize = 256).
If cx.buf already holds more than 256 bytes (e.g. PROXY header + a
1.8 KB TLS ClientHello, ~1865 bytes total in production), bufio's
single 256-byte fill drains only 256 from cx via cx.Read. cx.offset
advances to 256, but the reset branch in cx.Read (offset == len(buf))
does not fire because cx.buf is longer than 256. The result is:
- cx.buf[0:256] has been consumed by bufio (≈28 bytes go to
parseHeader; ~228 stay in bufio's private buffer);
- cx.buf[256:] is unread by anyone but still present in the slice.
cx.Wrap then carries the misaligned cx.buf into the wrapped
Connection. Subsequent matchers either:
- read from cx.buf[256:] directly (when their reads bypass bufio),
getting mid-stream junk and silently returning (matched=false,
err=nil) because hdr[0] != 0x16; or
- read through proxyproto.Conn (when their reads go via the wrapped
cx after a Reset), get bufio's ~228-byte leftover, then run out of
bytes waiting for the ~238 unread bytes of cx.buf that are now
unreachable — surfacing as ErrMatchingTimeout (3s) in production.
The fix is to size bufio's buffer via WithBufferSize to a value that
strictly exceeds cx.buf's upper bound. A new exported constant
layer4.MaxBufLen captures this bound: prefetch() gates with
`len(cx.buf) < MaxMatchingBytes` *before* the read, then appends up
to prefetchChunkSize bytes — so cx.buf can reach MaxMatchingBytes +
prefetchChunkSize - 1 bytes. Sizing only to MaxMatchingBytes leaves
a prefetchChunkSize-wide hole where the same misalignment bug
re-activates for any deployment that runs more than one prefetch
iteration before the proxy_protocol handler executes (e.g. when
proxy_protocol is not the first route).
With bufio = MaxBufLen, bufio's first fill drains all of cx.buf in
one cx.Read regardless of how many prefetch iterations preceded the
handler, the reset branch fires cleanly, and bufio ends up holding
the entire post-PROXY-header byte stream. Subsequent reads through
proxyproto.Conn return TLS bytes in stream order.
Adds two integration regression tests in handler_test.go:
TestProxyProtocolHandleDoesNotLeaveStaleBufferForNextMatcher
Small payload (~1.8 KB, the production case): catches the
original misalignment (hdr[0] would be 0xAA filler without
the fix) and byte-loss (full record body would EOF mid-read
without the fix).
TestProxyProtocolHandleDoesNotLeaveStaleBufferForNextMatcher_LargePrefetch
17 KiB payload split across two TLS records: catches the
over-MaxMatchingBytes regime. Sizing bufio to
MaxMatchingBytes (instead of MaxBufLen) fails this with
hdr=BB BB BB BB BB — the second TLS record's body, served
from cx.buf[MaxMatchingBytes:].
Toggling WithBufferSize argument from MaxBufLen → MaxMatchingBytes
fails the large-prefetch test cleanly. Toggling the option off
entirely fails both tests at the first assertion.
This bug is latent for any caddy-l4 deployment chaining the
proxy_protocol handler with subsequent matchers and receiving a
ClientHello larger than ~228 bytes after the PROXY header — i.e.
essentially every real HTTPS browser/curl client behind HAProxy,
AWS NLB, or Fly's proxy_proto handlers.
* chore: Apply Copilot's suggestions (1/3)
Co-authored-by: Copilot <copilot@github.com>
* chore: Apply Copilot's suggestions (2/3)
Co-authored-by: Copilot <copilot@github.com>
* chore: Apply Copilot's suggestions (3/3)
Co-authored-by: Copilot <copilot@github.com>
---------
Co-authored-by: Carter Bryden <carter.bryden@7sage.com>
Co-authored-by: Copilot <copilot@github.com>
|
||
|---|---|---|
| .github | ||
| docs | ||
| integration | ||
| layer4 | ||
| modules | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .golangci.yml | ||
| go.mod | ||
| go.sum | ||
| imports.go | ||
| LICENSE | ||
| README.md | ||
Project Conncept: a TCP/UDP app for Caddy
Project Conncept is an experimental layer 4 app for Caddy. It facilitates composable handling of raw TCP/UDP connections based on properties of the connection or the beginning of the stream.
With it, you can listen on sockets/ports and express logic such as:
- "Echo all input back to the client."
- "Proxy all the raw bytes to 10.0.3.14:1592."
- "If connection is TLS, terminate TLS then proxy all bytes to :5000."
- "Terminate TLS; then if it is HTTP, proxy to localhost:80; otherwise echo."
- "If connection is TLS, proxy to :443 without terminating; if HTTP, proxy to :80; if SSH, proxy to :22."
- "If the HTTP Host is
example.comor the TLS ServerName isexample.com, then proxy to 192.168.0.4." - "Block connections from these IP ranges: ..."
- "Throttle data flow to simulate slow connections."
- And much more!
⚠️ This app is very capable and flexible, but is still in development. Please expect breaking changes.
Because this is a Caddy app, it can be used alongside other Caddy apps such as the HTTP server or TLS certificate manager.
Note
This is not an official repository of the Caddy Web Server organization.
Introduction
Important
Documentation is available in the docs directory. For better understanding, you may also read the code, especially type definitions and their comments. It's actually a pretty simple code base. See below for tips and examples writing config.
This app works similarly to the http app. You define servers, and each server consists of
routes. A route has a set of matchers and handlers;
if a connection matches, the associated handlers are invoked.
Refer the docs for lists of matchers and handlers included in the package.
Compiling
The recommended way is to use xcaddy:
$ xcaddy build --with github.com/mholt/caddy-l4
Alternatively, to hack on the plugin code, you can clone it down, then build and run like so:
- Download or clone this repo:
git clone https://github.com/mholt/caddy-l4.git - In the project folder, run
xcaddyjust like you would runcaddy. For example:xcaddy list-modules --versions(you should see thelayer4modules).
Writing config
This app supports Caddyfile, but you may also use Caddy's native JSON format to configure it. I highly recommend this caddy-json-schema plugin by @abiosoft which can give you auto-complete and documentation right in your editor as you write your config!
See below for some examples to help you get started.
Config examples
The following configuration examples are included in the documentation:
- Combining Apps
- DNS-over-TLS
- Echo Server
- HTTP & HTTPS Mix
- IMAPS with Proxy Protocol
- Mutual TLS
- Postgres Matching
- Postgres-over-TLS
- SOCKS Proxy
- SSH-over-TLS
- TLS ALPN Routing
- TLS SNI Dynamic Upstreams
Other examples could be found in the documentation files describing specific matchers and handlers, as well as in issues and pull requests.