Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "blasthttp"
version = "0.7.0"
version = "0.8.0"
edition = "2024"
description = "Offensive-first HTTP library with Python bindings"
license = "GPL-3.0"
Expand Down
31 changes: 30 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ blasthttp https://example.com -L
# Through a proxy
blasthttp https://example.com -x http://127.0.0.1:8080

# ...but connect directly to certain hosts (host, *.suffix, IP, or CIDR)
blasthttp https://example.com -x http://127.0.0.1:8080 --no-proxy '*.internal.corp' --no-proxy 10.0.0.0/8

# Force specific TLS versions/ciphers
blasthttp https://legacy-server.com --min-tls 1.0 --ciphers "RC4-SHA"

Expand Down Expand Up @@ -106,6 +109,7 @@ Output is JSON (one object per response), including status, headers, redirect ch
| `--max-body-size` | Max response body (bytes) | 10 MB |
| `--verify` | Enable TLS cert validation | off |
| `-x, --proxy` | HTTP/SOCKS proxy URL | |
| `--no-proxy` | Host that bypasses the proxy: `host`, `*.suffix`, IP, CIDR, or `*` (repeatable) | |
| `--ciphers` | OpenSSL cipher string | all |
| `--min-tls` | Minimum TLS version (1.0–1.3) | |
| `--max-tls` | Maximum TLS version (1.0–1.3) | |
Expand Down Expand Up @@ -232,6 +236,31 @@ response = await client.request(
)
```

### Proxy exclusions (`no_proxy`)

`proxy` routes a request through an HTTP or SOCKS5 proxy; `no_proxy` is a per-request list of hosts that bypass it and connect directly — the `NO_PROXY` equivalent. It's accepted by `request()`, `download()`, `raw_connect()`, and `BatchConfig`, and as the repeatable `--no-proxy` CLI flag.

```python
# Proxy everything except internal hosts and the loopback range.
r = await client.request(
"https://example.com/",
proxy="http://127.0.0.1:8080",
no_proxy=["localhost", "*.internal.corp", "10.0.0.0/8"],
)
```

Each entry matches case-insensitively as:

- an exact hostname (`elastic.corp`);
- a domain suffix — `*.corp`, `.corp`, and `corp` are equivalent and match the domain itself plus any subdomain;
- a single IP (`127.0.0.1`, `::1`);
- a CIDR range (`10.0.0.0/8`, `fd00::/8`), matched only against IP hosts;
- or `*` to bypass the proxy for every host.

The proxy/`no_proxy` decision is re-evaluated on **every redirect hop**, not just the first URL: if a request starts on an excluded (direct) host and is redirected onto a non-excluded host, the later hop goes through the proxy — and vice versa.

`no_proxy` only does anything when a `proxy` is set, so passing it without one is rejected up front (the request errors rather than silently ignoring the exclusions).

### Global Rate Limiting

Set a client-level rate limit (requests per second) that applies to **all** request methods — `request()`, `request_batch()`, `request_batch_stream()`, and `download()`:
Expand Down Expand Up @@ -290,7 +319,7 @@ async def main():
asyncio.run(main())
```

`raw_connect()` takes all the same TLS knobs as `request()` — `verify_certs`, `cipher_string`, `min_tls_version`, `max_tls_version`, `resolve_ip`, `proxy`. `alpn_protocols` is a list of byte-strings (commonly `["h2", "http/1.1"]`) used in the TLS ALPN extension. After the handshake, `conn.negotiated_alpn` reports which one the server picked (or `None` if no ALPN was negotiated, including all plain-HTTP connections).
`raw_connect()` takes all the same TLS knobs as `request()` — `verify_certs`, `cipher_string`, `min_tls_version`, `max_tls_version`, `resolve_ip`, `proxy`, `no_proxy`. `alpn_protocols` is a list of byte-strings (commonly `["h2", "http/1.1"]`) used in the TLS ALPN extension. After the handshake, `conn.negotiated_alpn` reports which one the server picked (or `None` if no ALPN was negotiated, including all plain-HTTP connections).

`conn.peer_ip` returns the actual IP address the OS connected to, same as `Response.peer_ip`. `None` when the connection went through a proxy.

Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "maturin"

[project]
name = "blasthttp"
version = "0.7.0"
version = "0.8.0"
description = "Offensive-first HTTP library"
license = "GPL-3.0"
requires-python = ">=3.10"
Expand All @@ -20,7 +20,7 @@ managed = false
dev = [
"pytest>=8",
"pytest-asyncio>=0.23",
"ruff==0.15.13",
"ruff==0.15.15",
]

[tool.pytest.ini_options]
Expand Down
79 changes: 44 additions & 35 deletions src/client/hyper.rs
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,7 @@ impl HyperClient {

/// Determine the connection mode for a given request config + target URI.
fn conn_mode(config: &RequestConfig, target_uri: &http::Uri) -> Result<ConnMode, ClientError> {
match config.proxy.as_deref() {
match config.effective_proxy(target_uri.host().unwrap_or("")) {
None => Ok(ConnMode::Direct),
Some(proxy_url) => {
let proxy_uri: http::Uri =
Expand Down Expand Up @@ -684,6 +684,8 @@ pub(crate) async fn connect_stream(
> {
use super::proxy::{self, ProxyScheme};

config.validate_proxy().map_err(ClientError::other)?;

let v = config.verbosity;
let host = target_uri.host().unwrap_or("").to_string();
let is_https = target_uri.scheme_str() == Some("https");
Expand All @@ -694,7 +696,7 @@ pub(crate) async fn connect_stream(
// tunnel from there. resolve_ip is ignored when a proxy is set — DNS
// resolution happens at the proxy for SOCKS5, and CONNECT addresses the
// target by hostname.
let proxy_config = match config.proxy.as_ref() {
let proxy_config = match config.effective_proxy(&host) {
Some(p) => Some(proxy::parse_proxy_url(p)?),
None => None,
};
Expand Down Expand Up @@ -1183,6 +1185,7 @@ fn retry_backoff(attempt: u32, min_wait: Duration, max_wait: Duration) -> Durati

impl HttpClient for HyperClient {
async fn send(&self, config: &RequestConfig) -> Result<Response, ClientError> {
config.validate_proxy().map_err(ClientError::other)?;
let timeout_duration = Duration::from_secs(config.timeout());
let max_retries = config.max_retries();
let min_wait = config.retry_wait_min();
Expand Down Expand Up @@ -1267,7 +1270,7 @@ impl HyperClient {
})?;

debug_record(log, v, 1, &format!("-> {} {}", config.method(), uri));
if let Some(ref proxy) = config.proxy {
if let Some(proxy) = config.effective_proxy(uri.host().unwrap_or("")) {
debug_record(log, v, 1, &format!(" Proxy: {}", proxy));
}
if !config.should_verify_certs() {
Expand Down Expand Up @@ -1321,41 +1324,37 @@ impl HyperClient {
});
}

let mode = Self::conn_mode(config, &uri)?;

// Forward proxy: dispatch directly via TCP + http1::SendRequest
// (bypasses hyper Client's URI normalization to preserve absolute-form)
let is_forward_proxy = matches!(&mode, ConnMode::ForwardProxy(_));
let proxy_url_for_fwd = if let ConnMode::ForwardProxy(ref url) = mode {
Some(url.clone())
} else {
None
};

// For non-forward-proxy modes, get the cached hyper Client
let cached = if is_forward_proxy {
None
} else {
Some(self.get_or_build(config, &mode)?)
};

let mut redirect_chain: Vec<RedirectHop> = Vec::new();
let mut hops = 0u32;

// Per-hop peer IP lookup. Returns None when:
// • the request went through a proxy (the connector recorded
// the proxy IP under the proxy's authority, so the target's
// authority isn't in the map)
// • forward-proxy dispatch (bypasses the connector entirely)
// • peer_addr() failed at connect time (vanishingly rare)
let lookup_peer_ip = |target_uri: &http::Uri| -> Option<String> {
let key = peer_slot_key(target_uri)?;
let cached = cached.as_ref()?;
let map = cached.peer_slot.lock().ok()?;
map.get(&key).map(|ip| ip.to_string())
};

loop {
// Decide the connection mode for the *current* target host on every
// hop, not just the first. A redirect can send the request to a
// different host, and the proxy / no_proxy decision has to follow
// it. Freezing the first hop's choice would otherwise let a request
// that started direct keep connecting directly after a redirect onto
// a proxied host (leaking traffic past the proxy), and let a request
// that started proxied keep using the proxy after a redirect onto a
// no_proxy host. Clients are cached by mode, so hops that share a
// mode reuse the same client.
let mode = Self::conn_mode(config, &uri)?;

// Forward proxy: dispatch directly via TCP + http1::SendRequest
// (bypasses hyper Client's URI normalization to preserve absolute-form)
let is_forward_proxy = matches!(&mode, ConnMode::ForwardProxy(_));
let proxy_url_for_fwd = if let ConnMode::ForwardProxy(ref url) = mode {
Some(url.clone())
} else {
None
};

// For non-forward-proxy modes, get the cached hyper Client.
let cached = if is_forward_proxy {
None
} else {
Some(self.get_or_build(config, &mode)?)
};

let resp = if let Some(ref proxy_url) = proxy_url_for_fwd {
dispatch_forward_proxy(proxy_url, &uri, config, log).await?
} else {
Expand All @@ -1364,7 +1363,17 @@ impl HyperClient {
let hop_ms = start.elapsed().as_millis();
debug_record(log, v, 1, &format!("<- {} ({}ms)", resp.status, hop_ms));

let hop_peer_ip = lookup_peer_ip(&uri);
// Per-hop peer IP lookup. Returns None when:
// • the request went through a proxy (the connector recorded
// the proxy IP under the proxy's authority, so the target's
// authority isn't in the map)
// • forward-proxy dispatch (bypasses the connector entirely)
// • peer_addr() failed at connect time (vanishingly rare)
let hop_peer_ip = peer_slot_key(&uri).and_then(|key| {
let cached = cached.as_ref()?;
let map = cached.peer_slot.lock().ok()?;
map.get(&key).map(|ip| ip.to_string())
});

if is_redirect(resp.status) && config.should_follow_redirects() {
hops += 1;
Expand Down
Loading
Loading