Skip to content

query API maps ClickHouse caller-side errors (e.g. ACCESS_DENIED) to HTTP 502 instead of 4xx #403

Description

@EricAndrechek

Summary

The query handler classifies a non-200 ClickHouse response purely by ClickHouse's HTTP status — 4xx → 400, everything else → 502 — in internal/api/query.go (~L260–L286):

status := http.StatusBadGateway
if resp.StatusCode >= 400 && resp.StatusCode < 500 {
    status = http.StatusBadRequest
}

But ClickHouse's HTTP interface returns HTTP 500 for most caller-side errors, not 4xx — including ACCESS_DENIED, syntax errors, unknown table/identifier, and type mismatches. So a request that is the caller's fault comes back as a 502 Bad Gateway, which reads (to clients, proxies, and monitoring) as "the engine/upstream is down" rather than "your request was rejected."

Repro

Point a ClickHouse user that lacks a privilege at the query API and run a statement it can't perform — e.g. a user without CREATE TABLE on a database running CREATE TABLE db.events (...). ClickHouse rejects it:

Code: 497. DB::Exception: <user>: Not enough privileges. To execute this query,
it's necessary to have the grant CREATE TABLE ON db.events. (ACCESS_DENIED)

ClickHouse returns that over HTTP with status 500, so the handler maps it to 502. The caller sees a gateway/upstream error for what is really a 403-class problem. Same story for a syntax error, an unknown table, a type mismatch — all surface from ClickHouse as HTTP 500 and become 502.

Why it matters

  • A 502 is indistinguishable from a genuine engine outage: clients retry/back off, dashboards page, and the actionable message ("you're missing a grant" / "your SQL is malformed") is buried behind an "upstream failed" status.
  • Behind a proxy/CDN that replaces origin 5xx responses with its own error page, the forwarded ClickHouse message can be lost entirely, leaving the caller with only "Bad Gateway" and no diagnostic.

Suggested fix

ClickHouse returns its internal exception code in the X-ClickHouse-Exception-Code response header (and as Code: NNN at the start of the body). Parse it and map the caller-fault classes to 4xx, keeping 502 only for genuine upstream/transport failures:

switch chExceptionCode(resp.Header.Get("X-ClickHouse-Exception-Code")) {
case 497 /* ACCESS_DENIED */, 516 /* AUTHENTICATION_FAILED */:
    status = http.StatusForbidden
case 62 /* SYNTAX_ERROR */, 60 /* UNKNOWN_TABLE */,
     47 /* UNKNOWN_IDENTIFIER */, 53 /* TYPE_MISMATCH */:
    status = http.StatusBadRequest
default:
    status = http.StatusBadGateway // genuine upstream fault
}

The existing comment notes that distinguishing specific ClickHouse codes was "out of scope" — this is a proposal to bring the ACCESS_DENIED / bad-SQL classes into scope, precisely because ClickHouse's HTTP status alone can't separate them from a real outage (it returns 500 for all of them). The body already carries ClickHouse's verbatim message, so no diagnostic is lost.

Happy to send a PR if this direction sounds right.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/apiHTTP handlers, routing, middleware

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions