Support HTTP response compression

CMakeLists.txt

@@ -23,11 +23,13 @@ endif()
 find_package(CURL QUIET)
 find_package(OpenSSL QUIET)
 find_package(Threads REQUIRED)
+find_package(ZLIB QUIET)
 
 add_library(alternator_client_cpp
     src/attribute_value.cpp
     src/config.cpp
     src/go_random.cpp
+    src/http_compression.cpp
     src/http_client.cpp
     src/key_route_affinity.cpp
     src/live_nodes.cpp
@@ -48,6 +50,13 @@ target_link_libraries(alternator_client_cpp
     PUBLIC
         Threads::Threads)
 
+if(ZLIB_FOUND)
+    target_link_libraries(alternator_client_cpp PUBLIC ZLIB::ZLIB)
+    target_compile_definitions(alternator_client_cpp PUBLIC SCYLLADB_ALTERNATOR_CLIENT_CPP_HAS_ZLIB=1)
+else()
+    message(STATUS "ZLIB not found; gzip/deflate response decoding requires a caller-provided HttpContentEncodingDecoder")
+endif()
+
 if(CURL_FOUND)
     target_link_libraries(alternator_client_cpp PUBLIC CURL::libcurl)
     target_compile_definitions(alternator_client_cpp PUBLIC SCYLLADB_ALTERNATOR_CLIENT_CPP_HAS_CURL=1)
@@ -60,17 +69,7 @@ else()
 endif()
 
 if(ALTERNATOR_CLIENT_CPP_ENABLE_AWS)
-    find_package(ZLIB QUIET)
-    if(ZLIB_FOUND)
-        find_package(AWSSDK QUIET COMPONENTS dynamodb)
-    else()
-        set(AWSSDK_FOUND FALSE)
-        if(ALTERNATOR_CLIENT_CPP_REQUIRE_AWS)
-            message(FATAL_ERROR "ZLIB not found; AWS DynamoDB adapter target cannot be built")
-        else()
-            message(STATUS "ZLIB not found; AWS DynamoDB adapter target will not be built")
-        endif()
-    endif()
+    find_package(AWSSDK QUIET COMPONENTS dynamodb)
     if(AWSSDK_FOUND)
         add_library(alternator_client_cpp_aws
             src/aws_dynamodb_helper.cpp)

README.md

@@ -11,7 +11,7 @@ make build
 make test
 ```
 
-The core target requires C++17. The default discovery HTTP client uses libcurl when CMake can find the libcurl development package; otherwise it falls back to a small POSIX plain-HTTP client. HTTPS discovery requires libcurl or a caller-provided `HttpClient`. Tests use GoogleTest when it is installed. The AWS adapter target is built only when CMake can find `AWSSDK` with the `dynamodb` component. `make test-integration` requires the AWS adapter and its tests by configuring with `ALTERNATOR_CLIENT_CPP_REQUIRE_AWS=ON`.
+The core target requires C++17. The default discovery HTTP client uses libcurl when CMake can find the libcurl development package; otherwise it falls back to a small POSIX plain-HTTP client. HTTPS discovery requires libcurl or a caller-provided `HttpClient`. Built-in gzip and deflate response decoding is available when CMake finds zlib; callers can provide a custom decoder for other content encodings or for builds without zlib. Tests use GoogleTest when it is installed. The AWS adapter target is built only when CMake can find `AWSSDK` with the `dynamodb` component. `make test-integration` requires the AWS adapter and its tests by configuring with `ALTERNATOR_CLIENT_CPP_REQUIRE_AWS=ON`.
 
 ## Core Usage
 
@@ -87,6 +87,71 @@ The AWS adapter uses the SDK's per-client `DynamoDBEndpointProviderBase` hook by
 
 The HTTP client factory runs after request signing. This is the closest public AWS SDK for C++ hook for retry-aware endpoint rewriting, but it means applications that depend on strict SigV4 host validation should prefer endpoint-provider routing or implement their own SDK wrapper. Automatic middleware features such as transparent header optimization and request-content-based endpoint rewriting are not available in the same form. The core library does provide deterministic key-route affinity primitives, and `DynamoDBHelper` exposes them for applications that integrate request-specific routing in their own AWS SDK wrapper.
 
+### HTTP Response Compression
+
+Response compression is disabled by default. When configured, the built-in
+discovery client sends the configured `Accept-Encoding` value on `/localnodes`
+requests. If zlib was found at build time, configuring
+`ZlibContentEncodingDecoder` enables matching `Content-Encoding: gzip` and
+`Content-Encoding: deflate` responses.
+
+For DynamoDB operations, response compression is supported when
+`DynamoDBHelper::ApplyToSDKOptions()` installs the Alternator HTTP client
+factory before `Aws::InitAPI()`. The factory advertises the configured response
+encodings only for requests routed to Alternator nodes and decodes compressed
+responses before the AWS SDK parses the JSON body. A DynamoDB client that only
+uses the endpoint provider does not get this factory-level response decoding.
+
+Configure content-encoding decoders before constructing the helper or discovery
+client. The default empty decoder list disables compression advertisement.
+Each decoder advertises the encodings it supports, and each advertised encoding
+may be owned by only one decoder. Responses using an encoding outside the
+configured decoder list are rejected.
+
+```cpp
+scylladb::alternator::Config cfg;
+cfg.content_encoding_decoders.push_back(
+    std::make_shared<scylladb::alternator::ZlibContentEncodingDecoder>());
+```
+
+When zlib is available, `ZlibContentEncodingDecoder` implements gzip and deflate
+decoding and advertises both encodings by default. Pass a subset to advertise
+only one of them, for example `ZlibContentEncodingDecoder({"gzip"})`. zlib is
+optional at build time; without it, applications can still provide their own
+decoder implementations.
+
+For generic content-encoding support, provide another decoder. Its
+`AcceptedResponseEncodings()` method declares the encodings to advertise, and
+`Decode()` receives the encoded response body and the specific encoding token to
+decode:
+
+```cpp
+class BrotliDecoder final : public scylladb::alternator::HttpContentEncodingDecoder {
+public:
+    std::vector<std::string> AcceptedResponseEncodings() const override {
+        return {"br"};
+    }
+
+    std::string Decode(std::string body, const std::string& content_encoding) const override {
+        // Decode body according to content_encoding.
+        return body;
+    }
+};
+
+scylladb::alternator::Config cfg;
+cfg.content_encoding_decoders.push_back(std::make_shared<BrotliDecoder>());
+```
+
+To request gzip and zstd, configure zlib for gzip only and add a zstd decoder:
+
+```cpp
+scylladb::alternator::Config cfg;
+cfg.content_encoding_decoders.push_back(
+    std::make_shared<scylladb::alternator::ZlibContentEncodingDecoder>(
+        std::vector<std::string>{"gzip"}));
+cfg.content_encoding_decoders.push_back(std::make_shared<ZstdDecoder>());
+```
+
 ### DynamoDB HTTP Connections
 
 DynamoDB requests use the AWS SDK for C++ HTTP transport. The helper passes

cmake/scylladb-alternator-client-cpp-config.cmake.in

@@ -4,6 +4,10 @@ include(CMakeFindDependencyMacro)
 
 find_dependency(Threads)
 
+if(@ZLIB_FOUND@)
+    find_dependency(ZLIB)
+endif()
+
 if(@CURL_FOUND@)
     find_dependency(CURL)
 endif()
@@ -13,7 +17,6 @@ if(@OpenSSL_FOUND@)
 endif()
 
 if(@ALTERNATOR_CLIENT_CPP_AWS_TARGET_BUILT@)
-    find_dependency(ZLIB)
     find_dependency(AWSSDK COMPONENTS dynamodb)
 endif()
 

include/scylladb/alternator/config.h

@@ -2,7 +2,9 @@
 
 #include <chrono>
 #include <cstdint>
+#include <memory>
 #include <string>
+#include <vector>
 
 #include <scylladb/alternator/key_route_affinity.h>
 #include <scylladb/alternator/node_health.h>
@@ -15,6 +17,25 @@ struct Credentials {
     std::string secret_access_key;
 };
 
+class HttpContentEncodingDecoder {
+public:
+    virtual ~HttpContentEncodingDecoder() = default;
+
+    [[nodiscard]] virtual std::vector<std::string> AcceptedResponseEncodings() const = 0;
+    [[nodiscard]] virtual std::string Decode(std::string body, const std::string& content_encoding) const = 0;
+};
+
+class ZlibContentEncodingDecoder final : public HttpContentEncodingDecoder {
+public:
+    explicit ZlibContentEncodingDecoder(std::vector<std::string> accepted_response_encodings = {"gzip", "deflate"});
+
+    [[nodiscard]] std::vector<std::string> AcceptedResponseEncodings() const override;
+    [[nodiscard]] std::string Decode(std::string body, const std::string& content_encoding) const override;
+
+private:
+    std::vector<std::string> accepted_response_encodings_;
+};
+
 struct Config {
     std::uint16_t port = 8080;
     std::string scheme = "http";
@@ -37,6 +58,7 @@ struct Config {
 
     unsigned max_connections = 100;
     bool reuse_discovery_connections = true;
+    std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders;
     std::string user_agent = "scylladb-alternator-client-cpp/devel";
 
     NodeHealthStoreConfig node_health;

src/aws_dynamodb_helper.cpp

@@ -1,5 +1,7 @@
 #include <scylladb/alternator/aws/dynamodb_helper.h>
 
+#include "http_compression.h"
+
 #include <aws/core/Aws.h>
 #include <aws/core/auth/AWSCredentials.h>
 #include <aws/core/client/CoreErrors.h>
@@ -9,6 +11,7 @@
 #include <aws/core/http/HttpTypes.h>
 #include <aws/core/http/curl/CurlHttpClient.h>
 #include <aws/core/http/standard/StandardHttpRequest.h>
+#include <aws/core/http/standard/StandardHttpResponse.h>
 #include <aws/core/utils/logging/LogMacros.h>
 #include <aws/dynamodb/model/AttributeAction.h>
 #include <aws/dynamodb/model/AttributeValueUpdate.h>
@@ -26,6 +29,9 @@
 #include <algorithm>
 #include <chrono>
 #include <cstdint>
+#include <ios>
+#include <istream>
+#include <iterator>
 #include <set>
 #include <stdexcept>
 #include <thread>
@@ -108,6 +114,56 @@ NodeHealthObservation ObservationFromHttpResult(const std::shared_ptr<Aws::Http:
     return NodeHealthObservation::Success;
 }
 
+std::string ReadResponseBody(Aws::IOStream& body) {
+    body.clear();
+    body.seekg(0, std::ios::beg);
+    return std::string(std::istreambuf_iterator<char>(body), std::istreambuf_iterator<char>());
+}
+
+void WriteResponseBody(Aws::IOStream& body, const std::string& value) {
+    if (!value.empty()) {
+        body.write(value.data(), static_cast<std::streamsize>(value.size()));
+    }
+    body.flush();
+    body.clear();
+    body.seekg(0, std::ios::beg);
+}
+
+std::shared_ptr<Aws::Http::HttpResponse> DecodeCompressedAwsResponse(
+    const std::shared_ptr<Aws::Http::HttpRequest>& request,
+    const std::shared_ptr<Aws::Http::HttpResponse>& response,
+    const std::vector<std::shared_ptr<HttpContentEncodingDecoder>>& content_encoding_decoders) {
+    if (!request || !response || response->HasClientError() ||
+        !response->HasHeader(Aws::Http::CONTENT_ENCODING_HEADER)) {
+        return response;
+    }
+
+    const auto content_encoding = std::string(response->GetHeader(Aws::Http::CONTENT_ENCODING_HEADER).c_str());
+    auto decoded_body = detail::DecodeHttpResponseBody(
+        ReadResponseBody(response->GetResponseBody()),
+        content_encoding,
+        content_encoding_decoders);
+
+    auto decoded_response = Aws::MakeShared<Aws::Http::Standard::StandardHttpResponse>(
+        kAllocationTag,
+        request);
+    decoded_response->SetResponseCode(response->GetResponseCode());
+
+    for (const auto& header : response->GetHeaders()) {
+        const auto header_name = detail::ToLowerAscii(std::string(header.first.c_str()));
+        if (header_name == Aws::Http::CONTENT_ENCODING_HEADER ||
+            header_name == Aws::Http::CONTENT_LENGTH_HEADER) {
+            continue;
+        }
+        decoded_response->AddHeader(header.first, header.second);
+    }
+    decoded_response->AddHeader(
+        Aws::Http::CONTENT_LENGTH_HEADER,
+        Aws::String(std::to_string(decoded_body.size()).c_str()));
+    WriteResponseBody(decoded_response->GetResponseBody(), decoded_body);
+    return decoded_response;
+}
+
 bool IsFirstSdkAttempt(const Aws::Http::HttpRequest& request) {
     if (!request.HasHeader(Aws::Http::SDK_REQUEST_HEADER)) {
         return false;
@@ -151,9 +207,12 @@ class AlternatorHttpClient final : public Aws::Http::HttpClient {
 public:
     AlternatorHttpClient(std::shared_ptr<AlternatorLiveNodes> nodes,
                          std::uint16_t endpoint_override_port,
+                         std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders,
                          std::shared_ptr<Aws::Http::HttpClient> delegate)
         : nodes_(std::move(nodes))
         , endpoint_override_port_(endpoint_override_port)
+        , content_encoding_decoders_(std::move(content_encoding_decoders))
+        , accept_encoding_value_(detail::BuildAcceptEncodingValue(content_encoding_decoders_))
         , delegate_(std::move(delegate)) {
         if (!nodes_) {
             throw std::invalid_argument("nodes must not be null");
@@ -168,8 +227,17 @@ class AlternatorHttpClient final : public Aws::Http::HttpClient {
         Aws::Utils::RateLimits::RateLimiterInterface* read_limiter = nullptr,
         Aws::Utils::RateLimits::RateLimiterInterface* write_limiter = nullptr) const override {
         auto node = SelectAttemptNode(request, nodes_, endpoint_override_port_);
+        if (!node.Empty() && !accept_encoding_value_.empty()) {
+            request->SetAcceptEncoding(Aws::String(accept_encoding_value_.c_str()));
+        }
 
         auto response = delegate_->MakeRequest(request, read_limiter, write_limiter);
+        if (!node.Empty()) {
+            response = DecodeCompressedAwsResponse(
+                request,
+                response,
+                content_encoding_decoders_);
+        }
         if (!node.Empty()) {
             nodes_->ReportNodeResult(node, ObservationFromHttpResult(response));
         }
@@ -183,15 +251,19 @@ class AlternatorHttpClient final : public Aws::Http::HttpClient {
 private:
     std::shared_ptr<AlternatorLiveNodes> nodes_;
     std::uint16_t endpoint_override_port_ = 0;
+    std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders_;
+    std::string accept_encoding_value_;
     std::shared_ptr<Aws::Http::HttpClient> delegate_;
 };
 
 class AlternatorHttpClientFactory final : public Aws::Http::HttpClientFactory {
 public:
     AlternatorHttpClientFactory(std::shared_ptr<AlternatorLiveNodes> nodes,
-                                std::uint16_t endpoint_override_port)
+                                std::uint16_t endpoint_override_port,
+                                std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders)
         : nodes_(std::move(nodes))
-        , endpoint_override_port_(endpoint_override_port) {
+        , endpoint_override_port_(endpoint_override_port)
+        , content_encoding_decoders_(std::move(content_encoding_decoders)) {
         if (!nodes_) {
             throw std::invalid_argument("nodes must not be null");
         }
@@ -208,6 +280,7 @@ class AlternatorHttpClientFactory final : public Aws::Http::HttpClientFactory {
             kAllocationTag,
             nodes_,
             endpoint_override_port_,
+            content_encoding_decoders_,
             std::move(delegate));
     }
 
@@ -241,6 +314,7 @@ class AlternatorHttpClientFactory final : public Aws::Http::HttpClientFactory {
 private:
     std::shared_ptr<AlternatorLiveNodes> nodes_;
     std::uint16_t endpoint_override_port_ = 0;
+    std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders_;
 };
 
 std::string EndpointOverride(std::uint16_t port, const std::string& scheme) {
@@ -249,11 +323,13 @@ std::string EndpointOverride(std::uint16_t port, const std::string& scheme) {
 
 std::shared_ptr<Aws::Http::HttpClientFactory> NewAlternatorHttpClientFactory(
     std::shared_ptr<AlternatorLiveNodes> nodes,
-    std::uint16_t endpoint_override_port) {
+    std::uint16_t endpoint_override_port,
+    std::vector<std::shared_ptr<HttpContentEncodingDecoder>> content_encoding_decoders) {
     return Aws::MakeShared<AlternatorHttpClientFactory>(
         kAllocationTag,
         std::move(nodes),
-        endpoint_override_port);
+        endpoint_override_port,
+        std::move(content_encoding_decoders));
 }
 
 class FixedEndpointProvider final : public Aws::DynamoDB::Endpoint::DynamoDBEndpointProviderBase {
@@ -662,7 +738,10 @@ std::shared_ptr<AlternatorEndpointProvider> DynamoDBHelper::NewEndpointProvider(
 }
 
 std::shared_ptr<Aws::Http::HttpClientFactory> DynamoDBHelper::NewHttpClientFactory() const {
-    return NewAlternatorHttpClientFactory(nodes_, config_.port);
+    return NewAlternatorHttpClientFactory(
+        nodes_,
+        config_.port,
+        config_.content_encoding_decoders);
 }
 
 Aws::DynamoDB::DynamoDBClientConfiguration DynamoDBHelper::NewClientConfiguration() const {
@@ -692,8 +771,9 @@ std::shared_ptr<AlternatorLiveNodes> DynamoDBHelper::Nodes() const {
 void DynamoDBHelper::ApplyToSDKOptions(Aws::SDKOptions& options) const {
     auto nodes = nodes_;
     const auto port = config_.port;
-    options.httpOptions.httpClientFactory_create_fn = [nodes, port] {
-        return NewAlternatorHttpClientFactory(nodes, port);
+    const auto content_encoding_decoders = config_.content_encoding_decoders;
+    options.httpOptions.httpClientFactory_create_fn = [nodes, port, content_encoding_decoders] {
+        return NewAlternatorHttpClientFactory(nodes, port, content_encoding_decoders);
     };
 }
 

src/config.cpp

@@ -1,5 +1,7 @@
 #include <scylladb/alternator/config.h>
 
+#include "http_compression.h"
+
 #include <stdexcept>
 
 namespace scylladb::alternator {
@@ -17,6 +19,7 @@ void ValidateConfig(const Config& config) {
     if (config.max_connections == 0) {
         throw std::invalid_argument("max_connections must be > 0");
     }
+    (void)detail::BuildAcceptEncodingValue(config.content_encoding_decoders);
     if (config.key_route_affinity.partition_key_discovery_attempts == 0) {
         throw std::invalid_argument("partition_key_discovery_attempts must be > 0");
     }