Bump Confluent.Kafka from 1.8.2 to 2.14.0

[dependabot]

Updated Confluent.Kafka from 1.8.2 to 2.14.0.

Release notes

Sourced from Confluent.Kafka's releases.

2.14.0

Enhancements

• References librdkafka.redist 2.14.0. Refer to the librdkafka v2.14.0 release notes for more information.
• Add AssociatedNameStrategy (#​2577)
• Add enableAt to RuleSet (#​2593)
• Add config to validate before domain rules for JSON Schema (#​2596)
• Cache FieldAccessor to address CSFLE perf with JSON (#​2601)

Fixes

• Ensure dispose methods clean up properly (#​2595)

2.13.2

Enhancements

• Preserve inner exceptions in RestService (#​2571)

Fixes

• Support multi-certificate PEM bundles in Schema Registry SslCaLocation (#​2576)

2.13.1

Enhancements

• Unify build configurations and remove redundant properties (#​2464)
• JSONSchema Validation Errors: Include paths to nested properties (#​2554)
• Add Confluent-Client-Version header to requests to SR, C# client (#​2549)

Fixes

• Fix DeriveKey in Hkdf implementation (#​2559)
• Ensure single use of HttpRequestMessage for token fetch (#​2544)
• Fix race conditions when retrieving KEK/DEK (#​2565)
• Allowing for Confluent-Identity-Pool-Id to be optional (#​2564)
• Ensure JSON schema access is thread-safe (#​2567)
• Use lock on root schema when protecting access to subschemas (#​2568)
• Processing arrays and maps sequentially to avoid race conditions (#​2569)

2.13.0

Enhancements

• References librdkafka.redist 2.13.0. Refer to the librdkafka v2.13.0 release notes for more information.
• Support Avro schema references for C# client (#​2534)
• Add Accept-Version header (#​2536)
• Handle evolution during field transformation (#​2541)
• Re-enable deserialization optimization by removing unnecessary array conversions (#​2531)

Fixes

• Ensure schemaId initialization is thread-safe (#​2540)
• Ensure all deps are strongly named (#​2548)

2.12.0

KIP-848 – General Availability

Starting with Confluent.Kafka 2.12.0, the next generation consumer group rebalance protocol defined in KIP-848 is production-ready. Please refer to the following migration guide for moving from Classic to Consumer protocol.

Note: The new consumer group protocol defined in KIP-848 is not enabled by default. There are few contract change associated with the new protocol and might cause breaking changes. GroupProtocol configuration property dictates whether to use the new Consumer protocol or older Classic protocol. It defaults to Classic if not provided.

Enhancements

• References librdkafka.redist 2.12.0. Refer to the librdkafka v2.12.0 release notes for more information.
• OAuth OIDC method example for Kafka metadata based authentication with
  an Azure IMDS endpoint using an attached managed identity as principal (#​2526).

2.11.1

Enhancements

• References librdkafka.redist 2.11.1. Refer to the librdkafka v2.11.1 release notes for more information.

2.11.0

Enhancements

• References librdkafka.redist 2.11.0. Refer to the librdkafka v2.11.0 release notes for more information.

2.10.1

Enhancements

• References librdkafka.redist 2.10.1. Refer to the librdkafka v2.10.1 release notes for more information.

2.10.0

Enhancements

• References librdkafka.redist 2.10.0. Refer to the librdkafka v2.10.0 release notes for more information.
• [KIP-848] Group Config is now supported in AlterConfigs, IncrementalAlterConfigs and DescribeConfigs. (#​2366)

2.9.0

Enhancements

• Add utilities to convert decimals from/to Protobuf (#​2424)
• Add client credentials OAuth support (#​2426)
• Add support for passing schema ID during serialization (#​2428)
• Optimize schema registry client by caching results of LookupSchemaAsync (#​2429)
• Use ConcurrentDictionary as default cache for Schema Registry client (#​2433)

Fixes

• Fix JSON Schema validation to use latest schema if needed (#​2406)
• Support Protobuf oneof fields in Data Contract rules (#​2413)

2.8.0

v2.8.0 is a feature release with the following features, fixes and enhancements:

Enhancements

• References librdkafka.redist 2.8.0. Refer to the librdkafka v2.8.0 release notes for more information.
• Add retry logic to RestService for Schema Registry (#​2353)
• Add config for validating payloads against JSON Schema (#​2350)
• Add ability to override disable flag and actions on a rule (#​2377)
• Add AWS AssumeRole support to AWS KMS (#​2379)

Fixes

• Retrieve version for writer schema when getting migrations (#​2358)
• Add missing JSON Schema validate check (#​2372)
• Ensure different key ids use different client instances (#​2374)
• DGS-19409 Ensure Avro serde caches per subject (#​2387)
• Update hash code for schema references so they can be retrieved correctly (#​2390)

2.6.1

v2.6.1 is a maintenance release with the following features, fixes and enhancements:

Enhancements

• References librdkafka.redist 2.6.1. Refer to the librdkafka v2.6.1 release notes for more information.

Fixes

• Fix to continue supporting .NET Framework 4.6.2+ in core client library (#​2342).
• Fix JSON Schema handling to not require use of $id (#​2339).
• Update Caching.Memory to 8.0.1 to address CVE (#​23440.
• Added Qualified and Custom reference name strategy approaches for protobuf references (#​2345).
• Fix validate of SSL CA certs in Schema Registry client (#​2346).
• Skip SSL certs validation when configured in Schema Registry client (#​2347).
• Allow proxy to be specified in Schema Registry client (#​2348).

2.6.0

v2.6.0 is a feature release with the following features, fixes and enhancements:

Enhancements

• KIP-848 EA: Admin API for listing consumer groups now has an optional filter to return only groups of given types (#​2323).
• KIP-460 Admin Leader Election RPC (#​2320)
• .NET 8 support with NJsonSchema 11 (#​2314)

confluent-kafka-dotnet is based on librdkafka v2.6.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

2.5.3

v2.5.3 is a maintenance release with the following fixes and enhancements:

Fixes

• Properly handle messages with well-known types in Protobuf serializer
• Use AES128_GCM in the Local KMS client, for consistency with Java/go
• Include deleted schemas when getting schemas by subject and version
• Handle signed ints when transforming Protobuf payloads
• Allow null SchemaRegistryClient in AsyncSerde constructor

confluent-kafka-dotnet is based on librdkafka v2.5.3, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

2.5.2

[!WARNING]
Versions 2.5.0, 2.5.1 and 2.5.2 have a regression in which an assert is triggered during PushTelemetry call. This happens when no metric is matched on the client side among those requested by broker subscription.

You won't face any problem if:

• Broker doesn't support KIP-714.
• KIP-714 feature is disabled on the broker side.
• KIP-714 feature is disabled on the client side. This is enabled by default. Set configuration enable.metrics.push to false.
• If KIP-714 is enabled on the broker side and there is no subscription configured there.
• If KIP-714 is enabled on the broker side with subscriptions that match the KIP-714 metrics defined on the client.

Having said this, we strongly recommend using v2.5.3 and above to not face this regression at all.

This is a maintenance release.

Fixes

• Fix CSFLE (client-side field-level encryption) to use the Google Tink format for DEKs for interoperability with clients in other languages (Java, go, etc.).
• Improve error when specifying an invalid KMS type for CSFLE
• Enhance CSFLE examples with KMS configuration settings

2.5.1

This is a maintenance release.

Fixes

• Fix CSFLE (client-side field-level encryption) when using Azure Key Vault by specifying RsaOaep256 (instead of RsaOaep) for interoperability with clients in other languages (Java, go, etc.).
• Fix AvroSerializer configuration to allow using schema normalization.
• Upgrade Azure Identity library to 1.11.4 to address a vulnerability in previous versions.

2.5.0

This is a feature release

Enhancements

• References librdkafka.redist 2.5.0. Refer to the librdkafka v2.5.0 release notes for more information.
• Add support for metadata and ruleSet in the schema registry client, which together support data
  contracts.
• Add support for CSFLE (client-side field-level encryption) for AWS, Azure, GCP, and HashiCorp
  Vault. See the encryption examples in the examples directory.
• Add support for CEL, CEL_FIELD, and JSONata rules.

Fixes

• Switch license expression and other repo information. (#​2192, @​thompson-tomo)

2.4.0

This is a feature release.

Enhancements

• References librdkafka.redist 2.4.0. Refer to the librdkafka v2.4.0 release notes for more information.
• KIP-848 EA: Added KIP-848 based new consumer group rebalance protocol.
  Integration tests running with the new consumer group protocol. The feature is an Early Access: not production ready. Please refer
  detailed doc for more information. (#​2212).

2.3.0

This is a feature release.

Enhancements

• References librdkafka.redist 2.3.0. Refer to the librdkafka v2.3.0 release notes for more information.
• KIP-430:
  Return authorized operations in describe responses (#​2021, @​jainruchir).
• KIP-396: Added support for ListOffsets Admin API (#​2086).
• Add Rack to the Node type, so AdminAPI calls can expose racks for brokers (currently, all Describe
  Responses) (#​2021, @​jainruchir).
• Added support for external JSON schemas in JsonSerializer and JsonDeserializer (#​2042).
• Added compatibility methods to CachedSchemaRegistryClient (ISBronny, #​2097).
• Add support for AdminAPI DescribeCluster() and DescribeTopics() (#​2021, @​jainruchir).

2.2.1

librdkafka v2.2.1 is a maintenance release

Enhancements

• References librdkafka.redist 2.2.1. Refer to the librdkafka v2.2.1 release notes for more information.

2.2.0

This is a feature release.

Enhancements

• References librdkafka.redist 2.2.0. Refer to the librdkafka v2.2.0 release notes for more information.
• KIP-339
  IncrementalAlterConfigs API (#​2005).
• KIP-554:
  User SASL/SCRAM credentials alteration and description (#​2070).

Fixes

• Fix backwards compatability of TopicPartitionOffset constructor. (drinehimer, #​2066)
• Fix IConsumer breaking change. (ttd2089, #​2071)

2.1.1

This is a maintenance release.

Enhancements

• References librdkafka.redist 2.1.1. Refer to the librdkafka v2.1.1 release notes for more information.
• Less heap allocations when calling Produce (bjornbouetsmith, #​2020)

2.1.0

This is a feature release.

Enhancements

• References librdkafka.redist 2.1.0. Refer to the librdkafka v2.1.0 release notes and later ones for more information.
• Added SetSaslCredentials. This new method (on the Producer, Consumer, and AdminClient) allows modifying the stored
  SASL PLAIN/SCRAM credentials that will be used for subsequent (new) connections to a broker (#​1980).
• Changed the way the _SCHEMA filed is accessed internally from reflecting the static field to accessing it from the instance (AlexeyRaga).
• KIP-320: add offset leader epoch fields to the TopicPartitionOffset,
  TopicPartitionOffsetError and ConsumeResult classes (#​2027).

Fixes

• Fixed OverflowException thrown intermittently when using the ListGroup method (#​2003).

2.0.2

Upgrade considerations

OpenSSL 3.0.x upgrade in librdkafka requires a major version bump, as some legacy ciphers need to be explicitly configured to continue working, but it is highly recommended NOT to use them. The rest of the API remains backward compatible.

Enhancements

• References librdkafka.redist 2.0.2. Refer to the librdkafka v2.0.0 release notes and later ones for more information.
• Upgraded NJsonSchema to v10.6.3
• Added LatestCompatibilityStrict configuration property to JsonSerializerConfig to check the compatibility with latest schema when UseLatestVersion is set to true.
• Added DeleteConsumerGroupOffset to AdminClient.
• KIP-222 Finish remaining implementation: Add Consumer Group operations to Admin API (DeleteGroups is already present).
• KIP-518 Allow listing consumer groups per state.
• KIP-396 Partially implemented: support for AlterConsumerGroupOffsets.
• As result of the above KIPs, added (#​1981)
  • ListConsumerGroups Admin operation. Supports listing by state.
  • DescribeConsumerGroups Admin operation. Supports multiple groups.
  • ListConsumerGroupOffsets Admin operation. Currently, only supports
    1 group with multiple partitions. Supports the requireStable option.
  • AlterConsumerGroupOffsets Admin operation. Currently, only supports
    1 group with multiple offsets.

Fixes

• During a group rebalance, partitions are now always revoked as a side effect of a call to Consume, whether or not a partitions revoked handler has been specified. Previously, if no handler was specified, the timing of when the consumer lost ownership of partitions during a rebalance was arbitrarily, frequently resulting in an erroneous state exception when committing or storing offsets.
• Fixed 100% CPU usage with DependentAdminClientBuilder.

Note: There were no 2.0.0 and 2.0.1 releases.

1.9.3

Enhancements

• Added NormalizeSchemas configuration property to the Avro, Json and Protobuf serdes.

Fixes

• Schema Registry authentication now works with passwords that contain the ':' character (luismedel).
• Added missing librdkafka internal and broker error codes to the ErrorCode enum.

1.9.2

Enhancements

• References librdkafka.redist 1.9.2 which includes an Apple M1 librdkafka build.
• Added ACL AdminClient operations (CreateAcls, DescribeAcls, DeleteAcls) (emasab).
• Added DeleteGroups to AdminClient (3schwartz).
• Enhanced the Avro Specific Deserializer to ignore the type namespace (sergemat).
• Improved efficiency of the statistics handler (VladimirTyrin).

Fixes

• The AdminClient poll loop no longer terminates when a request results in an error (emasab).
• Upgraded Newtonsoft.Json to 13.0.1 to address a security vulnerability in 9.0.1.

1.9.0

Enhancements

• References librdkafka.redist 1.9.0. Refer to the librdkafka release notes
  for a complete list of changes, enhancements, fixes and upgrade considerations.
• References Apache.Avro 1.11.0. Refer to the release notes for further information (JanReimerD).
• Added support for serializing and deserializing null in Avro serdes (YairHalberstadt).
• Enhanced CachedSchemaRegistryClient to allow for user-implemented authentication schemes (henrydaly).
• Reduced memory use when producing with delivery reports disabled (TrickyCat).

Fixes

• Resolved incompatibility with Apple M1 processors (dkaukov).
• No longer crashes on Alpine Linux when MUSL is installed (shurivich).
• JSON validation exception messages now properly include failing paths (drinehimer).
• Upgraded Google.Protobuf dependency to 3.15.0 (CVE-2021-22570).
• Resolved memory leak in AdminClient response handler (emasab).
• Resolved memory leak in Producer.SendOffsetsToTransaction.

Upgrade Considerations

• The earliest supported .NET Framework version is now 4.6.2, previously this was 4.5 (bjornbouetsmith).

Commits viewable in compare view.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)