Kafka Month-to-month Digest – August 2021 – IBM Developer

That is the forty third version of the Kafka Month-to-month Digest. On this version, I’ll cowl what occurred within the Apache Kafka neighborhood in August 2021.

For final month’s digest, see Kafka Month-to-month Digest: July 2021.

Kafka Summit

The Kafka Summit Americas is on September 14 and 15. It’s a digital occasion with dozens of periods overlaying all elements of the Kafka ecosystem. You may register without cost on the Kafka Summit web site.

Releases

3.0.0

Kafka 3.0.0 is at the moment within the stabilization interval following code freeze. After a couple of extra Blocker JIRAs (KAFKA-13198, KAFKA-13219), Konstantine revealed the primary Launch Candidate, 3.0.0 RC0, on August 27. A vote on this RC is at the moment on going.

2.8.1

On August 12, David Jacot volunteered to run this bugfix launch. The discharge date is at the moment targetted for September. The launch plan is accessible on the wiki.

KIPs

Final month, the neighborhood submitted 5 KIPs (KIP-766 to KIP-770), and these are those that caught my eye.

• KIP-768: Prolong SASL/OAUTHBEARER with Help for OIDC. Kafka helps OAuth authentication since 2.0 (KIP-255). Nevertheless, KIP-255 solely launched a framework and organizations have to offer their very own implementations. This KIP proposes including a manufacturing prepared implementation of the interfaces outlined in KIP-255 to authenticate with Open ID id suppliers.

• KIP-770: Change “buffered.information.per.partition” with “enter.buffer.max.bytes”. This KIP goals at controlling the reminiscence utilized by a Kafka Streams software. It proposes including a brand new configuration enter.buffer.max.bytes to specify the overall reminiscence that can be utilized to buffer information. The present configuration buffered.information.per.partition which solely managed reminiscence per partition shall be deprecated.

On this part, I’ll cowl releases of some neighborhood tasks. This solely consists of tasks which might be open supply.

• kafka-connect-file-pulse 2.2.0. FilePulse is a connectors to stream any file into Kafka. This new model improves situations with a really massive variety of small recordsdata, improves help for XML recordsdata and introduces a brand new pluggable Partitioner interface to distribute recordsdata throughout duties.

• kcat 1.7.0. Beforehand referred to as kafkacat, kcat is a CLI consumer for Kafka. This launch improves multibytes delimiters and provides help for incremental rebalancing.

• node-rdkafka 2.11. This new launch of node-rdkafka is now primarily based on librdkafka 1.6.1. It now helps Nodejs v16 and in addition the idempotent and transactional producer.

• kcctl 1.0.0-EA. kcctl is a command line consumer for Kafka Join. It permits to simply work together (deploy, cease, restart, describe, and so on) with connectors.

Blogs

Get began with Kafka

IBM Occasion Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get began with IBM Occasion Streams right this moment.