Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions pages/clusters-for-kafka/concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Understand the key concepts for Clusters for Apache Kafka®.
tags: billing account pricing payment
dates:
posted: 2025-11-26
validation: 2025-09-15
validation: 2026-06-19
---

## Cluster
Expand All @@ -17,19 +17,19 @@ Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-man

## Credentials

Credentials consist of a username and password that allow you to connect to your cluster. These are defined during cluster creation. During Public Beta, a limit of one user (i.e. one set of credentials) per cluster is applied.
Credentials consist of a username and password that allow you to connect to your cluster. These are defined during cluster creation. Currently, a limit of one user (i.e. one set of credentials) per cluster is applied.

## Data replication

Data replication is the mechanism that ensures event data is copied across multiple brokers in a cluster, for fault tolerance and high availability. If a node fails, data replication means Apache Kafka® can keep serving data without downtime or data loss.

## Endpoint

During Public Beta, access to your Cluster for Apache Kafka® is via private endpoints only. When creating your cluster, you must attach it to a [Private Network](/vpc/concepts/#private-networks), and all nodes in the cluster receive a private IPv4 endpoint on this network. Endpoints are in a format similar to `bf610fb0-f7eb-4ec3-a893...634aea.kafka.fr-par.scw.cloud`.
Currently, access to your Cluster for Apache Kafka® is via private endpoints only. When creating your cluster, you must attach it to a [Private Network](/vpc/concepts/#private-networks), and all nodes in the cluster receive a private IPv4 endpoint on this network. Endpoints are in a format similar to `bf610fb0-f7eb-4ec3-a893...634aea.kafka.fr-par.scw.cloud`.

## Event streaming

Clusters for Apache Kafka® is specifically designed for implementing event streaming at scale. Event streaming is the process of capturing, storing and processing a continuous flow of events in real time. Each event represents something that happened, e.g.`User added item to cart`, `User clicked 'Pay now'`, `Payment was rejected by bank`. Events are stored in topics, e.g. `Inventory`, `Payment`, `Orders`. Typically, programs and microservices subscribe to these topics to consume their events and take action in relation to them, e.g. `Generate invoice`, `Check for fraud`, `Program delivery`.
Clusters for Apache Kafka® is specifically designed for implementing event streaming at scale. Event streaming is the process of capturing, storing, and processing a continuous flow of events in real time. Each event represents something that happened, e.g.,`User added item to cart`, `User clicked 'Pay now'`, `Payment was rejected by bank`. Events are stored in topics, e.g., `Inventory`, `Payment`, `Orders`. Typically, programs and microservices subscribe to these topics to consume their events and take action in relation to them, e.g., `Generate invoice`, `Check for fraud`, `Program delivery`.

## Node

Expand Down
24 changes: 12 additions & 12 deletions pages/clusters-for-kafka/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Clusters for Apache Kafka® FAQ
description: Get answers to all your questions about Scaleway Clusters for Apache Kafka®
dates:
validation: 2025-12-17
validation: 2026-06-19
posted: 2025-09-15
productIcon: ManagedKafkaProductIcon
---
Expand All @@ -11,7 +11,7 @@ productIcon: ManagedKafkaProductIcon

### What is Clusters for Apache Kafka®?

Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-managed infrastructure for your Apache Kafka® deployments. With our clusters, you can create data pipelines and event-driven architectures that run natively on Apache Kafka®, without having to manage the underlying infrastructure. This product is currently in Public Beta.
Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-managed infrastructure for your Apache Kafka® deployments. With our clusters, you can create data pipelines and event-driven architectures that run natively on Apache Kafka®, without having to manage the underlying infrastructure.

### Is Scaleway's Clusters for Apache Kafka® based on the open-source Kafka framework?

Expand All @@ -23,10 +23,14 @@ Apache Kafka, RabbitMQ, and NATS are all messaging systems but serve different p

- Apache Kafka is designed for high-throughput, real-time data streaming, and is topic-based, using the pull model. It can process large amounts of data in real-time with high concurrency and fault tolerance, and is ideal for stream processing, event sourcing, and big data applications.
- RabbitMQ is a versatile message broker that supports multiple protocols, and offers strong support for message delivery guarantees and reliability. It uses the push model, and is well-suited for traditional message queuing and task distribution.
- [NATS](/nats/) is a lightweight, high-performance subject-based messaging system focused on speed and simplicity, making it great for microservices.
- [NATS](/nats/) is a lightweight, high-performance, subject-based messaging system focused on speed and simplicity, making it great for microservices.

Each system has its own strengths and is suited to different use cases depending on the requirements of the application.

## How does a multi-AZ setup work?

In a multi‑AZ deployment, the nodes (brokers) of your Apache Kafka® cluster are spread across two Availability Zones (AZs) within the same region.

## Getting started

### How do I use my cluster, and how can I connect to it?
Expand All @@ -41,14 +45,10 @@ See our dedicated [pricing page](https://www.scaleway.com/en/pricing/) for infor

## Quotas and limitations

### What limitations apply during the Public Beta?

The following limitations apply to Clusters for Apache Kafka® in its current Public Beta phase:
### What limitations apply to Clusters for Apache Kafka®?

- Only one Kafka version is supported
- Clusters can only be provisioned with either one or three nodes (brokers)
- The number of nodes cannot be modified after cluster creation
- Connectivity to the cluster is facilitated exclusively through Private Networks
- Only one set of credentials (username and password) can be created per cluster
The following limitations apply to Clusters for Apache Kafka®:

We aim to improve and develop the product based on feedback received during the beta stage.
- The number of nodes cannot be modified after cluster creation.
- Connectivity to the cluster is facilitated exclusively through Private Networks.
- Only one set of credentials (username and password) can be created per cluster.
10 changes: 5 additions & 5 deletions pages/clusters-for-kafka/how-to/connect-to-cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: How to connect to a Cluster for Apache Kafka®
description: Learn how to connect to your Scaleway Cluster for Apache Kafka® with Scaleway, and understand how to securely establish a connection using TLS certificates and credentials.
dates:
validation: 2025-11-26
validation: 2026-06-19
posted: 2025-09-15
---
import Requirements from '@macros/iam/requirements.mdx'
Expand All @@ -29,7 +29,7 @@ All clients that want to connect to your cluster must:

3. In the **Connectivity** panel, click **Download** next to the TLS certificate.

The TLS certificate in `.pem` format is downloaded to your local device. If using Java, import it into a truststore and configure your `client.properties` appropriately. For other types of client (e.g. Python, Go, CLI), you may be able to use the certificate directly by setting its location in your Kafka consumer configuration code.
The TLS certificate in `.pem` format is downloaded to your local device. If using Java, import it into a truststore and configure your `client.properties` appropriately. For other types of client (e.g., Python, Go, CLI), you may be able to use the certificate directly by setting its location in your Kafka consumer configuration code.

<Message type="note">
TLS is server-side only, and no client certificate is required.
Expand All @@ -41,7 +41,7 @@ Any clients wanting to connect to the cluster must be [attached to a Private Net

## How to access the username and password

The username and password for a Cluster for Apache Kafka® are defined when [creating the cluster](/clusters-for-kafka/how-to/create-cluster/). During Public Beta, only one user can be created per cluster. The username cannot be modified after creation, but the password can be changed at any time.
The username and password for a Cluster for Apache Kafka® are defined when [creating the cluster](/clusters-for-kafka/how-to/create-cluster/). Currently, only one user can be created per cluster. The username cannot be modified after creation, but the password can be changed at any time.

If you need a reminder of the username after creating the cluster, follow the steps below. You cannot view the password you set, but you can change it to a new one.

Expand All @@ -55,15 +55,15 @@ The username displays in the list of users. Click **Change password** if you hav

## How to connect to the cluster

Once you have ensured that the client in question has the cluster's TLS certificate, appropriate network access and the correct username and password, you can facilitate connection via the Kafka-compatible framework of your choice. Code snippets are provided for Go, Python and Java; access them by following the steps below.
Once you have ensured that the client in question has the cluster's TLS certificate, appropriate network access, and the correct username and password, you can facilitate connection via the Kafka-compatible framework of your choice. Code snippets are provided for Go, Python and Java; access them by following the steps below.

1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays.

2. Click the cluster you want to connect to. Its **Overview** page displays.

3. In the **Connectivity** panel, click **Connect**.

A pop-up displays, with tabs for each sample framework: **Go**, **Python**, **Java**, **Node.js**, and **Mongoose**.
A pop-up displays, with tabs for each sample framework: **Go**, **Python**, and **Java**.

4. Click the tab of your choice to view the code sample.

33 changes: 24 additions & 9 deletions pages/clusters-for-kafka/how-to/create-cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: How to create a Cluster for Apache Kafka®
description: Learn how to create a Cluster for Apache Kafka® with Scaleway, and get your hosted, fully-managed Kafka infrastructure up and running with just a few clicks.
dates:
validation: 2025-11-26
validation: 2026-06-19
posted: 2025-09-15
---
import Requirements from '@macros/iam/requirements.mdx'
Expand All @@ -21,24 +21,39 @@ This page shows you how to create a Cluster for Apache Kafka® via the Scaleway

2. Click **Create cluster**. The creation wizard displays.

3. Select a **region** for the cluster. During Public Beta, clusters are only available in the Paris region.
3. Select a **region** for the cluster.

4. Select a **Kafka® version** to run on the cluster. Only one pre-selected Kafka® version is available during Public Beta.
4. Select a **Kafka® version** to run on the cluster.

5. Select the **number of nodes** (brokers) you want in the cluster, either **3 nodes** for a highly-available cluster enabling data replication, or **1 node** for a cluster suitable for testing and development. During Public Beta, the number of nodes cannot be changed after cluster creation.
5. Select an Availability Zone (AZ) configuration for the cluster:
- **Multi-AZ**: Select this configuration to create a highly available Apache Kafka® cluster. Data can be partitioned and replicated across three nodes (brokers) for maximum fault tolerance and availability. Currently, the multi-AZ offers replicate three nodes over two Availability Zones.
- **Single-AZ**: Select this configuration to provision an Apache Kafka® cluster in a single AZ without compromising on fault-tolerance and replication.

6. Select a **node type**. Choose between [dedicated or shared nodes](/clusters-for-kafka/concepts/#node), depending on your workload requirements and use case. For shared nodes, various types are available, each offering different cost-performance ratios.

7. Choose the **storage capacity** to define how much Block Storage to provision for each node.
7. Select the **number of nodes** (brokers) you want in the cluster.
- Allowed number of nodes: 1 - 30, in multiples of 3 (except for a single‑node option).
- For highly‑available clusters enabling data replication: use at least 3 nodes.
- For testing and development: 1 node is sufficient.

8. Configure **network connectivity**. During Public Beta, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to.
<Message type="note">
A single‑node cluster is not compatible with a multi‑AZ configuration.
</Message>

<Message type="note">
The number of nodes cannot be changed after cluster creation.
</Message>

8. Choose the **storage capacity** to define how much Block Storage to provision for each node.

9. Configure **network connectivity**. Currently, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to.

<VPCRoutingCompatibility />

9. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. During Public Beta, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed).
10. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. Currently, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed).

10. Enter a **name**, and (optionally), **tags** to identify your cluster.
11. Enter a **name**, and (optionally), **tags** to identify your cluster.

11. Check the estimated cost summary, then click **Create cluster**.
12. Check the estimated cost summary, then click **Create cluster**.

Your cluster is created, and you are taken to its **Overview** page in the console.
16 changes: 11 additions & 5 deletions pages/clusters-for-kafka/how-to/manage-cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: How to manage to a Cluster for Apache Kafka®
description: Learn how to manage to your Scaleway Cluster for Apache Kafka® with Scaleway, including editing its details, renewing its TLS certificate and more.
dates:
validation: 2025-11-26
validation: 2026-06-19
posted: 2025-09-15
---
import Requirements from '@macros/iam/requirements.mdx'
Expand All @@ -21,7 +21,7 @@ Other management actions for internally configuring your cluster, such as creati

## How to renew the TLS certificate

You may want or need to renew the cluster's TLS certificate for reasons of security, compliance or if the certificate expires. Follow the steps below to renew the certificate:
You may want or need to renew the cluster's TLS certificate for reasons of security, compliance, or if the certificate expires. Follow the steps below to renew the certificate:

1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays.

Expand Down Expand Up @@ -55,7 +55,7 @@ The cluster is detached, and you are returned to its **Overview** page. Click **

## How to change the password

During Public Beta, each cluster can have only one set of user credentials, which are set during cluster creation. You cannot change the username after creating the cluster, but you can change the password at any time:
Currently, each cluster can have only one set of user credentials, which are set during cluster creation. You cannot change the username after creating the cluster, but you can change the password at any time:

1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays.

Expand All @@ -69,6 +69,12 @@ During Public Beta, each cluster can have only one set of user credentials, whic

Ensure that you copy or remember the password, as it will not be displayed again after this screen.

5. Click **Update**.
5. Click **Change password**.

The password is changed and you are returned to your cluster's **Overview** page.
The password is changed and you are returned to your cluster's **Overview** page.

{/* ## How to view observability metrics for your cluster

Observability metrics are available for your cluster via Scaleway Cockpit.

For details on how to view them, see [How to access Grafana and your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/). */}
8 changes: 1 addition & 7 deletions pages/clusters-for-kafka/index.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Clusters for Apache Kafka® Documentation
description: Dive into Scaleway Clusters for Apache Kafka® with our quickstart guides, how-tos and more.
description: Dive into Scaleway Clusters for Apache Kafka® with our quickstart guides, how-tos, and more.
---

<ProductHeader
Expand All @@ -11,12 +11,6 @@ description: Dive into Scaleway Clusters for Apache Kafka® with our quickstart
label="Clusters for Apache Kafka® Quickstart"
/>

<Message
type="note"
title="Clusters for Apache Kafka® is now in Public Beta">
This product is currently in [Public Beta](https://www.scaleway.com/en/betas/). Test it out today in the [Scaleway console](https://console.scaleway.com/) or [API](https://www.scaleway.com/en/developers/api/clusters-for-kafka/)
</Message>

## Getting Started

<Grid>
Expand Down
Loading
Loading