diff --git a/pages/clusters-for-kafka/concepts.mdx b/pages/clusters-for-kafka/concepts.mdx index c65ec1afc5..cc8f27bc70 100644 --- a/pages/clusters-for-kafka/concepts.mdx +++ b/pages/clusters-for-kafka/concepts.mdx @@ -4,7 +4,7 @@ description: Understand the key concepts for Clusters for Apache Kafka®. tags: billing account pricing payment dates: posted: 2025-11-26 - validation: 2025-09-15 + validation: 2026-06-19 --- ## Cluster @@ -17,7 +17,7 @@ Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-man ## Credentials -Credentials consist of a username and password that allow you to connect to your cluster. These are defined during cluster creation. During Public Beta, a limit of one user (i.e. one set of credentials) per cluster is applied. +Credentials consist of a username and password that allow you to connect to your cluster. These are defined during cluster creation. Currently, a limit of one user (i.e. one set of credentials) per cluster is applied. ## Data replication @@ -25,11 +25,11 @@ Data replication is the mechanism that ensures event data is copied across multi ## Endpoint -During Public Beta, access to your Cluster for Apache Kafka® is via private endpoints only. When creating your cluster, you must attach it to a [Private Network](/vpc/concepts/#private-networks), and all nodes in the cluster receive a private IPv4 endpoint on this network. Endpoints are in a format similar to `bf610fb0-f7eb-4ec3-a893...634aea.kafka.fr-par.scw.cloud`. +Currently, access to your Cluster for Apache Kafka® is via private endpoints only. When creating your cluster, you must attach it to a [Private Network](/vpc/concepts/#private-networks), and all nodes in the cluster receive a private IPv4 endpoint on this network. Endpoints are in a format similar to `bf610fb0-f7eb-4ec3-a893...634aea.kafka.fr-par.scw.cloud`. ## Event streaming -Clusters for Apache Kafka® is specifically designed for implementing event streaming at scale. Event streaming is the process of capturing, storing and processing a continuous flow of events in real time. Each event represents something that happened, e.g.`User added item to cart`, `User clicked 'Pay now'`, `Payment was rejected by bank`. Events are stored in topics, e.g. `Inventory`, `Payment`, `Orders`. Typically, programs and microservices subscribe to these topics to consume their events and take action in relation to them, e.g. `Generate invoice`, `Check for fraud`, `Program delivery`. +Clusters for Apache Kafka® is specifically designed for implementing event streaming at scale. Event streaming is the process of capturing, storing, and processing a continuous flow of events in real time. Each event represents something that happened, e.g.,`User added item to cart`, `User clicked 'Pay now'`, `Payment was rejected by bank`. Events are stored in topics, e.g., `Inventory`, `Payment`, `Orders`. Typically, programs and microservices subscribe to these topics to consume their events and take action in relation to them, e.g., `Generate invoice`, `Check for fraud`, `Program delivery`. ## Node diff --git a/pages/clusters-for-kafka/faq.mdx b/pages/clusters-for-kafka/faq.mdx index 94cc11156b..4cc3577995 100644 --- a/pages/clusters-for-kafka/faq.mdx +++ b/pages/clusters-for-kafka/faq.mdx @@ -2,7 +2,7 @@ title: Clusters for Apache Kafka® FAQ description: Get answers to all your questions about Scaleway Clusters for Apache Kafka® dates: - validation: 2025-12-17 + validation: 2026-06-19 posted: 2025-09-15 productIcon: ManagedKafkaProductIcon --- @@ -11,7 +11,7 @@ productIcon: ManagedKafkaProductIcon ### What is Clusters for Apache Kafka®? -Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-managed infrastructure for your Apache Kafka® deployments. With our clusters, you can create data pipelines and event-driven architectures that run natively on Apache Kafka®, without having to manage the underlying infrastructure. This product is currently in Public Beta. +Clusters for Apache Kafka® is Scaleway's product to provide a secure, fully-managed infrastructure for your Apache Kafka® deployments. With our clusters, you can create data pipelines and event-driven architectures that run natively on Apache Kafka®, without having to manage the underlying infrastructure. ### Is Scaleway's Clusters for Apache Kafka® based on the open-source Kafka framework? @@ -23,10 +23,14 @@ Apache Kafka, RabbitMQ, and NATS are all messaging systems but serve different p - Apache Kafka is designed for high-throughput, real-time data streaming, and is topic-based, using the pull model. It can process large amounts of data in real-time with high concurrency and fault tolerance, and is ideal for stream processing, event sourcing, and big data applications. - RabbitMQ is a versatile message broker that supports multiple protocols, and offers strong support for message delivery guarantees and reliability. It uses the push model, and is well-suited for traditional message queuing and task distribution. -- [NATS](/nats/) is a lightweight, high-performance subject-based messaging system focused on speed and simplicity, making it great for microservices. +- [NATS](/nats/) is a lightweight, high-performance, subject-based messaging system focused on speed and simplicity, making it great for microservices. Each system has its own strengths and is suited to different use cases depending on the requirements of the application. +## How does a multi-AZ setup work? + +In a multi‑AZ deployment, the nodes (brokers) of your Apache Kafka® cluster are spread across two Availability Zones (AZs) within the same region. + ## Getting started ### How do I use my cluster, and how can I connect to it? @@ -41,14 +45,10 @@ See our dedicated [pricing page](https://www.scaleway.com/en/pricing/) for infor ## Quotas and limitations -### What limitations apply during the Public Beta? - -The following limitations apply to Clusters for Apache Kafka® in its current Public Beta phase: +### What limitations apply to Clusters for Apache Kafka®? -- Only one Kafka version is supported -- Clusters can only be provisioned with either one or three nodes (brokers) -- The number of nodes cannot be modified after cluster creation -- Connectivity to the cluster is facilitated exclusively through Private Networks -- Only one set of credentials (username and password) can be created per cluster +The following limitations apply to Clusters for Apache Kafka®: -We aim to improve and develop the product based on feedback received during the beta stage. \ No newline at end of file +- The number of nodes cannot be modified after cluster creation. +- Connectivity to the cluster is facilitated exclusively through Private Networks. +- Only one set of credentials (username and password) can be created per cluster. \ No newline at end of file diff --git a/pages/clusters-for-kafka/how-to/connect-to-cluster.mdx b/pages/clusters-for-kafka/how-to/connect-to-cluster.mdx index 0feae20fab..6d22e52271 100644 --- a/pages/clusters-for-kafka/how-to/connect-to-cluster.mdx +++ b/pages/clusters-for-kafka/how-to/connect-to-cluster.mdx @@ -2,7 +2,7 @@ title: How to connect to a Cluster for Apache Kafka® description: Learn how to connect to your Scaleway Cluster for Apache Kafka® with Scaleway, and understand how to securely establish a connection using TLS certificates and credentials. dates: - validation: 2025-11-26 + validation: 2026-06-19 posted: 2025-09-15 --- import Requirements from '@macros/iam/requirements.mdx' @@ -29,7 +29,7 @@ All clients that want to connect to your cluster must: 3. In the **Connectivity** panel, click **Download** next to the TLS certificate. -The TLS certificate in `.pem` format is downloaded to your local device. If using Java, import it into a truststore and configure your `client.properties` appropriately. For other types of client (e.g. Python, Go, CLI), you may be able to use the certificate directly by setting its location in your Kafka consumer configuration code. +The TLS certificate in `.pem` format is downloaded to your local device. If using Java, import it into a truststore and configure your `client.properties` appropriately. For other types of client (e.g., Python, Go, CLI), you may be able to use the certificate directly by setting its location in your Kafka consumer configuration code. TLS is server-side only, and no client certificate is required. @@ -41,7 +41,7 @@ Any clients wanting to connect to the cluster must be [attached to a Private Net ## How to access the username and password -The username and password for a Cluster for Apache Kafka® are defined when [creating the cluster](/clusters-for-kafka/how-to/create-cluster/). During Public Beta, only one user can be created per cluster. The username cannot be modified after creation, but the password can be changed at any time. +The username and password for a Cluster for Apache Kafka® are defined when [creating the cluster](/clusters-for-kafka/how-to/create-cluster/). Currently, only one user can be created per cluster. The username cannot be modified after creation, but the password can be changed at any time. If you need a reminder of the username after creating the cluster, follow the steps below. You cannot view the password you set, but you can change it to a new one. @@ -55,7 +55,7 @@ The username displays in the list of users. Click **Change password** if you hav ## How to connect to the cluster -Once you have ensured that the client in question has the cluster's TLS certificate, appropriate network access and the correct username and password, you can facilitate connection via the Kafka-compatible framework of your choice. Code snippets are provided for Go, Python and Java; access them by following the steps below. +Once you have ensured that the client in question has the cluster's TLS certificate, appropriate network access, and the correct username and password, you can facilitate connection via the Kafka-compatible framework of your choice. Code snippets are provided for Go, Python and Java; access them by following the steps below. 1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays. @@ -63,7 +63,7 @@ Once you have ensured that the client in question has the cluster's TLS certific 3. In the **Connectivity** panel, click **Connect**. - A pop-up displays, with tabs for each sample framework: **Go**, **Python**, **Java**, **Node.js**, and **Mongoose**. + A pop-up displays, with tabs for each sample framework: **Go**, **Python**, and **Java**. 4. Click the tab of your choice to view the code sample. diff --git a/pages/clusters-for-kafka/how-to/create-cluster.mdx b/pages/clusters-for-kafka/how-to/create-cluster.mdx index 528aaee5ca..f60f74a1bc 100644 --- a/pages/clusters-for-kafka/how-to/create-cluster.mdx +++ b/pages/clusters-for-kafka/how-to/create-cluster.mdx @@ -2,7 +2,7 @@ title: How to create a Cluster for Apache Kafka® description: Learn how to create a Cluster for Apache Kafka® with Scaleway, and get your hosted, fully-managed Kafka infrastructure up and running with just a few clicks. dates: - validation: 2025-11-26 + validation: 2026-06-19 posted: 2025-09-15 --- import Requirements from '@macros/iam/requirements.mdx' @@ -21,24 +21,39 @@ This page shows you how to create a Cluster for Apache Kafka® via the Scaleway 2. Click **Create cluster**. The creation wizard displays. -3. Select a **region** for the cluster. During Public Beta, clusters are only available in the Paris region. +3. Select a **region** for the cluster. -4. Select a **Kafka® version** to run on the cluster. Only one pre-selected Kafka® version is available during Public Beta. +4. Select a **Kafka® version** to run on the cluster. -5. Select the **number of nodes** (brokers) you want in the cluster, either **3 nodes** for a highly-available cluster enabling data replication, or **1 node** for a cluster suitable for testing and development. During Public Beta, the number of nodes cannot be changed after cluster creation. +5. Select an Availability Zone (AZ) configuration for the cluster: + - **Multi-AZ**: Select this configuration to create a highly available Apache Kafka® cluster. Data can be partitioned and replicated across three nodes (brokers) for maximum fault tolerance and availability. Currently, the multi-AZ offers replicate three nodes over two Availability Zones. + - **Single-AZ**: Select this configuration to provision an Apache Kafka® cluster in a single AZ without compromising on fault-tolerance and replication. 6. Select a **node type**. Choose between [dedicated or shared nodes](/clusters-for-kafka/concepts/#node), depending on your workload requirements and use case. For shared nodes, various types are available, each offering different cost-performance ratios. -7. Choose the **storage capacity** to define how much Block Storage to provision for each node. +7. Select the **number of nodes** (brokers) you want in the cluster. + - Allowed number of nodes: 1 - 30, in multiples of 3 (except for a single‑node option). + - For highly‑available clusters enabling data replication: use at least 3 nodes. + - For testing and development: 1 node is sufficient. -8. Configure **network connectivity**. During Public Beta, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to. + + A single‑node cluster is not compatible with a multi‑AZ configuration. + + + + The number of nodes cannot be changed after cluster creation. + + +8. Choose the **storage capacity** to define how much Block Storage to provision for each node. + +9. Configure **network connectivity**. Currently, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to. -9. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. During Public Beta, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed). +10. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. Currently, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed). -10. Enter a **name**, and (optionally), **tags** to identify your cluster. +11. Enter a **name**, and (optionally), **tags** to identify your cluster. -11. Check the estimated cost summary, then click **Create cluster**. +12. Check the estimated cost summary, then click **Create cluster**. Your cluster is created, and you are taken to its **Overview** page in the console. diff --git a/pages/clusters-for-kafka/how-to/manage-cluster.mdx b/pages/clusters-for-kafka/how-to/manage-cluster.mdx index 5585365e0f..c1ec39b5da 100644 --- a/pages/clusters-for-kafka/how-to/manage-cluster.mdx +++ b/pages/clusters-for-kafka/how-to/manage-cluster.mdx @@ -2,7 +2,7 @@ title: How to manage to a Cluster for Apache Kafka® description: Learn how to manage to your Scaleway Cluster for Apache Kafka® with Scaleway, including editing its details, renewing its TLS certificate and more. dates: - validation: 2025-11-26 + validation: 2026-06-19 posted: 2025-09-15 --- import Requirements from '@macros/iam/requirements.mdx' @@ -21,7 +21,7 @@ Other management actions for internally configuring your cluster, such as creati ## How to renew the TLS certificate -You may want or need to renew the cluster's TLS certificate for reasons of security, compliance or if the certificate expires. Follow the steps below to renew the certificate: +You may want or need to renew the cluster's TLS certificate for reasons of security, compliance, or if the certificate expires. Follow the steps below to renew the certificate: 1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays. @@ -55,7 +55,7 @@ The cluster is detached, and you are returned to its **Overview** page. Click ** ## How to change the password -During Public Beta, each cluster can have only one set of user credentials, which are set during cluster creation. You cannot change the username after creating the cluster, but you can change the password at any time: +Currently, each cluster can have only one set of user credentials, which are set during cluster creation. You cannot change the username after creating the cluster, but you can change the password at any time: 1. Click **Apache Kafka®** in the **Data and Analytics** section of the [console](https://console.scaleway.com/) side menu. A list of your clusters displays. @@ -69,6 +69,12 @@ During Public Beta, each cluster can have only one set of user credentials, whic Ensure that you copy or remember the password, as it will not be displayed again after this screen. -5. Click **Update**. +5. Click **Change password**. - The password is changed and you are returned to your cluster's **Overview** page. \ No newline at end of file + The password is changed and you are returned to your cluster's **Overview** page. + +{/* ## How to view observability metrics for your cluster + +Observability metrics are available for your cluster via Scaleway Cockpit. + +For details on how to view them, see [How to access Grafana and your preconfigured dashboards](/cockpit/how-to/access-grafana-and-managed-dashboards/). */} \ No newline at end of file diff --git a/pages/clusters-for-kafka/index.mdx b/pages/clusters-for-kafka/index.mdx index 3f93b82fbf..57ee765674 100644 --- a/pages/clusters-for-kafka/index.mdx +++ b/pages/clusters-for-kafka/index.mdx @@ -1,6 +1,6 @@ --- title: Clusters for Apache Kafka® Documentation -description: Dive into Scaleway Clusters for Apache Kafka® with our quickstart guides, how-tos and more. +description: Dive into Scaleway Clusters for Apache Kafka® with our quickstart guides, how-tos, and more. --- - - This product is currently in [Public Beta](https://www.scaleway.com/en/betas/). Test it out today in the [Scaleway console](https://console.scaleway.com/) or [API](https://www.scaleway.com/en/developers/api/clusters-for-kafka/) - - ## Getting Started diff --git a/pages/clusters-for-kafka/quickstart.mdx b/pages/clusters-for-kafka/quickstart.mdx index 241adaf660..4281e03e85 100644 --- a/pages/clusters-for-kafka/quickstart.mdx +++ b/pages/clusters-for-kafka/quickstart.mdx @@ -3,7 +3,7 @@ title: Clusters for Apache Kafka® - Quickstart description: Quickstart guide to setting up and managing Scaleway Clusters for Apache Kafka®. tags: cluster apache kafka dates: - validation: 2025-11-26 + validation: 2026-06-19 posted: 2025-09-15 --- import Requirements from '@macros/iam/requirements.mdx' @@ -23,23 +23,34 @@ Follow our quickstart guide to get your first cluster up and running efficiently 2. Click **Create cluster**. The creation wizard displays. -3. Select a **region** for the cluster. During Public Beta, clusters are only available in the Paris region. +3. Select a **region** for the cluster. -4. Select a **Kafka® version** to run on the cluster. Only one pre-selected Kafka® version is available during Public Beta. +4. Select a **Kafka® version** to run on the cluster. -5. Select the **number of nodes** (brokers) you want in the cluster, either **3 nodes** for a highly-available cluster enabling data replication, or **1 node** for a cluster suitable for testing and development. During Public Beta, the number of nodes cannot be changed after cluster creation. +5. Select an Availability Zone (AZ) configuration for the cluster: + - **Multi-AZ**: Select this configuration to create a highly available Apache Kafka® cluster. Currently, the multi-AZ offers replicate three nodes over two Availability Zones. + - **Single-AZ**: Select this configuration to provision an Apache Kafka® cluster in a single AZ without compromising on fault-tolerance and replication. 6. Select a **node type**. Choose between [dedicated or shared nodes](/clusters-for-kafka/concepts/#node), depending on your workload requirements and use case. For shared nodes, various types are available, each offering different cost-performance ratios. -7. Choose the **storage capacity** to define how much Block Storage to provision for each node. +7. Select the **number of nodes** (brokers) you want in the cluster. + - Allowed number of nodes: 1 - 30, in multiples of 3 (except for a single‑node option). + - For highly‑available clusters enabling data replication: use ≥ 3 nodes. + - For testing and development: 1 node is sufficient, but note that it is not compatible with a multi‑AZ configuration. -8. Configure **network connectivity**. During Public Beta, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to. + + The number of nodes cannot be changed after cluster creation. + -9. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. During Public Beta, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed). +8. Choose the **storage capacity** to define how much Block Storage to provision for each node. -10. Enter a **name**, and (optionally), **tags** to identify your cluster. +9. Configure **network connectivity**. Currently, connectivity to clusters is facilitated exclusively through **Private Networks**. Select a Private Network from the dropdown list to attach your cluster to. -11. Check the estimated cost summary, then click **Create cluster**. +10. Create **credentials** by defining a username and password. These will be used when connecting to your cluster. Currently, only one user can be created per cluster, and the username is not modifiable after creation (though the password can be changed). + +11. Enter a **name**, and (optionally), **tags** to identify your cluster. + +12. Check the estimated cost summary, then click **Create cluster**. Your cluster is created, and you are taken to its **Overview** page in the console. @@ -59,7 +70,7 @@ Once you have ensured the client satisfies the conditions above, you can facilit 3. In the **Connectivity** panel, click **Connect**. - A pop-up displays, with tabs for each sample framework: **Go**, **Python**, **Java**, **Node.js**, and **Mongoose**. + A pop-up displays, with tabs for each sample framework: **Go**, **Python**, and **Java**. 4. Click the tab of your choice to view the code sample. @@ -67,7 +78,7 @@ Once you have ensured the client satisfies the conditions above, you can facilit Configuration actions that cannot currently be carried out via the Scaleway console, API or devtools include: -- Creating topics, events and streams +- Creating topics, events, and streams - Configuring cluster parameters, such as replication factors, retention policies, and partitions - Integrating clients and consumers