From 48bd31fff407d72221b08e5d51536541f562f702 Mon Sep 17 00:00:00 2001 From: "itamar.marom" Date: Wed, 10 May 2023 01:16:06 +0300 Subject: [PATCH 1/5] add apache druid --- apache-druid/README.md | 12 + apache-druid/spec/README.md | 24 + apache-druid/spec/feature_matrix.md | 166 +++ apache-druid/spec/feature_matrix.yaml | 826 +++++++++++++ apache-druid/spec/yaml2md/README.md | 19 + apache-druid/spec/yaml2md/yaml2md.sh | 11 + apache-druid/spec/yaml2md/yaml2md.tmpl | 7 + apache-druid/vendor-submissions/README.md | 23 + .../vendor-submission-template-generator.sh | 11 + .../vendor-submission-template-generator.tmpl | 23 + .../vendor-submission-template.yaml | 1046 +++++++++++++++++ 11 files changed, 2168 insertions(+) create mode 100644 apache-druid/README.md create mode 100644 apache-druid/spec/README.md create mode 100644 apache-druid/spec/feature_matrix.md create mode 100644 apache-druid/spec/feature_matrix.yaml create mode 100644 apache-druid/spec/yaml2md/README.md create mode 100755 apache-druid/spec/yaml2md/yaml2md.sh create mode 100644 apache-druid/spec/yaml2md/yaml2md.tmpl create mode 100644 apache-druid/vendor-submissions/README.md create mode 100644 apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.sh create mode 100644 apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.tmpl create mode 100644 apache-druid/vendor-submissions/vendor-submission-template.yaml diff --git a/apache-druid/README.md b/apache-druid/README.md new file mode 100644 index 0000000..fcb7c91 --- /dev/null +++ b/apache-druid/README.md @@ -0,0 +1,12 @@ +# Apache Druid Operator Feature Matrix (Apache Druid OFM) + + +This directory contains the OFM for Apache Druid Kubernetes operators. +Please refer to the [main project's README.md](../README.md) for more information about the the OFM (Operator Feature Matrix) project. + + +The contents of this directory are structured as follows: + +* [spec](spec). Contains the specification of the Apache Druid OFM. +* [vendor-submissions](vendor-submissions). Contains the submission files that every vendor has contributed for each of the releases that they submitted a compliance OFM. Includes documentation for vendors on how to submit their compliance OFMs. + diff --git a/apache-druid/spec/README.md b/apache-druid/spec/README.md new file mode 100644 index 0000000..bfd6252 --- /dev/null +++ b/apache-druid/spec/README.md @@ -0,0 +1,24 @@ +# Apache Druid OFM Specification + + +The contents of this directory are structured as follows: + +* [feature_matrix.yaml](feature_matrix.yaml). The feature matrix itself. It is a YAML file that serves as the source of truth for the Apache Druid OFM features. It is encoded in a machine parsable format to allow other files to be derived from it, as well as to allow automatic processing of the features. +* [feature_matrix.md](feature_matrix.md). This file acts as a "view" to the matrix, providing a convenient way to represent it visually. This file should not be edited directly, as it is generated automatically from the [feature_matrix.yaml](feature_matrix.yaml) file. This render is intented to aid in reading and understanding the specification itself, but it is not expected to be consumed directly by end users. Instead, end users are expected to view the submissions made by vendors or, better, the Markdown or web renderings of such submissions. +* [yaml2md](yaml2md). Directory that contains the script to render the Markdown feature matrix from the YAML feature matrix. + + +## Description of the Apache Druid feature matrix + +The OFM is contained in the [feature_matrix.yaml](feature_matrix.yaml) file. The most relevant fields of the document are: +* `.metadata`. Contains information about the OFM Specification, including its version and version release date. +* `.categories`. Features are classified first into categories. It is an array of features. Each category has an `.id` and a `.name`. The `.id` is an immutable and unique four characters identifier, and the `.name` is a mutable and user-friendly definition of the category. +* `.categories[].features`. An array that contains all the features for every category. +* `.categories[].features[]`. Each feature. Similarly to categories, they contain an `.id` (in this case five characters) and a `.name`. They also contain other relevant fields: + * `type`. It is the data type of the value that vendors will provide on the submission for this particular feature. For most of the features this field will be `boolean`, which is the easiest way for end users to visually read a comprehensive feature matrix with many features. Some other features require more details, and may have `enum`, `string` or even `string_array` types for this field. + * `description`. It is the end user facing description of the feature. May include Markdown syntax to enhance readability. + * `vendor_compliance`. It is a field for vendors that provides clarification on how the feature is expected to be implemented in order to achieve compliance with the feature. It also accepts Markdown formatting. + * `main`. It is a cross-categories field to identify features that are considered more important or "core" to the whole OFM. They may be rendered independently of the others to signify their relevance. + + + diff --git a/apache-druid/spec/feature_matrix.md b/apache-druid/spec/feature_matrix.md new file mode 100644 index 0000000..1d3c099 --- /dev/null +++ b/apache-druid/spec/feature_matrix.md @@ -0,0 +1,166 @@ + +## General Characteristics – [genc](#genc) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| instm | Installation mechanism | string_array | Mechanism(s) provided to install the operator. | Provide a succinct value or list of values of the supported mechanisms.
If any of the following names apply, use them, and add other mechanism names, as needed:

* YAML
* Helm
* Kustomize
* Operator Bundle
| ✓ | +| mcrds | Management via CRDs | boolean | The operator manages CRDs (Custom Resource Definition). | If true, provide a list of the names of the CRDs managed by the operator. Names should be direct links to their documentation or API reference.
| ✓ | +| offin | Offline installation | boolean | Whether it is possible to install the operator on a (air-gapped) cluster without internet connection. | A link should be provided with documentation on how to install the operator on air-gapped environments.
| | +| scpua | Supported CPU architectures | string_array | Support for the operator to run on nodes with specified CPU architectures. | Provide a list of CPU architectures that are supported between:

* amd64
* arm
* arm64
* ppc64le
* s390x

In case you support additional architectures, add their name(s) in a similar fashion.
| | +| coios | Container images OSes | string_array | Indicate the base image of the images used by the operator. (e.g: scratch, ubi, fedora, ubuntu, centos, alpine, etc.).
Provide the image name without tags (e.g. debian instead of debian:8.11). | | | +| olmcl | Operator Capability Level | string | [Operator Capability Levels](https://sdk.operatorframework.io/docs/overview/operator-capabilities/) indicates the operator maturity
levels in regards to their lifecycle management capabilities for the application or workload they deliver. The capability models
aims to provide guidance in terminology to express what features users can expect from an operator. | Capability level must be set to one of the following levels:

1. `Basic Install`
2. `Seamless Upgrades`
3. `Full Lifecycle`
4. `Deep Insights`
5. `Auto Pilot`

A link should be provided with documentation on how the operator fulfil the declared capability level.

For a detailed description of each capability level refer to the Operator Framework documentation for
[Operator Capability Level](https://sdk.operatorframework.io/docs/overview/operator-capabilities/).
| | + + +## Versions – [vers](#vers) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| pgver | Supported PostgreSQL versions | string_array | Which major (and minor) versions does this version of the operator support. | Provide a list of every supported major version followed by a list in parenthesis of the minor versions supported for that major version.
E.g. '15 (15.1, 15.0), 14 (14.3, 14.2)'. All versions must be ordered by reverse chronological order.
Other components versions that are used may be described in the comments (e.g: Patroni, WAL-G, PgBackRest, PgBouncer, etc.)
| ✓ | +| k8ver | Supported Kubernetes versions | string_array | Which Kubernetes versions the operator can be installed into. | Provide a list of official Kubernetes minor versions numbers, ordered by reverse chronological order.
Use a range of versions, if possible, e.g. '1.26 - 1.23' or '1.22 - 1.19'.
| ✓ | +| pgfor | PostgreSQL OSS forks | string_array | PostgreSQL forks are considered derived projects that started from PostgreSQL codebase and add different features while maintaining compatibility with PostgreSQL.
Some operators may provide support for some of such forks, alongside with the original PostgreSQL. | Leave empty if no version other than the original PostgreSQL is supported. Otherwise, provide a list of names and links to the PostgreSQL forks supported.
If non-OSS forks are supported, leave empty but add a link to them in the comments section.
| | + + +## Licensing & Support – [lisu](#lisu) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| ossli | Open Source license | boolean | The operator is published, and its source code is made publicly available, under an OSI-approved open source license. | If true, provide the SPDX identifier(s) of the license.
| ✓ | +| fcimg | Free container images | boolean | The container images used for the operator's operation are free to distribute and use, not subject to commercial restrictions. | If false, provide a link with documentation on the usage requisites/restrictions of the container images.
| ✓ | +| nenof | No enterprise-only features | boolean | All the operator's features are provided as open source.
No derived, proprietary (often called 'enterprise') versions exist with additional, non open source features. | If false, provide links to all known derived, proprietary versions.
Linked documentation should clearly reflect the features that are only part of proprietary versions.
| | +| comms | Commercial support | boolean | Vendor and/or other entities offer paid-for, commercial support for the operator. | Provide a link or list of links to the entities that provide commercial support.
Links should be as specific as possible, pointing to informative documentation on the provided support.
| | +| pubit | Public issue tracker | boolean | Development happens in the open. There is a public issue tracker where users may view, comment and create issues. | Provide a specific link to the issue tracker.
| | +| pubch | Public chat/forums | boolean | There is/are public forums, mailing lists, chat groups (Slack, Discord, Matrix, etc) where users may freely join and participate in discussions. | Provide links to all relevant user forums.
| | + + +## PostgreSQL Clusters – [pgcl](#pgcl) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| depau | Deployment automation | boolean | The operator provides capabilities to automatically deploy production-ready clusters based on a provided configuration.
No user initiated commands must be required. | | | +| pomte | Pod management technology | string | Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. | Provide the most succinct possible name of the technology. Use the Kubernetes resource type name (e.g. 'Deployment'), if applicable.
| ✓ | +| pgcnf | PostgreSQL custom config | boolean | The operator allows the user to supply custom PostgreSQL configuration. | | | +| conpl | Integrated connection pooling | boolean | The operator provides options to deploy a connection pool in front of the database, automatically deployed and configured. | The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc.
All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler.
| ✓ | +| cpccf | Connection pool custom config | boolean | The operator allows the user to supply a custom connection pool configuration for the connection pool service. | Only applies if [pgcl/conpl] is true.
| | +| tlssp | TLS Support | boolean | PostgreSQL connections can be secured with Postgres SSL/TLS support. | | ✓ | +| tlscu | TLS user-provided certificates | boolean | Operators may chose by default to generate self-signed SSL certificates.
They may also offer the option to specify the CA and certificates that users want Postgres clusters to use. | | | +| crtmg | CertManager integration | boolean | The operator integrates with CertManager in order to generate the certificate to be used with Postgres. | Only applies if [pgcl/tlscu] is true.
| | +| insql | Initialization from SQL scripts | boolean | After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation.
The operator must properly inform the user of the execution result of the scripts. | | | +| inext | Initialization from external source | boolean | After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data.
The operator must properly inform the user of the execution result of the scripts. | | | +| cuhba | HBA custom config | boolean | The operator allows users to provide custom configuration for the Postgres HBA (Host-Based Authentication) mechanism.
It may involve directly generating the custom `pg_hba.conf` file or by using more cloud-native technologies. | | | +| mgmup | Management of db users/passwords | boolean | The operator provides a mechanism to declaratively create, modify or delete users, roles and/or password for the PostgreSQL cluster. | | | +| reqli | Customize computing resource requests, limits | boolean | The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the Postgres pods. | | ✓ | +| hugpa | Support for huge pages | boolean | The user may specify the request to use huge pages for Postgres (and/or potentially other sidecars).
Postgres must be able to be configured and start using huge pages. | | ✓ | +| pgsrv | PostgreSQL exposed via Service | string | The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the PostgreSQL connections.
Expected capabilities should include a RW (read-write) or RO (read-only in case of cascading replication) connection to the primary instance; and, optionally, a RO (read-only) to load balance read-only replicas instances in the cluster. | The value must be one of:

* PrimaryAndReplicas: Primary and replicas services must be offered.
* PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered.
* Balanced: A single service that load balances RW/RO traffic transparently.
* Primary: Primary service must be offered.
| | +| stosc | Automatic storage scaling | boolean | If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically
(either by setting some default thresholds or requiring explicit declarative configuration from the user). | | | +| scal0 | Scale down to zero | boolean | The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed.
Upon scaling up, the cluster should be brought back up without the need to restore a backup. This feature may also be known as "hibernation". | | | +| tblsp | Tablespaces | boolean | The user may specify one or more PostgreSQL tablespaces and their associated backing storage. | | | +| cupgi | Custom Postgres images | boolean | The operator allows the user to specify custom (user-provided) Postgres container images, instead of using the operator's provided images. | The operator should specify if the custom image needs to follow some minimal patterns to be able to work, or if it can work
with any postgres container image provided.
| | +| uside | User supplied sidecars | boolean | The user may specify custom sidecars (containers or init containers) to be created alongside the Postgres container (and, possibly, other operator sidecars).
User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. | | | +| usvol | User supplied volumes | boolean | The user may specify and mount custom volumes to be created alongside the volumes created for the Postgres container. | | | +| usprt | User supplied ports | boolean | The user may specify custom ports to be created alongside the services that expose PostgreSQL. | Only applies if [pgcl/pgsrv] is set.
| | +| srvbi | Support for Service Binding | boolean | [Service Binding](https://servicebinding.io/spec/core/1.0.0/) is a Kubernetes specification to "communicating service secrets to workloads in a consistent way".
If the operator supports Service Binding, applications that are bound to the Postgres clusters will be able to fetch database access credentials without requiring
the user to specify them manually in the application. | The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification.
| | +| cuann | Custom Annotations | boolean | The operator allows the user to provide custom [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)
(aside from any operator generated Annotations) to the objects generated by the operator. | The operator must provide configuration for setting Annotations on all objects generated by the operator.
Partial compliance is achieved if at least Pods can be annotated and this is clearly marked in the comments.
| | +| culab | Custom Labels | boolean | The operator allows the user to provide custom [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the objects generated by the operator. | The operator must provide configuration for setting Labels on all objects generated by the operator.
Partial compliance is achieved if at Labels can be added to at least Pods and this is clearly marked in the comments.
| | +| apdnd | Assign Pod to Node | string | The operator allows Postgres Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. | Indicate one of the following values:

* 'Complete': if the operator support all the rules to assign a Pod to a Node defined in the latest Kubernetes version specified under [vers/k8ver].
* 'Partial': if the operator support only partially the rules to assign a Pod to a Node.

A link for each supported rule or rules group to assign a Pod to a Node must be specified.
| | +| topsc | Topology Spread Constraints | boolean | The operator allows Postgres pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints),
to control how Pods are spread across the Kubernetes cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. | | | + + +## HA & Replication – [harp](#harp) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| autfa | Automated failover | boolean | The operator provides the facility that in the event of a node or pod failure where the Postgres primary is affected,
another Postgres instance (if available) will be promoted to primary. The operation must happen automatically without user intervention. | | ✓ | +| techa | HA Technology | string | The technology (software name or technology principles) that support the high availability and automated failover capabilities of the operator. | Provide a succinct software or technology name. E.g.: Patroni, Stolon, Custom.
| ✓ | +| asrep | Asynchronous replication | boolean | The operator allows to configure Postgres clusters using asynchronous streaming replication.
Asynchronous is the default streaming replication mode in Postgres. | | | +| syrep | Synchronous replication | boolean | The operator allows to configure Postgres clusters using synchronous streaming replication.
The operator must manage the configuration details based on the user's preferences on which nodes behave synchronously. | | | +| serep | Semi-synchronous replication | boolean | Semi-synchronous, or also called group or quorum replication, is a hybrid mode where a subset of the nodes is expected to replicate synchronously;
and once that level is reached, the remainding nodes replicate asynchronously.
That is, Postgres waits for confirmation of the write only from the synchronous nodes before considering the transaction committed. | | | +| derep | Delayed replicas | boolean | The operator allows the user to configure one or more replicas subject to a user-specified (intentional) replication lag.
This is useful to have always an online instance with data "in the past" for analytics or data recovery purposes. | | | +| carep | Cascading replication | boolean | Postgres allows to create replicas that are fed from another replica, instead of replicating from the primary instance.
This is interesting to alleviate the effects of replication on the primary; and allows to create arbitrary replication tree-like topologies. | | | +| exrep | Replication from external origin | boolean | The operator allows a Postgres cluster to be setup in recovery mode while replicating from an external (non-operator managed) Postgres origin.
This allows migrations to the operator using streaming replication. | To implement the feature, the user should be able to provide arbitrary host, port, username and password (or other means of credential passing) of the origin.
| | +| walsh | WAL Shipping | boolean | WAL Shipping is a Postgres technique that allows replication via a shared storage mechanism where WAL files produced on the primary node are copied ("shipped") to the replica.
This feature is useful for having replicas for example on DR sites, replicate over global storage object stores and others. | Implementing the feature implies both generating WAL files to a suitable location; as well as creating clusters that permanently read those WAL files and replicate from them.
| | +| prstb | Promote standby cluster | boolean | The operator allows the user to promote a standby cluster that is a cluster with primary in recovery mode so that the primary becomes RW. This feature is used in conjunction with [harp/exrep] and/or [harp/walsh] to make GEO-replicated and/or DR (Disaster Recovery) clusters on other regions, where a region is essentially an independent Kubernetes cluster. | Only applies if [harp/exrep] or [harp/walsh] are true.
The operator must allow to promote a standby clusters replicating either via WAL Shipping and/or streaming replication.
| ✓ | +| lorep | Managed logical replication | boolean | Postgres supports logical replication as well as streaming replication. It requires some configuration and commands to be run by the user.
This feature represents operator capabilities to perform these operations in a managed way, without the user having to type commands or create configurations directly. | The operator is considered compliant if it allows the user to provide logical replication without having to setup it directly, just specifying (declarative) preferences.
The operator is still considered compliant if it cannot proceed until certain operations are performed by the users that are dependent on their environment
(e.g. adding PKs, UNIQUEs or REPLICA IDENTITY to the DDL).
| | + + +## PostgreSQL Extensions – [pgex](#pgex) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| extme | Extensions distribution mechanism | string | This feature is set to describe how extensions are shipped in a containerized environment.
Typically they are part of the same Postgres container image, but they may also be distributed via other mechanisms. | Indicate 'built-in' if the extensions come included with the same Postgres container; or a succinct word or few words naming the distribution mechanism.
More details may be provided, if needed, via the links and comments fields.
| | +| coext | Core/contrib extensions | integer | The number of core+contrib extensions provided by the operator for the latest Postgres version provided by the operator. | Submission must provide the total number of core+contrib extensions supported for the latest Postgres version provided by the operator.
It should also be provided, when available, a link with a detailed list of those extensions supported (via links field).
| ✓ | +| thext | Third-party extensions | integer | The number of extensions not included in the Postgres core+contrib (i.e. created by third parties, outside of Postgres repository) for the latest Postgres version provided by the operator. | Submission must provide the total number of third-party extensions supported for the latest Postgres version provided by the operator.
A link with a detailed list of those extensions supported (via links field) should also be provided, if available.
| ✓ | +| byext | Bring your own extension | boolean | The operator provides mechanisms for users to add (or upload) third-party extensions not initially provided by the operator. | The operator must support providing some mechanism to include custom extensions additionally or independently of the extension distribution mechanism,
if a user builds a local extension, this could be uploaded or included on the postgres container with an automatic copy to all replicas related.
| | + + +## Backups – [bkup](#bkup) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| bktch | Backup Technology | string | What technology (pgbasebackup, PgBackRest, WAL-e/g, Barman, custom, etc) is used to support creation and restoration of backups. | Provide the most succinct possible name of the technology.
| ✓ | +| bkdst | Backup destinations | string_array | Where backups can be stored (typically this may include object stores, PVs, etc). | If any of the following technology names apply, use them, and add other names, as needed:

* PersistentVolume
* NFS
* S3
* GCS
* Azure Blob
* Local file system
| | +| autbk | Automated backup management | boolean | The operator provides mechanisms for performing backups automatically and providing lifecycle mechanisms (delete old backups according to a user-supplied policy - often known as retention policies). | True response for the feature implies both automatic backups and lifecycle management.
If only the former is provided, answer should be false but this capability should be mentioned in the comments field.
| ✓ | +| encbk | Backups encryption | boolean | Backups can be performed with user-supplied encryption keys. | | | +| ptrbk | Point In Time Recovery | boolean | The operator provides support for the user to specify a recovery point (in the past) to which a backup should be recovered to
(if unspecified, backup will be recovered in full). | True response implies that at least time-based recovery is supported.
If the operator also supports PITR by xid and label, clarify in the comments field.
| | +| mulbk | Multiple backup configurations | boolean | The operator supports managing more than one backup configuration.
This is typically used to store backups on different object stores (for protection purposes) or to send them to different sites.
It may also include different lifecycle policies. | | | + + +## Management Interfaces – [mgmi](#mgmi) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| cucol | Resource display columns | boolean | Operator-managed objects (CRDs) include specific fields that provide additional information about the Postgres instances.
These fields are shown when querying the Kubernetes resources using tools like `kubectl`. | In order to be compliant, operators have to provide at least the following information:

* Replication status (primary/replica/etc)
* Cluster status (healthy/not healthy)
* Postgres major/minor version
* Latest base backup date
* SSL on/off
| | +| kbplg | kubectl plugin | boolean | The users may download an additional kubectl plugin which provides custom commands to manage the operator. | If the feature is provided, include a link to the downloadable plugin in the links field.
| | +| bagui | Basic management GUI | boolean | The operator bundles some GUI (web interface or other) that allows to perform basic operations (e.g. cluster creation) graphically. | | ✓ | +| fugui | Fully-featured GUI | boolean | The operator bundles a GUI that is fully-featured (that is, in feature-parity with the capabilities provided via the other management interfaces -- typically CRDs).
The GUI may provide additional features that cannot be otherwise provided via text interfaces (e.g. graphical representation of resources or operations). | | ✓ | +| urgui | GUI users & roles | boolean | The bundled management GUI has authentication and authorization baked in and provides mechanisms to support multiple users and authorize/deny them to perform operations. | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided.
Submission should detail in the link and/or comments field how users and roles for the GUI can be managed.
| | +| guiss | GUI Single-Sign On | boolean | The bundled management GUI supports integration with Single-Sign On (SSO), like OIDC, identity providers (e.g. Google, Github or Twitter login). | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided.
| | +| cogui | GUI database console | boolean | The bundled management GUI includes a console that can connect to any database managed by the operator and send command (e.g. like `psql`). | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided.
| | + + +## Observability – [o11y](#o11y) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| mtech | Metrics technology | string_array | If the operator supports extracting metrics from Postgres, define how they are handled, which technology receives and processes them.

If any of the following technology names apply, use them, and add other names, as needed:

* Prometheus
* OTEL
* OpenTSDB
* Nagios
* Sensu | If supported, provide in the comments information about whether the technology is a dependency, is built-in, external, etc.
Provide link(s) to the documentation for further information.
| ✓ | +| expme | Export metrics | boolean | Regardless of how metrics are processed (e.g. as part of the operator),
this feature is implemented when the operator allows the user to configure sending metrics to external services, like a metrics SaaS. | | | +| cudas | Custom dashboards | boolean | In order to display the captured Postgres metrics, the operator provides specialized Postgres dashboards for the users. | | | +| cuale | Custom alerts | boolean | The operator provides bundled specific Postgres alerts to be triggered on the Postgres metrics processed.
E.g. there is an alert for transaction wraparound or for unused replication slots. | | | +| exdel | Exposed decorated logs | boolean | The operator provides a mechanism to expose all the logs of the managed Postgres instances to a centralized logging tool.
The logs must be decorated with extra metadata in order to provide semantic meaning, including the Pod name and namespace, the cluster name, the role of the Postgres instance (e.g. primary, replica, standby-leader, etc.) and the timestamp that will be available to be used to filter logs entries.
There is no need to configure the tool in order to obtain required extra metadata from the logs. | | | +| explg | Export logs | boolean | The operator allows the user to configure an external sink for the Postgres logs (e.g. a SaaS service). | | | +| oo11y | Operator Observability | boolean | The operator is itself a source of telemetry data, potentially including metrics, traces and logs, about its own performance. | | | + + +## Security – [secy](#secy) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| opcsc | Operator code security scanning | boolean | The operator code is always scanned for security vulnerabilities. | | ✓ | +| imgsc | Image scanning | boolean | Operator-provided images are always scanned for security vulnerabilities. | | ✓ | +| sigim | Signed images | boolean | Container images are digitally signed according to the [sigstore](https://www.sigstore.dev/) project. | Compliance may also be achieved by using a technology other than sigstore, as long as it provides an equivalent set of security capabilities.
| | +| isbom | Software Bill of Materials | boolean | The operator releases include the SBOM (Software Bill of Materials), a detailed description of all the components, modules, and their dependencies. | SBOM is expected to be in accordance to the [Kubernetes SIG BOM](https://github.com/kubernetes-sigs/bom).
| | +| fgopp | Fine-grained RBAC permissions | boolean | The operator uses a separate serviceaccount that has RBAC permissions that only require the access that is actually needed to create and manage the Kubernetes resources, not more. | | | +| noprm | No or justified privileged mode | boolean | The operator-provided containers do not require privileged mode.
The container processes do not run as root. | Reasonable exceptions to this rule can be made for features that require or do not diminish the container's security, e.g. when using eBPF.
| | + + +## Day 2 Operations – [day2](#day2) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| amiup | Automated minor upgrades | boolean | The operator can perform a minor version upgrade of a Postgres cluster automatically.
This operation can be managed by the user declaratively. | The operator must provide proper information to the user as to the status and final result of the operation.
The operator should provide ongoing status information, and perform the operation with the minimum downtime required.
Provide information about the update strategy (i.e. restart of the pods or rolling update followed by a switchover or a restart).
| ✓ | +| amaup | Automated major upgrades | boolean | The operator can perform a major version upgrade of a Postgres cluster automatically.
This operation can be managed by the user declaratively. | The operator must provide proper information to the user as to the status and final result of the operation.
The operator should provide ongoing status information, and perform the operation with the minimum downtime required.
| ✓ | +| crest | Controlled cluster restart | boolean | Sometimes Postgres needs to be restarted (e.g. changing of a parameter that requires restart).
The operator provides means to perform this operation automatically and in a controlled manner (rolling restart) so that the cluster faces a minimal downtime only. | | | +| ociup | Container images upgrade | boolean | Similarly to the controlled restart operation, the operator is capable of updating the running container images (which require a pod restart) automatically and with minimal cluster impact. | | | +| swtch | Switchover | boolean | If HA capabilities are provided, the operator also provides a mechanism for manual switchover.
The user may specify the configuration declaratively and the operator will perform the desired switchover automatically, by demoting the current primary, promoting the a replica, and updating the endpoints/services as required. | | | +| sqlmi | SQL Migrations | boolean | The operator provides managed SQL migration capabilities.
The user may specify SQL scripts that contain migrations (DDL changes, etc) to be deployed to a given database, having the operator apply them automatically. | The operator must report back to the user detailed information about the results of the execution(s) of the script(s) provided by the user.
| | +| oday2 | Other Day 2 Operations | string_array | The operator provides support for other managed Day 2 operations. | All the mentioned additional day 2 operations need to be possible via declarative configuration and the operator to fully execute them without further user intervention.
| | + + +## Data Integration – [dain](#dain) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| kfkin | Kafka integration | boolean | The operator provides a managed solution for automation to export CDC (Change Data Capture) events to Kafka. | Kafka could be an external dependency to the operator.
The whole operation is expected to be fully automated, with the user only providing a declarative configuration.
| | +| migpg | Migration from Postgres | string_array | The operator provides a managed capability to automatically migrate data from another (external) Postgres database.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations. | Provide one or more of the following values: `offline`, `cdc`.
| | +| migmy | Migration from MySQL | string_array | The operator provides a managed capability to automatically migrate data from a MySQL database.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations.
Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. | Provide one or more of the following values: `offline`, `cdc`, `schema conversion`.
| | +| migor | Migration from Oracle | string_array | The operator provides a managed capability to automatically migrate data from an Oracle database.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations.
Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. | Provide one or more of the following values: `offline`, `cdc`, `schema conversion`.
| | +| migms | Migration from SQL Server | string_array | The operator provides a managed capability to automatically migrate data from a SQL Server database.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations.
Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. | Provide one or more of the following values: `offline`, `cdc`, `schema conversion`.
| | +| migmo | Migration from MongoDB | string_array | The operator provides a managed capability to automatically migrate data from a MongoDB database.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations.
Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. | Provide one or more of the following values: `offline`, `cdc`, `schema conversion`.
| | +| migot | Migration from other databases | string_array | The operator provides a managed capability to automatically migrate data from other relational or non-relational databases.
Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),
which allows for near-zero downtime migrations.
Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. | Provide the names of the other database from where migration is supported.
| | + + +## Sharding – [shrd](#shrd) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +| mshrd | Managed sharding support | boolean | The operator understands horizontal scalability and is capable to automatically deploying sets of clusters (shards) with their accompanying infrastructure (coordinators, services, etc).
The user should be able to write a declarative configuration of a sharded cluster and the operator to fully deploy it without any further manual user intervention. | | ✓ | +| shrdt | Sharding technology | string_array | If managed sharding support is provided, which technology or technologies are used to provide sharding support (which is not included in Postgres core). | | | +| aggds | Aggregated monitoring dashboards | boolean | Since a sharded cluster is a set of (independent) Postgres clusters, an aggregated dashboard is one that provides Observability information across all shards of the cluster.
E.g. total database size, tps processed or total number of client connections. | | | +| cobkp | Coordinated backups | boolean | A sharded cluster can be backed up by backing up each shard independently, but those backups are not necessarily consistent with each other.
If sharding is used, an additional capability is to coordinate backups on a given timestamp so that they are performed or can be restored at the same logical timestamp,
so they become consistent with each other. | | | + diff --git a/apache-druid/spec/feature_matrix.yaml b/apache-druid/spec/feature_matrix.yaml new file mode 100644 index 0000000..1b30434 --- /dev/null +++ b/apache-druid/spec/feature_matrix.yaml @@ -0,0 +1,826 @@ +--- +metadata: + version: v1.1.0 + date: 26.04.2023 +categories: + - id: genc + name: General Characteristics + features: + - id: instm + name: Installation mechanism + type: string_array + description: | + Mechanism(s) provided to install the operator. + vendor_compliance: | + * Helm + * Kustomize + main: true + - id: mcrds + name: Management via CRDs + type: boolean + description: | + The operator manages CRDs (Custom Resource Definition). + vendor_compliance: | + * druid.apache.org/v1alpha1/Druid - https://github.com/datainfrahq/druid-operator/blob/v1.1.0/apis/druid/v1alpha1/druid_types.go + main: true + - id: offin + name: Offline installation + type: boolean + description: | + Whether it is possible to install the operator on a (air-gapped) cluster without internet connection. + vendor_compliance: | + Druid operator can be installed in air-gapped environment the same as with any other type of environment. + Installation documentation - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + - id: scpua + name: Supported CPU architectures + type: string_array + description: | + Support for the operator to run on nodes with specified CPU architectures. + vendor_compliance: | + * amd64 + - id: coios + name: Container images OSes + type: string_array + description: | + Druid's base image is depends on the JDK version required: + gcr.io/distroless/java$JDK_VERSION-debian11 + - id: olmcl + name: Operator Capability Level + type: string + description: | + [Operator Capability Levels](https://sdk.operatorframework.io/docs/overview/operator-capabilities/) indicates the operator maturity + levels in regards to their lifecycle management capabilities for the application or workload they deliver. The capability models + aims to provide guidance in terminology to express what features users can expect from an operator. + vendor_compliance: | + Capability level must be set to one of the following levels: + + 1. `Basic Install` + 2. `Seamless Upgrades` + 3. `Full Lifecycle` + 4. `Deep Insights` + 5. `Auto Pilot` + + A link should be provided with documentation on how the operator fulfil the declared capability level. + + For a detailed description of each capability level refer to the Operator Framework documentation for + [Operator Capability Level](https://sdk.operatorframework.io/docs/overview/operator-capabilities/). + - id: vers + name: Versions + features: + - id: pgver + name: Supported PostgreSQL versions + type: string_array + description: | + Which major (and minor) versions does this version of the operator support. + vendor_compliance: | + Provide a list of every supported major version followed by a list in parenthesis of the minor versions supported for that major version. + E.g. '15 (15.1, 15.0), 14 (14.3, 14.2)'. All versions must be ordered by reverse chronological order. + Other components versions that are used may be described in the comments (e.g: Patroni, WAL-G, PgBackRest, PgBouncer, etc.) + main: true + - id: k8ver + name: Supported Kubernetes versions + type: string_array + description: | + Which Kubernetes versions the operator can be installed into. + vendor_compliance: | + Provide a list of official Kubernetes minor versions numbers, ordered by reverse chronological order. + Use a range of versions, if possible, e.g. '1.26 - 1.23' or '1.22 - 1.19'. + main: true + - id: pgfor + name: PostgreSQL OSS forks + type: string_array + description: | + PostgreSQL forks are considered derived projects that started from PostgreSQL codebase and add different features while maintaining compatibility with PostgreSQL. + Some operators may provide support for some of such forks, alongside with the original PostgreSQL. + vendor_compliance: | + Leave empty if no version other than the original PostgreSQL is supported. Otherwise, provide a list of names and links to the PostgreSQL forks supported. + If non-OSS forks are supported, leave empty but add a link to them in the comments section. + - id: lisu + name: Licensing & Support + features: + - id: ossli + name: Open Source license + type: boolean + description: | + The operator is published, and its source code is made publicly available, under an OSI-approved open source license. + vendor_compliance: | + If true, provide the SPDX identifier(s) of the license. + main: true + - id: fcimg + name: Free container images + type: boolean + description: | + The container images used for the operator's operation are free to distribute and use, not subject to commercial restrictions. + vendor_compliance: | + If false, provide a link with documentation on the usage requisites/restrictions of the container images. + main: true + - id: nenof + name: No enterprise-only features + type: boolean + description: | + All the operator's features are provided as open source. + No derived, proprietary (often called 'enterprise') versions exist with additional, non open source features. + vendor_compliance: | + If false, provide links to all known derived, proprietary versions. + Linked documentation should clearly reflect the features that are only part of proprietary versions. + - id: comms + name: Commercial support + type: boolean + description: | + Vendor and/or other entities offer paid-for, commercial support for the operator. + vendor_compliance: | + Provide a link or list of links to the entities that provide commercial support. + Links should be as specific as possible, pointing to informative documentation on the provided support. + - id: pubit + name: Public issue tracker + type: boolean + description: | + Development happens in the open. There is a public issue tracker where users may view, comment and create issues. + vendor_compliance: | + Provide a specific link to the issue tracker. + - id: pubch + name: Public chat/forums + type: boolean + description: | + There is/are public forums, mailing lists, chat groups (Slack, Discord, Matrix, etc) where users may freely join and participate in discussions. + vendor_compliance: | + Provide links to all relevant user forums. + - id: pgcl + name: PostgreSQL Clusters + features: + - id: depau + name: Deployment automation + type: boolean + description: | + The operator provides capabilities to automatically deploy production-ready clusters based on a provided configuration. + No user initiated commands must be required. + - id: pomte + name: Pod management technology + type: string + description: | + Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. + vendor_compliance: | + Provide the most succinct possible name of the technology. Use the Kubernetes resource type name (e.g. 'Deployment'), if applicable. + main: true + - id: pgcnf + name: PostgreSQL custom config + type: boolean + description: | + The operator allows the user to supply custom PostgreSQL configuration. + - id: conpl + name: Integrated connection pooling + type: boolean + description: | + The operator provides options to deploy a connection pool in front of the database, automatically deployed and configured. + vendor_compliance: | + The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc. + All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. + main: true + - id: cpccf + name: Connection pool custom config + type: boolean + description: | + The operator allows the user to supply a custom connection pool configuration for the connection pool service. + vendor_compliance: | + Only applies if [pgcl/conpl] is true. + - id: tlssp + name: TLS Support + type: boolean + description: | + PostgreSQL connections can be secured with Postgres SSL/TLS support. + main: true + - id: tlscu + name: TLS user-provided certificates + type: boolean + description: | + Operators may chose by default to generate self-signed SSL certificates. + They may also offer the option to specify the CA and certificates that users want Postgres clusters to use. + - id: crtmg + name: CertManager integration + type: boolean + description: | + The operator integrates with CertManager in order to generate the certificate to be used with Postgres. + vendor_compliance: | + Only applies if [pgcl/tlscu] is true. + - id: insql + name: Initialization from SQL scripts + type: boolean + description: | + After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation. + The operator must properly inform the user of the execution result of the scripts. + - id: inext + name: Initialization from external source + type: boolean + description: | + After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data. + The operator must properly inform the user of the execution result of the scripts. + - id: cuhba + name: HBA custom config + type: boolean + description: | + The operator allows users to provide custom configuration for the Postgres HBA (Host-Based Authentication) mechanism. + It may involve directly generating the custom `pg_hba.conf` file or by using more cloud-native technologies. + - id: mgmup + name: Management of db users/passwords + type: boolean + description: | + The operator provides a mechanism to declaratively create, modify or delete users, roles and/or password for the PostgreSQL cluster. + - id: reqli + name: Customize computing resource requests, limits + type: boolean + description: | + The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the Postgres pods. + main: true + - id: hugpa + name: Support for huge pages + type: boolean + description: | + The user may specify the request to use huge pages for Postgres (and/or potentially other sidecars). + Postgres must be able to be configured and start using huge pages. + main: true + - id: pgsrv + name: PostgreSQL exposed via Service + type: string + description: | + The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the PostgreSQL connections. + Expected capabilities should include a RW (read-write) or RO (read-only in case of cascading replication) connection to the primary instance; and, optionally, a RO (read-only) to load balance read-only replicas instances in the cluster. + vendor_compliance: | + The value must be one of: + + * PrimaryAndReplicas: Primary and replicas services must be offered. + * PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered. + * Balanced: A single service that load balances RW/RO traffic transparently. + * Primary: Primary service must be offered. + - id: stosc + name: Automatic storage scaling + type: boolean + description: | + If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically + (either by setting some default thresholds or requiring explicit declarative configuration from the user). + - id: scal0 + name: Scale down to zero + type: boolean + description: | + The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed. + Upon scaling up, the cluster should be brought back up without the need to restore a backup. This feature may also be known as "hibernation". + - id: tblsp + name: Tablespaces + type: boolean + description: | + The user may specify one or more PostgreSQL tablespaces and their associated backing storage. + - id: cupgi + name: Custom Postgres images + type: boolean + description: | + The operator allows the user to specify custom (user-provided) Postgres container images, instead of using the operator's provided images. + vendor_compliance: | + The operator should specify if the custom image needs to follow some minimal patterns to be able to work, or if it can work + with any postgres container image provided. + - id: uside + name: User supplied sidecars + type: boolean + description: | + The user may specify custom sidecars (containers or init containers) to be created alongside the Postgres container (and, possibly, other operator sidecars). + User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. + - id: usvol + name: User supplied volumes + type: boolean + description: | + The user may specify and mount custom volumes to be created alongside the volumes created for the Postgres container. + - id: usprt + name: User supplied ports + type: boolean + description: | + The user may specify custom ports to be created alongside the services that expose PostgreSQL. + vendor_compliance: | + Only applies if [pgcl/pgsrv] is set. + - id: srvbi + name: Support for Service Binding + type: boolean + description: | + [Service Binding](https://servicebinding.io/spec/core/1.0.0/) is a Kubernetes specification to "communicating service secrets to workloads in a consistent way". + If the operator supports Service Binding, applications that are bound to the Postgres clusters will be able to fetch database access credentials without requiring + the user to specify them manually in the application. + vendor_compliance: | + The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification. + - id: cuann + name: Custom Annotations + type: boolean + description: | + The operator allows the user to provide custom [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + (aside from any operator generated Annotations) to the objects generated by the operator. + vendor_compliance: | + The operator must provide configuration for setting Annotations on all objects generated by the operator. + Partial compliance is achieved if at least Pods can be annotated and this is clearly marked in the comments. + - id: culab + name: Custom Labels + type: boolean + description: | + The operator allows the user to provide custom [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the objects generated by the operator. + vendor_compliance: | + The operator must provide configuration for setting Labels on all objects generated by the operator. + Partial compliance is achieved if at Labels can be added to at least Pods and this is clearly marked in the comments. + - id: apdnd + name: Assign Pod to Node + type: string + description: | + The operator allows Postgres Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. + vendor_compliance: | + Indicate one of the following values: + + * 'Complete': if the operator support all the rules to assign a Pod to a Node defined in the latest Kubernetes version specified under [vers/k8ver]. + * 'Partial': if the operator support only partially the rules to assign a Pod to a Node. + + A link for each supported rule or rules group to assign a Pod to a Node must be specified. + - id: topsc + name: Topology Spread Constraints + type: boolean + description: | + The operator allows Postgres pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints), + to control how Pods are spread across the Kubernetes cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. + - id: harp + name: HA & Replication + features: + - id: autfa + name: Automated failover + type: boolean + description: | + The operator provides the facility that in the event of a node or pod failure where the Postgres primary is affected, + another Postgres instance (if available) will be promoted to primary. The operation must happen automatically without user intervention. + main: true + - id: techa + name: HA Technology + type: string + description: | + The technology (software name or technology principles) that support the high availability and automated failover capabilities of the operator. + vendor_compliance: | + Provide a succinct software or technology name. E.g.: Patroni, Stolon, Custom. + main: true + - id: asrep + name: Asynchronous replication + type: boolean + description: | + The operator allows to configure Postgres clusters using asynchronous streaming replication. + Asynchronous is the default streaming replication mode in Postgres. + - id: syrep + name: Synchronous replication + type: boolean + description: | + The operator allows to configure Postgres clusters using synchronous streaming replication. + The operator must manage the configuration details based on the user's preferences on which nodes behave synchronously. + - id: serep + name: Semi-synchronous replication + type: boolean + description: | + Semi-synchronous, or also called group or quorum replication, is a hybrid mode where a subset of the nodes is expected to replicate synchronously; + and once that level is reached, the remainding nodes replicate asynchronously. + That is, Postgres waits for confirmation of the write only from the synchronous nodes before considering the transaction committed. + - id: derep + name: Delayed replicas + type: boolean + description: | + The operator allows the user to configure one or more replicas subject to a user-specified (intentional) replication lag. + This is useful to have always an online instance with data "in the past" for analytics or data recovery purposes. + - id: carep + name: Cascading replication + type: boolean + description: | + Postgres allows to create replicas that are fed from another replica, instead of replicating from the primary instance. + This is interesting to alleviate the effects of replication on the primary; and allows to create arbitrary replication tree-like topologies. + - id: exrep + name: Replication from external origin + type: boolean + description: | + The operator allows a Postgres cluster to be setup in recovery mode while replicating from an external (non-operator managed) Postgres origin. + This allows migrations to the operator using streaming replication. + vendor_compliance: | + To implement the feature, the user should be able to provide arbitrary host, port, username and password (or other means of credential passing) of the origin. + - id: walsh + name: WAL Shipping + type: boolean + description: | + WAL Shipping is a Postgres technique that allows replication via a shared storage mechanism where WAL files produced on the primary node are copied ("shipped") to the replica. + This feature is useful for having replicas for example on DR sites, replicate over global storage object stores and others. + vendor_compliance: | + Implementing the feature implies both generating WAL files to a suitable location; as well as creating clusters that permanently read those WAL files and replicate from them. + - id: prstb + name: Promote standby cluster + type: boolean + description: | + The operator allows the user to promote a standby cluster that is a cluster with primary in recovery mode so that the primary becomes RW. This feature is used in conjunction with [harp/exrep] and/or [harp/walsh] to make GEO-replicated and/or DR (Disaster Recovery) clusters on other regions, where a region is essentially an independent Kubernetes cluster. + vendor_compliance: | + Only applies if [harp/exrep] or [harp/walsh] are true. + The operator must allow to promote a standby clusters replicating either via WAL Shipping and/or streaming replication. + main: true + - id: lorep + name: Managed logical replication + type: boolean + description: | + Postgres supports logical replication as well as streaming replication. It requires some configuration and commands to be run by the user. + This feature represents operator capabilities to perform these operations in a managed way, without the user having to type commands or create configurations directly. + vendor_compliance: | + The operator is considered compliant if it allows the user to provide logical replication without having to setup it directly, just specifying (declarative) preferences. + The operator is still considered compliant if it cannot proceed until certain operations are performed by the users that are dependent on their environment + (e.g. adding PKs, UNIQUEs or REPLICA IDENTITY to the DDL). + - id: pgex + name: PostgreSQL Extensions + features: + - id: extme + name: Extensions distribution mechanism + type: string + description: | + This feature is set to describe how extensions are shipped in a containerized environment. + Typically they are part of the same Postgres container image, but they may also be distributed via other mechanisms. + vendor_compliance: | + Indicate 'built-in' if the extensions come included with the same Postgres container; or a succinct word or few words naming the distribution mechanism. + More details may be provided, if needed, via the links and comments fields. + - id: coext + name: Core/contrib extensions + type: integer + description: | + The number of core+contrib extensions provided by the operator for the latest Postgres version provided by the operator. + vendor_compliance: | + Submission must provide the total number of core+contrib extensions supported for the latest Postgres version provided by the operator. + It should also be provided, when available, a link with a detailed list of those extensions supported (via links field). + main: true + - id: thext + name: Third-party extensions + type: integer + description: | + The number of extensions not included in the Postgres core+contrib (i.e. created by third parties, outside of Postgres repository) for the latest Postgres version provided by the operator. + vendor_compliance: | + Submission must provide the total number of third-party extensions supported for the latest Postgres version provided by the operator. + A link with a detailed list of those extensions supported (via links field) should also be provided, if available. + main: true + - id: byext + name: Bring your own extension + type: boolean + description: | + The operator provides mechanisms for users to add (or upload) third-party extensions not initially provided by the operator. + vendor_compliance: | + The operator must support providing some mechanism to include custom extensions additionally or independently of the extension distribution mechanism, + if a user builds a local extension, this could be uploaded or included on the postgres container with an automatic copy to all replicas related. + - id: bkup + name: Backups + features: + - id: bktch + name: Backup Technology + type: string + description: | + What technology (pgbasebackup, PgBackRest, WAL-e/g, Barman, custom, etc) is used to support creation and restoration of backups. + vendor_compliance: | + Provide the most succinct possible name of the technology. + main: true + - id: bkdst + name: Backup destinations + type: string_array + description: | + Where backups can be stored (typically this may include object stores, PVs, etc). + vendor_compliance: | + If any of the following technology names apply, use them, and add other names, as needed: + + * PersistentVolume + * NFS + * S3 + * GCS + * Azure Blob + * Local file system + - id: autbk + name: Automated backup management + type: boolean + description: | + The operator provides mechanisms for performing backups automatically and providing lifecycle mechanisms (delete old backups according to a user-supplied policy - often known as retention policies). + vendor_compliance: | + True response for the feature implies both automatic backups and lifecycle management. + If only the former is provided, answer should be false but this capability should be mentioned in the comments field. + main: true + - id: encbk + name: Backups encryption + type: boolean + description: | + Backups can be performed with user-supplied encryption keys. + - id: ptrbk + name: Point In Time Recovery + type: boolean + description: | + The operator provides support for the user to specify a recovery point (in the past) to which a backup should be recovered to + (if unspecified, backup will be recovered in full). + vendor_compliance: | + True response implies that at least time-based recovery is supported. + If the operator also supports PITR by xid and label, clarify in the comments field. + - id: mulbk + name: Multiple backup configurations + type: boolean + description: | + The operator supports managing more than one backup configuration. + This is typically used to store backups on different object stores (for protection purposes) or to send them to different sites. + It may also include different lifecycle policies. + - id: mgmi + name: Management Interfaces + features: + - id: cucol + name: Resource display columns + type: boolean + description: | + Operator-managed objects (CRDs) include specific fields that provide additional information about the Postgres instances. + These fields are shown when querying the Kubernetes resources using tools like `kubectl`. + vendor_compliance: | + In order to be compliant, operators have to provide at least the following information: + + * Replication status (primary/replica/etc) + * Cluster status (healthy/not healthy) + * Postgres major/minor version + * Latest base backup date + * SSL on/off + - id: kbplg + name: kubectl plugin + type: boolean + description: | + The users may download an additional kubectl plugin which provides custom commands to manage the operator. + vendor_compliance: | + If the feature is provided, include a link to the downloadable plugin in the links field. + - id: bagui + name: Basic management GUI + type: boolean + description: | + The operator bundles some GUI (web interface or other) that allows to perform basic operations (e.g. cluster creation) graphically. + main: true + - id: fugui + name: Fully-featured GUI + type: boolean + description: | + The operator bundles a GUI that is fully-featured (that is, in feature-parity with the capabilities provided via the other management interfaces -- typically CRDs). + The GUI may provide additional features that cannot be otherwise provided via text interfaces (e.g. graphical representation of resources or operations). + main: true + - id: urgui + name: GUI users & roles + type: boolean + description: | + The bundled management GUI has authentication and authorization baked in and provides mechanisms to support multiple users and authorize/deny them to perform operations. + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + Submission should detail in the link and/or comments field how users and roles for the GUI can be managed. + - id: guiss + name: GUI Single-Sign On + type: boolean + description: | + The bundled management GUI supports integration with Single-Sign On (SSO), like OIDC, identity providers (e.g. Google, Github or Twitter login). + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + - id: cogui + name: GUI database console + type: boolean + description: | + The bundled management GUI includes a console that can connect to any database managed by the operator and send command (e.g. like `psql`). + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + - id: o11y + name: Observability + features: + - id: mtech + name: Metrics technology + type: string_array + description: | + If the operator supports extracting metrics from Postgres, define how they are handled, which technology receives and processes them. + + If any of the following technology names apply, use them, and add other names, as needed: + + * Prometheus + * OTEL + * OpenTSDB + * Nagios + * Sensu + vendor_compliance: | + If supported, provide in the comments information about whether the technology is a dependency, is built-in, external, etc. + Provide link(s) to the documentation for further information. + main: true + - id: expme + name: Export metrics + type: boolean + description: | + Regardless of how metrics are processed (e.g. as part of the operator), + this feature is implemented when the operator allows the user to configure sending metrics to external services, like a metrics SaaS. + - id: cudas + name: Custom dashboards + type: boolean + description: | + In order to display the captured Postgres metrics, the operator provides specialized Postgres dashboards for the users. + - id: cuale + name: Custom alerts + type: boolean + description: | + The operator provides bundled specific Postgres alerts to be triggered on the Postgres metrics processed. + E.g. there is an alert for transaction wraparound or for unused replication slots. + - id: exdel + name: Exposed decorated logs + type: boolean + description: | + The operator provides a mechanism to expose all the logs of the managed Postgres instances to a centralized logging tool. + The logs must be decorated with extra metadata in order to provide semantic meaning, including the Pod name and namespace, the cluster name, the role of the Postgres instance (e.g. primary, replica, standby-leader, etc.) and the timestamp that will be available to be used to filter logs entries. + There is no need to configure the tool in order to obtain required extra metadata from the logs. + - id: explg + name: Export logs + type: boolean + description: | + The operator allows the user to configure an external sink for the Postgres logs (e.g. a SaaS service). + - id: oo11y + name: Operator Observability + type: boolean + description: | + The operator is itself a source of telemetry data, potentially including metrics, traces and logs, about its own performance. + - id: secy + name: Security + features: + - id: opcsc + name: Operator code security scanning + type: boolean + description: | + The operator code is always scanned for security vulnerabilities. + main: true + - id: imgsc + name: Image scanning + type: boolean + description: | + Operator-provided images are always scanned for security vulnerabilities. + main: true + - id: sigim + name: Signed images + type: boolean + description: | + Container images are digitally signed according to the [sigstore](https://www.sigstore.dev/) project. + vendor_compliance: | + Compliance may also be achieved by using a technology other than sigstore, as long as it provides an equivalent set of security capabilities. + - id: isbom + name: Software Bill of Materials + type: boolean + description: | + The operator releases include the SBOM (Software Bill of Materials), a detailed description of all the components, modules, and their dependencies. + vendor_compliance: | + SBOM is expected to be in accordance to the [Kubernetes SIG BOM](https://github.com/kubernetes-sigs/bom). + - id: fgopp + name: Fine-grained RBAC permissions + type: boolean + description: | + The operator uses a separate serviceaccount that has RBAC permissions that only require the access that is actually needed to create and manage the Kubernetes resources, not more. + - id: noprm + name: No or justified privileged mode + type: boolean + description: | + The operator-provided containers do not require privileged mode. + The container processes do not run as root. + vendor_compliance: | + Reasonable exceptions to this rule can be made for features that require or do not diminish the container's security, e.g. when using eBPF. + - id: day2 + name: Day 2 Operations + features: + - id: amiup + name: Automated minor upgrades + type: boolean + description: | + The operator can perform a minor version upgrade of a Postgres cluster automatically. + This operation can be managed by the user declaratively. + vendor_compliance: | + The operator must provide proper information to the user as to the status and final result of the operation. + The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + Provide information about the update strategy (i.e. restart of the pods or rolling update followed by a switchover or a restart). + main: true + - id: amaup + name: Automated major upgrades + type: boolean + description: | + The operator can perform a major version upgrade of a Postgres cluster automatically. + This operation can be managed by the user declaratively. + vendor_compliance: | + The operator must provide proper information to the user as to the status and final result of the operation. + The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + main: true + - id: crest + name: Controlled cluster restart + type: boolean + description: | + Sometimes Postgres needs to be restarted (e.g. changing of a parameter that requires restart). + The operator provides means to perform this operation automatically and in a controlled manner (rolling restart) so that the cluster faces a minimal downtime only. + - id: ociup + name: Container images upgrade + type: boolean + description: | + Similarly to the controlled restart operation, the operator is capable of updating the running container images (which require a pod restart) automatically and with minimal cluster impact. + - id: swtch + name: Switchover + type: boolean + description: | + If HA capabilities are provided, the operator also provides a mechanism for manual switchover. + The user may specify the configuration declaratively and the operator will perform the desired switchover automatically, by demoting the current primary, promoting the a replica, and updating the endpoints/services as required. + - id: sqlmi + name: SQL Migrations + type: boolean + description: | + The operator provides managed SQL migration capabilities. + The user may specify SQL scripts that contain migrations (DDL changes, etc) to be deployed to a given database, having the operator apply them automatically. + vendor_compliance: | + The operator must report back to the user detailed information about the results of the execution(s) of the script(s) provided by the user. + - id: oday2 + name: Other Day 2 Operations + type: string_array + description: | + The operator provides support for other managed Day 2 operations. + vendor_compliance: | + All the mentioned additional day 2 operations need to be possible via declarative configuration and the operator to fully execute them without further user intervention. + - id: dain + name: Data Integration + features: + - id: kfkin + name: Kafka integration + type: boolean + description: | + The operator provides a managed solution for automation to export CDC (Change Data Capture) events to Kafka. + vendor_compliance: | + Kafka could be an external dependency to the operator. + The whole operation is expected to be fully automated, with the user only providing a declarative configuration. + - id: migpg + name: Migration from Postgres + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from another (external) Postgres database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`. + - id: migmy + name: Migration from MySQL + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a MySQL database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migor + name: Migration from Oracle + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from an Oracle database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migms + name: Migration from SQL Server + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a SQL Server database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migmo + name: Migration from MongoDB + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a MongoDB database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migot + name: Migration from other databases + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from other relational or non-relational databases. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide the names of the other database from where migration is supported. + - id: shrd + name: Sharding + features: + - id: mshrd + name: Managed sharding support + type: boolean + description: | + The operator understands horizontal scalability and is capable to automatically deploying sets of clusters (shards) with their accompanying infrastructure (coordinators, services, etc). + The user should be able to write a declarative configuration of a sharded cluster and the operator to fully deploy it without any further manual user intervention. + main: true + - id: shrdt + name: Sharding technology + type: string_array + description: | + If managed sharding support is provided, which technology or technologies are used to provide sharding support (which is not included in Postgres core). + - id: aggds + name: Aggregated monitoring dashboards + type: boolean + description: | + Since a sharded cluster is a set of (independent) Postgres clusters, an aggregated dashboard is one that provides Observability information across all shards of the cluster. + E.g. total database size, tps processed or total number of client connections. + - id: cobkp + name: Coordinated backups + type: boolean + description: | + A sharded cluster can be backed up by backing up each shard independently, but those backups are not necessarily consistent with each other. + If sharding is used, an additional capability is to coordinate backups on a given timestamp so that they are performed or can be restored at the same logical timestamp, + so they become consistent with each other. diff --git a/apache-druid/spec/yaml2md/README.md b/apache-druid/spec/yaml2md/README.md new file mode 100644 index 0000000..b398359 --- /dev/null +++ b/apache-druid/spec/yaml2md/README.md @@ -0,0 +1,19 @@ +# README + +In order to export the markdown file containing the table version of the origin YAML, you must have [gomplate](https://github.com/hairyhenderson/gomplate) installed. + + +## Installing gomplate + +Information on how to install the required command line tool can be found here: [https://docs.gomplate.ca/installing/](https://docs.gomplate.ca/installing/) + +## Convert yaml to markdown table + +``` +sh yaml2md.sh +``` + +## Notes + +- Make sure that the file `feature_matrix.yaml` is stored on the parent folder before running the script. +- The html file `feature_matrix.md` will be created on the parent folder. diff --git a/apache-druid/spec/yaml2md/yaml2md.sh b/apache-druid/spec/yaml2md/yaml2md.sh new file mode 100755 index 0000000..9ac7df5 --- /dev/null +++ b/apache-druid/spec/yaml2md/yaml2md.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +> ../feature_matrix.md + +if ! command -v gomplate &> /dev/null +then + echo "The required tool \"gomplate\" could not be found. Please make sure to install it and try again." + exit +fi + +gomplate -d fm='../feature_matrix.yaml' -o '../feature_matrix.md' -f 'yaml2md.tmpl' \ No newline at end of file diff --git a/apache-druid/spec/yaml2md/yaml2md.tmpl b/apache-druid/spec/yaml2md/yaml2md.tmpl new file mode 100644 index 0000000..8b43fab --- /dev/null +++ b/apache-druid/spec/yaml2md/yaml2md.tmpl @@ -0,0 +1,7 @@ +{{ range (ds "fm").categories }} +## {{ .name }} – [{{ .id }}](#{{ .id }}) +| **ID** | **NAME** | **TYPE** | **DESCRIPTION** | **VENDOR COMPLIANCE** | **MAIN CATEGORY** | +|---|---|---|---|---|---| +{{ range .features }}| {{ .id }} | {{ .name }} | {{ .type }} | {{ if (has . "description") }}{{ .description | strings.Trim "\n" | strings.ReplaceAll "\n" "
" }}{{ end }} | {{ if (has . "vendor_compliance") }}{{ .vendor_compliance | strings.ReplaceAll "\n" "
" }}{{ end }} | {{ if (has . "main") }} ✓ {{ end }} | +{{ end }} +{{ end }} \ No newline at end of file diff --git a/apache-druid/vendor-submissions/README.md b/apache-druid/vendor-submissions/README.md new file mode 100644 index 0000000..a87b406 --- /dev/null +++ b/apache-druid/vendor-submissions/README.md @@ -0,0 +1,23 @@ +# Apache Druid OFM Vendor Submission + +This directory contains: + +* [vendor-submission-template.yaml](vendor-submission-template.yaml). This is the file that should be copied as a template to fill in the details of each vendor's submission. Note that this file should not be edited directly, as it is derived from the source [feature matrix](../spec/feature_matrix.yaml). +* [vendor-submission-template-generator](vendor-submission-template-generator). Script and helper file to generate the `vendor-submission-template.yaml` from the source feature matrix. + + +## Vendor Submission Guidelines + +To produce a vendor submission, please observe the following rules: +* Submissions are tied to a particular release of the vendor's operator. Vendors are expected to produce a new submission for every major release of their operator. If no significant change across operator releases is produced in terms of the features supported, submission may be skipped. The rendering of the vendor's OFM for end users will do a best effort to always show the "latest" version (if semantic versioning or similar versioning style is use) of a given vendor's submissions. It is however preferrable to always submit a version with each operator's release. It is suggested for vendors to treat the YAML file for the submission as another artifact of their release and add it to their standard release procedure. +* Create a pull request against this repository with the submission. The filename for the submission must comply with the following naming format: `Apache Druid/vendor-submissions/${vendor_name_lowercase}/${vendor_name_lowercase}-${ofm_version}-${operator_release_version}.yaml`, and be based on the `vendor-submission-template.yaml` of the given OFM version. +* Observe the indications in the `.vendor_compliance` field of the OFM to determine whether the operator's release is compliant (implements) or not a given feature. +* No bullshit, no marketing. Make objective assessments, and don't include marketing or subjective observations as part of the submission. + +The relevant fields of the submission template are: +* Comments. They are provided as indications for the vendor. They must be left as part of the submission, without removing the comment character. +* `.value`. It is the main value to provide for the feature, and it must be valid according to the `.type` field. Required. +* `.references`. A string array. Ideally, a single string, provide more than one only if strictly necessary. Each reference must be a Markdown-formatted URL. The text part of the URL should be a short or very short text and the location the address of a webpage or section of a webpage that shows or demonstrates the implementation of the given feature. To keep the text short (for readability purposes) either a short version of the title of the documentation's section may be used, or simple words like `Documentation` or `Doc`. This field is not required, but of high value to end users if provided. +* `.comments`. An optional free-form text field for the vendor to provide more detail information about the compliance/non compliance and possible justifications. Please observe the no bullshit and no marketing rules. Comments should be as concise and objective as possible. Markdown formatting may be used. +* Unimplemented features can be left empty; be removed from the submission; or be left unimplemented (setting the `.value` field to `false` for boolean values or empty for every other type) and providing further information in the `.comments` field about why it is not (or partially) implemented. Empty (null) value is therefore treated as "unimplemented". + diff --git a/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.sh b/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.sh new file mode 100644 index 0000000..7c250e1 --- /dev/null +++ b/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +> vendor-submission-template.yaml + +if ! command -v gomplate &> /dev/null +then + echo "The required tool \"gomplate\" could not be found. Please make sure to install it and try again." + exit +fi + +gomplate -d fm='../../spec/feature_matrix.yaml' -o '../vendor-submission-template.yaml' -f 'vendor-submission-template-generator.tmpl' diff --git a/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.tmpl b/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.tmpl new file mode 100644 index 0000000..7f562fc --- /dev/null +++ b/apache-druid/vendor-submissions/vendor-submission-template-generator/vendor-submission-template-generator.tmpl @@ -0,0 +1,23 @@ +--- +metadata: + featureMatrixSpecVersion: + submission: + version: + datePublished: + releaseNotes: + operator: + name: + url: +featureMatrix: + categories:{{ range (ds "fm").categories }} + - id: {{ .id }} + # name: {{ .name }}{{ range .features }} + - id: {{ .id }} + # name: {{ .name }} + # type: {{ .type }} + # description: | + # {{ .description | strings.ReplaceAll "\n" "" }}{{ if (has . "vendor_compliance") }}{{ "\n " }}# vendor_compliance: |{{ "\n " }}# {{ .vendor_compliance | strings.TrimSuffix "\n" | strings.ReplaceAll "\n" "\n # " }}{{ end }} + value: + references: + comments:{{ end }} +{{ end }} \ No newline at end of file diff --git a/apache-druid/vendor-submissions/vendor-submission-template.yaml b/apache-druid/vendor-submissions/vendor-submission-template.yaml new file mode 100644 index 0000000..9930811 --- /dev/null +++ b/apache-druid/vendor-submissions/vendor-submission-template.yaml @@ -0,0 +1,1046 @@ +--- +metadata: + featureMatrixSpecVersion: + submission: + version: + datePublished: + releaseNotes: + operator: + name: + url: +featureMatrix: + categories: + - id: genc + # name: General Characteristics + - id: instm + # name: Installation mechanism + # type: string_array + # description: | + # Mechanism(s) provided to install the operator. + # vendor_compliance: | + # Provide a succinct value or list of values of the mechanisms supported. + # If any of the following names apply, use them, and add other mechanism names, as needed: + # + # * YAML + # * Helm + # * Kustomize + # * Operator Bundle + value: + references: + comments: + - id: mcrds + # name: Management via CRDs + # type: boolean + # description: | + # The operator can be managed via CRDs (Custom Resource Definition). + # vendor_compliance: | + # If true, provide a list of the names of the CRDs supported by the operator. Names should be direct links to their documentation. + value: + references: + comments: + - id: offin + # name: Offline installation + # type: boolean + # description: | + # Whether it is possible to install the operator on a (air-gapped) cluster without internet connection. + # vendor_compliance: | + # A link should be provided with documentation on how to install the operator on air-gapped environments. + value: + references: + comments: + - id: scpua + # name: Supported CPU architectures + # type: string_array + # description: | + # Support for the operator to run on nodes with specified CPU architectures. + # vendor_compliance: | + # Provide a list of CPU architectures that are supported between: + # + # * amd64 + # * arm + # * arm64 + # * ppc64le + # * s390x + # + # In case you support additional architectures, add their name(s) in a similar fashion. + value: + references: + comments: + - id: coios + # name: Container images OSes + # type: string_array + # description: | + # Indicate the base image of the images used by the operator. (e.g: scratch, ubi, fedora, ubuntu, centos, alpine, etc.).Provide the image name without tags (e.g. debian instead of debian:8.11) + value: + references: + comments: + - id: olmcl + # name: Operator Capability Level + # type: string + # description: | + # [Operator Capability Levels](https://sdk.operatorframework.io/docs/overview/operator-capabilities/) indicates the operator maturitylevels in regards to their lifecycle management capabilities for the application or workload they deliver. The capability modelsaims to provide guidance in terminology to express what features users can expect from an operator. + # vendor_compliance: | + # Capability level must be set one of the following levels: + # + # 1. `Basic Install` + # 2. `Seamless Upgrades` + # 3. `Full Lifecycle` + # 4. `Deep Insights` + # 5. `Auto Pilot` + # + # A link should be provided with documentation on how the operator fulfil the declared capability level. + # + # For a detailed description of each capability level refer to the Operator Framework documentation for + # [Operator Capability Level](https://sdk.operatorframework.io/docs/overview/operator-capabilities/). + value: + references: + comments: + + - id: vers + # name: Versions + - id: pgver + # name: Supported PostgreSQL versions + # type: string_array + # description: | + # Which major (and minor) versions does this version of the operator support. + # vendor_compliance: | + # Provide a list of every major version supported followed by a list in parenthesis of the minor versions supported for that major version. + # E.g. '15 (15.1, 15.0), 14 (14.3, 14.2)'. All versions must be ordered by reverse chronological order. + # Other components versions that are used may be described in the comments (e.g: Patroni, WAL-G, PgBackRest, PgBouncer, etc.) + value: + references: + comments: + - id: k8ver + # name: Supported Kubernetes versions + # type: string_array + # description: | + # Which Kubernetes versions the operator can be installed into. + # vendor_compliance: | + # Provide a list of official Kubernetes minor versions numbers, ordered by reverse chronological order. + # Use a range of versions, if possible, e.g. '1.26 - 1.23' or '1.22 - 1.19'. + value: + references: + comments: + - id: pgfor + # name: PostgreSQL OSS forks + # type: string_array + # description: | + # PostgreSQL forks are considered derived projects that started from PostgreSQL codebase and add different features while maintaining compatibility with PostgreSQL.Some operators may provide support for some of such forks, alongside with the original PostgreSQL. + # vendor_compliance: | + # Leave empty if no version other than the original PostgreSQL is supported. Otherwise, provide a list of names and links to the PostgreSQL forks supported. + # If non-OSS forks are supported, leave empty but add a link to them in the comments section. + value: + references: + comments: + + - id: lisu + # name: Licensing & Support + - id: ossli + # name: Open Source license + # type: boolean + # description: | + # The operator is published, and its source code is made publicly available, under an OSI-approved open source license. + # vendor_compliance: | + # If true, provide the SPDX identifier(s) of the license. + value: + references: + comments: + - id: fcimg + # name: Free container images + # type: boolean + # description: | + # The container images used for the operator's operation are free to distribute and use, not subject to commercial restrictions. + # vendor_compliance: | + # If false, provide a link with documentation on the usage requisites/restrictions of the container images. + value: + references: + comments: + - id: nenof + # name: No enterprise-only features + # type: boolean + # description: | + # All the operator's features are provided as open source.No derived, proprietary (often called 'enterprise') versions exist with additional, non open source features. + # vendor_compliance: | + # If false, provide links to all known derived, proprietary versions. + # Linked documentation should clearly reflect the features that are only part of proprietary versions. + value: + references: + comments: + - id: comms + # name: Commercial support + # type: boolean + # description: | + # Vendor and/or other entities offer paid-for, commercial support for the operator. + # vendor_compliance: | + # Provide a link or list of links to the entities that provide commercial support. + # Links should be as specific as possible, pointing to informative documentation on the provided support. + value: + references: + comments: + - id: pubit + # name: Public issue tracker + # type: boolean + # description: | + # Development happens in the open. There is a public issue tracker where users may view, comment and create issues. + # vendor_compliance: | + # Provide a specific link to the issue tracker. + value: + references: + comments: + - id: pubch + # name: Public chat/forums + # type: boolean + # description: | + # There is/are public forums, mailing lists, chat groups (Slack, Discord, Matrix, etc) where users may freely join and participate in discussions. + # vendor_compliance: | + # Provide links to all relevant user forums. + value: + references: + comments: + + - id: pgcl + # name: PostgreSQL Clusters + - id: depau + # name: Deployment automation + # type: boolean + # description: | + # The operator provides capabilities to deploy production-ready clusters automatically based on a provided configuration.No user initiated commands must be required. + value: + references: + comments: + - id: pomte + # name: Pod management technology + # type: string + # description: | + # Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. + # vendor_compliance: | + # Provide the most succinct possible name of the technology. Use the Kubernetes resource type name (e.g. 'Deployment'), if applicable. + value: + references: + comments: + - id: pgcnf + # name: PostgreSQL custom config + # type: boolean + # description: | + # The operator allows the user to supply custom PostgreSQL configuration. + value: + references: + comments: + - id: conpl + # name: Integrated connection pooling + # type: boolean + # description: | + # The operator provides options to deploy a connection pool in front of the database, automatically deployed and configured. + # vendor_compliance: | + # The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc. + # All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. + value: + references: + comments: + - id: cpccf + # name: Connection pool custom config + # type: boolean + # description: | + # The operator allows the user to supply a custom connection pool configuration for the connection pool service. + # vendor_compliance: | + # Only applies if [pgcl/conpl] is true. + value: + references: + comments: + - id: tlssp + # name: TLS Support + # type: boolean + # description: | + # PostgreSQL connections can be secured with Postgres SSL/TLS support. + value: + references: + comments: + - id: tlscu + # name: TLS user-provided certificates + # type: boolean + # description: | + # Operators may chose by default to generate self-signed SSL certificates.They may also offer the option to specify the CA and certificates that users want Postgres clusters to use. + value: + references: + comments: + - id: crtmg + # name: CertManager integration + # type: boolean + # description: | + # The operator integrates with CertManager in order to generate the certificate to be used with Postgres. + # vendor_compliance: | + # Only applies if [pgcl/tlscu] is true. + value: + references: + comments: + - id: insql + # name: Initialization from SQL scripts + # type: boolean + # description: | + # After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation.The operator must properly inform the user of the execution result of the scripts. + value: + references: + comments: + - id: inext + # name: Initialization from external source + # type: boolean + # description: | + # After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data.The operator must properly inform the user of the execution result of the scripts. + value: + references: + comments: + - id: cuhba + # name: HBA custom config + # type: boolean + # description: | + # The operator allows users to provide custom configuration for the Postgres HBA (Host-Based Authentication) mechanism.It may involve directly generating the custom `pg_hba.conf` file or by using more cloud-native technologies. + value: + references: + comments: + - id: mgmup + # name: Management of db users/passwords + # type: boolean + # description: | + # The operator provides a mechanism to declaratively create, modify or delete users, roles and/or password for the PostgreSQL cluster. + value: + references: + comments: + - id: reqli + # name: Customize computing resource requests, limits + # type: boolean + # description: | + # The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the Postgres pods. + value: + references: + comments: + - id: hugpa + # name: Support for huge pages + # type: boolean + # description: | + # The user may specify the request to use huge pages for Postgres (and/or potentially other sidecars).Postgres must be able to be configured and start using huge pages. + value: + references: + comments: + - id: pgsrv + # name: PostgreSQL exposed via Service + # type: string + # description: | + # The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the PostgreSQL connections.Expected capabilities should include a RW (read-write) or RO (read-only in case of cascading replication) connection to the primary instance; and, optionally, a RO (read-only) to load balance read-only replicas instances in the cluster. + # vendor_compliance: | + # The value must be one of: + # + # * PrimaryAndReplicas: Primary and replicas services must be offered. + # * PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered. + # * Balanced: A single service that load balances RW/RO traffic transparently. + # * Primary: Primary service must be offered. + value: + references: + comments: + - id: stosc + # name: Automatic storage scaling + # type: boolean + # description: | + # If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically(either by setting some default thresholds or requiring explicit declarative configuration from the user). + value: + references: + comments: + - id: scal0 + # name: Scale down to zero + # type: boolean + # description: | + # The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed.Upon scaling up, the cluster should be brought back up without the need to restore a backup. + value: + references: + comments: + - id: tblsp + # name: Tablespaces + # type: boolean + # description: | + # The user may specify one or more PostgreSQL tablespaces and their associated backing storage. + value: + references: + comments: + - id: cupgi + # name: Custom Postgres images + # type: boolean + # description: | + # The operator allows the user to specify custom (user-provided) Postgres container images, instead of using the operator's provided images. + # vendor_compliance: | + # The operator should specify if the custom image needs to follow some minimal patterns to be able to work, or if it can work + # with any postgres container image provided. + value: + references: + comments: + - id: uside + # name: User supplied sidecars + # type: boolean + # description: | + # The user may specify custom sidecars (containers or init containers) to be created alongside the Postgres container (and, possibly, other operator sidecars).User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. + value: + references: + comments: + - id: usvol + # name: User supplied volumes + # type: boolean + # description: | + # The user may specify and mount custom volumes to be created alongside the volumes created for the Postgres container. + value: + references: + comments: + - id: usprt + # name: User supplied ports + # type: boolean + # description: | + # The user may specify custom ports to be created alongside the services that expose PostgreSQL. + # vendor_compliance: | + # Only applies if [pgcl/pgsrv] is set. + value: + references: + comments: + - id: srvbi + # name: Support for Service Binding + # type: boolean + # description: | + # [Service Binding](https://servicebinding.io/spec/core/1.0.0/) is a Kubernetes specification to "communicating service secrets to workloads in a consistent way".If the operator supports Service Binding, applications that are bound to the Postgres clusters will be able to fetch database access credentials without requiringthe user to specify them manually in the application. + # vendor_compliance: | + # The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification. + value: + references: + comments: + - id: cuann + # name: Custom Annotations + # type: boolean + # description: | + # The operator allows the user to provide custom [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/)(aside from any operator generated Annotations) to the objects generated by the operator. + # vendor_compliance: | + # The operator must provide configuration for setting Annotations on all objects generated by the operator. + # Partial compliance is achieved if at least Pods can be annotated and this is clearly marked in the comments. + value: + references: + comments: + - id: culab + # name: Custom Labels + # type: boolean + # description: | + # The operator allows the user to provide custom [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the objects generated by the operator. + # vendor_compliance: | + # The operator must provide configuration for setting Labels on all objects generated by the operator. + # Partial compliance is achieved if at Labels can be added to at least Pods and this is clearly marked in the comments. + value: + references: + comments: + - id: apdnd + # name: Assign Pod to Node + # type: string + # description: | + # The operator allows Postgres Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. + # vendor_compliance: | + # Indicate one of the following values: + # + # * 'Complete': if the operator support all the rules to assign a Pod to a Node defined in the latest Kubernetes version specified under [vers/k8ver]. + # * 'Partial': if the operator support only partially the rules to assign a Pod to a Node. + # + # A link for each supported rule or rules group to assign a Pod to a Node must be specified. + value: + references: + comments: + - id: topsc + # name: Topology Spread Constraints + # type: boolean + # description: | + # The operator allows Postgres pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints),to control how Pods are spread across the Kubernetes cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. + value: + references: + comments: + + - id: harp + # name: HA & Replication + - id: autfa + # name: Automated failover + # type: boolean + # description: | + # The operator provides the facility that in the event of a node or pod failure where the Postgres primary is affected,another Postgres pod (if it exists) will be promoted to primary. The operation must happen automatically without user intervention. + value: + references: + comments: + - id: techa + # name: HA Technology + # type: string + # description: | + # The technology (software name or technology principles) that support the high availability and automated failover capabilities of the operator. + # vendor_compliance: | + # Provide a succinct software or technology name. E.g.: Patroni, Stolon, Custom. + value: + references: + comments: + - id: asrep + # name: Asynchronous replication + # type: boolean + # description: | + # The operator allows to configure Postgres clusters using asynchronous streaming replication.Asynchronous is the default streaming replication mode in Postgres. + value: + references: + comments: + - id: syrep + # name: Synchronous replication + # type: boolean + # description: | + # The operator allows to configure Postgres clusters using synchronous streaming replication.The operator must manage the configuration details based on the user's preferences on which nodes behave synchronously. + value: + references: + comments: + - id: serep + # name: Semi-synchronous replication + # type: boolean + # description: | + # Semi-synchronous, or also called group or quorum replication, is a hybrid mode where a subset of the nodes is expected to replicate synchronously;and once that level is reached, the remainding nodes replicate asynchronously.That is, Postgres waits for confirmation of the write only from the synchronous nodes before considering the transaction committed. + value: + references: + comments: + - id: derep + # name: Delayed replicas + # type: boolean + # description: | + # The operator allows the user to configure one or more replicas subject to a user-specified (intentional) replication lag.This is useful to have always an online instance with data "in the past" for analytics or data recovery purposes. + value: + references: + comments: + - id: carep + # name: Cascading replication + # type: boolean + # description: | + # Postgres allows to create replicas that are fed from another replica, instead of replicating from the primary instance.This is interesting to alleviate the effects of replication on the primary; and allows to create arbitrary replication tree-like topologies. + value: + references: + comments: + - id: exrep + # name: Replication from external origin + # type: boolean + # description: | + # The operator allows a Postgres cluster to be setup in recovery mode while replicating from an external (non-operator managed) Postgres origin.This allows migrations to the operator using streaming replication. + # vendor_compliance: | + # To implement the feature, the user should be able to provide arbitrary host, port, username and password (or other means of credential passing) of the origin. + value: + references: + comments: + - id: walsh + # name: WAL Shipping + # type: boolean + # description: | + # WAL Shipping is a Postgres technique that allows replication via a shared storage mechanism where WAL files produced on the primary node are copied ("shipped") to the replica.This feature is useful for having replicas for example on DR sites, replicate over global storage object stores and others. + # vendor_compliance: | + # Implementing the feature implies both generating WAL files to a suitable location; as well as creating clusters that permanently read those WAL files and replicate from them. + value: + references: + comments: + - id: prstb + # name: Promote standby cluster + # type: boolean + # description: | + # The operator allows the user to promote a standby cluster that is a cluster with primary in recovery mode so that the primary becomes RW. This feature is used in conjunction with [harp/exrep] and/or [harp/walsh] to make GEO-replicated and/or DR (Disaster Recovery) clusters on other regions, where a region is essentially an independent Kubernetes cluster. + # vendor_compliance: | + # Only applies if [harp/exrep] or [harp/walsh] are true. + # The operator must allow to promote a standby clusters replicating either via WAL Shipping and/or streaming replication. + value: + references: + comments: + - id: lorep + # name: Managed logical replication + # type: boolean + # description: | + # Postgres supports logical replication as well as streaming replication. It requires some configuration and commands to be run by the user.This feature represents operator capabilities to perform these operations in a managed way, without the user having to type commands or create configurations directly. + # vendor_compliance: | + # The operator is considered compliant if it allows the user to provide logical replication without having to setup it directly, just specifying (declarative) preferences. + # The operator is still considered compliant if it cannot proceed until certain operations are performed by the users that are dependent on their environment + # (e.g. adding PKs, UNIQUEs or REPLICA IDENTITY to the DDL). + value: + references: + comments: + + - id: pgex + # name: PostgreSQL Extensions + - id: extme + # name: Extensions distribution mechanism + # type: string + # description: | + # This feature is set to describe how extensions are shipped in a containerized environment.Typically they are part of the same Postgres container image, but they may also be distributed via other mechanisms. + # vendor_compliance: | + # Indicate 'built-in' if the extensions come included with the same Postgres container; or a succinct word or few words naming the distribution mechanism. + # More details may be provided, if needed, via the links and comments fields. + value: + references: + comments: + - id: coext + # name: Core/contrib extensions + # type: integer + # description: | + # The number of core+contrib extensions provided by the operator for the latest Postgres version provided by the operator. + # vendor_compliance: | + # Submission must provide the total number of core+contrib extensions supported for the latest Postgres version provided by the operator. + # It should also be provided, when available, a link with a detailed list of those extensions supported (via links field). + value: + references: + comments: + - id: thext + # name: Third-party extensions + # type: integer + # description: | + # The number of extensions not included in the Postgres core+contrib (i.e. created by third parties, outside of Postgres repository) for the latest Postgres version provided by the operator. + # vendor_compliance: | + # Submission must provide the total number of third-party extensions supported for the latest Postgres version provided by the operator. + # A link with a detailed list of those extensions supported (via links field) should also be provided, if available. + value: + references: + comments: + - id: byext + # name: Bring your own extension + # type: boolean + # description: | + # The opertor provides mechanisms for users to add (or upload) third-party extensions not initially provided by the operator. + # vendor_compliance: | + # The operator must support providing some mechanism to include custom extensions additionally or independently of the extension distribution mechanism, + # if a user build a local extension it could be uploaded or included on the postgres container with an automatic copy to all replicas related. + value: + references: + comments: + + - id: bkup + # name: Backups + - id: bktch + # name: Backup Technology + # type: string + # description: | + # What technology (pgbasebackup, PgBackRest, WAL-e/g, Barman, custom, etc) is used to support creation and restoration of backups. + # vendor_compliance: | + # Provide the most succinct possible name of the technology. + value: + references: + comments: + - id: bkdst + # name: Backup destinations + # type: string_array + # description: | + # Where backups can be stored (typically this may include object stores, PVs, etc). + # vendor_compliance: | + # If any of the following technology names apply, use them, and add other names, as needed: + # + # * PersistentVolume + # * NFS + # * S3 + # * GCS + # * Azure Blob + # * Local file system + value: + references: + comments: + - id: autbk + # name: Automated backup management + # type: boolean + # description: | + # The operator provides mechanisms for performing backups automatically and providing lifecycle mechanisms (delete old backups according to a user-supplied policy). + # vendor_compliance: | + # True response for the feature implies both automatic backups and lifecycle management. + # If only the former is provided, answer should be false but this capability should be mentioned in the comments field. + value: + references: + comments: + - id: encbk + # name: Backups encryption + # type: boolean + # description: | + # Backups can be performed with user-supplied encryption keys. + value: + references: + comments: + - id: ptrbk + # name: Point In Time Recovery + # type: boolean + # description: | + # The operator provides support for the user to specify a recovery point (in the past) to which a backup should be recovered to(if unspecified, backup will be recovered in full). + # vendor_compliance: | + # True response implies that at least time-based recovery is supported. + # If the operator also supports PITR by xid and label, clarify in the comments field. + value: + references: + comments: + - id: mulbk + # name: Multiple backup configurations + # type: boolean + # description: | + # The operator supports managing more than one backup configuration.This is typically used to store backups on different object stores (for protection purposes) or to send them to different sites.It may also include different lifecycle policies. + value: + references: + comments: + + - id: mgmi + # name: Management Interfaces + - id: cucol + # name: Resource display columns + # type: boolean + # description: | + # Operator-managed objects (CRDs) include specific fields that provide additional information about the Postgres instances.These fields are shown when querying the Kubernetes resources using tools like `kubectl`. + # vendor_compliance: | + # In order to be compliant, operators have to provide at least the following information: + # + # * Replication status (primary/replica/etc) + # * Cluster status (healthy/not healthy) + # * Postgres major/minor version + # * Latest base backup date + # * SSL on/off + value: + references: + comments: + - id: kbplg + # name: kubectl plugin + # type: boolean + # description: | + # The users may download an additional kubectl plugin which provides custom commands to manage the operator. + # vendor_compliance: | + # If the feature is provided, include a link to the downloadable plugin in the links field. + value: + references: + comments: + - id: bagui + # name: Basic management GUI + # type: boolean + # description: | + # The operator bundles some GUI (web interface or other) that allows to perform basic operations (e.g. cluster creation) graphically. + value: + references: + comments: + - id: fugui + # name: Fully-featured GUI + # type: boolean + # description: | + # The operator bundles a GUI that is fully-featured (that is, in feature-parity with the capabilities provided via the other management interfaces -- typically CRDs).The GUI may provide additional features that cannot be otherwise provided via text interfaces (e.g. graphical representation of resources or operations). + value: + references: + comments: + - id: urgui + # name: GUI users & roles + # type: boolean + # description: | + # The bundled management GUI has authentication and authorization baked in and provides mechanisms to support multiple users and authorize/deny them to perform operations. + # vendor_compliance: | + # Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + # Submission should detail in the link and/or comments field how users and roles for the GUI can be managed. + value: + references: + comments: + - id: guiss + # name: GUI Single-Sign On + # type: boolean + # description: | + # The bundled management GUI supports integration with Single-Sign On (SSO), like OIDC, identity providers (e.g. Google, Github or Twitter login). + # vendor_compliance: | + # Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + value: + references: + comments: + - id: cogui + # name: GUI database console + # type: boolean + # description: | + # The bundled management GUI includes a console that can connect to any database managed by the operator and send command (e.g. like `psql`). + # vendor_compliance: | + # Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + value: + references: + comments: + + - id: o11y + # name: Observability + - id: mtech + # name: Metrics technology + # type: string_array + # description: | + # If the operator supports extracting metrics from Postgres, how are they handled, which technology receives and process them?If any of the following technology names apply, use them, and add other names, as needed:* Prometheus* OpenTSDB* Nagios* Sensu + # vendor_compliance: | + # If supported, provide in the comments information about whether the technology is a dependency, is built-in, external, etc. + # Provide link(s) to the documentation for further information. + value: + references: + comments: + - id: expme + # name: Export metrics + # type: boolean + # description: | + # Regardless of how metrics are processed (e.g. as part of the operator),this feature is implemented when the operator allows the user to configure sending metrics to external services, like a metrics SaaS. + value: + references: + comments: + - id: cudas + # name: Custom dashboards + # type: boolean + # description: | + # In order to display the captured Postgres metrics, the operator provides specialized Postgres dashboards for the users. + value: + references: + comments: + - id: cuale + # name: Custom alerts + # type: boolean + # description: | + # The operator provides bundled specific Postgres alerts to be triggered on the Postgres metrics processed.E.g. there is an alert for transaction wraparound or for unused replication slots. + value: + references: + comments: + - id: exdel + # name: Exposed decorated logs + # type: boolean + # description: | + # The operator provides a mechanism to expose all the logs of the managed Postgres instances to a centralized logging tool.The logs must be decorated with extra metadata that includes the Pod name and namespace, the cluster name, the role of the Postgres instance (e.g. primary, replica, standby-leader, etc.) and the timestamp that will be available to be used to filter logs entries.There is no need to configure the tool in order to obtain required extra metadata from the logs. + value: + references: + comments: + - id: explg + # name: Export logs + # type: boolean + # description: | + # The operator allows the user to configure an external sink for the Postgres logs (e.g. a SaaS service). + value: + references: + comments: + - id: oo11y + # name: Operator Observability + # type: boolean + # description: | + # The operator is itself a source of telemetry data, potentially including metrics, traces and logs, about its own performance. + value: + references: + comments: + + - id: secy + # name: Security + - id: opcsc + # name: Operator code security scanning + # type: boolean + # description: | + # The operator code is always scanned for security vulnerabilities. + value: + references: + comments: + - id: imgsc + # name: Image scanning + # type: boolean + # description: | + # Operator-provided images are always scanned for security vulnerabilities. + value: + references: + comments: + - id: sigim + # name: Signed images + # type: boolean + # description: | + # Container images are digitally signed according to the [sigstore](https://www.sigstore.dev/) project. + # vendor_compliance: | + # Compliance may also be achieved by using a technology other than sigstore, as long as it provides an equivalent set of security capabilities. + value: + references: + comments: + - id: isbom + # name: Software Bill of Materials + # type: boolean + # description: | + # The operator releases include the SBOM (Software Bill of Materials), a detailed description of all the components, modules, and their dependencies. + # vendor_compliance: | + # SBOM is expected to be in accordance to the [Kubernetes SIG BOM](https://github.com/kubernetes-sigs/bom). + value: + references: + comments: + - id: fgopp + # name: Fine-grained RBAC permissions + # type: boolean + # description: | + # The operator uses a separate serviceaccount that has RBAC permissions that only require the access that is actually needed to create and manage the Kubernetes resources, not more. + value: + references: + comments: + - id: noprm + # name: No or justified privileged mode + # type: boolean + # description: | + # The operator-provided containers do not require privileged mode.The container processes do not run as root. + # vendor_compliance: | + # Reasonable exceptions to this rule can be made for features that require or do not diminish the container's security, e.g. when using eBPF. + value: + references: + comments: + + - id: day2 + # name: Day 2 Operations + - id: amiup + # name: Automated minor upgrades + # type: boolean + # description: | + # The operator can perform a minor version upgrade of a Postgres cluster automatically.This operation can be managed by the user declaratively. + # vendor_compliance: | + # The operator must provide proper information to the user as to the status and final result of the operation. + # The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + value: + references: + comments: + - id: amaup + # name: Automated major upgrades + # type: boolean + # description: | + # The operator can perform a major version upgrade of a Postgres cluster automatically.This operation can be managed by the user declaratively. + # vendor_compliance: | + # The operator must provide proper information to the user as to the status and final result of the operation. + # The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + value: + references: + comments: + - id: crest + # name: Controlled cluster restart + # type: boolean + # description: | + # Sometimes Postgres needs to be restarted (e.g. changing of a parameter that requires restart).The operator provides means to perform this operation automatically and in a controlled manner (rolling restart) so that the cluster faces a minimal downtime only. + value: + references: + comments: + - id: ociup + # name: Container images upgrade + # type: boolean + # description: | + # Similarly to the controlled restart operation, the operator is capable of updating the running container images (which require a pod restart) automatically and with minimal cluster impact. + value: + references: + comments: + - id: swtch + # name: Switchover + # type: boolean + # description: | + # If HA capabilities are provided, the operator also provides a mechanism for manual switchover.The user may specify the configuration declaratively and the operator will perform the desired switchover automatically, updating the endpoints/services as required. + value: + references: + comments: + - id: sqlmi + # name: SQL Migrations + # type: boolean + # description: | + # The operator provides managed SQL migration capabilities.The user may specify SQL scripts that contain migrations (DDL changes, etc) to be deployed to a given database, having the operator apply them automatically. + # vendor_compliance: | + # The operator must report back to the user detailed information about the results of the execution(s) of the script(s) provided by the user. + value: + references: + comments: + - id: oday2 + # name: Other Day 2 Operations + # type: string_array + # description: | + # The operator provides support for other managed Day 2 operations. + # vendor_compliance: | + # All the mentioned additional day 2 operations need to be possible via declarative configuration and the operator to fully execute them without further user intervention. + value: + references: + comments: + + - id: dain + # name: Data Integration + - id: kfkin + # name: Kafka integration + # type: boolean + # description: | + # The operator provides a managed solution for automation to export CDC (Change Data Capture) events to Kafka. + # vendor_compliance: | + # Kafka could be an external dependency to the operator. + # The whole operation is expected to be fully automated, with the user only providing a declarative configuration. + value: + references: + comments: + - id: migpg + # name: Migration from Postgres + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from another (external) Postgres database.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations. + # vendor_compliance: | + # Provide one or more of the following values: `offline`, `cdc`. + value: + references: + comments: + - id: migmy + # name: Migration from MySQL + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from a MySQL database.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations.Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + # vendor_compliance: | + # Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + value: + references: + comments: + - id: migor + # name: Migration from Oracle + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from an Oracle database.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations.Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + # vendor_compliance: | + # Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + value: + references: + comments: + - id: migms + # name: Migration from SQL Server + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from a SQL Server database.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations.Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + # vendor_compliance: | + # Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + value: + references: + comments: + - id: migmo + # name: Migration from MongoDB + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from a MongoDB database.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations.Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + # vendor_compliance: | + # Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + value: + references: + comments: + - id: migot + # name: Migration from other databases + # type: string_array + # description: | + # The operator provides a managed capability to automatically migrate data from other relational or non-relational databases.Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture),which allows for near-zero downtime migrations.Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + # vendor_compliance: | + # Provide the names of the other database from where migration is supported. + value: + references: + comments: + + - id: shrd + # name: Sharding + - id: mshrd + # name: Managed sharding support + # type: boolean + # description: | + # The operator understands horizontal scalability and is capable to automatically deploying sets of clusters (shards) with their accompanying infrastructure (coordinators, services, etc).The user should be able to write a declarative configuration of a sharded cluster and the operator to fully deploy it without any further manual user intervention. + value: + references: + comments: + - id: shrdt + # name: Sharding technology + # type: string_array + # description: | + # If managed sharding support is provided, which technology or technologies are used to provide sharding support (which is not included in Postgres core). + value: + references: + comments: + - id: aggds + # name: Aggregated monitoring dashboards + # type: boolean + # description: | + # Since a sharded cluster is a set of (independent) Postgres clusters, an aggregated dashboard is one that provides Observability information across all shards of the cluster.E.g. total database size, tps processed or total number of client connections. + value: + references: + comments: + - id: cobkp + # name: Coordinated backups + # type: boolean + # description: | + # A sharded cluster can be backed up by backing up each shard independently, but those backups are not necessarily consistent with each other.If sharding is used, an additional capability is to coordinate backups on a given timestamp so that they are performed or can be restored at the same logical timestamp,so they become consistent with each other. + value: + references: + comments: From c7314e06377008babceb8b1b85cdbe4570755d43 Mon Sep 17 00:00:00 2001 From: "itamar.marom" Date: Mon, 15 May 2023 21:00:30 +0300 Subject: [PATCH 2/5] fixed yaml and filled more fields --- apache-druid/spec/feature_matrix.bck.yaml | 807 ++++++++++++++++++ apache-druid/spec/feature_matrix.yaml | 164 ++-- .../vendor-submission-template.yaml | 6 +- 3 files changed, 914 insertions(+), 63 deletions(-) create mode 100644 apache-druid/spec/feature_matrix.bck.yaml diff --git a/apache-druid/spec/feature_matrix.bck.yaml b/apache-druid/spec/feature_matrix.bck.yaml new file mode 100644 index 0000000..496f98d --- /dev/null +++ b/apache-druid/spec/feature_matrix.bck.yaml @@ -0,0 +1,807 @@ +--- +metadata: + version: v1.1.0 + date: 26.04.2023 +categories: + - id: genc + name: General Characteristics + features: + - id: instm + name: Installation mechanism + type: string_array + description: | + Mechanism(s) provided to install the operator. + vendor_compliance: | + * Helm + * Kustomize + main: true + - id: mcrds + name: Management via CRDs + type: boolean + description: | + The operator manages CRDs (Custom Resource Definition). + vendor_compliance: | + * druid.apache.org/v1alpha1/Druid - https://github.com/datainfrahq/druid-operator/blob/v1.1.0/apis/druid/v1alpha1/druid_types.go + main: true + - id: offin + name: Offline installation + type: boolean + description: | + Whether it is possible to install the operator on a (air-gapped) cluster without internet connection. + vendor_compliance: | + Druid operator can be installed in air-gapped environment the same as with any other type of environment. + Installation documentation - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + - id: scpua + name: Supported CPU architectures + type: string_array + description: | + Support for the operator to run on nodes with specified CPU architectures. + vendor_compliance: | + * amd64 + - id: coios + name: Container images OSes + type: string_array + description: | + Druid's base image is depends on the JDK version required: + gcr.io/distroless/java$JDK_VERSION-debian11 + - id: olmcl + name: Operator Capability Level + type: string + description: | + [Operator Capability Levels](https://sdk.operatorframework.io/docs/overview/operator-capabilities/) indicates the operator maturity + levels in regards to their lifecycle management capabilities for the application or workload they deliver. The capability models + aims to provide guidance in terminology to express what features users can expect from an operator. + vendor_compliance: | + `Seamless Upgrades` + Druid's feature list is documented in the following [page](https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md) + - id: vers + name: Versions + features: + - id: drver + name: Supported Apache Druid versions + type: string_array + description: | + Which major (and minor) versions does this version of the operator support. + vendor_compliance: | + 25 (25.0) + main: true + - id: k8ver + name: Supported Kubernetes versions + type: string_array + description: | + Which Kubernetes versions the operator can be installed into. + vendor_compliance: | + 1.26 - 1.25 + main: true + - id: lisu + name: Licensing & Support + features: + - id: ossli + name: Open Source license + type: boolean + description: | + The operator is published, and its source code is made publicly available, under an OSI-approved open source license. + vendor_compliance: | + true - Apache-2.0 + main: true + - id: fcimg + name: Free container images + type: boolean + description: | + The container images used for the operator's operation are free to distribute and use, not subject to commercial restrictions. + vendor_compliance: | + true + main: true + - id: nenof + name: No enterprise-only features + type: boolean + description: | + All the operator's features are provided as open source. + No derived, proprietary (often called 'enterprise') versions exist with additional, non open source features. + vendor_compliance: | + true + - id: comms + name: Commercial support + type: boolean + description: | + Vendor and/or other entities offer paid-for, commercial support for the operator. + vendor_compliance: | + false + - id: pubit + name: Public issue tracker + type: boolean + description: | + Development happens in the open. There is a public issue tracker where users may view, comment and create issues. + vendor_compliance: | + https://github.com/datainfrahq/druid-operator/issues + - id: pubch + name: Public chat/forums + type: boolean + description: | + There is/are public forums, mailing lists, chat groups (Slack, Discord, Matrix, etc) where users may freely join and participate in discussions. + vendor_compliance: | + druid-operator channel in Kubernetes Slack Workspace - https://kubernetes.slack.com/archives/C04F4M6HT2L + DataInfra slack workspace - https://launchpass.com/datainfra-workspace + - id: drcl + name: Apache Druid Clusters + features: + - id: depau + name: Deployment automation + type: boolean + description: | + The operator provides capabilities to automatically deploy production-ready clusters based on a provided configuration. + No user initiated commands must be required. + vendor_compliance: | + true + - id: pomte + name: Pod management technology + type: string + description: | + Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. + vendor_compliance: | + 'StatefulSet', 'Deployment'. + main: true + - id: drcnf + name: Apache Druid custom config + type: boolean + description: | + The operator allows the user to supply custom Druid configuration. + vendor_compliance: | + true + - id: conpl + name: Integrated connection pooling + type: boolean + description: | + The operator provides options to deploy a connection pool in front of the database, automatically deployed and configured. + vendor_compliance: | ??? + The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc. + All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. + main: true + - id: cpccf + name: Connection pool custom config + type: boolean + description: | ??? + The operator allows the user to supply a custom connection pool configuration for the connection pool service. + vendor_compliance: | + Only applies if [pgcl/conpl] is true. + - id: tlssp + name: TLS Support + type: boolean + description: | ??? + PostgreSQL connections can be secured with Postgres SSL/TLS support. + main: true + - id: tlscu + name: TLS user-provided certificates + type: boolean + description: | ??? + Operators may chose by default to generate self-signed SSL certificates. + They may also offer the option to specify the CA and certificates that users want Postgres clusters to use. + - id: crtmg + name: CertManager integration + type: boolean + description: | + The operator integrates with CertManager in order to generate the certificate to be used with Postgres. + vendor_compliance: | ??? + Only applies if [pgcl/tlscu] is true. + - id: insql + name: Initialization from SQL scripts + type: boolean + description: | ??? + After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation. + The operator must properly inform the user of the execution result of the scripts. + - id: inext + name: Initialization from external source + type: boolean + description: | + After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data. + The operator must properly inform the user of the execution result of the scripts. + - id: cuhba + name: HBA custom config + type: boolean + description: | + The operator allows users to provide custom configuration for the Postgres HBA (Host-Based Authentication) mechanism. + It may involve directly generating the custom `pg_hba.conf` file or by using more cloud-native technologies. + - id: mgmup + name: Management of db users/passwords + type: boolean + description: | + The operator provides a mechanism to declaratively create, modify or delete users, roles and/or password for the PostgreSQL cluster. + - id: reqli + name: Customize computing resource requests, limits + type: boolean + description: | + The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the Postgres pods. + main: true + - id: hugpa + name: Support for huge pages + type: boolean + description: | + The user may specify the request to use huge pages for Postgres (and/or potentially other sidecars). + Postgres must be able to be configured and start using huge pages. + main: true + - id: pgsrv + name: PostgreSQL exposed via Service + type: string + description: | + The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the PostgreSQL connections. + Expected capabilities should include a RW (read-write) or RO (read-only in case of cascading replication) connection to the primary instance; and, optionally, a RO (read-only) to load balance read-only replicas instances in the cluster. + vendor_compliance: | + The value must be one of: + + * PrimaryAndReplicas: Primary and replicas services must be offered. + * PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered. + * Balanced: A single service that load balances RW/RO traffic transparently. + * Primary: Primary service must be offered. + - id: stosc + name: Automatic storage scaling + type: boolean + description: | + If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically + (either by setting some default thresholds or requiring explicit declarative configuration from the user). + - id: scal0 + name: Scale down to zero + type: boolean + description: | + The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed. + Upon scaling up, the cluster should be brought back up without the need to restore a backup. This feature may also be known as "hibernation". + - id: tblsp + name: Tablespaces + type: boolean + description: | + The user may specify one or more PostgreSQL tablespaces and their associated backing storage. + - id: cupgi + name: Custom Postgres images + type: boolean + description: | + The operator allows the user to specify custom (user-provided) Postgres container images, instead of using the operator's provided images. + vendor_compliance: | + The operator should specify if the custom image needs to follow some minimal patterns to be able to work, or if it can work + with any postgres container image provided. + - id: uside + name: User supplied sidecars + type: boolean + description: | + The user may specify custom sidecars (containers or init containers) to be created alongside the Postgres container (and, possibly, other operator sidecars). + User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. + - id: usvol + name: User supplied volumes + type: boolean + description: | + The user may specify and mount custom volumes to be created alongside the volumes created for the Postgres container. + - id: usprt + name: User supplied ports + type: boolean + description: | + The user may specify custom ports to be created alongside the services that expose PostgreSQL. + vendor_compliance: | + Only applies if [pgcl/pgsrv] is set. + - id: srvbi + name: Support for Service Binding + type: boolean + description: | + [Service Binding](https://servicebinding.io/spec/core/1.0.0/) is a Kubernetes specification to "communicating service secrets to workloads in a consistent way". + If the operator supports Service Binding, applications that are bound to the Postgres clusters will be able to fetch database access credentials without requiring + the user to specify them manually in the application. + vendor_compliance: | + The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification. + - id: cuann + name: Custom Annotations + type: boolean + description: | + The operator allows the user to provide custom [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + (aside from any operator generated Annotations) to the objects generated by the operator. + vendor_compliance: | + The operator must provide configuration for setting Annotations on all objects generated by the operator. + Partial compliance is achieved if at least Pods can be annotated and this is clearly marked in the comments. + - id: culab + name: Custom Labels + type: boolean + description: | + The operator allows the user to provide custom [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the objects generated by the operator. + vendor_compliance: | + The operator must provide configuration for setting Labels on all objects generated by the operator. + Partial compliance is achieved if at Labels can be added to at least Pods and this is clearly marked in the comments. + - id: apdnd + name: Assign Pod to Node + type: string + description: | + The operator allows Postgres Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. + vendor_compliance: | + Indicate one of the following values: + + * 'Complete': if the operator support all the rules to assign a Pod to a Node defined in the latest Kubernetes version specified under [vers/k8ver]. + * 'Partial': if the operator support only partially the rules to assign a Pod to a Node. + + A link for each supported rule or rules group to assign a Pod to a Node must be specified. + - id: topsc + name: Topology Spread Constraints + type: boolean + description: | + The operator allows Postgres pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints), + to control how Pods are spread across the Kubernetes cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. + - id: harp + name: HA & Replication + features: + - id: autfa + name: Automated failover + type: boolean + description: | + The operator provides the facility that in the event of a node or pod failure where the Postgres primary is affected, + another Postgres instance (if available) will be promoted to primary. The operation must happen automatically without user intervention. + main: true + - id: techa + name: HA Technology + type: string + description: | + The technology (software name or technology principles) that support the high availability and automated failover capabilities of the operator. + vendor_compliance: | + Provide a succinct software or technology name. E.g.: Patroni, Stolon, Custom. + main: true + - id: asrep + name: Asynchronous replication + type: boolean + description: | + The operator allows to configure Postgres clusters using asynchronous streaming replication. + Asynchronous is the default streaming replication mode in Postgres. + - id: syrep + name: Synchronous replication + type: boolean + description: | + The operator allows to configure Postgres clusters using synchronous streaming replication. + The operator must manage the configuration details based on the user's preferences on which nodes behave synchronously. + - id: serep + name: Semi-synchronous replication + type: boolean + description: | + Semi-synchronous, or also called group or quorum replication, is a hybrid mode where a subset of the nodes is expected to replicate synchronously; + and once that level is reached, the remainding nodes replicate asynchronously. + That is, Postgres waits for confirmation of the write only from the synchronous nodes before considering the transaction committed. + - id: derep + name: Delayed replicas + type: boolean + description: | + The operator allows the user to configure one or more replicas subject to a user-specified (intentional) replication lag. + This is useful to have always an online instance with data "in the past" for analytics or data recovery purposes. + - id: carep + name: Cascading replication + type: boolean + description: | + Postgres allows to create replicas that are fed from another replica, instead of replicating from the primary instance. + This is interesting to alleviate the effects of replication on the primary; and allows to create arbitrary replication tree-like topologies. + - id: exrep + name: Replication from external origin + type: boolean + description: | + The operator allows a Postgres cluster to be setup in recovery mode while replicating from an external (non-operator managed) Postgres origin. + This allows migrations to the operator using streaming replication. + vendor_compliance: | + To implement the feature, the user should be able to provide arbitrary host, port, username and password (or other means of credential passing) of the origin. + - id: walsh + name: WAL Shipping + type: boolean + description: | + WAL Shipping is a Postgres technique that allows replication via a shared storage mechanism where WAL files produced on the primary node are copied ("shipped") to the replica. + This feature is useful for having replicas for example on DR sites, replicate over global storage object stores and others. + vendor_compliance: | + Implementing the feature implies both generating WAL files to a suitable location; as well as creating clusters that permanently read those WAL files and replicate from them. + - id: prstb + name: Promote standby cluster + type: boolean + description: | + The operator allows the user to promote a standby cluster that is a cluster with primary in recovery mode so that the primary becomes RW. This feature is used in conjunction with [harp/exrep] and/or [harp/walsh] to make GEO-replicated and/or DR (Disaster Recovery) clusters on other regions, where a region is essentially an independent Kubernetes cluster. + vendor_compliance: | + Only applies if [harp/exrep] or [harp/walsh] are true. + The operator must allow to promote a standby clusters replicating either via WAL Shipping and/or streaming replication. + main: true + - id: lorep + name: Managed logical replication + type: boolean + description: | + Postgres supports logical replication as well as streaming replication. It requires some configuration and commands to be run by the user. + This feature represents operator capabilities to perform these operations in a managed way, without the user having to type commands or create configurations directly. + vendor_compliance: | + The operator is considered compliant if it allows the user to provide logical replication without having to setup it directly, just specifying (declarative) preferences. + The operator is still considered compliant if it cannot proceed until certain operations are performed by the users that are dependent on their environment + (e.g. adding PKs, UNIQUEs or REPLICA IDENTITY to the DDL). + - id: pgex + name: PostgreSQL Extensions + features: + - id: extme + name: Extensions distribution mechanism + type: string + description: | + This feature is set to describe how extensions are shipped in a containerized environment. + Typically they are part of the same Postgres container image, but they may also be distributed via other mechanisms. + vendor_compliance: | + Indicate 'built-in' if the extensions come included with the same Postgres container; or a succinct word or few words naming the distribution mechanism. + More details may be provided, if needed, via the links and comments fields. + - id: coext + name: Core/contrib extensions + type: integer + description: | + The number of core+contrib extensions provided by the operator for the latest Postgres version provided by the operator. + vendor_compliance: | + Submission must provide the total number of core+contrib extensions supported for the latest Postgres version provided by the operator. + It should also be provided, when available, a link with a detailed list of those extensions supported (via links field). + main: true + - id: thext + name: Third-party extensions + type: integer + description: | + The number of extensions not included in the Postgres core+contrib (i.e. created by third parties, outside of Postgres repository) for the latest Postgres version provided by the operator. + vendor_compliance: | + Submission must provide the total number of third-party extensions supported for the latest Postgres version provided by the operator. + A link with a detailed list of those extensions supported (via links field) should also be provided, if available. + main: true + - id: byext + name: Bring your own extension + type: boolean + description: | + The operator provides mechanisms for users to add (or upload) third-party extensions not initially provided by the operator. + vendor_compliance: | + The operator must support providing some mechanism to include custom extensions additionally or independently of the extension distribution mechanism, + if a user builds a local extension, this could be uploaded or included on the postgres container with an automatic copy to all replicas related. + - id: bkup + name: Backups + features: + - id: bktch + name: Backup Technology + type: string + description: | + What technology (pgbasebackup, PgBackRest, WAL-e/g, Barman, custom, etc) is used to support creation and restoration of backups. + vendor_compliance: | + Provide the most succinct possible name of the technology. + main: true + - id: bkdst + name: Backup destinations + type: string_array + description: | + Where backups can be stored (typically this may include object stores, PVs, etc). + vendor_compliance: | + If any of the following technology names apply, use them, and add other names, as needed: + + * PersistentVolume + * NFS + * S3 + * GCS + * Azure Blob + * Local file system + - id: autbk + name: Automated backup management + type: boolean + description: | + The operator provides mechanisms for performing backups automatically and providing lifecycle mechanisms (delete old backups according to a user-supplied policy - often known as retention policies). + vendor_compliance: | + True response for the feature implies both automatic backups and lifecycle management. + If only the former is provided, answer should be false but this capability should be mentioned in the comments field. + main: true + - id: encbk + name: Backups encryption + type: boolean + description: | + Backups can be performed with user-supplied encryption keys. + - id: ptrbk + name: Point In Time Recovery + type: boolean + description: | + The operator provides support for the user to specify a recovery point (in the past) to which a backup should be recovered to + (if unspecified, backup will be recovered in full). + vendor_compliance: | + True response implies that at least time-based recovery is supported. + If the operator also supports PITR by xid and label, clarify in the comments field. + - id: mulbk + name: Multiple backup configurations + type: boolean + description: | + The operator supports managing more than one backup configuration. + This is typically used to store backups on different object stores (for protection purposes) or to send them to different sites. + It may also include different lifecycle policies. + - id: mgmi + name: Management Interfaces + features: + - id: cucol + name: Resource display columns + type: boolean + description: | + Operator-managed objects (CRDs) include specific fields that provide additional information about the Postgres instances. + These fields are shown when querying the Kubernetes resources using tools like `kubectl`. + vendor_compliance: | + In order to be compliant, operators have to provide at least the following information: + + * Replication status (primary/replica/etc) + * Cluster status (healthy/not healthy) + * Postgres major/minor version + * Latest base backup date + * SSL on/off + - id: kbplg + name: kubectl plugin + type: boolean + description: | + The users may download an additional kubectl plugin which provides custom commands to manage the operator. + vendor_compliance: | + If the feature is provided, include a link to the downloadable plugin in the links field. + - id: bagui + name: Basic management GUI + type: boolean + description: | + The operator bundles some GUI (web interface or other) that allows to perform basic operations (e.g. cluster creation) graphically. + main: true + - id: fugui + name: Fully-featured GUI + type: boolean + description: | + The operator bundles a GUI that is fully-featured (that is, in feature-parity with the capabilities provided via the other management interfaces -- typically CRDs). + The GUI may provide additional features that cannot be otherwise provided via text interfaces (e.g. graphical representation of resources or operations). + main: true + - id: urgui + name: GUI users & roles + type: boolean + description: | + The bundled management GUI has authentication and authorization baked in and provides mechanisms to support multiple users and authorize/deny them to perform operations. + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + Submission should detail in the link and/or comments field how users and roles for the GUI can be managed. + - id: guiss + name: GUI Single-Sign On + type: boolean + description: | + The bundled management GUI supports integration with Single-Sign On (SSO), like OIDC, identity providers (e.g. Google, Github or Twitter login). + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + - id: cogui + name: GUI database console + type: boolean + description: | + The bundled management GUI includes a console that can connect to any database managed by the operator and send command (e.g. like `psql`). + vendor_compliance: | + Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + - id: o11y + name: Observability + features: + - id: mtech + name: Metrics technology + type: string_array + description: | + If the operator supports extracting metrics from Postgres, define how they are handled, which technology receives and processes them. + + If any of the following technology names apply, use them, and add other names, as needed: + + * Prometheus + * OTEL + * OpenTSDB + * Nagios + * Sensu + vendor_compliance: | + If supported, provide in the comments information about whether the technology is a dependency, is built-in, external, etc. + Provide link(s) to the documentation for further information. + main: true + - id: expme + name: Export metrics + type: boolean + description: | + Regardless of how metrics are processed (e.g. as part of the operator), + this feature is implemented when the operator allows the user to configure sending metrics to external services, like a metrics SaaS. + - id: cudas + name: Custom dashboards + type: boolean + description: | + In order to display the captured Postgres metrics, the operator provides specialized Postgres dashboards for the users. + - id: cuale + name: Custom alerts + type: boolean + description: | + The operator provides bundled specific Postgres alerts to be triggered on the Postgres metrics processed. + E.g. there is an alert for transaction wraparound or for unused replication slots. + - id: exdel + name: Exposed decorated logs + type: boolean + description: | + The operator provides a mechanism to expose all the logs of the managed Postgres instances to a centralized logging tool. + The logs must be decorated with extra metadata in order to provide semantic meaning, including the Pod name and namespace, the cluster name, the role of the Postgres instance (e.g. primary, replica, standby-leader, etc.) and the timestamp that will be available to be used to filter logs entries. + There is no need to configure the tool in order to obtain required extra metadata from the logs. + - id: explg + name: Export logs + type: boolean + description: | + The operator allows the user to configure an external sink for the Postgres logs (e.g. a SaaS service). + - id: oo11y + name: Operator Observability + type: boolean + description: | + The operator is itself a source of telemetry data, potentially including metrics, traces and logs, about its own performance. + - id: secy + name: Security + features: + - id: opcsc + name: Operator code security scanning + type: boolean + description: | + The operator code is always scanned for security vulnerabilities. + main: true + - id: imgsc + name: Image scanning + type: boolean + description: | + Operator-provided images are always scanned for security vulnerabilities. + main: true + - id: sigim + name: Signed images + type: boolean + description: | + Container images are digitally signed according to the [sigstore](https://www.sigstore.dev/) project. + vendor_compliance: | + Compliance may also be achieved by using a technology other than sigstore, as long as it provides an equivalent set of security capabilities. + - id: isbom + name: Software Bill of Materials + type: boolean + description: | + The operator releases include the SBOM (Software Bill of Materials), a detailed description of all the components, modules, and their dependencies. + vendor_compliance: | + SBOM is expected to be in accordance to the [Kubernetes SIG BOM](https://github.com/kubernetes-sigs/bom). + - id: fgopp + name: Fine-grained RBAC permissions + type: boolean + description: | + The operator uses a separate serviceaccount that has RBAC permissions that only require the access that is actually needed to create and manage the Kubernetes resources, not more. + - id: noprm + name: No or justified privileged mode + type: boolean + description: | + The operator-provided containers do not require privileged mode. + The container processes do not run as root. + vendor_compliance: | + Reasonable exceptions to this rule can be made for features that require or do not diminish the container's security, e.g. when using eBPF. + - id: day2 + name: Day 2 Operations + features: + - id: amiup + name: Automated minor upgrades + type: boolean + description: | + The operator can perform a minor version upgrade of a Postgres cluster automatically. + This operation can be managed by the user declaratively. + vendor_compliance: | + The operator must provide proper information to the user as to the status and final result of the operation. + The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + Provide information about the update strategy (i.e. restart of the pods or rolling update followed by a switchover or a restart). + main: true + - id: amaup + name: Automated major upgrades + type: boolean + description: | + The operator can perform a major version upgrade of a Postgres cluster automatically. + This operation can be managed by the user declaratively. + vendor_compliance: | + The operator must provide proper information to the user as to the status and final result of the operation. + The operator should provide ongoing status information, and perform the operation with the minimum downtime required. + main: true + - id: crest + name: Controlled cluster restart + type: boolean + description: | + Sometimes Postgres needs to be restarted (e.g. changing of a parameter that requires restart). + The operator provides means to perform this operation automatically and in a controlled manner (rolling restart) so that the cluster faces a minimal downtime only. + - id: ociup + name: Container images upgrade + type: boolean + description: | + Similarly to the controlled restart operation, the operator is capable of updating the running container images (which require a pod restart) automatically and with minimal cluster impact. + - id: swtch + name: Switchover + type: boolean + description: | + If HA capabilities are provided, the operator also provides a mechanism for manual switchover. + The user may specify the configuration declaratively and the operator will perform the desired switchover automatically, by demoting the current primary, promoting the a replica, and updating the endpoints/services as required. + - id: sqlmi + name: SQL Migrations + type: boolean + description: | + The operator provides managed SQL migration capabilities. + The user may specify SQL scripts that contain migrations (DDL changes, etc) to be deployed to a given database, having the operator apply them automatically. + vendor_compliance: | + The operator must report back to the user detailed information about the results of the execution(s) of the script(s) provided by the user. + - id: oday2 + name: Other Day 2 Operations + type: string_array + description: | + The operator provides support for other managed Day 2 operations. + vendor_compliance: | + All the mentioned additional day 2 operations need to be possible via declarative configuration and the operator to fully execute them without further user intervention. + - id: dain + name: Data Integration + features: + - id: kfkin + name: Kafka integration + type: boolean + description: | + The operator provides a managed solution for automation to export CDC (Change Data Capture) events to Kafka. + vendor_compliance: | + Kafka could be an external dependency to the operator. + The whole operation is expected to be fully automated, with the user only providing a declarative configuration. + - id: migpg + name: Migration from Postgres + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from another (external) Postgres database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`. + - id: migmy + name: Migration from MySQL + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a MySQL database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migor + name: Migration from Oracle + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from an Oracle database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migms + name: Migration from SQL Server + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a SQL Server database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migmo + name: Migration from MongoDB + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from a MongoDB database. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. + - id: migot + name: Migration from other databases + type: string_array + description: | + The operator provides a managed capability to automatically migrate data from other relational or non-relational databases. + Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), + which allows for near-zero downtime migrations. + Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. + vendor_compliance: | + Provide the names of the other database from where migration is supported. + - id: shrd + name: Sharding + features: + - id: mshrd + name: Managed sharding support + type: boolean + description: | + The operator understands horizontal scalability and is capable to automatically deploying sets of clusters (shards) with their accompanying infrastructure (coordinators, services, etc). + The user should be able to write a declarative configuration of a sharded cluster and the operator to fully deploy it without any further manual user intervention. + main: true + - id: shrdt + name: Sharding technology + type: string_array + description: | + If managed sharding support is provided, which technology or technologies are used to provide sharding support (which is not included in Postgres core). + - id: aggds + name: Aggregated monitoring dashboards + type: boolean + description: | + Since a sharded cluster is a set of (independent) Postgres clusters, an aggregated dashboard is one that provides Observability information across all shards of the cluster. + E.g. total database size, tps processed or total number of client connections. + - id: cobkp + name: Coordinated backups + type: boolean + description: | + A sharded cluster can be backed up by backing up each shard independently, but those backups are not necessarily consistent with each other. + If sharding is used, an additional capability is to coordinate backups on a given timestamp so that they are performed or can be restored at the same logical timestamp, + so they become consistent with each other. diff --git a/apache-druid/spec/feature_matrix.yaml b/apache-druid/spec/feature_matrix.yaml index 1b30434..e3ac1c6 100644 --- a/apache-druid/spec/feature_matrix.yaml +++ b/apache-druid/spec/feature_matrix.yaml @@ -12,38 +12,63 @@ categories: description: | Mechanism(s) provided to install the operator. vendor_compliance: | + Provide a succinct value or list of values of the supported mechanisms. + If any of the following names apply, use them, and add other mechanism names, as needed: + + * YAML * Helm * Kustomize - main: true + * Operator Bundle + value: | + * Helm + * Kustomize + references: https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md - id: mcrds name: Management via CRDs type: boolean description: | The operator manages CRDs (Custom Resource Definition). vendor_compliance: | + If true, provide a list of the names of the CRDs managed by the operator. Names should be direct links to their documentation or API reference. + value: true + references: | * druid.apache.org/v1alpha1/Druid - https://github.com/datainfrahq/druid-operator/blob/v1.1.0/apis/druid/v1alpha1/druid_types.go - main: true - id: offin name: Offline installation type: boolean description: | Whether it is possible to install the operator on a (air-gapped) cluster without internet connection. vendor_compliance: | - Druid operator can be installed in air-gapped environment the same as with any other type of environment. - Installation documentation - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + A link should be provided with documentation on how to install the operator on air-gapped environments. + value: true + references: Installation documentation - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + comments: Druid operator can be installed in air-gapped environment the same as with any other type of environment. - id: scpua name: Supported CPU architectures type: string_array description: | Support for the operator to run on nodes with specified CPU architectures. vendor_compliance: | + Provide a list of CPU architectures that are supported between: + + * amd64 + * arm + * arm64 + * ppc64le + * s390x + + In case you support additional architectures, add their name(s) in a similar fashion. + value: | * amd64 - id: coios name: Container images OSes type: string_array description: | - Druid's base image is depends on the JDK version required: - gcr.io/distroless/java$JDK_VERSION-debian11 + Indicate the base image of the images used by the operator. (e.g: scratch, ubi, fedora, ubuntu, centos, alpine, etc.). + Provide the image name without tags (e.g. debian instead of debian:8.11). + value: | + * gcr.io/distroless/java$JDK_VERSION-debian11 + comments: Druid's base image is depends on the JDK version required - id: olmcl name: Operator Capability Level type: string @@ -64,11 +89,14 @@ categories: For a detailed description of each capability level refer to the Operator Framework documentation for [Operator Capability Level](https://sdk.operatorframework.io/docs/overview/operator-capabilities/). + value: `Seamless Upgrades` + references: | + [Druid's feature list](https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md) - id: vers name: Versions features: - - id: pgver - name: Supported PostgreSQL versions + - id: drver + name: Supported Apache Druid versions type: string_array description: | Which major (and minor) versions does this version of the operator support. @@ -76,7 +104,11 @@ categories: Provide a list of every supported major version followed by a list in parenthesis of the minor versions supported for that major version. E.g. '15 (15.1, 15.0), 14 (14.3, 14.2)'. All versions must be ordered by reverse chronological order. Other components versions that are used may be described in the comments (e.g: Patroni, WAL-G, PgBackRest, PgBouncer, etc.) - main: true + value: | + * 25 (25.0) + comments: | + Druid image is customizable thus you can use different versions. + The version of the operator has been tested with this version of Druid - id: k8ver name: Supported Kubernetes versions type: string_array @@ -85,16 +117,8 @@ categories: vendor_compliance: | Provide a list of official Kubernetes minor versions numbers, ordered by reverse chronological order. Use a range of versions, if possible, e.g. '1.26 - 1.23' or '1.22 - 1.19'. - main: true - - id: pgfor - name: PostgreSQL OSS forks - type: string_array - description: | - PostgreSQL forks are considered derived projects that started from PostgreSQL codebase and add different features while maintaining compatibility with PostgreSQL. - Some operators may provide support for some of such forks, alongside with the original PostgreSQL. - vendor_compliance: | - Leave empty if no version other than the original PostgreSQL is supported. Otherwise, provide a list of names and links to the PostgreSQL forks supported. - If non-OSS forks are supported, leave empty but add a link to them in the comments section. + value: | + 1.26 - 1.25 - id: lisu name: Licensing & Support features: @@ -105,7 +129,7 @@ categories: The operator is published, and its source code is made publicly available, under an OSI-approved open source license. vendor_compliance: | If true, provide the SPDX identifier(s) of the license. - main: true + value: true - id: fcimg name: Free container images type: boolean @@ -113,7 +137,7 @@ categories: The container images used for the operator's operation are free to distribute and use, not subject to commercial restrictions. vendor_compliance: | If false, provide a link with documentation on the usage requisites/restrictions of the container images. - main: true + value: true - id: nenof name: No enterprise-only features type: boolean @@ -123,6 +147,7 @@ categories: vendor_compliance: | If false, provide links to all known derived, proprietary versions. Linked documentation should clearly reflect the features that are only part of proprietary versions. + value: true - id: comms name: Commercial support type: boolean @@ -131,6 +156,7 @@ categories: vendor_compliance: | Provide a link or list of links to the entities that provide commercial support. Links should be as specific as possible, pointing to informative documentation on the provided support. + value: false - id: pubit name: Public issue tracker type: boolean @@ -138,6 +164,8 @@ categories: Development happens in the open. There is a public issue tracker where users may view, comment and create issues. vendor_compliance: | Provide a specific link to the issue tracker. + value: true + references: https://github.com/datainfrahq/druid-operator/issues - id: pubch name: Public chat/forums type: boolean @@ -145,8 +173,12 @@ categories: There is/are public forums, mailing lists, chat groups (Slack, Discord, Matrix, etc) where users may freely join and participate in discussions. vendor_compliance: | Provide links to all relevant user forums. - - id: pgcl - name: PostgreSQL Clusters + value: true + references: | + - druid-operator channel in Kubernetes Slack Workspace - https://kubernetes.slack.com/archives/C04F4M6HT2L + - DataInfra slack workspace - https://launchpass.com/datainfra-workspace + - id: drcl + name: Apache Druid Clusters features: - id: depau name: Deployment automation @@ -154,19 +186,23 @@ categories: description: | The operator provides capabilities to automatically deploy production-ready clusters based on a provided configuration. No user initiated commands must be required. + value: true - id: pomte name: Pod management technology - type: string + type: string_array description: | Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. vendor_compliance: | Provide the most succinct possible name of the technology. Use the Kubernetes resource type name (e.g. 'Deployment'), if applicable. - main: true - - id: pgcnf - name: PostgreSQL custom config + valur: | + - StatefulSet + - Deployment + - id: drcnf + name: Apache Druid custom config type: boolean description: | - The operator allows the user to supply custom PostgreSQL configuration. + The operator allows the user to supply custom configuration. + value: true - id: conpl name: Integrated connection pooling type: boolean @@ -175,74 +211,74 @@ categories: vendor_compliance: | The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc. All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. - main: true + value: ??? - id: cpccf name: Connection pool custom config type: boolean description: | The operator allows the user to supply a custom connection pool configuration for the connection pool service. vendor_compliance: | - Only applies if [pgcl/conpl] is true. + Only applies if [drcl/conpl] is true. + value: ??? - id: tlssp name: TLS Support type: boolean description: | - PostgreSQL connections can be secured with Postgres SSL/TLS support. - main: true + Connections can be secured with SSL/TLS support. + value: ??? - id: tlscu name: TLS user-provided certificates type: boolean description: | Operators may chose by default to generate self-signed SSL certificates. - They may also offer the option to specify the CA and certificates that users want Postgres clusters to use. + They may also offer the option to specify the CA and certificates that users want the clusters to use. + value: ??? - id: crtmg name: CertManager integration type: boolean description: | - The operator integrates with CertManager in order to generate the certificate to be used with Postgres. + The operator integrates with CertManager in order to generate the certificate to be used with. vendor_compliance: | - Only applies if [pgcl/tlscu] is true. + Only applies if [drcl/tlscu] is true. + value: ??? - id: insql name: Initialization from SQL scripts type: boolean description: | After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation. The operator must properly inform the user of the execution result of the scripts. + value: ??? - id: inext name: Initialization from external source type: boolean description: | After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data. The operator must properly inform the user of the execution result of the scripts. - - id: cuhba - name: HBA custom config - type: boolean - description: | - The operator allows users to provide custom configuration for the Postgres HBA (Host-Based Authentication) mechanism. - It may involve directly generating the custom `pg_hba.conf` file or by using more cloud-native technologies. + value: ??? - id: mgmup name: Management of db users/passwords type: boolean description: | - The operator provides a mechanism to declaratively create, modify or delete users, roles and/or password for the PostgreSQL cluster. + The operator provides a mechanism to declarative create, modify or delete users, roles and/or password for the cluster. + value: ??? - id: reqli name: Customize computing resource requests, limits type: boolean description: | - The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the Postgres pods. - main: true + The user may specify/customize the requests, the limits or both for computing resources (CPU, memory) of the pods. + value: true - id: hugpa name: Support for huge pages type: boolean description: | - The user may specify the request to use huge pages for Postgres (and/or potentially other sidecars). - Postgres must be able to be configured and start using huge pages. - main: true - - id: pgsrv - name: PostgreSQL exposed via Service + The user may specify the request to use huge pages (and/or potentially for other sidecars). + It must be able to be configured and start using huge pages. + value: ??? + - id: drsrv + name: Apache Druid exposed via Service type: string description: | - The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the PostgreSQL connections. + The operator creates by default or allows the user to request one or more Kubernetes Service(s) to be created to expose the system connections. Expected capabilities should include a RW (read-write) or RO (read-only in case of cascading replication) connection to the primary instance; and, optionally, a RO (read-only) to load balance read-only replicas instances in the cluster. vendor_compliance: | The value must be one of: @@ -251,42 +287,49 @@ categories: * PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered. * Balanced: A single service that load balances RW/RO traffic transparently. * Primary: Primary service must be offered. + value: ??? - id: stosc name: Automatic storage scaling type: boolean description: | If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically (either by setting some default thresholds or requiring explicit declarative configuration from the user). + value: true + references: | + - https://github.com/datainfrahq/druid-operator/blob/f3f44e332e2d08ea463c1ca49517177a71bcf4b7/docs/features.md#volume-expansion-of-druid-nodes-running-as-statefulsets + comments: | + NOTE: This feature has been tested only on cloud environments and storage classes which have supported volume expansion. This feature uses cascade=orphan strategy to make sure only Stateful is deleted and recreated and pods are not deleted. - id: scal0 name: Scale down to zero type: boolean description: | The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed. Upon scaling up, the cluster should be brought back up without the need to restore a backup. This feature may also be known as "hibernation". - - id: tblsp - name: Tablespaces - type: boolean - description: | - The user may specify one or more PostgreSQL tablespaces and their associated backing storage. - - id: cupgi - name: Custom Postgres images + value: ??? + - id: cupdi + name: Custom Apache Druid images type: boolean description: | - The operator allows the user to specify custom (user-provided) Postgres container images, instead of using the operator's provided images. + The operator allows the user to specify custom (user-provided) Druid container images, instead of using the operator's provided images. vendor_compliance: | The operator should specify if the custom image needs to follow some minimal patterns to be able to work, or if it can work with any postgres container image provided. + value: true - id: uside name: User supplied sidecars type: boolean description: | - The user may specify custom sidecars (containers or init containers) to be created alongside the Postgres container (and, possibly, other operator sidecars). + The user may specify custom sidecars (containers or init containers) to be created alongside the system container (and, possibly, other operator sidecars). User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. + value: true + references: | + - https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md#Add-Additional-Containers-in-Druid-Nodes - id: usvol name: User supplied volumes type: boolean description: | - The user may specify and mount custom volumes to be created alongside the volumes created for the Postgres container. + The user may specify and mount custom volumes to be created alongside the volumes created for the system container. + value: true - id: usprt name: User supplied ports type: boolean @@ -294,6 +337,7 @@ categories: The user may specify custom ports to be created alongside the services that expose PostgreSQL. vendor_compliance: | Only applies if [pgcl/pgsrv] is set. + value: ??? - id: srvbi name: Support for Service Binding type: boolean diff --git a/postgres/vendor-submissions/vendor-submission-template.yaml b/postgres/vendor-submissions/vendor-submission-template.yaml index 9930811..78aff98 100644 --- a/postgres/vendor-submissions/vendor-submission-template.yaml +++ b/postgres/vendor-submissions/vendor-submission-template.yaml @@ -35,9 +35,9 @@ featureMatrix: # The operator can be managed via CRDs (Custom Resource Definition). # vendor_compliance: | # If true, provide a list of the names of the CRDs supported by the operator. Names should be direct links to their documentation. - value: - references: - comments: + value: + references: + comments: - id: offin # name: Offline installation # type: boolean From d3c9912fc27e6b3f36035aea8b8cdbfc42542058 Mon Sep 17 00:00:00 2001 From: "itamar.marom" Date: Mon, 22 May 2023 20:32:13 +0300 Subject: [PATCH 3/5] more druid configs --- apache-druid/spec/feature_matrix.yaml | 35 +++++++++++++++------------ 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/apache-druid/spec/feature_matrix.yaml b/apache-druid/spec/feature_matrix.yaml index e3ac1c6..051659f 100644 --- a/apache-druid/spec/feature_matrix.yaml +++ b/apache-druid/spec/feature_matrix.yaml @@ -211,7 +211,9 @@ categories: vendor_compliance: | The connection pool may be deployed in several ways, like a Deployment layer, a side car, etc. All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. - value: ??? + value: true + comments: | + this is not relevant to the operator itself. Druid configures it with defaults. You can override them through the operator - id: cpccf name: Connection pool custom config type: boolean @@ -219,7 +221,7 @@ categories: The operator allows the user to supply a custom connection pool configuration for the connection pool service. vendor_compliance: | Only applies if [drcl/conpl] is true. - value: ??? + value: true - id: tlssp name: TLS Support type: boolean @@ -241,20 +243,15 @@ categories: vendor_compliance: | Only applies if [drcl/tlscu] is true. value: ??? - - id: insql - name: Initialization from SQL scripts - type: boolean - description: | - After the database cluster creation, the operator will run automatically one or more user-supplied scripts for initial DDL or data (possibly limited in size) creation. - The operator must properly inform the user of the execution result of the scripts. - value: ??? - id: inext name: Initialization from external source type: boolean description: | After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data. The operator must properly inform the user of the execution result of the scripts. - value: ??? + value: false + comments: | + This will be available in the near future with the Druid Ingestion controller - id: mgmup name: Management of db users/passwords type: boolean @@ -287,7 +284,10 @@ categories: * PrimaryAndReplicasAndBalanced: Primary service, replicas services and a service that load balance RW/RO traffic transparently must be offered. * Balanced: A single service that load balances RW/RO traffic transparently. * Primary: Primary service must be offered. - value: ??? + value: "Balanced" + comments: | + Druid is built with microservices architecture. There is no really primary-secondary. There is an option to deploy + a Kubernetes Service for each component with then references to all pods from that type. - id: stosc name: Automatic storage scaling type: boolean @@ -295,7 +295,7 @@ categories: If the user's selected storage technology supports transparent scaling, the operator will take care of scaling the storage automatically (either by setting some default thresholds or requiring explicit declarative configuration from the user). value: true - references: | + references: - https://github.com/datainfrahq/druid-operator/blob/f3f44e332e2d08ea463c1ca49517177a71bcf4b7/docs/features.md#volume-expansion-of-druid-nodes-running-as-statefulsets comments: | NOTE: This feature has been tested only on cloud environments and storage classes which have supported volume expansion. This feature uses cascade=orphan strategy to make sure only Stateful is deleted and recreated and pods are not deleted. @@ -305,7 +305,10 @@ categories: description: | The number of pods in the cluster can be set to 0. This implies that no pods (no compute) would be used, but storage is not removed. Upon scaling up, the cluster should be brought back up without the need to restore a backup. This feature may also be known as "hibernation". - value: ??? + value: true + comments: | + Druid saves its data in a remote storage (like S3 bucket), in case of scaling to zero and then scaling back up, the data will be loaded back + from the remote store. - id: cupdi name: Custom Apache Druid images type: boolean @@ -334,10 +337,10 @@ categories: name: User supplied ports type: boolean description: | - The user may specify custom ports to be created alongside the services that expose PostgreSQL. + The user may specify custom ports to be created alongside the services that expose Apache Druid. vendor_compliance: | - Only applies if [pgcl/pgsrv] is set. - value: ??? + Only applies if [drcl/drsrv] is set. + value: true - id: srvbi name: Support for Service Binding type: boolean From a60fc00b77e06b22fdb430cd851d80812e51f944 Mon Sep 17 00:00:00 2001 From: "itamar.marom" Date: Mon, 22 May 2023 21:08:58 +0300 Subject: [PATCH 4/5] more druid configs --- apache-druid/spec/feature_matrix.yaml | 131 ++------------------------ 1 file changed, 7 insertions(+), 124 deletions(-) diff --git a/apache-druid/spec/feature_matrix.yaml b/apache-druid/spec/feature_matrix.yaml index 051659f..8a8226c 100644 --- a/apache-druid/spec/feature_matrix.yaml +++ b/apache-druid/spec/feature_matrix.yaml @@ -350,6 +350,7 @@ categories: the user to specify them manually in the application. vendor_compliance: | The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification. + value: false - id: cuann name: Custom Annotations type: boolean @@ -359,6 +360,7 @@ categories: vendor_compliance: | The operator must provide configuration for setting Annotations on all objects generated by the operator. Partial compliance is achieved if at least Pods can be annotated and this is clearly marked in the comments. + value: true - id: culab name: Custom Labels type: boolean @@ -367,11 +369,12 @@ categories: vendor_compliance: | The operator must provide configuration for setting Labels on all objects generated by the operator. Partial compliance is achieved if at Labels can be added to at least Pods and this is clearly marked in the comments. + value: true - id: apdnd name: Assign Pod to Node type: string description: | - The operator allows Postgres Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. + The operator allows Pods to be assigned to a specific set of Nodes based on some [Kubernetes known rules](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/). E.g. Node Labels, Affinity, etc. vendor_compliance: | Indicate one of the following values: @@ -379,134 +382,14 @@ categories: * 'Partial': if the operator support only partially the rules to assign a Pod to a Node. A link for each supported rule or rules group to assign a Pod to a Node must be specified. + value: 'Complete' - id: topsc name: Topology Spread Constraints type: boolean description: | - The operator allows Postgres pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints), + The operator allows pods to specify [Topology Spread Constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints), to control how Pods are spread across the Kubernetes cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. - - id: harp - name: HA & Replication - features: - - id: autfa - name: Automated failover - type: boolean - description: | - The operator provides the facility that in the event of a node or pod failure where the Postgres primary is affected, - another Postgres instance (if available) will be promoted to primary. The operation must happen automatically without user intervention. - main: true - - id: techa - name: HA Technology - type: string - description: | - The technology (software name or technology principles) that support the high availability and automated failover capabilities of the operator. - vendor_compliance: | - Provide a succinct software or technology name. E.g.: Patroni, Stolon, Custom. - main: true - - id: asrep - name: Asynchronous replication - type: boolean - description: | - The operator allows to configure Postgres clusters using asynchronous streaming replication. - Asynchronous is the default streaming replication mode in Postgres. - - id: syrep - name: Synchronous replication - type: boolean - description: | - The operator allows to configure Postgres clusters using synchronous streaming replication. - The operator must manage the configuration details based on the user's preferences on which nodes behave synchronously. - - id: serep - name: Semi-synchronous replication - type: boolean - description: | - Semi-synchronous, or also called group or quorum replication, is a hybrid mode where a subset of the nodes is expected to replicate synchronously; - and once that level is reached, the remainding nodes replicate asynchronously. - That is, Postgres waits for confirmation of the write only from the synchronous nodes before considering the transaction committed. - - id: derep - name: Delayed replicas - type: boolean - description: | - The operator allows the user to configure one or more replicas subject to a user-specified (intentional) replication lag. - This is useful to have always an online instance with data "in the past" for analytics or data recovery purposes. - - id: carep - name: Cascading replication - type: boolean - description: | - Postgres allows to create replicas that are fed from another replica, instead of replicating from the primary instance. - This is interesting to alleviate the effects of replication on the primary; and allows to create arbitrary replication tree-like topologies. - - id: exrep - name: Replication from external origin - type: boolean - description: | - The operator allows a Postgres cluster to be setup in recovery mode while replicating from an external (non-operator managed) Postgres origin. - This allows migrations to the operator using streaming replication. - vendor_compliance: | - To implement the feature, the user should be able to provide arbitrary host, port, username and password (or other means of credential passing) of the origin. - - id: walsh - name: WAL Shipping - type: boolean - description: | - WAL Shipping is a Postgres technique that allows replication via a shared storage mechanism where WAL files produced on the primary node are copied ("shipped") to the replica. - This feature is useful for having replicas for example on DR sites, replicate over global storage object stores and others. - vendor_compliance: | - Implementing the feature implies both generating WAL files to a suitable location; as well as creating clusters that permanently read those WAL files and replicate from them. - - id: prstb - name: Promote standby cluster - type: boolean - description: | - The operator allows the user to promote a standby cluster that is a cluster with primary in recovery mode so that the primary becomes RW. This feature is used in conjunction with [harp/exrep] and/or [harp/walsh] to make GEO-replicated and/or DR (Disaster Recovery) clusters on other regions, where a region is essentially an independent Kubernetes cluster. - vendor_compliance: | - Only applies if [harp/exrep] or [harp/walsh] are true. - The operator must allow to promote a standby clusters replicating either via WAL Shipping and/or streaming replication. - main: true - - id: lorep - name: Managed logical replication - type: boolean - description: | - Postgres supports logical replication as well as streaming replication. It requires some configuration and commands to be run by the user. - This feature represents operator capabilities to perform these operations in a managed way, without the user having to type commands or create configurations directly. - vendor_compliance: | - The operator is considered compliant if it allows the user to provide logical replication without having to setup it directly, just specifying (declarative) preferences. - The operator is still considered compliant if it cannot proceed until certain operations are performed by the users that are dependent on their environment - (e.g. adding PKs, UNIQUEs or REPLICA IDENTITY to the DDL). - - id: pgex - name: PostgreSQL Extensions - features: - - id: extme - name: Extensions distribution mechanism - type: string - description: | - This feature is set to describe how extensions are shipped in a containerized environment. - Typically they are part of the same Postgres container image, but they may also be distributed via other mechanisms. - vendor_compliance: | - Indicate 'built-in' if the extensions come included with the same Postgres container; or a succinct word or few words naming the distribution mechanism. - More details may be provided, if needed, via the links and comments fields. - - id: coext - name: Core/contrib extensions - type: integer - description: | - The number of core+contrib extensions provided by the operator for the latest Postgres version provided by the operator. - vendor_compliance: | - Submission must provide the total number of core+contrib extensions supported for the latest Postgres version provided by the operator. - It should also be provided, when available, a link with a detailed list of those extensions supported (via links field). - main: true - - id: thext - name: Third-party extensions - type: integer - description: | - The number of extensions not included in the Postgres core+contrib (i.e. created by third parties, outside of Postgres repository) for the latest Postgres version provided by the operator. - vendor_compliance: | - Submission must provide the total number of third-party extensions supported for the latest Postgres version provided by the operator. - A link with a detailed list of those extensions supported (via links field) should also be provided, if available. - main: true - - id: byext - name: Bring your own extension - type: boolean - description: | - The operator provides mechanisms for users to add (or upload) third-party extensions not initially provided by the operator. - vendor_compliance: | - The operator must support providing some mechanism to include custom extensions additionally or independently of the extension distribution mechanism, - if a user builds a local extension, this could be uploaded or included on the postgres container with an automatic copy to all replicas related. + value: false - id: bkup name: Backups features: From 76e9a52ed32aae856609e9e98113b43faca5ecb2 Mon Sep 17 00:00:00 2001 From: "itamar.marom" Date: Mon, 31 Jul 2023 22:52:17 +0300 Subject: [PATCH 5/5] druid updates --- apache-druid/spec/feature_matrix.yaml | 276 +++++++++++--------------- 1 file changed, 112 insertions(+), 164 deletions(-) diff --git a/apache-druid/spec/feature_matrix.yaml b/apache-druid/spec/feature_matrix.yaml index 8a8226c..0234cba 100644 --- a/apache-druid/spec/feature_matrix.yaml +++ b/apache-druid/spec/feature_matrix.yaml @@ -19,10 +19,9 @@ categories: * Helm * Kustomize * Operator Bundle - value: | - * Helm - * Kustomize - references: https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + value: ["Helm"] + references: + - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md - id: mcrds name: Management via CRDs type: boolean @@ -31,8 +30,8 @@ categories: vendor_compliance: | If true, provide a list of the names of the CRDs managed by the operator. Names should be direct links to their documentation or API reference. value: true - references: | - * druid.apache.org/v1alpha1/Druid - https://github.com/datainfrahq/druid-operator/blob/v1.1.0/apis/druid/v1alpha1/druid_types.go + references: + - druid.apache.org/v1alpha1/Druid - https://github.com/datainfrahq/druid-operator/blob/v1.1.0/apis/druid/v1alpha1/druid_types.go - id: offin name: Offline installation type: boolean @@ -41,7 +40,8 @@ categories: vendor_compliance: | A link should be provided with documentation on how to install the operator on air-gapped environments. value: true - references: Installation documentation - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md + references: + - https://github.com/datainfrahq/druid-operator/blob/master/docs/getting_started.md comments: Druid operator can be installed in air-gapped environment the same as with any other type of environment. - id: scpua name: Supported CPU architectures @@ -58,16 +58,14 @@ categories: * s390x In case you support additional architectures, add their name(s) in a similar fashion. - value: | - * amd64 + value: ["amd64"] - id: coios name: Container images OSes type: string_array description: | Indicate the base image of the images used by the operator. (e.g: scratch, ubi, fedora, ubuntu, centos, alpine, etc.). Provide the image name without tags (e.g. debian instead of debian:8.11). - value: | - * gcr.io/distroless/java$JDK_VERSION-debian11 + value: ["gcr.io/distroless/java$JDK_VERSION-debian11"] comments: Druid's base image is depends on the JDK version required - id: olmcl name: Operator Capability Level @@ -89,9 +87,9 @@ categories: For a detailed description of each capability level refer to the Operator Framework documentation for [Operator Capability Level](https://sdk.operatorframework.io/docs/overview/operator-capabilities/). - value: `Seamless Upgrades` - references: | - [Druid's feature list](https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md) + value: "Seamless Upgrades" + references: + - https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md - id: vers name: Versions features: @@ -104,8 +102,7 @@ categories: Provide a list of every supported major version followed by a list in parenthesis of the minor versions supported for that major version. E.g. '15 (15.1, 15.0), 14 (14.3, 14.2)'. All versions must be ordered by reverse chronological order. Other components versions that are used may be described in the comments (e.g: Patroni, WAL-G, PgBackRest, PgBouncer, etc.) - value: | - * 25 (25.0) + value: ["25 (25.0)"] comments: | Druid image is customizable thus you can use different versions. The version of the operator has been tested with this version of Druid @@ -117,8 +114,7 @@ categories: vendor_compliance: | Provide a list of official Kubernetes minor versions numbers, ordered by reverse chronological order. Use a range of versions, if possible, e.g. '1.26 - 1.23' or '1.22 - 1.19'. - value: | - 1.26 - 1.25 + value: "1.27 - 1.25" - id: lisu name: Licensing & Support features: @@ -165,7 +161,8 @@ categories: vendor_compliance: | Provide a specific link to the issue tracker. value: true - references: https://github.com/datainfrahq/druid-operator/issues + references: + - https://github.com/datainfrahq/druid-operator/issues - id: pubch name: Public chat/forums type: boolean @@ -174,7 +171,7 @@ categories: vendor_compliance: | Provide links to all relevant user forums. value: true - references: | + references: - druid-operator channel in Kubernetes Slack Workspace - https://kubernetes.slack.com/archives/C04F4M6HT2L - DataInfra slack workspace - https://launchpass.com/datainfra-workspace - id: drcl @@ -194,9 +191,7 @@ categories: Which Pod management technology is used to handle database's Pods. E.g. 'StatefulSet', 'Deployment', 'Custom', etc. vendor_compliance: | Provide the most succinct possible name of the technology. Use the Kubernetes resource type name (e.g. 'Deployment'), if applicable. - valur: | - - StatefulSet - - Deployment + valur: ["StatefulSet", "Deployment"] - id: drcnf name: Apache Druid custom config type: boolean @@ -213,7 +208,11 @@ categories: All should be valid towards this feature as long as they are deployed automatically and offer an entrypoint for the user to the connection pooler. value: true comments: | - this is not relevant to the operator itself. Druid configures it with defaults. You can override them through the operator + This is not relevant to the operator itself. + Druid configures it with defaults. + You can override them through the CRD. + references: + - https://druid.apache.org/docs/latest/operations/basic-cluster-tuning.html#connection-pool-guidelines - id: cpccf name: Connection pool custom config type: boolean @@ -227,14 +226,20 @@ categories: type: boolean description: | Connections can be secured with SSL/TLS support. - value: ??? + value: true + comments: | + This is not relevant to the operator itself. + Druid does not configure it by default. + You can configure it through the CRD. + references: + - https://druid.apache.org/docs/latest/operations/tls-support.html - id: tlscu name: TLS user-provided certificates type: boolean description: | Operators may chose by default to generate self-signed SSL certificates. They may also offer the option to specify the CA and certificates that users want the clusters to use. - value: ??? + value: true - id: crtmg name: CertManager integration type: boolean @@ -242,22 +247,20 @@ categories: The operator integrates with CertManager in order to generate the certificate to be used with. vendor_compliance: | Only applies if [drcl/tlscu] is true. - value: ??? + value: ?@adheip - id: inext name: Initialization from external source type: boolean description: | After the database cluster creation, the operator will automatically connect to an external data source (like an object storage or a public repo) and fetch the DDL/data. The operator must properly inform the user of the execution result of the scripts. - value: false - comments: | - This will be available in the near future with the Druid Ingestion controller + value: ?@adheip - id: mgmup name: Management of db users/passwords type: boolean description: | The operator provides a mechanism to declarative create, modify or delete users, roles and/or password for the cluster. - value: ??? + value: ?@adheip - id: reqli name: Customize computing resource requests, limits type: boolean @@ -270,7 +273,7 @@ categories: description: | The user may specify the request to use huge pages (and/or potentially for other sidecars). It must be able to be configured and start using huge pages. - value: ??? + value: ?@adheip - id: drsrv name: Apache Druid exposed via Service type: string @@ -299,6 +302,13 @@ categories: - https://github.com/datainfrahq/druid-operator/blob/f3f44e332e2d08ea463c1ca49517177a71bcf4b7/docs/features.md#volume-expansion-of-druid-nodes-running-as-statefulsets comments: | NOTE: This feature has been tested only on cloud environments and storage classes which have supported volume expansion. This feature uses cascade=orphan strategy to make sure only Stateful is deleted and recreated and pods are not deleted. + - id: dovol + name: Orphan Volume Deletion + type: boolean + description: | + When deleting and/or scaling a StatefulSet down, Kubernetes will not delete the volumes associated with the StatefulSet. + The operator allows to delete orphan volumes to handle cases like scaling down. + value: true - id: scal0 name: Scale down to zero type: boolean @@ -309,6 +319,14 @@ categories: comments: | Druid saves its data in a remote storage (like S3 bucket), in case of scaling to zero and then scaling back up, the data will be loaded back from the remote store. + - id: hascl + name: Horizontal Auto Scaling + type: boolean + description: | + The operator allow to configure horizontal pod autoscaling to its relevant components. + value: true + comments: | + The Druid operator allows to configure HPA to all its components. - id: cupdi name: Custom Apache Druid images type: boolean @@ -325,8 +343,18 @@ categories: The user may specify custom sidecars (containers or init containers) to be created alongside the system container (and, possibly, other operator sidecars). User supplied sidecars must be able, by default or by configuration, to access the Postgres container filesystem and Unix Domain Sockets file. value: true - references: | + references: - https://github.com/datainfrahq/druid-operator/blob/master/docs/features.md#Add-Additional-Containers-in-Druid-Nodes + - id: adcnt + name: User supplied additional containers + type: boolean + description: | + The operator supports adding additional containers to run along with the druid pods. This helps support + co-located, co-managed helper processes for the primary druid application. + This can be used for init containers, sidecars, proxies etc. + This is different then current Kubernetes implementation and will be part of Kubernetes in the near future: + https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/753-sidecar-containers/README.md#summary + value: true - id: usvol name: User supplied volumes type: boolean @@ -350,7 +378,7 @@ categories: the user to specify them manually in the application. vendor_compliance: | The operator must implement the [Provisioned Service](https://servicebinding.io/spec/core/1.0.0/#provisioned-service) part of the specification. - value: false + value: ?@adheip - id: cuann name: Custom Annotations type: boolean @@ -395,7 +423,7 @@ categories: features: - id: bktch name: Backup Technology - type: string + type: string_array description: | What technology (pgbasebackup, PgBackRest, WAL-e/g, Barman, custom, etc) is used to support creation and restoration of backups. vendor_compliance: | @@ -415,6 +443,13 @@ categories: * GCS * Azure Blob * Local file system + comments: | + Druid deep storage is not something to backup. + Druid can backup its metadata storage which can be on of the following: + - Apache Derby + - PostgreSQL + - MySQL + value: [] - id: autbk name: Automated backup management type: boolean @@ -423,12 +458,16 @@ categories: vendor_compliance: | True response for the feature implies both automatic backups and lifecycle management. If only the former is provided, answer should be false but this capability should be mentioned in the comments field. - main: true + comments: | + Druid deep storage is not something to backup. + Druid can backup its metadata storage. + main: false - id: encbk name: Backups encryption type: boolean description: | Backups can be performed with user-supplied encryption keys. + value: false - id: ptrbk name: Point In Time Recovery type: boolean @@ -438,6 +477,7 @@ categories: vendor_compliance: | True response implies that at least time-based recovery is supported. If the operator also supports PITR by xid and label, clarify in the comments field. + value: false - id: mulbk name: Multiple backup configurations type: boolean @@ -445,6 +485,7 @@ categories: The operator supports managing more than one backup configuration. This is typically used to store backups on different object stores (for protection purposes) or to send them to different sites. It may also include different lifecycle policies. + value: false - id: mgmi name: Management Interfaces features: @@ -452,7 +493,7 @@ categories: name: Resource display columns type: boolean description: | - Operator-managed objects (CRDs) include specific fields that provide additional information about the Postgres instances. + Operator-managed objects (CRDs) include specific fields that provide additional information about the CRD instances. These fields are shown when querying the Kubernetes resources using tools like `kubectl`. vendor_compliance: | In order to be compliant, operators have to provide at least the following information: @@ -462,6 +503,7 @@ categories: * Postgres major/minor version * Latest base backup date * SSL on/off + value: false - id: kbplg name: kubectl plugin type: boolean @@ -469,19 +511,20 @@ categories: The users may download an additional kubectl plugin which provides custom commands to manage the operator. vendor_compliance: | If the feature is provided, include a link to the downloadable plugin in the links field. + value: false - id: bagui name: Basic management GUI type: boolean description: | The operator bundles some GUI (web interface or other) that allows to perform basic operations (e.g. cluster creation) graphically. - main: true + value: false - id: fugui name: Fully-featured GUI type: boolean description: | The operator bundles a GUI that is fully-featured (that is, in feature-parity with the capabilities provided via the other management interfaces -- typically CRDs). The GUI may provide additional features that cannot be otherwise provided via text interfaces (e.g. graphical representation of resources or operations). - main: true + value: false - id: urgui name: GUI users & roles type: boolean @@ -490,6 +533,7 @@ categories: vendor_compliance: | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. Submission should detail in the link and/or comments field how users and roles for the GUI can be managed. + value: false - id: guiss name: GUI Single-Sign On type: boolean @@ -497,6 +541,7 @@ categories: The bundled management GUI supports integration with Single-Sign On (SSO), like OIDC, identity providers (e.g. Google, Github or Twitter login). vendor_compliance: | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + value: false - id: cogui name: GUI database console type: boolean @@ -504,6 +549,7 @@ categories: The bundled management GUI includes a console that can connect to any database managed by the operator and send command (e.g. like `psql`). vendor_compliance: | Only applies if [mgmi/bagui] or [mgmi/fugui] are provided. + value: false - id: o11y name: Observability features: @@ -511,7 +557,7 @@ categories: name: Metrics technology type: string_array description: | - If the operator supports extracting metrics from Postgres, define how they are handled, which technology receives and processes them. + If the operator supports extracting metrics from Druid, define how they are handled, which technology receives and processes them. If any of the following technology names apply, use them, and add other names, as needed: @@ -523,41 +569,47 @@ categories: vendor_compliance: | If supported, provide in the comments information about whether the technology is a dependency, is built-in, external, etc. Provide link(s) to the documentation for further information. - main: true + value: ?@adheip - id: expme name: Export metrics type: boolean description: | Regardless of how metrics are processed (e.g. as part of the operator), this feature is implemented when the operator allows the user to configure sending metrics to external services, like a metrics SaaS. + value: ?@adheip - id: cudas name: Custom dashboards type: boolean description: | - In order to display the captured Postgres metrics, the operator provides specialized Postgres dashboards for the users. + In order to display the captured metrics, the operator provides specialized dashboards for the users. + value: ?@adheip - id: cuale name: Custom alerts type: boolean description: | - The operator provides bundled specific Postgres alerts to be triggered on the Postgres metrics processed. - E.g. there is an alert for transaction wraparound or for unused replication slots. + The operator provides bundled specific alerts to be triggered on the metrics processed. + value: false - id: exdel name: Exposed decorated logs type: boolean description: | - The operator provides a mechanism to expose all the logs of the managed Postgres instances to a centralized logging tool. + The operator provides a mechanism to expose all the logs of the managed instances to a centralized logging tool. The logs must be decorated with extra metadata in order to provide semantic meaning, including the Pod name and namespace, the cluster name, the role of the Postgres instance (e.g. primary, replica, standby-leader, etc.) and the timestamp that will be available to be used to filter logs entries. There is no need to configure the tool in order to obtain required extra metadata from the logs. + value: ?@adheip - id: explg name: Export logs type: boolean description: | - The operator allows the user to configure an external sink for the Postgres logs (e.g. a SaaS service). + The operator allows the user to configure an external sink for the logs (e.g. a SaaS service). + value: ?@adheip - id: oo11y name: Operator Observability type: boolean description: | The operator is itself a source of telemetry data, potentially including metrics, traces and logs, about its own performance. + value: true + comments: Provides Kubebuilder built-in metrics & logs - id: secy name: Security features: @@ -566,13 +618,13 @@ categories: type: boolean description: | The operator code is always scanned for security vulnerabilities. - main: true + value: false - id: imgsc name: Image scanning type: boolean description: | Operator-provided images are always scanned for security vulnerabilities. - main: true + value: false - id: sigim name: Signed images type: boolean @@ -580,6 +632,7 @@ categories: Container images are digitally signed according to the [sigstore](https://www.sigstore.dev/) project. vendor_compliance: | Compliance may also be achieved by using a technology other than sigstore, as long as it provides an equivalent set of security capabilities. + value: ?@adheip - id: isbom name: Software Bill of Materials type: boolean @@ -587,11 +640,13 @@ categories: The operator releases include the SBOM (Software Bill of Materials), a detailed description of all the components, modules, and their dependencies. vendor_compliance: | SBOM is expected to be in accordance to the [Kubernetes SIG BOM](https://github.com/kubernetes-sigs/bom). + value: false - id: fgopp name: Fine-grained RBAC permissions type: boolean description: | The operator uses a separate serviceaccount that has RBAC permissions that only require the access that is actually needed to create and manage the Kubernetes resources, not more. + value: true - id: noprm name: No or justified privileged mode type: boolean @@ -600,6 +655,7 @@ categories: The container processes do not run as root. vendor_compliance: | Reasonable exceptions to this rule can be made for features that require or do not diminish the container's security, e.g. when using eBPF. + value: true - id: day2 name: Day 2 Operations features: @@ -613,7 +669,7 @@ categories: The operator must provide proper information to the user as to the status and final result of the operation. The operator should provide ongoing status information, and perform the operation with the minimum downtime required. Provide information about the update strategy (i.e. restart of the pods or rolling update followed by a switchover or a restart). - main: true + value: ?@adheip - id: amaup name: Automated major upgrades type: boolean @@ -623,32 +679,21 @@ categories: vendor_compliance: | The operator must provide proper information to the user as to the status and final result of the operation. The operator should provide ongoing status information, and perform the operation with the minimum downtime required. - main: true + value: ?@adheip - id: crest name: Controlled cluster restart type: boolean description: | - Sometimes Postgres needs to be restarted (e.g. changing of a parameter that requires restart). - The operator provides means to perform this operation automatically and in a controlled manner (rolling restart) so that the cluster faces a minimal downtime only. + Sometimes Druid needs to be restarted (e.g. changing of a parameter that requires restart). + The operator provides means to perform this operation automatically and in a controlled manner + (rolling restart / custom) so that the cluster faces a minimal to non downtime only. + value: true - id: ociup name: Container images upgrade type: boolean description: | Similarly to the controlled restart operation, the operator is capable of updating the running container images (which require a pod restart) automatically and with minimal cluster impact. - - id: swtch - name: Switchover - type: boolean - description: | - If HA capabilities are provided, the operator also provides a mechanism for manual switchover. - The user may specify the configuration declaratively and the operator will perform the desired switchover automatically, by demoting the current primary, promoting the a replica, and updating the endpoints/services as required. - - id: sqlmi - name: SQL Migrations - type: boolean - description: | - The operator provides managed SQL migration capabilities. - The user may specify SQL scripts that contain migrations (DDL changes, etc) to be deployed to a given database, having the operator apply them automatically. - vendor_compliance: | - The operator must report back to the user detailed information about the results of the execution(s) of the script(s) provided by the user. + value: true - id: oday2 name: Other Day 2 Operations type: string_array @@ -656,101 +701,4 @@ categories: The operator provides support for other managed Day 2 operations. vendor_compliance: | All the mentioned additional day 2 operations need to be possible via declarative configuration and the operator to fully execute them without further user intervention. - - id: dain - name: Data Integration - features: - - id: kfkin - name: Kafka integration - type: boolean - description: | - The operator provides a managed solution for automation to export CDC (Change Data Capture) events to Kafka. - vendor_compliance: | - Kafka could be an external dependency to the operator. - The whole operation is expected to be fully automated, with the user only providing a declarative configuration. - - id: migpg - name: Migration from Postgres - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from another (external) Postgres database. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - vendor_compliance: | - Provide one or more of the following values: `offline`, `cdc`. - - id: migmy - name: Migration from MySQL - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from a MySQL database. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. - vendor_compliance: | - Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. - - id: migor - name: Migration from Oracle - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from an Oracle database. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. - vendor_compliance: | - Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. - - id: migms - name: Migration from SQL Server - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from a SQL Server database. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. - vendor_compliance: | - Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. - - id: migmo - name: Migration from MongoDB - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from a MongoDB database. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. - vendor_compliance: | - Provide one or more of the following values: `offline`, `cdc`, `schema conversion`. - - id: migot - name: Migration from other databases - type: string_array - description: | - The operator provides a managed capability to automatically migrate data from other relational or non-relational databases. - Migrations may be offline (source needs to be disconnected from clients while the migration happens) and/or via CDC (Change Data Capture), - which allows for near-zero downtime migrations. - Optionally, the operator may provide additional capabilities to convert the schema from the source database into the target Postgres database. - vendor_compliance: | - Provide the names of the other database from where migration is supported. - - id: shrd - name: Sharding - features: - - id: mshrd - name: Managed sharding support - type: boolean - description: | - The operator understands horizontal scalability and is capable to automatically deploying sets of clusters (shards) with their accompanying infrastructure (coordinators, services, etc). - The user should be able to write a declarative configuration of a sharded cluster and the operator to fully deploy it without any further manual user intervention. - main: true - - id: shrdt - name: Sharding technology - type: string_array - description: | - If managed sharding support is provided, which technology or technologies are used to provide sharding support (which is not included in Postgres core). - - id: aggds - name: Aggregated monitoring dashboards - type: boolean - description: | - Since a sharded cluster is a set of (independent) Postgres clusters, an aggregated dashboard is one that provides Observability information across all shards of the cluster. - E.g. total database size, tps processed or total number of client connections. - - id: cobkp - name: Coordinated backups - type: boolean - description: | - A sharded cluster can be backed up by backing up each shard independently, but those backups are not necessarily consistent with each other. - If sharding is used, an additional capability is to coordinate backups on a given timestamp so that they are performed or can be restored at the same logical timestamp, - so they become consistent with each other. + value: []