From 3e40c80c8041c7fa16b2d26313c364a5daa899fe Mon Sep 17 00:00:00 2001 From: Sam Crauwels Date: Thu, 12 Mar 2026 20:14:35 +0100 Subject: [PATCH 1/2] Update docs: cert format detection is content-based, not extension-based The cert_validate.yml shared task probes files with openssl to detect PEM vs PKCS12 format rather than relying on file extensions. The docs in several places incorrectly stated the format was auto-detected from the extension. Also adds cert_validate.yml to the shared certificate tasks table in the elasticstack role docs, where it was missing. --- docs/architecture.md | 2 +- docs/roles/elasticsearch.md | 2 +- docs/roles/elasticstack.md | 1 + docs/roles/kibana.md | 2 +- 4 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 8b62e66c..e34c6432 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -91,7 +91,7 @@ Certificate renewal is handled automatically: each role checks certificate expir ### External certificates -All roles support `*_cert_source: external` to use certificates from any CA (corporate, ACME, Vault PKI). External certs can be provided as file paths or as inline PEM content in variables. The format (PEM vs PKCS12) is auto-detected for file paths; content mode is always PEM. +All roles support `*_cert_source: external` to use certificates from any CA (corporate, ACME, Vault PKI). External certs can be provided as file paths or as inline PEM content in variables. For file paths, the format (PEM vs PKCS12) is auto-detected by probing the file content with `openssl` — not from the file extension. Content mode is always PEM. Elasticsearch supports separate transport and HTTP layer certificates — useful when transport uses an internal CA while HTTP uses a public ACME cert. HTTP falls back to transport if not specified. diff --git a/docs/roles/elasticsearch.md b/docs/roles/elasticsearch.md index 59ee995a..4fa2cc60 100644 --- a/docs/roles/elasticsearch.md +++ b/docs/roles/elasticsearch.md @@ -306,7 +306,7 @@ elasticsearch_tls_remote_src: false elasticsearch_http_ssl_keystore_path: "" ``` -`elasticsearch_transport_tls_certificate` is the path to the transport layer (port 9300) TLS certificate. Accepts PEM (`.crt`, `.pem`) or PKCS12 (`.p12`, `.pfx`) -- the format is auto-detected from the file extension. +`elasticsearch_transport_tls_certificate` is the path to the transport layer (port 9300) TLS certificate. Accepts PEM (`.crt`, `.pem`) or PKCS12 (`.p12`, `.pfx`) -- the format is auto-detected by probing the file content with `openssl` (not from the file extension). `elasticsearch_transport_tls_key` is the transport private key. For PEM format, this is auto-derived from the certificate path (`.crt` replaced with `.key`) if left empty. Ignored for P12 bundles where the key is included in the keystore. diff --git a/docs/roles/elasticstack.md b/docs/roles/elasticstack.md index ed64b781..74896bf3 100644 --- a/docs/roles/elasticstack.md +++ b/docs/roles/elasticstack.md @@ -211,6 +211,7 @@ The `certs/` directory contains reusable tasks imported by all roles: | `cert_check_expiry.yml` | Check certificate expiry and set `*_will_expire_soon` fact | | `cert_backup.yml` | Create timestamped backup of certificates before renewal | | `cert_distribute.yml` | Fetch cert from CA host to controller, then copy to target node | +| `cert_validate.yml` | Validate external certificates: format detection (PEM/P12 via openssl probing), key match, expiry check, SAN warning | | `fetch_password.yml` | Extract a user's password from the initial passwords file | ## Tags diff --git a/docs/roles/kibana.md b/docs/roles/kibana.md index dce0a997..8da2f35f 100644 --- a/docs/roles/kibana.md +++ b/docs/roles/kibana.md @@ -137,7 +137,7 @@ kibana_tls_ca_file: "" kibana_tls_remote_src: false ``` -`kibana_tls_certificate_file` is the path to a TLS certificate in PEM (`.crt`/`.pem`) or PKCS12 (`.p12`/`.pfx`) format. The format is auto-detected from the file extension. +`kibana_tls_certificate_file` is the path to a TLS certificate in PEM (`.crt`/`.pem`) or PKCS12 (`.p12`/`.pfx`) format. The format is auto-detected by probing the file content with `openssl` (not from the file extension). `kibana_tls_key_file` is the path to the corresponding private key. For PEM certificates, if left empty, the role derives the key path from the certificate path by changing the extension. For PKCS12, this is ignored (the key is inside the P12 file). From 3b9a85e57aa3c870f1b8da30b3e0cc18f64163c7 Mon Sep 17 00:00:00 2001 From: Sam Crauwels Date: Thu, 12 Mar 2026 20:17:43 +0100 Subject: [PATCH 2/2] Improve docs for first-time users The getting-started guide now explains how to retrieve initial passwords and where they live in multi-node deployments. The security-disabling section correctly shows that elasticstack_security cascades in full-stack mode. Added a note that elasticstack_release defaults to 8 since the examples all use 9. Beats security auto-inheritance from the global flag is now in the main description instead of buried in a note. ES heap docs mention container cgroup recalculation. --- docs/getting-started.md | 18 +++++++++++++++--- docs/roles/beats.md | 7 ++----- docs/roles/elasticsearch.md | 2 +- 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/docs/getting-started.md b/docs/getting-started.md index aaa2f297..7941043a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -21,6 +21,9 @@ collections: - name: oddly.elasticstack ``` +!!! note + The default `elasticstack_release` is `8`. All examples below explicitly set it to `9` for new deployments. If you omit it, the collection installs Elastic 8.x packages. + ## Single-node deployment (simplest) This deploys everything on one host — useful for development and testing. @@ -53,7 +56,13 @@ all: ansible-playbook -i inventory.yml playbook.yml ``` -After the run completes, Elasticsearch will be listening on `https://localhost:9200` with security enabled. The initial passwords are stored in `/usr/share/elasticsearch/initial_passwords` on the host. +After the run completes, Elasticsearch will be listening on `https://localhost:9200` with security enabled. The `elastic` superuser password and all built-in user passwords are stored in `/usr/share/elasticsearch/initial_passwords` on the host. You can retrieve the `elastic` password with: + +```bash +grep "PASSWORD elastic" /usr/share/elasticsearch/initial_passwords | awk '{print $4}' +``` + +In multi-node deployments, this file lives on the CA host (the first node in the `elasticsearch` group) and other roles fetch passwords from it automatically via `delegate_to`. ## Multi-node full-stack deployment @@ -110,12 +119,15 @@ For internal networks or development environments where TLS is not needed: ```yaml elasticstack_security: false +``` + +In full-stack mode (`elasticstack_full_stack: true`), this cascades to all roles — Elasticsearch, Kibana, Logstash, and Beats all inherit the setting. If you are running roles individually (`elasticstack_full_stack: false`), you also need to set the per-role flags: + +```yaml elasticsearch_security: false beats_security: false ``` -This disables TLS, authentication, and HTTPS across all roles. - ## Using a package mirror If your hosts can't reach `artifacts.elastic.co`, point the repo at a local mirror: diff --git a/docs/roles/beats.md b/docs/roles/beats.md index 7fad26c1..a8985fd7 100644 --- a/docs/roles/beats.md +++ b/docs/roles/beats.md @@ -68,16 +68,13 @@ beats_security: false beats_ssl_verification_mode: certificate ``` -`beats_security` enables TLS for all Beat-to-Elasticsearch and Beat-to-Logstash connections. When enabled, each Beat uses client certificates to authenticate. +`beats_security` enables TLS for all Beat-to-Elasticsearch and Beat-to-Logstash connections. When enabled, each Beat uses client certificates to authenticate. Although the default is `false`, in full-stack mode (`elasticstack_full_stack: true`) this is automatically set to `true` when `elasticstack_security` is `true`. Set `elasticstack_override_beats_tls: true` to prevent this automatic inheritance. -`beats_ssl_verification_mode` controls the TLS verification mode used for Elasticsearch output connections. Valid values are `full` (verify certificate and hostname), `certificate` (verify certificate only), and `none` (skip verification). Logstash output always uses `full` verification regardless of this setting. +`beats_ssl_verification_mode` controls the TLS verification mode used for Elasticsearch output connections when `beats_security` is enabled. Valid values are `full` (verify certificate and hostname), `certificate` (verify certificate only), and `none` (skip verification). Logstash output always uses `full` verification regardless of this setting. !!! warning The default passphrase `BeatsChangeMe` (see `beats_tls_key_passphrase` below) should be changed in any non-test deployment. -!!! note - In full-stack mode (`elasticstack_full_stack: true`), `beats_security` is automatically set to `true` when `elasticstack_security` is `true`. Set `elasticstack_override_beats_tls: true` to prevent this automatic inheritance. - ### Logging These settings apply to all three Beat types (Filebeat, Metricbeat, Auditbeat). diff --git a/docs/roles/elasticsearch.md b/docs/roles/elasticsearch.md index 4fa2cc60..6642f77a 100644 --- a/docs/roles/elasticsearch.md +++ b/docs/roles/elasticsearch.md @@ -174,7 +174,7 @@ elasticsearch_jna_workaround: false elasticsearch_memory_lock: false ``` -`elasticsearch_heap` is the JVM heap size in gigabytes. Both `-Xms` and `-Xmx` are set to this value (Elastic recommends equal initial and maximum heap). The default formula calculates half of system RAM, capped at 30 GB, with a floor of 1 GB. The 30 GB cap keeps the JVM within compressed ordinary object pointers (oops) territory. For production nodes, 8-16 GB is typical; the default of 2 GB (on a 4 GB host) is fine for development. Override with an integer: `elasticsearch_heap: 16`. +`elasticsearch_heap` is the JVM heap size in gigabytes. Both `-Xms` and `-Xmx` are set to this value (Elastic recommends equal initial and maximum heap). The default formula calculates half of system RAM, capped at 30 GB, with a floor of 1 GB. The 30 GB cap keeps the JVM within compressed ordinary object pointers (oops) territory. In containers with a cgroup memory limit lower than the host's physical RAM, the role recalculates heap from the cgroup limit instead. For production nodes, 8-16 GB is typical; the default of 2 GB (on a 4 GB host) is fine for development. Override with an integer: `elasticsearch_heap: 16`. !!! tip Set `elasticsearch_check_calculation: true` to print the calculated heap value during a run without making any changes. Useful for verifying auto-calculation on heterogeneous hardware before committing.