HIP: adding expose deployed chart

hips/README.md

@@ -37,3 +37,4 @@ restricted markdown format and can be found in the
 - [hip-0025: Better Support for Resource Creation Sequencing](hip-0025.md)
 - [hip-0026: H4HIP: Wasm plugin system](hip-0026.md)
 - [hip-0027: Bring .helmignore to parity with .gitignore file targeting syntax](hip-0027.md)
+- [hip-0029: Expose Release History During Template Rendering](hip-0029.md)

hips/hip-0029.md

@@ -0,0 +1,306 @@
+---
+hip: 0029
+title: "Expose Release History During Template Rendering"
+authors: ["Andrew Shoell <mrlunchbox777@gmail.com>"]
+created: "2025-11-12"
+type: "feature"
+status: "draft"
+---
+
+## Abstract
+
+This HIP proposes exposing release history metadata during template rendering. Currently, Helm templates have access to `.Chart` for the chart being installed but no equivalent access to deployed release history. This forces chart authors to use complex workarounds like post-renderers, pre-upgrade hooks, or manual values conventions to implement version-aware upgrade logic.
+
+The proposal introduces `.Release.History` (array of historical releases) available in template contexts, populated during `helm upgrade` and `helm rollback` operations (including `--dry-run=server`) when the `--release-history-max` flag is provided. The flag controls how many historical releases to retrieve (default: 0), requiring explicit opt-in. Chart authors can check `len .Release.History` to determine if sufficient historical data is available and use `.Release.Revision` (or compare `len .Release.History` to expected minimum) to detect upgrade scenarios requiring migration logic.
+
+## Motivation
+
+### Current Limitations
+
+Helm provides comprehensive chart metadata through `.Chart` but offers no native way to access deployed release metadata during template evaluation. Chart developers must resort to problematic workarounds:
+
+**Post-Renderers:** External tools that query the cluster, parse manifests, and make version-aware modifications. This moves upgrade logic outside the chart, requires additional tooling, and breaks Helm's self-contained design.
+
+**Pre-Upgrade Hooks:** Store version metadata in ConfigMaps via hooks, creating ordering dependencies and potential failure points.
+
+**Manual Values:** Require users to specify previous versions in values files—error-prone and defeats Helm's release tracking.
+
+### Real-World Impact
+
+This limitation prevents or complicates legitimate use cases:
+
+- **Breaking Changes:** No clean migration path for renamed resources or changed structures
+- **Conditional Resources:** Cannot create migration Jobs based on version deltas
+- **Smart Defaults:** Cannot distinguish fresh installs from upgrades for intelligent defaults
+- **Advanced Deployments:** Blue-green and similar strategies require external orchestration
+
+Post-rendering solutions violate Helm's design philosophy that template rendering should be deterministic and self-contained. Making deployed chart metadata available at template time keeps upgrade logic in the chart itself, maintaining Helm's portability, testability, and transparency.
+
+## Rationale
+
+### Naming: `.Release.History`
+
+`.Release.History` extends the existing `.Release` built-in object with a history array, making it immediately intuitive and discoverable for Helm users. This follows the established pattern of `.Release.Name`, `.Release.Namespace`, `.Release.Revision`, etc., and feels like a natural part of Helm's API.
+
+The history array is ordered in reverse chronological order (index 0 is most recent deployed release). This provides ergonomic access to the most recent release while enabling multi-version migrations when needed.
+
+Alternatives considered and rejected:
+
+- `.PreviousChart` - Ambiguous during rollbacks
+- `.DeployedChart`/`.DeployedCharts` - Less discoverable, doesn't extend existing objects
+- `.InstalledChart` - Confusing with current installation
+- `.CurrentChart` - Ambiguous which is "current"
+- `.Release.Deployed.Chart` - Unnecessarily nested
+
+### Always Available as Template Object
+
+`.Release.History` (empty array or populated) is always present to ensure consistent template behavior, prevent undefined variable errors, and enable testing with `helm template`.
+
+### Populated During Upgrades/Rollbacks (and Server Dry-Runs)
+
+`.Release.History` contains release metadata during `helm upgrade` and `helm rollback` when deployed releases exist and `--release-history-max` is greater than 0. During rollback, history follows Helm's normal revision behavior: rollback creates a new revision and prior revisions remain in history.
+
+For `--dry-run=server`, behavior matches the corresponding live command because cluster context is available (for example, `helm upgrade --dry-run=server --release-history-max N` can populate `.Release.History`). It's empty for:
+
+- `helm install` - No deployed release
+- `helm template` and `--dry-run=client` - No cluster context
+- When `--release-history-max 0` is used (default)
+
+### Filtered Release Data
+
+This proposal exposes a filtered subset of release data to balance utility with performance and security:
+
+**Included by default:**
+
+- Chart metadata (Name, Version, AppVersion, and other Chart.yaml fields)
+- Release metadata (Name, Namespace, Revision, Status)
+- Timestamps (FirstDeployed, LastDeployed)
+
+**Excluded by default (opt-in via flag):**
+
+- Values (may contain secrets; use `--include-history-values` to include)
+- Manifests (can be very large; future consideration)
+- Templates (can be very large; no clear use case - templates are static per chart version)
+- Hooks (implementation detail)
+
+The conservative default excludes potentially sensitive or large data. Users who need historical values for complex migration scenarios can opt-in explicitly with `--include-history-values`, accepting the security and performance tradeoffs. See Security Implications section for detailed rationale.
+
+**Future consideration:** Historical manifests could be made available via `--include-history-manifests` if demand exists, though manifests can be quite large and increase memory/performance overhead significantly.
+
+### Max Control Flag
+
+The `--release-history-max` flag controls how many historical releases to retrieve (default: 0, requiring explicit opt-in). This conservative default protects users from accidental performance impact. Setting `--release-history-max 0` explicitly disables the feature (though this is already the default). Higher max values may impact performance and should only be used for specific multi-version migration scenarios.
+
+`--release-history-max` is a retrieval limit, not a guarantee that many revisions exist. Template checks should rely on actual available data (`len .Release.History`).
+
+### Design Decisions
+
+- **Different Chart Names:** Still populates `.Release.History` even if chart names differ—templates can detect and handle this
+- **Helm's Record:** Reflects Helm's stored release record, not actual cluster state (use `lookup()` for that)
+- **Relationship to `helm history`:** `.Release.History` is derived from the same release records used by `helm history <release>`, but exposed as a reverse-chronological, template-friendly projection with field filtering and `--release-history-max` limits.
+- **Dry-Run Behavior:** `helm template` and `--dry-run=client` always return an empty array (cluster-agnostic). `--dry-run=server` follows live command behavior and can populate history when available.
+- **Opt-In by Default:** Default max of 0 requires explicit user choice, preventing accidental performance impact
+
+## Specification
+
+### New Template Objects
+
+**`.Release.History`**: Array of release objects in reverse chronological order (most recent first). Empty array if no deployed releases exist or `--release-history-max 0` is used (default).
+
+**Note:** Use `.Release.Revision` to detect if this is an upgrade (revision > 1) and `len .Release.History` to check how much historical data is available.
+
+**Note:** Template logic should rely on actual availability (`len .Release.History`), not requested max. Requested max may be larger or smaller than existing revisions.
+
+**Note:** `.Release.Revision` provides an upper bound on possible prior history (`revision - 1`). Chart maintainers can combine `.Release.Revision` with `len .Release.History` for stricter migration checks.
+
+**Note:** `HistoricalRelease` is a runtime projection created for template rendering from Helm's stored release records. It is not a new persisted release schema.
+
+**Release Object Structure:** Each release object contains:
+
+```go
+type HistoricalRelease struct {
+    Name         string    // Release name
+    Namespace    string    // Release namespace
+    Revision     int       // Revision number
+    Status       string    // Release status (deployed, superseded, failed, etc.)
+    Chart        Chart     // Chart metadata (same structure as .Chart)
+    Values       map[string]interface{} // Optional: only included when --include-history-values is set
+    FirstDeployed time.Time // When this release was first deployed
+    LastDeployed  time.Time // When this release was last deployed
+}
+```
+
+**Usage Examples:**
+
+```yaml
+# Check if upgrading from a version that needs migration
+{{- if gt (len .Release.History) 0 }}
+{{- $lastRelease := index .Release.History 0 }}
+{{- if and (semverCompare ">=2.0.0" .Chart.Version) (semverCompare "<2.0.0" $lastRelease.Chart.Version) }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ include "mychart.fullname" . }}-migration
+  annotations:
+    "helm.sh/hook": pre-upgrade
+spec:
+  template:
+    spec:
+      containers:
+      - name: migrate
+        image: myapp/migrator:{{ .Chart.AppVersion }}
+        command: ["migrate", "v1-to-v2"]
+{{- end }}
+{{- end }}
+
+# Require minimum history for safe upgrades
+{{- if and (gt .Release.Revision 1) (lt (len .Release.History) 3) }}
+  {{- fail "This chart requires --release-history-max=3 for safe upgrades from v2.x" }}
+{{- end }}
+
+# Multi-version migration: handle complex upgrade paths
+{{- range .Release.History }}
+{{- if and (eq .Status "deployed") (semverCompare "<1.5.0" .Chart.Version) }}
+# Run migration for successfully deployed versions before 1.5.0
+{{- end }}
+{{- end }}
+
+# Check if last deployment was successful
+{{- if gt (len .Release.History) 0 }}
+{{- $lastRelease := index .Release.History 0 }}
+{{- if ne $lastRelease.Status "deployed" }}
+  {{- fail (printf "Previous release was %s - manual intervention required" $lastRelease.Status) }}
+{{- end }}
+{{- end }}
+```
+
+### Command-Line Flag
+
+```bash
+# Default: no history retrieved (max 0)
+helm upgrade myrelease mychart
+
+# Retrieve latest deployed release
+helm upgrade myrelease mychart --release-history-max 1
+
+# Retrieve last 3 releases
+helm upgrade myrelease mychart --release-history-max 3
+
+# Explicitly disable (same as default)
+helm upgrade myrelease mychart --release-history-max 0
+```
+
+### Behavior Matrix
+
+The following table shows what values are available in template context for different operations:
+
+| Operation                                                | `.Release.History`                    | `.Release.Revision` |
+| -------------------------------------------------------- | ------------------------------------- | ------------------- |
+| `helm install`                                           | `[]`                                  | 1                   |
+| `helm upgrade` (first)                                   | `[]` (default, no flag)               | 2                   |
+| `helm upgrade --release-history-max 1` (first)           | Populated with 1 release              | 2                   |
+| `helm upgrade --release-history-max N`                   | Up to N releases                      | varies              |
+| `helm rollback --release-history-max N`                  | Populated with releases before target | varies              |
+| `helm upgrade --dry-run=server --release-history-max N`  | Same as live `upgrade`                | varies              |
+| `helm rollback --dry-run=server --release-history-max N` | Same as live `rollback`               | varies              |
+| `helm install --dry-run=server`                          | `[]`                                  | 1                   |
+| `helm template` / `--dry-run=client`                     | `[]`                                  | 1                   |
+
+**Note:** Use `.Release.Revision` to distinguish installs (revision=1) from upgrades (revision>1).
+
+**Note:** `.Release.History` and `helm history <release>` both come from Helm's stored release records. For template ergonomics, `.Release.History` is reverse chronological (most recent first), filtered to HIP-defined fields, and capped by `--release-history-max`. During rollback, Helm creates a new revision; prior revisions remain in history.
+
+## Backwards Compatibility
+
+Fully backwards compatible. The `.Release.History` field is purely additive—existing charts work unchanged. Go templates handle empty arrays safely; the recommended `{{ if gt (len .Release.History) 0 }}` pattern works in all scenarios. Default max of 0 means existing behavior is unchanged unless users explicitly opt in.
+
+## Security Implications
+
+**Not Exposed:** Previous values (may contain secrets) or previous manifests (may contain sensitive data). Only filtered release metadata is exposed (chart metadata, release status, timestamps, revision numbers).
+
+**Considerations:** Chart authors should not store sensitive data in Chart.yaml. The default `--release-history-max 0` provides opt-out by default. Higher max values increase data exposure; use the minimum required. Release status and metadata are already stored in cluster secrets by Helm, so this doesn't expose data that isn't already persisted.
+
+## How to Teach This
+
+### Documentation Additions
+
+1. **Template Objects Reference:** Add `.Release.History` to built-in objects documentation with availability details and usage patterns with `.Release.Revision`
+2. **Upgrade Guide:** "Implementing Version-Aware Upgrades" covering empty array checks, version comparisons, status checking, and best practices
+3. **Migration Examples:** Show replacement of post-renderers and pre-upgrade hooks, including use of opt-in flag for values when needed, with practical patterns (check last release, check last successful)
+4. **Performance Note:** Document that `--release-history-max` should be kept minimal; opt-in by default protects users
+5. **Chart Linting:** Update `helm lint` to warn on `.Release.History` usage without empty array checks, and suggest using `.Release.Revision` for upgrade detection
+6. **Security Guide:** Document the opt-in flag `--include-history-values` with clear warnings about security implications
+
+### Key Example Pattern
+
+```yaml
+# Pattern 1: Defensive check before accessing history
+{{- if gt (len .Release.History) 0 }}
+{{- $lastRelease := index .Release.History 0 }}
+{{- if semverCompare "<3.0.0" $lastRelease.Chart.Version }}
+# Handle breaking change from versions < 3.0.0
+{{- end }}
+{{- end }}
+
+# Pattern 2: Check only last successful deployment
+{{- $lastSuccessful := dict }}
+{{- range .Release.History }}
+  {{- if eq .Status "deployed" }}
+    {{- $lastSuccessful = . }}
+    {{- break }}
+  {{- end }}
+{{- end }}
+{{- if $lastSuccessful }}
+  # Use $lastSuccessful for migration logic
+{{- end }}
+
+# Pattern 3: Require history for upgrades (not installs)
+{{- if gt .Release.Revision 1 }}
+  {{- if eq (len .Release.History) 0 }}
+    {{- fail "Upgrades require --release-history-max=1 for continuity checks" }}
+  {{- end }}
+{{- end }}
+
+# Pattern 4: Check for sufficient history depth (less common)
+{{- if and (gt .Release.Revision 1) (lt (len .Release.History) 2) }}
+  {{- fail "This complex migration requires --release-history-max=2 for full validation" }}
+{{- end }}
+```
+
+**Guidance:** Treat `--release-history-max` as a retrieval cap, not a guarantee. Use `.Release.Revision` to detect install vs upgrade and `len .Release.History` to validate actual available history.
+
+## Reference Implementation
+
+A future pull request will:
+
+1. Extend template rendering context to include `.Release.History`
+2. Populate `.Release.History` from release records during upgrade/rollback, including `--dry-run=server` parity (reverse chronological order)
+3. Add `--release-history-max` flag (default: 0)
+4. Add opt-in flag: `--include-history-values`
+5. Filter release objects by default to include: Chart, Name, Namespace, Revision, Status, FirstDeployed, LastDeployed
+6. When opt-in flag is used, include Values in historical releases
+7. Include comprehensive unit and integration tests covering flag behavior, filtering, and edge cases
+
+## Rejected Ideas
+
+- **Full Release Object:** Security/performance concerns; filtered release metadata sufficient
+- **Chart Metadata Only:** Missing release status/revision limits utility for migration logic
+- **Only Version Strings:** Inconsistent with `.Chart`; prevents access to other metadata
+- **`.DeployedChart`/`.DeployedCharts` Naming:** Less discoverable than extending `.Release` object
+- **Default Max of 1:** Opt-in by default (max 0) is more conservative and safer
+- **Environment Variable Control:** Less explicit than CLI flag
+- **Cluster Query During `helm template`:** Violates cluster-agnostic design principle
+- **Mutable Objects:** Violates read-only template model; no clear use case
+- **Separate `--disable-release-history` Flag:** Unified `--release-history-max` with 0 value is cleaner
+- **Unlimited History:** Performance implications; requiring explicit max prevents accidental overhead
+- **Including Values/Manifests by Default:** While historical values could be useful for migration scenarios, and users already have access via `helm get values --revision N` or `lookup()`, making them automatically available in templates creates additional surface area for accidental exposure. The filtered metadata approach with opt-in flag (`--include-history-values`) serves both conservative defaults and advanced use cases. Manifests moved to future consideration (`--include-history-manifests`) as they are very large and less commonly needed.
+- **Including Templates:** Templates are static per chart version; if you need old templates, retrieve the old chart version. No flag needed.
+- **`.Release.HistoryMax` Field:** Redundant with `len .Release.History` and `.Release.Revision`. Chart authors can use `.Release.Revision > 1` to detect upgrades and `len .Release.History` to check available data.
+
+## References
+
+- [Helm Built-in Objects](https://helm.sh/docs/chart_template_guide/builtin_objects/)
+- [Helm Chart.yaml](https://helm.sh/docs/topics/charts/#the-chartyaml-file)
+- [Go Templates](https://pkg.go.dev/text/template)
+- [Semantic Versioning](https://semver.org/)
+- [Example of current workaround](https://github.com/helm/community/pull/421#issuecomment-3662769874)