feat: Cross-track upgrades by MichaelThamm · Pull Request #174 · canonical/observability-stack

MichaelThamm · 2026-01-15T14:20:49Z

Should every charm have a juju_charm datasource i.e., no upgrades.tf file, or only one if there is a breaking change i.e., a upgrades.tf file.
Remove the manual refreshing from our upgrade itests
- Should we add any other kind of tests?
Add this feature for COS, once COS Lite is robust

Blocked by:

[Documentation] "Cannot upgrade application X: would break relation" juju/terraform-provider-juju#1022
- howto/manage-charms/#update-a-charm-when-a-relation-would-break
juju charm data source juju/terraform-provider-juju#1042
Failure to remove an oci-resource from a kubernetes based charm juju/juju#21648
- This will likely not be resolved any time soon, so we can have a lifecycle for the grafana application to remove it in case an upgrade takes place from track 2 to 3. We would not need this in future tracks.

This PR will need coordination with:

Relates to:

"Unable to create integration, got error: no relations found" with the current terraform deployment. #165

Issue

We want a smooth UX between the TF provider and the Juju client for upgrading the product's channels.

Solution

Use the juju_charm datasource to provide the latest revisions for a charm in a specific track.
Use lifecycle {replace_triggered_by = [terraform_data.grafana_ingress_interface]} to replace the juju_integration in the event that the interface changes.

Checklist

Convert the doc to a How-to
I have added or updated relevant documentation.
- Add an upgrade guide
- Update the charmhub source to a GH source
Add the charmhub module to COS
How to get past the "relation will break" Juju client errors so upgrades are more seamless. Lifecycle resources?
PR title makes an appropriate release note and follows conventional commits syntax.
Merge target is the correct branch, and relevant tandem backport PRs opened.

Context

In the future charms will have unique tracks which the products needs to map to:

Tagging for TF modules in charm repos observability#427

Testing Instructions

COS Lite manual

Deploy COS Lite 2/stable

Apply complete! Resources: 33 added, 0 changed, 0 destroyed.

Refresh to dev/edge

Apply complete! Resources: 1 added, 6 changed, 1 destroyed.

Revert to 2/stable

Apply complete! Resources: 1 added, 6 changed, 1 destroyed.

Documentation

See the CI job for the documentation changes

terraform/charmhub/main.tf

sed-i · 2026-01-16T05:02:01Z

docs/tutorial/installation/cos-lite-microk8s-sandbox.md

        --overlay ./storage-small-overlay.yaml
 ```

+(deploy-cos-ref)=


Iirc we haven't started using these our docs yet.
Remind me - it's an anchor or cross-sphinx ref?
I think juju had this and it turned out to be brittle and difficult to maintain, but maybe I'm missing something.

It's an anchor, but it can also be used as a cross-sphinx/project ref for projects that have that enabled.

You haven't started using them in the COS docs yet, but it is the standard approach to cross-linking for sphinx docs projects, so at some point, you should change your links to use these refs. Juju uses them in their docs, but I don't know what conversations happened around it

Using ref targets/anchors is nicer because otherwise anytime you change the filename/path, it'll break any of those links in your docs.

IMO, it could go either way in this PR (add it or don't add it), but the future goal should be that all docs have a ref target, and you use those for linking instead of file path. (Copilot should be able to handle it well when you do make this initiative)

docs/tutorial/index.rst

docs/tutorial/upgrade-product-module.md

sed-i · 2026-01-16T05:07:29Z

docs/tutorial/upgrade-product-module.md

+```diff
+   http = {
+     source  = "hashicorp/http"
+     version = "~> 3.0"
+   }
+```


Do we need to sepcify http here? Can it be a derived dependency from the charmhub module?

docs/tutorial/refresh-product-module.md

terraform/charmhub/main.tf

MichaelThamm · 2026-01-16T17:05:57Z

Hi @YanisaHS, could you please review this PR with me? I would appreciate some opinions on the clarity and structure of the document.

YanisaHS · 2026-01-16T19:15:34Z

@MichaelThamm Yea, I'd be happy to! 😊 I likely won't have time to review it until Tuesday though (I'm off Monday)

MichaelThamm · 2026-01-20T03:48:15Z

@MichaelThamm Yea, I'd be happy to! 😊 I likely won't have time to review it until Tuesday though (I'm off Monday)

@YanisaHS This PR is still a work in progress. It may be best to wait until I make some more progress before reviewing this PR. The docs still need updating before first review.

YanisaHS · 2026-01-20T15:02:17Z

@MichaelThamm Ok, sounds good! Just tag me when you're ready again

MichaelThamm · 2026-01-21T15:13:14Z

@YanisaHS

I think it is ready for review now: docs link

Some input I would like from you is:

This feature (updating channel updates charms to latest revision) is not guaranteed to work across tracks. In the tutorial, it shows 2/stable to 2/edge which is within the same track so it should be guaranteed to work.
Would you convert this into a How-to instead of a tutorial?

I need to add a comment to the doc about not guaranteed to work across tracks.

YanisaHS · 2026-01-26T14:25:16Z

@MichaelThamm Ok! I'll review it soon (was off the end of last week)

YanisaHS · 2026-01-28T15:35:58Z

@MichaelThamm I'll go through and actually provide feedback in the doc, but first addressing your two issues:

This feature (updating channel updates charms to latest revision) is not guaranteed to work across tracks. In the tutorial, it shows 2/stable to 2/edge which is within the same track so it should be guaranteed to work.

For a tutorial, this is fine. I'd still recommend you add a note (as you suggested) describing this condition, mostly for any driveby users / people who find this without realizing they're in a Diataxis ✨ tutorial ✨. Tutorials are more of a "sandboxed" experience, so it's okay to feed the user the configurations they need.

But...

Would you convert this into a How-to instead of a tutorial?

Yes. The Tutorials section in the Observability docs gives the docs in order a new user should follow, so now it reads like (1) Deploy, (2) Refresh, (3) Sync .. etc. The experience feels odd for a new user - if it's important they're on this track, then that should be given to them in the previous Deploy steps, rather than deploying and refreshing right after.

Actually this is something we'll discuss as a team this cycle: From a user's perspective, one of the most immediate concerns I have with the docs is that they don't clearly state how to deploy COS in the How-to guides (examples: Charmed Kafka, Charmed Postgres, Charmed K8s). When this is the first thing a user needs to do to access the product (and rest of the documentation).

I'll provide feedback with this in mind as I go through your PR - assuming you'll refactor it into a how-to guide.

YanisaHS

I was mostly just looking at the Refresh product module doc, although I quick scanned the README and didn't have any comments. If you want me to provide feedback on something else in the PR then LMK

YanisaHS · 2026-01-28T15:47:10Z

docs/tutorial/installation/cos-lite-microk8s-sandbox.md

        --overlay ./storage-small-overlay.yaml
 ```

+(deploy-cos-ref)=


It's an anchor, but it can also be used as a cross-sphinx/project ref for projects that have that enabled.

You haven't started using them in the COS docs yet, but it is the standard approach to cross-linking for sphinx docs projects, so at some point, you should change your links to use these refs. Juju uses them in their docs, but I don't know what conversations happened around it

Using ref targets/anchors is nicer because otherwise anytime you change the filename/path, it'll break any of those links in your docs.

IMO, it could go either way in this PR (add it or don't add it), but the future goal should be that all docs have a ref target, and you use those for linking instead of file path. (Copilot should be able to handle it well when you do make this initiative)

YanisaHS · 2026-01-28T15:53:55Z

docs/tutorial/refresh-product-module.md

@@ -0,0 +1,111 @@
+# Refresh COS to a new channel
+
+In this example, you will learn how to deploy COS Lite and refresh from channel `2/stable` to `2/edge`. To do this, we can deploy COS Lite via Terraform in the same way as [in the tutorial](https://documentation.ubuntu.com/observability/track-2/tutorial/installation/cos-lite-microk8s-sandbox).


Assuming this gets refactored into a tutorial, the language/framing should change so it's not like "In this example, you'll learn...". How-to guides aren't really "learning" experiences, it's more just like "here's the steps you need to do XYZ".

Example: Charmed Kafka: How to upgrade

YanisaHS · 2026-01-28T15:55:17Z

docs/tutorial/refresh-product-module.md

+
+- Know how to deploy {ref}`COS Lite with Terraform <deploy-cos-ref>`
+
+## Introduction


Similar to a comment I made above, this section can be less narrative if being refactored into a how-to guide

(btw I'm happy to meet and discuss my comments with you if you want!)

YanisaHS · 2026-01-28T15:58:44Z

docs/tutorial/refresh-product-module.md

+terraform apply
+```
+
+At this point, you will have successfully upgraded COS Lite from `2/stable` to `2/edge`!


The steps overall seem fine and pretty straightforward. I can provide some better feedback once it gets refactored into a how-to

MichaelThamm · 2026-02-09T16:47:00Z

Blocked by this issue:

juju charm data source juju/terraform-provider-juju#1042

* Separates the storage directives for different worker roles Signed-off-by: Bartlomiej Gmerek <bartlomiej.gmerek@canonical.com> * chore: TODOs for tests * Separates storage directives for Tempo workers Signed-off-by: Bartlomiej Gmerek <bartlomiej.gmerek@canonical.com> * Separates storage directives for Tempo workers Signed-off-by: Bartlomiej Gmerek <bartlomiej.gmerek@canonical.com> * Cleans up after testing Signed-off-by: Bartlomiej Gmerek <bartlomiej.gmerek@canonical.com> --------- Signed-off-by: Bartlomiej Gmerek <bartlomiej.gmerek@canonical.com> Co-authored-by: Michael Thamm <mike.thamm@canonical.com>

This PR fixes #188 Signed-off-by: Jose C. Massón <939888+Abuelodelanada@users.noreply.github.com>

…#209)

…nd versioning (#217) * docs: add explanation for dashboard upgrades and version dedup behavior * add "dedup*" to custom wordlist --------- Signed-off-by: Leon <82407168+sed-i@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sed-i <82407168+sed-i@users.noreply.github.com>

MichaelThamm added 2 commits January 15, 2026 08:55

feat: Charmhub module for upgrades

aa244a2

chore: README

6a5e192

sed-i approved these changes Jan 15, 2026

View reviewed changes

terraform/charmhub/main.tf Outdated Show resolved Hide resolved

feat: add an upgrade doc

d8b0fbd

sed-i reviewed Jan 16, 2026

View reviewed changes

chore: doc improvements

0634ec8

MichaelThamm added 3 commits January 19, 2026 08:54

feat: charmhub inside COS Lite

2c634d1

chore: update tutorial

424447a

chore: revert apps to remote source

b3b6492

chore

91dcf5f

YanisaHS suggested changes Jan 28, 2026

View reviewed changes

chore: dump ideas

188e4f5

chore: remove charmhub module

97f585a

MichaelThamm mentioned this pull request Mar 21, 2026

Tagging for TF modules in charm repos canonical/observability#427

Open

3 tasks

MichaelThamm and others added 8 commits March 22, 2026 19:29

chore: implement juju_data

440dbee

fix: remove ubuntu/litestream from list of oci images (#177)

f4ffc38

fix: Outdated docs (#178)

5c7bf3a

chore: add blackbox exporter machine to list of charms in docs (#176)

055a627

docs: fix extra double quote in cloud-init script (#181)

42999c2

Update source URL for cos-lite module (#190)

3f6cffc

This PR fixes #188 Signed-off-by: Jose C. Massón <939888+Abuelodelanada@users.noreply.github.com>

fix: READMEs (#191)

7078c6e

sed-i and others added 11 commits March 23, 2026 10:22

feat(doc): troubleshoot grafana admin password (#194)

2170e02

feat(docs): update otelcol docs (#199)

61f716f

feat: Tests for product module channel input (#202)

1d27584

docs: point to correct file for cos-lite variables.tf (#205)

dba3cef

docs: deprecation notice in tutorial for migrating from GA to otelcol (…

51dd4b7

…#209)

fix: Tiering HowTo doc with OTLP endpoints (#211)

db994ce

add cookie banner and Google Analytics tags (#213)

c7cfe08

docs: Troubleshooting compressed alerts and OTLP topology labels (#215)

b76242e

chore: cleanup

42ac9a5

Merge remote-tracking branch 'origin/main' into feat/charmhub-module

0187773

MichaelThamm changed the title ~~feat: Charmhub module for upgrades without revision pins~~ feat: Cross-track upgrades Mar 23, 2026

MichaelThamm added 2 commits March 23, 2026 16:41

chore: cleanup

8903f5b

chore: cleanup

fa6561d

MichaelThamm mentioned this pull request Mar 23, 2026

[Documentation] "Cannot upgrade application X: would break relation" juju/terraform-provider-juju#1022

Closed

		@@ -0,0 +1,111 @@
		# Refresh COS to a new channel

		In this example, you will learn how to deploy COS Lite and refresh from channel `2/stable` to `2/edge`. To do this, we can deploy COS Lite via Terraform in the same way as [in the tutorial](https://documentation.ubuntu.com/observability/track-2/tutorial/installation/cos-lite-microk8s-sandbox).


		- Know how to deploy {ref}`COS Lite with Terraform <deploy-cos-ref>`

		## Introduction

Conversation

MichaelThamm commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue

Solution

Checklist

Context

Testing Instructions

COS Lite manual

Documentation

Uh oh!

Uh oh!

sed-i Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

YanisaHS Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sed-i Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MichaelThamm commented Jan 16, 2026

Uh oh!

YanisaHS commented Jan 16, 2026

Uh oh!

MichaelThamm commented Jan 20, 2026

Uh oh!

YanisaHS commented Jan 20, 2026

Uh oh!

MichaelThamm commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

YanisaHS commented Jan 26, 2026

Uh oh!

YanisaHS commented Jan 28, 2026

Uh oh!

YanisaHS left a comment

Choose a reason for hiding this comment

Uh oh!

YanisaHS Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

YanisaHS Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

YanisaHS Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

YanisaHS Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

MichaelThamm commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

MichaelThamm commented Jan 15, 2026 •

edited

Loading

MichaelThamm commented Jan 21, 2026 •

edited

Loading