Skip to content

feat: add openlineage support #726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
razvan wants to merge 20 commits into
base: main
Choose a base branch
from
Draft

feat: add openlineage support #726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
739b49a
feat: add basic openlineage support
sweb Jul 9, 2026
43235f5
tests: better testing of the hbase connectors (#717)
razvan Jul 9, 2026
c7313a8
Revert "fix: make application templates namespaced (#694)" (#719)
razvan Jul 10, 2026
87da728
Reapply "fix: make application templates namespaced (#694)" (#719) (#…
razvan Jul 10, 2026
4335b15
extract OL structs into openlineage.rs
razvan Jul 16, 2026
454e8cf
refactor!: remove openlineage.enabled field
razvan Jul 16, 2026
6cdec1a
refactor!: replace OL config map with host:port fields
razvan Jul 16, 2026
24dfb75
feat: add tls field
razvan Jul 16, 2026
43ccff2
refactor!: use crd::openlineage module from op-rs
razvan Jul 17, 2026
5f63d94
Merge branch 'main' into feat/openlineage-from-op-rs
razvan Jul 17, 2026
7f079f3
chore: regenerate-nix ... again
razvan Jul 17, 2026
ed2c33e
feat: support Spark 3 as well
sweb Jul 17, 2026
b85328a
refactor!: remove ResolvedOpenLineageAppName struct
razvan Jul 17, 2026
93f09bd
feat: openlineage.tls now also installs the server CA if necesary
razvan Jul 17, 2026
c448ebc
test: add integration test for openlineage events
razvan Jul 17, 2026
92535f1
add authentication with bearer token
razvan Jul 20, 2026
70a7d9f
refactor(openlineage): use credentialsSecretName instead of Authentic…
razvan Jul 21, 2026
846effc
merge main
razvan Jul 22, 2026
4ee4a1e
refactor(openlineage): adopt job_name field and lineage.stackable.tec…
razvan Jul 24, 2026
8693558
refactor(openlineage): rename the openLineage spec field to lineage
razvan Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ All notable changes to this project will be documented in this file.

- BREAKING: Add required CLI argument and env var to set the image repository used to construct final product image names: `IMAGE_REPOSITORY` (`--image-repository`), eg. `oci.example.org/my/namespace` ([#684]).
- Support for Spark `4.1.2` ([#708])
- Add `spec.lineage` to enable OpenLineage lineage emission: injects the OpenLineage Spark listener, HTTP transport (endpoint built from the `host` and `port` fields), a stable job name, and the required `--add-opens` JVM flag. When the connection configures TLS server verification against a `secretClass` CA, that certificate is mounted into the driver and added to its trust store ([#XXXX]).

### Fixed

Expand Down
Loading