Docker Compose environments for developing modern data platform architectures using Kafka, Flink, Spark, Iceberg, OpenLineage, OpenMetadata, Pinot, ClickHouse, StarRocks + Kpow & Flex by Factor House

Tools for processing sql queries metadata.

NextGen DataMotion Lineage

Covid19 and Iowa Liquor Sales analysis at BigQuery using dbt, Airflow, Marquez, Google Cloud and other modern data stack tools

Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.

This Project aims to enhance Spark Lineage Generation in Open Metadata by creating Lineage edges from Table-to-Table with Column level lineage as well as Container to Table Lineage.

Testing a Docker deployment of Marquez and OpenLineage

Deploy MWAA with OpenLineage data lineage tracking using AWS CDK - Airflow 3.0.6, Marquez HA

A walkthrough of Correlator — an incident correlation engine that connects test failures to producing job runs to downstream impact across your data stack. The demo runs a Jaffle Shop pipeline with Airflow, dbt, and Great Expectations.

Data Reliability & Governance Framework — quality controls, observability evidence, incident playbooks, and ISO-ready audit mapping.

dbt plugin that accelerates incident resolution with automated test failure correlation

Exploring how to surface dlt pipeline metadata via Airflow's OpenLineage integration.

Scan AWS Glue PySpark jobs in GitHub repositories to generate data lineage using AST analysis and LLMs.

Incident correlation engine for data teams — connects test failures to job runs via OpenLineage

Airflow plugin that accelerates incident resolution with automated pipeline correlation

Accelerate Great Expectations incident resolution with automated correlation. Links validation failures to upstream data issues and provides direct navigation from incident to root cause.