Skip to content

Re-design model and replace projects.yaml with postgres database#40

Merged
lukpueh merged 3 commits into
eclipse-csi:mainfrom
lukpueh:model-redesign
May 12, 2026
Merged

Re-design model and replace projects.yaml with postgres database#40
lukpueh merged 3 commits into
eclipse-csi:mainfrom
lukpueh:model-redesign

Conversation

@lukpueh
Copy link
Copy Markdown
Contributor

@lukpueh lukpueh commented May 4, 2026 •
edited
Loading

fixes #5 #6 #29 #39

Disclaimer

This PR tackles two big features:

  1. Model redesign
    1. plus consequential and incidental auth and upload flow updates
  2. DB integration

It may have been possible to introduce these features in incremental PRs, for more bite-sized reviews. However, that would have made the total diff much larger, and I would have had to do a lot of retroactive git history mangling, which didn't seem worth the hassle. I recommend to review commit by commit:

  1. design doc
  2. implementation + tests
  3. misc janitorial work

And I am happy to walk reviewers through the PR, if needed!

Change overview

  • New model classes

    • Workload: Issuer aware CI/CD entity that can upload SBOMs (e.g. eclipse-csi/pia on GitHub)
    • DependencyTrackProject: Hierarchical SBOM upload slot (e.g. Eclipse CSI --> pia)
    • EclipseFoundationProject: A project can have multiple Workloads and DependencyTrackProjects.

  • OIDC auth

    • Preliminary issuer check is now less expensive (see #6)
    • New required claim for GitHub workloads: numeric owner id
    • Auth fails, if more than one workloads are matched by claims (previously first match was returned)
    • Result is a single authenticated Github or Jenkins workload

  • SBOM upload

    • Get authenticated workload
    • Find DT project, which maps to the same EF project as the Workload, and
    • whose "name" matches the "product_name" from the SBOM upload payload
    • Upload using the DT project's parent_uuid

  • DB integration

    • SQLAlchemy ORM for new model
    • On startup: create db engine and a db session factory
    • On request: create db session

  • Misc janitorial work

    • Update docker files and ci.yml
    • Remove exemplary projects.yaml

Upcoming related PRs

  • DB management CLI
  • DB migration with alembic

@lukpueh lukpueh force-pushed the model-redesign branch 2 times, most recently from 938767c to 574a60a Compare May 11, 2026 13:36
@lukpueh @claude
docs: Update model and db integration in DESIGN.md
a3cccab 
Replace YAML-based project configuration with PostgreSQL database
integration.

Introduce new data model with ORM:
 * EclipseFoundationProject (string PK),
 * Workload (polymorphic: GitHubWorkload, JenkinsWorkload)
 * DependencyTrackProject.

Update auth flow, settings, and all affected sections accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Lukas Puehringer <lukas.puehringer@eclipse-foundation.org>
lukpueh and others added 2 commits May 12, 2026 10:30
@lukpueh @claude
Update model and add db integration as per DESIGN.md
2bd37ad 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Lukas Puehringer <lukas.puehringer@eclipse-foundation.org>
@lukpueh
build: replace dummy projects.yaml with dummy db
1c7d3e7 
* Remove projects.yaml.example and *ignore entries.
* ci.yml -- Run pia with a dummy db URL:
  The docs smoke test still works, because SQLAlchemy only opens a real
  connection on the first db query.
* docker-compose.yml -- Start service with dummy db URL:
  This is a non-functional stop-gap to not break `docker compose up`
  with the model change.  In the future, we will likely add a working db
  service.

Signed-off-by: Lukas Puehringer <lukas.puehringer@eclipse-foundation.org>
lukpueh
lukpueh commented May 12, 2026
View reviewed changes
Comment thread docs/DESIGN.md
Comment thread pia/models.py
Comment thread tests/test_main.py
@lukpueh lukpueh marked this pull request as ready for review May 12, 2026 10:45
@lukpueh lukpueh mentioned this pull request May 12, 2026
Open
@lukpueh lukpueh merged commit 06218c6 into eclipse-csi:main May 12, 2026
3 checks passed
This was referenced May 18, 2026
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mbarbero mbarbero mbarbero approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Make model issuer aware

2 participants

@lukpueh @mbarbero