Skip to content

[WIP] Add 14-day time bound to failed-tests matviews#3651

Draft
mstaeble wants to merge 1 commit into
openshift:mainfrom
mstaeble:optimize-failed-tests-matview-refresh
Draft

[WIP] Add 14-day time bound to failed-tests matviews#3651
mstaeble wants to merge 1 commit into
openshift:mainfrom
mstaeble:optimize-failed-tests-matview-refresh

Conversation

@mstaeble

@mstaeble mstaeble commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The prow_job_failed_tests_by_day and prow_job_failed_tests_by_hour materialized views scanned the entire prow_job_run_tests table with no time filter, taking ~3 minutes per refresh. The only consumer (job analysis page) uses a 14-day window.
  • Adding a 15-day time bound (14 days plus 1 day of buffer for refresh timing) enables RANGE partition pruning within each release, reducing refresh time and row count.

Benchmarks (staging, enable_partitionwise_join = on)

Approach Refresh time Row count
No time bound (current) ~3 minutes 1,280,753
15-day bound ~1 minute 133,295

Matview read performance is unchanged (~46ms).

Alternative

PR #3647 removes the matviews entirely and queries the partitioned table directly (~249ms warm reads, no refresh needed). This PR is a smaller, lower-risk change that keeps the matview pattern.

Test plan

  • Full project compiles with go build ./...
  • go vet clean
  • Benchmarked refresh time on staging
  • CI passes

🤖 Generated with Claude Code

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 18, 2026
@openshift-ci

openshift-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@mstaeble, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 15 minutes and 5 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 7ec5d119-c983-4d63-986e-7984e69511fd

📥 Commits

Reviewing files that changed from the base of the PR and between dd64c4f and d6528a0.

📒 Files selected for processing (2)
  • pkg/api/job_analysis.go
  • pkg/db/views.go
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mstaeble
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mstaeble mstaeble changed the title Add 14-day time bound to failed-tests matviews [WIP] Add 14-day time bound to failed-tests matviews Jun 18, 2026
@mstaeble mstaeble force-pushed the optimize-failed-tests-matview-refresh branch 3 times, most recently from bfdb21f to 1444997 Compare June 18, 2026 14:20
The prow_job_failed_tests_by_day and prow_job_failed_tests_by_hour
materialized views scanned the entire prow_job_run_tests table with
no time filter, taking ~3 minutes per refresh. The only consumer
(job analysis page) uses a 14-day window.

Adding a 14-day time bound enables RANGE partition pruning within
each release, reducing refresh time to ~1 minute and row count
from 1.28M to 133K.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mstaeble mstaeble force-pushed the optimize-failed-tests-matview-refresh branch from 1444997 to d6528a0 Compare June 18, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant