Skip to content

Add progress tracking during tuple copy phase#477

Open
dreadushka wants to merge 2 commits intoreorg:masterfrom
dreadushka:copy-tuples-with-progress-tracking-via-sequence
Open

Add progress tracking during tuple copy phase#477
dreadushka wants to merge 2 commits intoreorg:masterfrom
dreadushka:copy-tuples-with-progress-tracking-via-sequence

Conversation

@dreadushka
Copy link

Progress tracking for long-running repacks

The problem
Repacking multi-terabyte tables can take days. Currently, there is no reliable way to determine whether the operation is progressing or stuck. As a result, teams are forced to:

  • Make guesses using fragile workarounds
  • Cancel repacks due to uncertainty
  • Wait blindly without being able to estimate completion time

What this adds
A progress counter that increments for every row copied during the COPY phase. It uses a dedicated per-table sequence (repack.track_insert_<oid>). No additional locks or scans are required.

Progress is exposed via a simple function:

SELECT * FROM repack.get_progress_all();

 table_oid |  table_full_name  | current_rows | estimated_total | progress_percent | status  
-----------+-------------------+--------------+-----------------+------------------+---------
     16425 | public.test_table |      3116631 |        10000006 |            31.17 | COPYING

Why it helps
We used this feature during holiday maintenance on tables exceeding 12 TB. Being able to state "we're at 73% after 36 hours of work" allowed us to:

  • Measure repack speed
  • Make informed decisions about extending the maintenance window
  • Forecast completion and provide management with realistic time estimates
  • Rule out stalls caused by locks or I/O issues

Compatibility
Fully backward compatible—existing repack workflows remain unchanged. Cleanup occurs automatically through the existing repack_drop() mechanism.

This small improvement makes long-running repack operations observable without adding operational complexity.

user added 2 commits February 8, 2026 18:24
Create repack.track_insert_<oid> sequence to count copied rows and expose progress via get_progress_all().
…uery

The RETURNING clause for progress tracking sequence was incorrectly placed
before ORDER BY in INSERT ... SELECT statements, causing syntax errors.
Moved RETURNING to the end of the query after ORDER BY processing in
pg_repack.c and removed it from the base query template in pg_repack.sql.in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant