PostgreSQL PANIC due to SMgrRelation hashtable corruption under pgactive active-active replication

[umutoguz]

Description

When running pgactive in an active-active topology, PostgreSQL can crash with a PANIC during TRUNCATE TABLE or other DDL-like operations.
The crash is caused by internal relation state corruption, reported as:
ERROR: SMgrRelation hashtable corrupted
PANIC: cannot abort transaction , it was already committed
This issue has been observed on PostgreSQL 17 and PostgreSQL 18, and only when pgactive is enabled. The same operations do not cause crashes on the same PostgreSQL versions when pgactive is not in use.

Steps to reproduce

1- Set up a multi-node PostgreSQL cluster (4 nodes in our case) using pgactive active-active replication.

2- Ensure pgactive background workers and logical decoding slots are running normally.

3- On one node, execute a TRUNCATE TABLE statement (or other DDL in PostgreSQL 17) on a replicated table, for example:

TRUNCATE TABLE inventory.test_table;

4- Observe that the PostgreSQL backend immediately crashes and triggers a full server restart.

Note: This reproduces even in a low-load test environment with no external concurrency.

Expected outcome

• The TRUNCATE TABLE (or DDL) operation should either:
  • Complete successfully, or
  • Fail with a regular SQL error.
• PostgreSQL should never reach a PANIC state or crash due to such operations.
• pgactive should not corrupt internal PostgreSQL relation or transaction state.

Actual outcome

• PostgreSQL reports:

ERROR: SMgrRelation hashtable corrupted
WARNING: AbortTransaction while in COMMIT state
PANIC: cannot abort transaction , it was already committed

• The backend process is terminated with SIGABRT.
• PostgreSQL forcibly terminates all remaining backends.
• Crash recovery is initiated and the database restarts.
• pgactive supervisor and workers are restarted after recovery.

This results in temporary database unavailability and poses a serious risk in active-active setups.

Analysis

Several observations suggest this is a pgactive-related bug, not a PostgreSQL core issue:

• The error indicates internal SMgrRelation / relation cache corruption, typically caused by extension-level misuse or race conditions.
• The crash occurs only when pgactive background workers are active and logical decoding is running.
• The issue reproduces across:
  • PostgreSQL 17 (with DDL present)
  • PostgreSQL 18 (with TRUNCATE TABLE only, even when DDL replication is disabled)
• pgactive.skip_ddl_replication = on does not prevent the crash.

This points to a potential race condition between pgactive background workers and DDL/TRUNCATE execution, possibly involving relfilenode changes and stale relation references, leading to an inconsistent transaction commit/abort state.

Logs

ERROR: SMgrRelation hashtable corrupted
STATEMENT: truncate table inventory.test_table;
WARNING: AbortTransaction while in COMMIT state
PANIC: cannot abort transaction 109231405, it was already committed
LOG: client backend was terminated by signal 6: Aborted
LOG: terminating any other active server processes
LOG: database system was not properly shut down; automatic recovery in progress

[umutoguz]

Just to update: I called the pause function to stop pgactive on the node where I executed the TRUNCATE command. However, I still encountered the same crash during the TRUNCATE execution.

Looking forward to your feedback.

[bdrouvotAWS]

Thanks for the report! Can you describe the exact sequence of events that leads to the PANICs? (Please provide the sequences for both the "DDL" and the "TRUNCATE" cases). Asking, because I'm not able to reproduce (though I suspect that #311 could fix the "TRUNCATE" issue, as it could/would corrupt the memory on a non assert enabled build).

Thanks
Bertrand

[umutoguz]

Hi,
Thanks for the update regarding PR #311.
We have isolated the exact sequence that triggers the PANIC. It specifically happens when we execute two or more consecutive TRUNCATE commands on the same table. A single truncate works fine, but repeating it immediately triggers the crash.

[bdrouvotAWS]

#311 will fix the TRUNCATE case then. What about the DDL case?

[umutoguz]

Thanks for the update and the fix in #311.
Regarding the DDL case: We have not encountered any PANICs or crashes with DDL operations (like ALTER TABLE) in our environment so far. Our report is specifically for the consecutive TRUNCATE commands. If we encounter any DDL-related issues, we will report them separately.
Regarding the fix in #311: What is the recommended way for us to apply this fix? Should we wait for a new release/tag, or do we need to build from the source (branch) manually to test it?

[bdrouvotAWS]

You could build from source and apply the proposed fix to verify it fixes the issue (please don't do that on a database you care about though). As far waiting for a new release/tag: I'd say it all depends if your database in on-premises and how critical it is for you to have this bug "quickly" fixed. Avoiding multiple TRUNCATE in the same session could also be a workaround.

[tokuda-takashi]

Please tell me.
The issue is closed.
However, the following explanation is provided:
Please don't do that on a database you care about though.

Was the fix in #311 sufficient?
Is there an estimated timeframe for a release containing this fix?

[bdrouvotAWS]

The comment was about applying the fix manually not about its content.
Yes, I think #311 should be sufficient.
I'll check and come back regarding the timeframe.

[tokuda-takashi]

Thank you for your prompt response.
It was very helpful.