Hi,
I've been testing some setups with REDIS_CHECKPOINT=true and noticed that my keys in redis contain txid_current but not checkpoint info most of the time. Sometimes the situation reverts and I only have checkpoint there.
After checking some code my theory is that status thread overwrites the complete document at regular interval in
|
self.redis.set_meta({"txid_current": self.txid_current}) |
As set_meta just runs redis SET command:
|
def set_meta(self, value: t.Any) -> None: |
|
"""Store an arbitrary JSON-serialisable value in a dedicated key.""" |
|
self.__db.set(self._meta_key, json.dumps(value)) |
Checkpointer's setter does the same:
|
self.redis.set_meta({"checkpoint": value}) |
This is not a problem when REDIS_CHECKPOINT is disabled as far as I can tell, because both commands operate on 2 different files.
If the above is correct, I believe the code either needs to use 2 different redis keys, or use HSET / HGET for field level updates to prevent interference
Hi,
I've been testing some setups with
REDIS_CHECKPOINT=trueand noticed that my keys in redis containtxid_currentbut notcheckpointinfo most of the time. Sometimes the situation reverts and I only havecheckpointthere.After checking some code my theory is that
statusthread overwrites the complete document at regular interval inpgsync/pgsync/sync.py
Line 2032 in 33eae21
As
set_metajust runs redis SET command:pgsync/pgsync/redisqueue.py
Lines 114 to 116 in 33eae21
Checkpointer's setter does the same:
pgsync/pgsync/sync.py
Line 1614 in 33eae21
This is not a problem when REDIS_CHECKPOINT is disabled as far as I can tell, because both commands operate on 2 different files.
If the above is correct, I believe the code either needs to use 2 different redis keys, or use HSET / HGET for field level updates to prevent interference