I have found various errors in regards to data, and after realizing they were unlikely due to the Gitaly version issue, since as you said those are present in upstream, I dug some more and found out the PostgreSQL 16->17 update via docker.io/kkimurak/sameersbn-postgresql:17 appears to have introduced a change similar to what is described in https://wiki.postgresql.org/wiki/Locale_data_changes though I don't think it is that exact problem.
@kkimurak I hope it is okay to tag you here, or we can open a new issue here or on that repo, or go to discussions. I am terribly unfamiliar with Postgres, so I am reluctant to make a proper issue out of my findings which I just very barely understand at all. It appears that the upgrade from 16 to 17 corrupted indexes, particular indexes on columns with textual values. I initially discovered this due to a bug that I narrowed down to there being a difference between SELECT count(*) FROM namespaces WHERE type LIKE 'Group' and SELECT count(*) FROM namespaces WHERE type = 'Group'. Rebuilding the indexes for the whole database fixed this issue and the bug, but I later started to discover more and more issues. My current working theory is that the 16-17 upgrade process is somehow subtly flawed; though it is possible that this somehow due to our specific data or setup. My theory continues that migrations running on the corrupted database then introduced further damage to our data, and while rebuilding the indexes helped mitigate some of the problems, others caused by migrations running with broken indexes led to actual data loss that can not be recovered. Let me know if and how you would like to further discuss my findings
Originally posted by @JKHSDTV in sameersbn/docker-gitlab#3270
Originally posted by @JKHSDTV in sameersbn/docker-gitlab#3270