Skip to content

Add thread-safe clear_cache suppressor and extract_cot_id_from_model_name#467

Merged
arthanson merged 18 commits intofeaturefrom
utilities-cache-suppressor-and-model-name
Apr 20, 2026
Merged

Add thread-safe clear_cache suppressor and extract_cot_id_from_model_name#467
arthanson merged 18 commits intofeaturefrom
utilities-cache-suppressor-and-model-name

Conversation

@bctiemann
Copy link
Copy Markdown
Contributor

@bctiemann bctiemann commented Apr 14, 2026

Summary

Separates two infrastructural utilities from PR #442 (polymorphic object fields) into a standalone, reviewable PR.

Changes

utilities.py

extract_cot_id_from_model_name(model_name)
Replaces the fragile chained .replace("table", "").replace("model", "") pattern used throughout the codebase to extract a COT primary key from an internal model name (e.g. table3model"3"). The old approach silently corrupts names that happen to contain the substrings "table" or "model". The new function uses a strict regex (^table(\d+)model$) and returns None for non-matching names.

Thread-safe apps.clear_cache suppressor
Django's apps.register_model() calls apps.clear_cache(), which triggers our get_models() override → get_model() for every known COT → potential infinite recursion and spurious cache invalidations during dynamic model registration.

The old fix was a non-thread-safe module-level monkey-patch (apps.clear_cache = lambda: None) with a broken restore (apps.clear_cache = apps.clear_cache). The new approach:

  • install_clear_cache_suppressor() — called once from AppConfig.ready(), installs a single wrapper on apps.clear_cache (idempotent).
  • _suppress_clear_cache() — private context manager using a thread-local depth counter. Only the calling thread sees suppression; all other threads continue to receive real cache invalidation.
  • generate_model() — updated to use _suppress_clear_cache() instead of the lambda patch.

__init__.py

  • Calls install_clear_cache_suppressor() at the top of ready().
  • Replaces the chained .replace() calls in get_model() with extract_cot_id_from_model_name().

Relationship to PR #442

PR #442 (polymorphic object fields) uses both of these utilities extensively. Once this PR merges, #442 should be rebased so it picks them up from feature rather than defining them itself. Note: models.py in #442 imports suppress_clear_cache directly — that import should be updated to use the private _suppress_clear_cache after rebase.

Migrations

This PR also fixes the migration numbering conflict present in feature.

Generated with Claude Code

@bctiemann bctiemann changed the base branch from main to feature April 14, 2026 09:52
apps.register_model() internally calls apps.clear_cache(), which (with the
thread-safe wrapper now installed) triggers get_models() -> get_model() ->
generate_model() before the model is in _model_cache — infinite recursion.

Fix: wrap register_model() + _model_cache write in _suppress_clear_cache()
so the wrapper is suppressed during registration. The explicit apps.clear_cache()
immediately after the block fires safely because the model is now cached.

Also corrects test_get_model_returns_model_when_not_skipping: the model name
was '{pk}tablemodel' (wrong) instead of 'table{pk}model'. The old chained
.replace() silently handled the wrong format; extract_cot_id_from_model_name
uses a strict regex that exposed the bug.
@bctiemann
Copy link
Copy Markdown
Contributor Author

@claude /review

@claude
Copy link
Copy Markdown

claude Bot commented Apr 14, 2026

Claude finished @bctiemann's task in 1m 40s —— View job


PR Review: Add thread-safe clear_cache suppressor and extract_cot_id_from_model_name

  • Read changed files
  • Analyze code quality and correctness
  • Post review feedback

Overview

This PR cleanly extracts two focused utilities from the larger PR #442. The design is sound — replacing the non-thread-safe monkey-patch with a thread-local depth counter is the right approach, and the regex-based extraction is strictly better than chained .replace().

However, there are a couple of bugs introduced in the callers that need addressing before merge.


Bugs

1. TypeError when extract_cot_id_from_model_name returns None__init__.py:238

custom_object_type_id = int(extract_cot_id_from_model_name(model_name))

If model_name passes the pre-guard at line 225 (which only checks for the substrings "table" and "model") but fails the strict regex (e.g. "sometablemodel", "table_model"), extract_cot_id_from_model_name returns None and int(None) raises TypeError — not LookupError. Callers expecting a LookupError will see an uncaught exception instead.

Fix: check the return value before calling int(). Fix this →

The pre-guard at line 225 is also inconsistent with the regex — it should either use extract_cot_id_from_model_name itself for the early-out check, or be documented as a fast-path that is not exhaustive.

2. AttributeError on None.isdigit()forms.py:111

custom_object_type_id = extract_cot_id_from_model_name(obj.model)
if custom_object_type_id.isdigit():

Previously the chained .replace() calls always returned a string (possibly ""), so .isdigit() was safe. Now extract_cot_id_from_model_name can return None, making .isdigit() an AttributeError. Since the function only matches the strict ^table(\d+)model$ pattern, the returned string is always all-digits when non-None, so the .isdigit() check is also now redundant. Fix this →


Minor Issues

models.py imports private _suppress_clear_cache directly

models.py:57 imports and uses _suppress_clear_cache at line 588, rather than going through generate_model(). The PR description acknowledges the analogous issue in #442, but this file is changed in the current PR diff. The usage pattern is different (calling apps.register_model() directly, not type()), so direct use of the context manager is arguably correct — but it would be worth a brief inline comment explaining why generate_model() isn't used here.

views.py:532 and field_types.py — no None guard before CustomObjectType.objects.get(pk=...)

Many call sites pass the return value of extract_cot_id_from_model_name directly to a DB query without checking for None. In practice these code paths are only reached when content_type.app_label == APP_LABEL, which means the model name should always match the pattern — but this is an implicit invariant. A defensive check or an assertion would make the assumption explicit.

install_clear_cache_suppressor has a minor TOCTOU race on the idempotency check

if apps.clear_cache is _wrapped_clear_cache:
    return
_real_clear_cache = apps.clear_cache
apps.clear_cache = _wrapped_clear_cache

Two concurrent callers could both pass the is _wrapped_clear_cache check and then both attempt to assign _real_clear_cache. The second assignment would store _wrapped_clear_cache itself as _real_clear_cache, effectively breaking suppression. Since this is called from AppConfig.ready() (single-threaded startup), this is not a real-world issue, but a comment noting the assumption would be helpful.


Positives

  • Thread-local depth counter design is clean and reentrant — the finally block ensures decrement even on exception.
  • The regex ^table(\d+)model$ is strict and self-documenting; the compiled module-level _COT_MODEL_RE is the right approach.
  • install_clear_cache_suppressor idempotency check is good.
  • Suppression happening at the type() call in generate_model() and around apps.register_model() in models.py covers both registration paths.
  • The __all__ export list correctly keeps _suppress_clear_cache private.

Tests

No new unit tests for extract_cot_id_from_model_name (e.g. valid inputs, None-returning edge cases, empty string, names with embedded "table"/"model" substrings). A few parametrized tests would be valuable given this function replaces a pattern used in ~10 places. Same for the thread-safety behavior of the suppressor.

bctiemann and others added 5 commits April 14, 2026 07:16
…model()

int(None) raises TypeError, not LookupError. Replace the old substring
pre-guard (which only checked for 'table'/'model' substrings) with a direct
None check on the extract result, which is both correct and sufficient.
… in forms.py

The old .replace() always returned a string, so .isdigit() was safe.
extract_cot_id_from_model_name returns None for non-matching names, making
.isdigit() an AttributeError. Since the regex guarantees the return is
either None or a digit-only string, replace .isdigit() with a None check.

Tests cover valid names, missing prefix/suffix, non-digit IDs, substring
matches, empty string, and case sensitivity.
- Add assertion after every call to extract_cot_id_from_model_name inside
  `if content_type.app_label == APP_LABEL:` blocks in field_types.py and
  views.py. The model name must match table<id>model when app_label is
  APP_LABEL; the assertion makes this implicit invariant explicit and gives
  a clear error if it is ever violated.
- Update models.py comment explaining why _suppress_clear_cache() is used
  directly (rather than going through generate_model()) to extend the
  suppression window through the _model_cache write.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The idempotency check and two-step assignment are not atomic. Add a comment
explaining that this is safe because AppConfig.ready() runs during
single-threaded Django startup, and noting that a lock would be needed if
the function were ever called outside that context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bctiemann
Copy link
Copy Markdown
Contributor Author

Addressed all surfaced issues. PR #442 will need to rebased on feature once this is merged.

@bctiemann bctiemann requested a review from arthanson April 14, 2026 12:09
On a fresh install 0003 runs before 0004, so the live CustomObjectType
model already declares group_name but the column does not exist yet. The
previous savepoint approach caught the resulting ProgrammingError but
still printed a confusing warning on every clean install.

Add a raw-SQL existence check at the start of the migration function.
If the table has no rows (fresh install), return immediately. The ORM
is never called and no warning is printed. The savepoint fallback is
retained for the edge case of an upgrade where 0003 and 0004 are applied
together against a non-empty table.
Copy link
Copy Markdown
Contributor

@arthanson arthanson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question on if assert is correct of if we want to Raise the error on these checks.

@bctiemann bctiemann requested a review from arthanson April 20, 2026 21:51
@bctiemann
Copy link
Copy Markdown
Contributor Author

Changed the asserts to ValueErrors.

@arthanson arthanson merged commit d7bd8b8 into feature Apr 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants