MAINT: Workflow simplifications

[pastewka]

• Removed WorkflowTemplate
• Integrated SubjectDispatch into WorkflowResult, removing one auxiliary table.
• Workflow is no longer a database model: Workflows are now identified (in the database) by their unique name and not by a foreign key to the workflow table. This allows to remove the workflow synchronization process entirely. This is less brittle than the manual sync before Django startup and removes an auxiliary database table.
• Moved workflow engine to a legacy subdirectory to indicate replacement in the (near) future
• Added mock User, Organization and PermissionSet models for testing without need for a dependency on e.g. topobank-orcid

[Copilot]

Pull request overview

This PR simplifies the workflow system by removing database-backed workflow metadata (and the former WorkflowTemplate / WorkflowSubject auxiliary tables) in favor of registry-derived workflow definitions keyed by workflow name. It also introduces a lightweight mock authentication stack to run tests without depending on topobank-orcid, and moves the legacy workflow engine into a legacy/ namespace while keeping backwards-compatible imports.

Changes:

• Replace Workflow DB model + FK usage with workflow_name string identifiers and a plain-Python Workflow wrapper backed by the workflow registry.
• Collapse WorkflowSubject (subject dispatch table) into direct nullable subject FKs on WorkflowResult + migration + query updates.
• Add topobank.testing.mock_auth apps (users/organizations/authorization) and update test settings / CI to remove the topobank-orcid dependency.

Reviewed changes

Copilot reviewed 79 out of 91 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
topobank/testing/workflows.py	Update Timer import; instantiate workflows by name instead of DB lookup.
topobank/testing/utils.py	Add DRF-like UTC datetime formatting helper for tests.
topobank/testing/mock_auth/users/models.py	Add mock User model compatible with existing auth expectations.
topobank/testing/mock_auth/users/migrations/0001_initial.py	Initial migration for mock users app.
topobank/testing/mock_auth/users/migrations/init.py	Mock users migrations package init.
topobank/testing/mock_auth/users/middleware.py	Middleware to ensure an anonymous user is present in tests.
topobank/testing/mock_auth/users/management/commands/notify_users.py	Mock management command for notifications in test auth stack.
topobank/testing/mock_auth/users/management/commands/init.py	Management commands package init.
topobank/testing/mock_auth/users/management/init.py	Management package init.
topobank/testing/mock_auth/users/apps.py	AppConfig for mock users app.
topobank/testing/mock_auth/users/anonymous.py	Helper to get/create an “anonymous” user.
topobank/testing/mock_auth/users/admin.py	Provide MyUserCreationForm for admin tests.
topobank/testing/mock_auth/users/init.py	Mock users package init.
topobank/testing/mock_auth/organizations/models.py	Add mock Organization model with Group integration.
topobank/testing/mock_auth/organizations/migrations/0006_organization_group.py	Migration to add Group relation (mock org app).
topobank/testing/mock_auth/organizations/migrations/0005_fix_plugins_available_data.py	Stub migration to satisfy cross-app deps.
topobank/testing/mock_auth/organizations/migrations/0004_stub.py	Stub migration replacing real org migration.
topobank/testing/mock_auth/organizations/migrations/0003_stub.py	Stub migration replacing real org migration.
topobank/testing/mock_auth/organizations/migrations/0002_alter_organization_name.py	Stub migration to satisfy dependencies.
topobank/testing/mock_auth/organizations/migrations/0001_initial.py	Initial migration for mock organizations.
topobank/testing/mock_auth/organizations/migrations/init.py	Mock org migrations package init.
topobank/testing/mock_auth/organizations/apps.py	AppConfig for mock organizations app.
topobank/testing/mock_auth/organizations/init.py	Mock organizations package init.
topobank/testing/mock_auth/authorization/models.py	Add mock PermissionSet + permission link models.
topobank/testing/mock_auth/authorization/migrations/0006_alter_userpermission_allow_organizationpermission.py	Add org permissions + adjust allow choices in mock auth.
topobank/testing/mock_auth/authorization/migrations/0005_alter_userpermission_related_name.py	Fix related_name + uniqueness for mock permissions.
topobank/testing/mock_auth/authorization/migrations/0004_alter_userpermission_user.py	Stub migration to satisfy dependencies.
topobank/testing/mock_auth/authorization/migrations/0003_add_permission_performance_indexes.py	Stub migration to satisfy dependencies.
topobank/testing/mock_auth/authorization/migrations/0002_organizationpermission.py	Stub migration to satisfy dependencies.
topobank/testing/mock_auth/authorization/migrations/0001_initial.py	Initial migration for mock authorization app.
topobank/testing/mock_auth/authorization/migrations/init.py	Mock authorization migrations package init.
topobank/testing/mock_auth/authorization/apps.py	AppConfig for mock authorization app.
topobank/testing/mock_auth/authorization/init.py	Mock authorization package init.
topobank/testing/mock_auth/apps.py	Convenience AppConfig classes used by test settings.
topobank/testing/mock_auth/init.py	Mock auth package init.
topobank/testing/fixtures.py	Remove workflow DB sync; rename fixtures to workflow-name based APIs.
topobank/testing/factories.py	Update factories to use workflow_name and remove WorkflowTemplate/Subject factories.
topobank/test_settings.py	Add dedicated test settings module using mock auth apps.
topobank/taskapp/utils.py	Minor slicing formatting change.
topobank/taskapp/models.py	Switch Timer import to muTimer.
topobank/taskapp/migrations/0001_initial.py	Normalize migration file formatting.
topobank/manager/zip_model.py	Remove explicit folder=None when creating Manifests.
topobank/manager/utils.py	Minor slicing formatting change.
topobank/manager/models.py	Use safe getattr(..., 'orcid_id', None) and remove redundant folder=None in Manifest creation.
topobank/manager/import_zip.py	Remove explicit folder=None when creating Manifests.
topobank/files/models.py	Refactor ManifestSet file lookup/save logic; tighten typing and docstrings.
topobank/files/admin.py	Simplify admin list display for Manifest.
topobank/authorization/models.py	Update docstring to reflect current concrete auth implementation location.
topobank/authorization/init.py	Update anonymous-user getter configuration guidance for tests.
topobank/analysis/workflows.py	Re-export workflow engine symbols from analysis.legacy.
topobank/analysis/utils.py	Update analysis ordering to use new subject FK fields; remove WorkflowTemplate filtering.
topobank/analysis/tasks.py	Update task execution/scheduling to use workflow_name + new subject FK fields.
topobank/analysis/signals.py	Update invalidation/deprecation queries to use new subject FK fields.
topobank/analysis/registry.py	Re-export legacy registry; rename workflow listing helper.
topobank/analysis/models.py	Replace Workflow DB model with plain class; add workflow_name and subject FKs to WorkflowResult.
topobank/analysis/migrations/0062_collapse_workflowsubject.py	Data/schema migration to inline subjects onto WorkflowResult and delete WorkflowSubject.
topobank/analysis/migrations/0061_merge_20260508_1756.py	Merge migration for divergent histories.
topobank/analysis/migrations/0060_delete_workflowtemplate.py	Delete WorkflowTemplate model.
topobank/analysis/migrations/0059_remove_workflow_model.py	Remove Workflow DB model/FKs and add workflow_name-based indexes.
topobank/analysis/migrations/0058_copy_workflow_names.py	Data migration copying FK workflow names into string fields.
topobank/analysis/migrations/0057_add_workflow_name_fields.py	Add workflow_name/implementation_name CharFields.
topobank/analysis/migrations/0056_workflowresult_permissions_and_more.py	Restore permissions fields for analysis entities.
topobank/analysis/migrations/0055_remove_workflowresult_permissions_and_more.py	Remove permissions fields (later re-added).
topobank/analysis/management/commands/save_default_function_kwargs.py	Update to registry-based workflows rather than DB-backed workflows.
topobank/analysis/management/commands/register_analysis_functions.py	Remove DB registration management command.
topobank/analysis/legacy/workflows.py	Move previous workflow implementation base + helpers under legacy.
topobank/analysis/legacy/registry.py	Move prior registry storage + exceptions under legacy.
topobank/analysis/legacy/init.py	Legacy package init.
topobank/analysis/custodian.py	Update cleanup logic for new subject FK fields.
topobank/analysis/controller.py	Resolve workflows via registry; update subject filtering to new FK fields.
topobank/analysis/apps.py	Keep app initialization but remove redundant comments.
topobank/analysis/admin.py	Update admin filters/displays to use workflow_name and inlined subject fields.
tests/users/test_admin.py	Point admin form test to mock auth admin form.
tests/taskapp/test_submit.py	Update tests to use workflow_name and test_workflow fixture.
tests/taskapp/test_signal_handlers.py	Update tests to use mock auth PermissionSet and new workflow storage fields.
tests/taskapp/test_perform.py	Update test factories to use workflow_name.
tests/organizations/test_models.py	Switch org tests to mock auth organization model/constants.
tests/manager/test_models.py	Switch PermissionSet import to mock auth.
tests/files/test_models.py	Switch PermissionSet import to mock auth and adjust folder file deletion flow.
tests/analysis/test_workflows.py	Add regression tests for registry-backed workflow listing and schema generation.
tests/analysis/test_utils.py	Update subject filtering in tests to new subject FK fields.
tests/analysis/test_surface_set_workflows.py	Update surface-set tests to use workflow_name and new subject FK clearing.
tests/analysis/test_sharing.py	Update sharing tests to use updated analysis controller/workflow APIs.
tests/analysis/test_outputs.py	Update workflow construction for outputs tests to registry-backed Workflow.
tests/analysis/test_models.py	Update model tests to registry-backed workflow behavior and remove DB uniqueness test.
tests/analysis/conftest.py	Ensure test workflow implementations are registered at import time.
test_settings.py	Update installed apps and migration modules to mock auth; adjust default DB URL.
pytest.ini	Point pytest to topobank.test_settings.
pyproject.toml	Add muTimer dependency and tidy Django comment.
docs/conf.py	Remove trailing whitespace.
.github/workflows/test.yml	Remove topobank-orcid install; add curl -L for MinIO client download.

Comments suppressed due to low confidence (2)

topobank/files/models.py:92

• ManifestSet.save_file() now creates a Manifest without folder and then assigns folder in a second save. This introduces a race: concurrent uploads of the same filename into the same folder can both pass self.files.get(...), both create a row, and then one will fail with an IntegrityError when saving the (folder, filename) uniqueness constraint. Consider restoring an atomic get_or_create(folder=self, filename=...) (or catching IntegrityError and retrying the lookup) to keep the operation concurrency-safe.
  topobank/analysis/models.py:748
• Workflow.eval_surfaces() calls self.implementation(**analysis.kwargs) without handling the case where the workflow name is not registered. In that situation it will raise a TypeError: 'NoneType' object is not callable instead of the expected WorkflowNotImplementedException (as eval() does). Consider mirroring the eval() guard (if impl is None: raise WorkflowNotImplementedException(...)) for consistent error semantics.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[RGClary]

@pastewka Approved, but it looks like some of the tests need db decorators.

[pastewka]

@RGClary - decorators added

[pastewka]

Closes #1300

[pastewka]

Closes #1299