Skip to content

Add a manual purge action for completed Workflows #5

Description

@daviddavis

Summary

Add a manual API for purging completed Workflow records, mirroring the existing pulpcore POST /pulp/api/v3/tasks/purge/ action.

Motivation

Workflows accumulate the same way tasks do — once completed, the rows stay around for audit/history but eventually become noise. Pulpcore already provides this for tasks via the tasks_purge action; workflows should have the equivalent so operators have a way to keep the workflow table tidy.

Explicitly manual, not automatic: like pulpcore's task purge, the action should only run when an operator (or a scheduled job they configure themselves) invokes it. No background sweeping should be added.

How pulpcore does it (for reference)

  • POST /pulp/api/v3/tasks/purge/ is an @action(detail=False, methods=["post"]) on TaskViewSet (pulpcore/app/viewsets/task.py).
  • The action accepts a PurgeSerializer body with two fields (pulpcore/app/serializers/purge.py):
    • finished_before — datetime; defaults to 30 days ago.
    • states — set of allowed final states; defaults to ["completed"], restricted to TASK_FINAL_STATES.
  • The action dispatches pulpcore.app.tasks.purge.purge and returns an OperationPostponedResponse (202).
  • The purge task is itself a regular pulp task: it filters in chunks of 1000, scopes by current domain and core.delete_task permission, and emits ProgressReports for totals, per-type counts, and errors.
  • A tasks entry is registered in INITIAL_GUARDED_SCHEDULES (pulpcore/app/util.py) so a default schedule exists, but the user is in control of whether it actually runs.

Proposed design for pulp_workflow

Add a purge action on WorkflowViewSet:

  • POST /pulp/api/v3/workflows/purge/ (action, detail=False, methods=["post"]).
  • Request body — WorkflowPurgeSerializer:
    • finished_beforeDateTimeField, default now() - 30d. Only workflows whose finished_at is strictly less than this are eligible.
    • statesMultipleChoiceField over the workflow's terminal states (COMPLETED, FAILED, CANCELED); default ["completed"].
  • Dispatch a new task pulp_workflow.app.tasks.purge_workflows and return OperationPostponedResponse (202). Do not run synchronously.
  • Permission model:
    • Add a delete_workflow Django permission to Workflow.Meta.permissions (or rely on the default delete permission once added) and gate the action on it via DEFAULT_ACCESS_POLICY, mirroring pulpcore's pattern of filtering by delete_task inside the purge task.
    • Scope by current domain (pulp_domain=get_domain()) when the task was dispatched by a user (i.e. not from a schedule).
  • Inside purge_workflows:
    • Build a queryset of Workflow.objects.filter(finished_at__lt=finished_before, state__in=states).
    • Delete in chunks of 1000 to keep memory bounded (same DELETE_LIMIT pattern as pulpcore).
    • Emit ProgressReports with codes such as purge.workflows.total, purge.workflows.key.<model>, purge.workflows.error for parity with the task purge.
  • Cascade behavior: WorkflowTask, WorkflowTaskArg, and WorkflowTaskKwarg already cascade off Workflow via on_delete=CASCADE, so deleting the Workflow cleans those up automatically. The associated TaskGroup (once Associate Workflows with a pulpcore TaskGroup #3 lands) and the child Task rows are not owned by the workflow and should be left alone — they are governed by tasks_purge. Document this in the action's help text so users understand they need to run both.

Explicitly out of scope

  • No automatic/scheduled purging by default. Do not add a TaskSchedule entry that runs on its own. Operators who want recurring purges can create their own TaskSchedule pointing at pulp_workflow.app.tasks.purge_workflows.
  • No purge of running or waiting workflows. The states field must reject non-terminal states (validate in the serializer the same way pulpcore's PurgeSerializer restricts to TASK_FINAL_STATES).
  • No deletion of the Task rows the workflow dispatched. Those are pulpcore's responsibility.

Acceptance criteria

  • POST /pulp/api/v3/workflows/purge/ exists, accepts finished_before and states, returns 202 with a task reference.
  • Dispatched task removes only workflows whose finished_at < finished_before and whose state is in the (validated terminal) states list.
  • Workflows in non-terminal states are never deleted.
  • Cascading delete of WorkflowTask / WorkflowTaskArg / WorkflowTaskKwarg works.
  • Default finished_before is "30 days ago" and default states is ["completed"].
  • Domain scoping matches pulpcore's task purge (current-domain only when dispatched by a user).
  • Permission check prevents users without delete_workflow from purging.
  • Functional test covers: defaults spare recent workflows; explicit finished_before in the future deletes everything terminal; non-terminal workflows are spared regardless.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions