diff --git a/docs/explanations/converters.md b/docs/explanations/converters.md index bace142..5238ba2 100644 --- a/docs/explanations/converters.md +++ b/docs/explanations/converters.md @@ -1,6 +1,6 @@ # Converters -The queue contains ulims experiment definitions, which need to be converted to blueapi calls for anything to run. This conversion is beamline specific and configurable. The default converter simply passes the parameters of the experiment definition directly to blueapi with no conversion taking place. +The queue contains ulims experiments and/or TaskRequests. While TaskRequests are already in the right format for blueapi to receive them, experiments need to be converted to blueapi calls for anything to run. This conversion is beamline specific and configurable. The default converter simply passes the parameters of the experiment definition directly to blueapi with no conversion taking place. Converters take the list of queued tasks, the queue history, and the blueapi call history (previous calls made to blueapi) and use this information to construct a list of future blueapi calls. Not all of this information needs to be used by the converter (for example, the default converter only uses the list of queued tasks to inform its 'conversion') but the interface of a converter function needs to be as follows: @@ -13,7 +13,9 @@ def my_converter( ... ``` -Conversion is run any time the queue changes and right before a call is made to blueapi, so the blueapi call list stays up to date. This means converters can make use of current information on a beamline such as PV values to inform the conversion. +Conversion is run any time the queue changes and right before a call is made to blueapi, so the blueapi call list stays in sync. This means converters can make use of current information on a beamline such as PV values to inform the conversion. + +Note that since the queue can contain TaskRequest objects, a converter should be able to handle this. It should almost always pass them through with no conversion In config, under the converter section, add the path and name of the converter you want to use. For example: diff --git a/docs/explanations/queue.md b/docs/explanations/queue.md deleted file mode 100644 index 798dbc1..0000000 --- a/docs/explanations/queue.md +++ /dev/null @@ -1,10 +0,0 @@ -Queue -===== - -The queue is essentially a list of [tasks](./tasks.md). Tasks can be added, deleted, and moved around the queue through the REST API. A worker takes the task at the top of the queue, if one is available, and sends it to blueapi. Once this task is in progress, it can't be moved or deleted, and no other task can be sent to blueapi until this one has finished. - -The queue can be paused, meaning no task can be taken and sent to blueapi until the queue is unpaused. This will not pause a task that is already in progress. While the queue is paused, tasks can still be added, deleted and moved around. - -Once a task is complete, it is added to a history list, which is a chronological list of completed tasks. - -See the [API reference](../reference/rest-api.rst) for how to interact with the queue. diff --git a/docs/explanations/queue_items.md b/docs/explanations/queue_items.md new file mode 100644 index 0000000..d0e58e3 --- /dev/null +++ b/docs/explanations/queue_items.md @@ -0,0 +1,7 @@ +# Queue Items + +The items inside the queue contain either an `Experiment` object or a `TaskRequest` object. These determine which bluesky plans will run. Experiment objects match the format of an experiment in ulims, and can involve multiple bluesky plans. It always contains a sample, experiment definition and instrument session. A TaskRequest object matches the format blueapi expects to receive to run a single plan. + +Allowing both in the queue means you can queue up whole experiments as well as individual bluesky plans. + +The queue will convert the entire list of experiments and task requests to a list of blueapi calls (the task request objects are already in the right format). See [converters](./converters.md) on how this is done. diff --git a/docs/explanations/tasks.md b/docs/explanations/tasks.md deleted file mode 100644 index cc6a038..0000000 --- a/docs/explanations/tasks.md +++ /dev/null @@ -1,14 +0,0 @@ -Tasks -===== - -The items contained inside the queue are called tasks. Tasks are produced by sending an experiment definition to the queue, which will validate it, assign it an ID it and create a task for it. Throughout the lifecycle of a task, information will be added to the task by the queue when it becomes available. This will include things like: -- Status of the task (Waiting, In Progress, Success etc) -- Time started -- Time completed -- Result of the task -- Any errors that occurred during the task -- A blueapi task ID (distinct from the ID assigned by the queue) - -When the queue is asked to retrieve a task, it will calculate its position in the queue and include that in the task object. - -Once a task reaches the top of the queue, the experiment definition it contains will be converted to a blueapi call and sent to blueapi. Converter plugins can be written and interchanged, allowing for flexibility in how experiment definitions are written by different beamlines. Once a task is complete, it will leave the queue and enter a history list. diff --git a/docs/reference/rest_api.json b/docs/reference/rest_api.json index 2ab291e..5ea59f7 100644 --- a/docs/reference/rest_api.json +++ b/docs/reference/rest_api.json @@ -186,9 +186,16 @@ "schema": { "type": "array", "items": { - "$ref": "#/components/schemas/ExperimentDefinition" + "anyOf": [ + { + "$ref": "#/components/schemas/TaskRequest" + }, + { + "$ref": "#/components/schemas/Experiment" + } + ] }, - "title": "Experiment Definitions" + "title": "Experiments" } } } @@ -1001,32 +1008,53 @@ "title": "EnvironmentConfig", "description": "Config for the RunEngine environment" }, + "Experiment": { + "properties": { + "name": { + "type": "string", + "title": "Name" + }, + "instrument_session": { + "type": "string", + "title": "Instrument Session" + }, + "sample": { + "$ref": "#/components/schemas/Sample" + }, + "experiment_definition": { + "$ref": "#/components/schemas/ExperimentDefinition" + } + }, + "type": "object", + "required": [ + "name", + "instrument_session", + "sample", + "experiment_definition" + ], + "title": "Experiment" + }, "ExperimentDefinition": { "properties": { - "plan_name": { + "name": { "type": "string", - "title": "Plan Name" + "title": "Name" }, - "sample_id": { + "id": { "type": "string", - "title": "Sample Id" + "title": "Id" }, - "params": { + "data": { "additionalProperties": true, "type": "object", - "title": "Params", - "description": "Values for parameters to plan, if any" - }, - "instrument_session": { - "type": "string", - "title": "Instrument Session" + "title": "Data" } }, "type": "object", "required": [ - "plan_name", - "sample_id", - "instrument_session" + "name", + "id", + "data" ], "title": "ExperimentDefinition" }, @@ -1232,6 +1260,30 @@ "type": "object", "title": "RestConfig" }, + "Sample": { + "properties": { + "name": { + "type": "string", + "title": "Name" + }, + "id": { + "type": "string", + "title": "Id" + }, + "data": { + "additionalProperties": true, + "type": "object", + "title": "Data" + } + }, + "type": "object", + "required": [ + "name", + "id", + "data" + ], + "title": "Sample" + }, "ScratchConfig": { "properties": { "root": { @@ -1396,6 +1448,14 @@ "title": "TaskError", "description": "Wrapper around an exception raised by a plan" }, + "TaskKind": { + "type": "string", + "enum": [ + "Experiment", + "Plan" + ], + "title": "TaskKind" + }, "TaskRequest": { "properties": { "name": { @@ -1450,8 +1510,16 @@ }, "TaskWithPosition": { "properties": { - "experiment_definition": { - "$ref": "#/components/schemas/ExperimentDefinition" + "experiment": { + "anyOf": [ + { + "$ref": "#/components/schemas/Experiment" + }, + { + "$ref": "#/components/schemas/TaskRequest" + } + ], + "title": "Experiment" }, "id": { "type": "string", @@ -1477,15 +1545,19 @@ } ], "title": "Position" + }, + "kind": { + "$ref": "#/components/schemas/TaskKind" } }, "type": "object", "required": [ - "experiment_definition", + "experiment", "id", "status", "blueapi_calls", - "position" + "position", + "kind" ], "title": "TaskWithPosition" }, diff --git a/src/daq_queuing_service/api/api.py b/src/daq_queuing_service/api/api.py index 6cc9bb4..da3aec3 100644 --- a/src/daq_queuing_service/api/api.py +++ b/src/daq_queuing_service/api/api.py @@ -2,6 +2,7 @@ import json from collections.abc import AsyncGenerator +from blueapi.service.model import TaskRequest from fastapi import APIRouter, Request, Response from fastapi.responses import EventSourceResponse from pydantic import BaseModel @@ -9,7 +10,7 @@ from daq_queuing_service.app._config import AppConfig from daq_queuing_service.blueapi_interaction.blueapi_call import BlueapiCallResponse from daq_queuing_service.broadcaster import Broadcaster -from daq_queuing_service.task import ExperimentDefinition, Status, Task +from daq_queuing_service.task import Experiment, Status, Task from daq_queuing_service.task_queue.queue import ( QUEUE_EVENTS, QueueState, @@ -72,13 +73,10 @@ async def get_queued_tasks(status: Status | None = None) -> list[TaskWithPositio @router.post("/queue") async def add_tasks_to_queue( - experiment_definitions: list[ExperimentDefinition], + experiments: list[TaskRequest | Experiment], position: int | None = None, ) -> list[str]: - tasks = [ - Task(experiment_definition=experiment_definition) - for experiment_definition in experiment_definitions - ] + tasks = [Task(experiment=experiment) for experiment in experiments] task_ids = [task.id for task in tasks] await queue.add_tasks(tasks, position) return task_ids diff --git a/src/daq_queuing_service/plugins/construct_task_request.py b/src/daq_queuing_service/plugins/construct_task_request.py index 79e1f48..595b614 100644 --- a/src/daq_queuing_service/plugins/construct_task_request.py +++ b/src/daq_queuing_service/plugins/construct_task_request.py @@ -1,17 +1,21 @@ from blueapi.service.model import TaskRequest from daq_queuing_service.blueapi_interaction.blueapi_call import BlueapiCall -from daq_queuing_service.task import ExperimentDefinition, TaskWithPosition +from daq_queuing_service.task import Experiment, TaskWithPosition def _construct_blueapi_task_request( - experiment_definition: ExperimentDefinition, + experiment: Experiment | TaskRequest, ) -> TaskRequest: - return TaskRequest( - name=experiment_definition.plan_name, - params=experiment_definition.params, - instrument_session=experiment_definition.instrument_session, - ) + match experiment: + case TaskRequest(): + return experiment + case Experiment(): + return TaskRequest( + instrument_session=experiment.instrument_session, + name=experiment.experiment_definition.name, + params=experiment.experiment_definition.data, + ) def construct_blueapi_call_list( @@ -23,7 +27,7 @@ def construct_blueapi_call_list( call_list = [ BlueapiCall( parent_task_id=task.id, - task_request=_construct_blueapi_task_request(task.experiment_definition), + task_request=_construct_blueapi_task_request(task.experiment), ) for task in queue ] diff --git a/src/daq_queuing_service/plugins/i15_1_converter.py b/src/daq_queuing_service/plugins/i15_1_converter.py index 6fff767..308cf8d 100644 --- a/src/daq_queuing_service/plugins/i15_1_converter.py +++ b/src/daq_queuing_service/plugins/i15_1_converter.py @@ -1,13 +1,13 @@ from blueapi.service.model import TaskRequest from daq_queuing_service.blueapi_interaction.blueapi_call import BlueapiCall -from daq_queuing_service.task import ExperimentDefinition, TaskWithPosition +from daq_queuing_service.task import Experiment, TaskWithPosition def construct_blueapi_tasks_from_i15_1_experiment( - experiment_definition: ExperimentDefinition, + experiment: Experiment, ) -> list[TaskRequest]: - sample_name = experiment_definition.params["sampleName"] + sample_name = experiment.sample.name # Assume sample name is of form test_8_1 to load from position 8 on puck 1 _, position, puck = sample_name.split("_") @@ -15,17 +15,17 @@ def construct_blueapi_tasks_from_i15_1_experiment( TaskRequest( name="robot_load", params={"puck": puck, "position": position}, - instrument_session=experiment_definition.instrument_session, + instrument_session=experiment.instrument_session, ), TaskRequest( name="centre_sample", params={"start_z": -5, "end_z": 5, "steps": 20, "exposure_time": 0.01}, - instrument_session=experiment_definition.instrument_session, + instrument_session=experiment.instrument_session, ), TaskRequest( name="robot_unload", params={}, - instrument_session=experiment_definition.instrument_session, + instrument_session=experiment.instrument_session, ), ] @@ -39,15 +39,19 @@ def construct_i15_1_blueapi_call_list( call_list: list[BlueapiCall] = [] for task in queue: - # Remove plan name from here and UI in https://github.com/DiamondLightSource/daq-queuing-service/issues/51 - if task.experiment_definition.plan_name == "run_full_collection": - call_list.extend( - [ - BlueapiCall(task_request=blueapi_task, parent_task_id=task.id) - for blueapi_task in construct_blueapi_tasks_from_i15_1_experiment( - task.experiment_definition - ) - ] - ) + match task.experiment: + case TaskRequest(): + call_list.append( + BlueapiCall(task_request=task.experiment, parent_task_id=task.id) + ) + case Experiment(): + call_list.extend( + [ + BlueapiCall(task_request=b_api_task, parent_task_id=task.id) + for b_api_task in construct_blueapi_tasks_from_i15_1_experiment( + task.experiment + ) + ] + ) return call_list diff --git a/src/daq_queuing_service/task.py b/src/daq_queuing_service/task.py index cb56824..6303fa0 100644 --- a/src/daq_queuing_service/task.py +++ b/src/daq_queuing_service/task.py @@ -1,8 +1,9 @@ from collections.abc import Mapping +from enum import StrEnum from typing import Any, Self from uuid import uuid4 -from blueapi.service.model import StrEnum +from blueapi.service.model import TaskRequest from pydantic import BaseModel, Field, computed_field from daq_queuing_service.blueapi_interaction.blueapi_call import ( @@ -12,6 +13,25 @@ ) +class Sample(BaseModel): + name: str + id: str + data: Mapping[str, Any] + + +class ExperimentDefinition(BaseModel): + name: str + id: str + data: Mapping[str, Any] + + +class Experiment(BaseModel): + name: str + instrument_session: str + sample: Sample + experiment_definition: ExperimentDefinition + + def create_uuid_str() -> str: return str(uuid4()) @@ -24,19 +44,15 @@ class Status(StrEnum): CANCELLED = "Cancelled" -class ExperimentDefinition(BaseModel): - plan_name: str - sample_id: str - params: Mapping[str, Any] = Field( - description="Values for parameters to plan, if any", default_factory=dict - ) - instrument_session: str +class TaskKind(StrEnum): + EXPERIMENT = "Experiment" + PLAN = "Plan" class Task(BaseModel): - experiment_definition: ExperimentDefinition + experiment: Experiment | TaskRequest id: str = Field(default_factory=create_uuid_str) - blueapi_calls: list[BlueapiCall] = [] + blueapi_calls: list[BlueapiCall] = Field(default_factory=lambda: []) _cancelled: bool = False def cancel(self): @@ -65,13 +81,23 @@ def status(self) -> Status: return Status.IN_PROGRESS return Status.QUEUED + @computed_field + @property + def kind(self) -> TaskKind: + match self.experiment: + case Experiment(): + return TaskKind.EXPERIMENT + case TaskRequest(): + return TaskKind.PLAN + class TaskWithPosition(BaseModel): - experiment_definition: ExperimentDefinition + experiment: Experiment | TaskRequest id: str status: Status blueapi_calls: list[BlueapiCallResponse] position: int | None + kind: TaskKind @classmethod def from_task(cls, task: Task, position: int | None = None) -> Self: diff --git a/tests/unit_tests/conftest.py b/tests/unit_tests/conftest.py index e09b10c..bf045e7 100644 --- a/tests/unit_tests/conftest.py +++ b/tests/unit_tests/conftest.py @@ -9,7 +9,13 @@ from daq_queuing_service.plugins.construct_task_request import ( construct_blueapi_call_list, ) -from daq_queuing_service.task import ExperimentDefinition, Task, TaskWithPosition +from daq_queuing_service.task import ( + Experiment, + ExperimentDefinition, + Sample, + Task, + TaskWithPosition, +) from daq_queuing_service.task_queue.queue import TaskQueue @@ -24,8 +30,13 @@ def propagate_logs(monkeypatch: MonkeyPatch): def tasks() -> list[Task]: return [ Task( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id=str(i), params={}, instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id=str(i), data={} + ), + sample=Sample(name=f"test_8_{i}", id=str(i), data={}), ), id=str(i), ) @@ -51,13 +62,14 @@ def construct_blueapi_call_list( call_list: list[BlueapiCall] = [] for task in queue: for _ in range(2): + assert isinstance(task.experiment, Experiment) call_list.append( BlueapiCall( parent_task_id=task.id, task_request=TaskRequest( - name=task.experiment_definition.plan_name, - params=task.experiment_definition.params, - instrument_session=task.experiment_definition.instrument_session, + name=task.experiment.experiment_definition.name, + params=task.experiment.experiment_definition.data, + instrument_session=task.experiment.instrument_session, ), ) ) diff --git a/tests/unit_tests/plugins/test_construct_task_request.py b/tests/unit_tests/plugins/test_construct_task_request.py index 71947ed..60b1afb 100644 --- a/tests/unit_tests/plugins/test_construct_task_request.py +++ b/tests/unit_tests/plugins/test_construct_task_request.py @@ -3,16 +3,20 @@ from daq_queuing_service.plugins.construct_task_request import ( _construct_blueapi_task_request, ) -from daq_queuing_service.task import ExperimentDefinition +from daq_queuing_service.task import Experiment, ExperimentDefinition, Sample def test_construct_task_request_produces_expected_task_request_from_exp_definition(): result = _construct_blueapi_task_request( - experiment_definition=ExperimentDefinition( - plan_name="sleep", - sample_id="test_sample", - params={"time": 10}, + experiment=Experiment( + name="test_experiment", instrument_session="cm12345-1", + experiment_definition=ExperimentDefinition( + name="sleep", + id="", + data={"time": 10}, + ), + sample=Sample(name="test_sample", id="test_sample", data={}), ) ) assert result == TaskRequest( diff --git a/tests/unit_tests/plugins/test_i15_1_converter.py b/tests/unit_tests/plugins/test_i15_1_converter.py index cf5d1f0..16ecabd 100644 --- a/tests/unit_tests/plugins/test_i15_1_converter.py +++ b/tests/unit_tests/plugins/test_i15_1_converter.py @@ -1,33 +1,42 @@ from copy import deepcopy +from blueapi.service.model import TaskRequest + from daq_queuing_service.plugins.i15_1_converter import ( construct_blueapi_tasks_from_i15_1_experiment, construct_i15_1_blueapi_call_list, ) -from daq_queuing_service.task import ExperimentDefinition, Status, TaskWithPosition +from daq_queuing_service.task import ( + Experiment, + ExperimentDefinition, + Sample, + Status, + TaskKind, + TaskWithPosition, +) def test_given_sample_name_in_correct_format_then_correct_sample_loaded(): - experiment_definition = ExperimentDefinition( - plan_name="_", - sample_id="test_sample", - params={"sampleName": "test_8_1"}, + experiment = Experiment( + name="test_experiment", + experiment_definition=ExperimentDefinition(name=" ", id="", data={}), + sample=Sample(name="test_8_1", id="", data={}), instrument_session="cm12345-1", ) - tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment_definition) + tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment) assert tasks[0].name == "robot_load" assert tasks[0].params["position"] == "8" assert tasks[0].params["puck"] == "1" def test_sample_centre_uses_expected_params(): - experiment_definition = ExperimentDefinition( - plan_name="_", - sample_id="test_sample", - params={"sampleName": "test_8_1"}, + experiment = Experiment( + name="test_experiment", + experiment_definition=ExperimentDefinition(name=" ", id="", data={}), + sample=Sample(name="test_8_1", id="", data={}), instrument_session="cm12345-1", ) - tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment_definition) + tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment) assert tasks[1].name == "centre_sample" assert tasks[1].params == { "start_z": -5, @@ -38,53 +47,64 @@ def test_sample_centre_uses_expected_params(): def test_session_and_number_of_tasks_per_experiment_is_expected(): - experiment_definition = ExperimentDefinition( - plan_name="_", - sample_id="test_sample", - params={"sampleName": "test_8_1"}, + experiment = Experiment( + name="test_experiment", + experiment_definition=ExperimentDefinition(name=" ", id="", data={}), + sample=Sample(name="test_8_1", id="", data={}), instrument_session="cm12345-1", ) - tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment_definition) + tasks = construct_blueapi_tasks_from_i15_1_experiment(experiment) assert len(tasks) == 3 for task in tasks: assert task.instrument_session == "cm12345-1" -def test_experiment_with_correct_plan_name_are_converted(): - experiment_definition = ExperimentDefinition( - plan_name="run_full_collection", - sample_id="test_sample", - params={"sampleName": "test_8_1"}, +def test_experiment_with_correct_experiment_type_are_converted(): + experiment = Experiment( + name="test_experiment", + experiment_definition=ExperimentDefinition( + name="run_full_collection", id="", data={} + ), + sample=Sample(name="test_8_1", id="", data={}), instrument_session="cm12345-1", ) task = TaskWithPosition( - experiment_definition=experiment_definition, + experiment=experiment, id="1", status=Status.QUEUED, blueapi_calls=[], position=None, + kind=TaskKind.EXPERIMENT, ) call_list = construct_i15_1_blueapi_call_list([task], [], []) assert len(call_list) == 3 -def test_mix_of_experiments_with_correct_plan_name_are_converted(): - good_experiment_definition = ExperimentDefinition( - plan_name="run_full_collection", - sample_id="test_sample", - params={"sampleName": "test_8_1"}, +def test_mix_of_experiments_with_correct_experiment_type_are_converted(): + good_experiment = Experiment( + name="test_experiment", + experiment_definition=ExperimentDefinition( + name="run_full_collection", id="", data={} + ), + sample=Sample(name="test_8_1", id="", data={}), instrument_session="cm12345-1", ) good_task = TaskWithPosition( - experiment_definition=good_experiment_definition, + experiment=good_experiment, id="1", status=Status.QUEUED, blueapi_calls=[], position=None, + kind=TaskKind.EXPERIMENT, ) + class BadExperiment: ... + bad_task = deepcopy(good_task) - bad_task.experiment_definition.plan_name = "_" - tasks = [good_task, bad_task, good_task] + bad_task.experiment = BadExperiment() # type: ignore + plan_task = deepcopy(good_task) + plan_task.experiment = TaskRequest(name="", instrument_session="") + plan_task.kind = TaskKind.PLAN + tasks = [good_task, bad_task, plan_task, good_task] call_list = construct_i15_1_blueapi_call_list(tasks, [], []) - assert len(call_list) == 6 + assert len(call_list) == 7 diff --git a/tests/unit_tests/test_api.py b/tests/unit_tests/test_api.py index e4d648c..e7bce36 100644 --- a/tests/unit_tests/test_api.py +++ b/tests/unit_tests/test_api.py @@ -29,8 +29,8 @@ ) from daq_queuing_service.broadcaster import Broadcaster, Event from daq_queuing_service.task import ( - ExperimentDefinition, Status, + TaskKind, TaskWithPosition, ) from daq_queuing_service.task_queue.queue import QUEUE_EVENTS, TaskQueue @@ -111,11 +111,11 @@ def test_get_queued_tasks_returns_queued_task(test_client: TestClient): assert response.status_code == 200 assert response.json() == [ { - "experiment_definition": { - "plan_name": "test", - "sample_id": "2", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_2", "id": "2", "data": {}}, + "experiment_definition": {"name": "test", "id": "2", "data": {}}, }, "id": "2", "status": "In progress", @@ -136,13 +136,14 @@ def test_get_queued_tasks_returns_queued_task(test_client: TestClient): } ], "position": 0, + "kind": "Experiment", }, { - "experiment_definition": { - "plan_name": "test", - "sample_id": "3", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_3", "id": "3", "data": {}}, + "experiment_definition": {"name": "test", "id": "3", "data": {}}, }, "id": "3", "status": "Queued", @@ -163,13 +164,14 @@ def test_get_queued_tasks_returns_queued_task(test_client: TestClient): } ], "position": 1, + "kind": "Experiment", }, { - "experiment_definition": { - "plan_name": "test", - "sample_id": "4", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_4", "id": "4", "data": {}}, + "experiment_definition": {"name": "test", "id": "4", "data": {}}, }, "id": "4", "status": "Queued", @@ -190,6 +192,7 @@ def test_get_queued_tasks_returns_queued_task(test_client: TestClient): } ], "position": 2, + "kind": "Experiment", }, ] @@ -199,13 +202,14 @@ def test_get_queued_tasks_can_filter_by_task_status(test_client: TestClient): assert response.status_code == 200 assert response.json() == [ { - "experiment_definition": { - "plan_name": "test", - "sample_id": "2", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_2", "id": "2", "data": {}}, + "experiment_definition": {"name": "test", "id": "2", "data": {}}, }, "id": "2", + "status": "In progress", "blueapi_calls": [ { "task_request": { @@ -223,7 +227,7 @@ def test_get_queued_tasks_can_filter_by_task_status(test_client: TestClient): } ], "position": 0, - "status": "In progress", + "kind": "Experiment", } ] @@ -243,11 +247,11 @@ async def test_get_all_tasks_can_filter_by_task_status(test_client: TestClient): assert response.status_code == 200 assert response.json() == [ { - "experiment_definition": { - "plan_name": "test", - "sample_id": "1", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_1", "id": "1", "data": {}}, + "experiment_definition": {"name": "test", "id": "1", "data": {}}, }, "id": "1", "status": "Complete", @@ -272,7 +276,8 @@ async def test_get_all_tasks_can_filter_by_task_status(test_client: TestClient): } ], "position": None, - }, + "kind": "Experiment", + } ] @@ -293,8 +298,7 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( "/queue", json=[ { - "plan_name": "add_tasks", - "sample_id": "1", + "name": "add_tasks", "params": {"time": 10}, "instrument_session": "abc", } @@ -306,9 +310,8 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( [uuid.UUID(task_id) for task_id in task_ids] assert await task_queue_with_history.get_task_by_position(-1) == TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="add_tasks", - sample_id="1", + experiment=TaskRequest( + name="add_tasks", params={"time": 10}, instrument_session="abc", ), @@ -329,6 +332,7 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( ], position=3, status=Status.QUEUED, + kind=TaskKind.PLAN, ) @@ -338,8 +342,7 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( [ [ { - "plan_name": "add_tasks", - "sample_id": "1", + "name": "test", "params": {"time": 10}, "instrument_session": "abc", } @@ -351,9 +354,8 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( [ [ { - "plan_name": "add_tasks", + "name": "test", "params": {"time": 10}, - "instrument_session": "abc", } ], 0, @@ -362,14 +364,28 @@ async def test_add_tasks_to_queue_adds_to_queue_and_and_returns_task_ids( "detail": [ { "type": "missing", - "loc": ["body", 0, "sample_id"], + "loc": ["body", 0, "TaskRequest", "instrument_session"], "msg": "Field required", - "input": { - "plan_name": "add_tasks", - "params": {"time": 10}, - "instrument_session": "abc", - }, - } + "input": {"name": "test", "params": {"time": 10}}, + }, + { + "type": "missing", + "loc": ["body", 0, "Experiment", "instrument_session"], + "msg": "Field required", + "input": {"name": "test", "params": {"time": 10}}, + }, + { + "type": "missing", + "loc": ["body", 0, "Experiment", "sample"], + "msg": "Field required", + "input": {"name": "test", "params": {"time": 10}}, + }, + { + "type": "missing", + "loc": ["body", 0, "Experiment", "experiment_definition"], + "msg": "Field required", + "input": {"name": "test", "params": {"time": 10}}, + }, ] }, ], @@ -490,11 +506,11 @@ async def test_cancel_tasks_removes_task_from_queue_and_returns_tasks( assert not any(task_id in task_ids_after for task_id in last_two_task_ids) assert response.json() == [ { - "experiment_definition": { - "plan_name": "test", - "sample_id": "3", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_3", "id": "3", "data": {}}, + "experiment_definition": {"name": "test", "id": "3", "data": {}}, }, "id": "3", "status": "Cancelled", @@ -515,13 +531,14 @@ async def test_cancel_tasks_removes_task_from_queue_and_returns_tasks( } ], "position": None, + "kind": "Experiment", }, { - "experiment_definition": { - "plan_name": "test", - "sample_id": "4", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_4", "id": "4", "data": {}}, + "experiment_definition": {"name": "test", "id": "4", "data": {}}, }, "id": "4", "status": "Cancelled", @@ -542,6 +559,7 @@ async def test_cancel_tasks_removes_task_from_queue_and_returns_tasks( } ], "position": None, + "kind": "Experiment", }, ] @@ -616,11 +634,11 @@ async def test_cancel_all_tasks_removes_all_queued_tasks_from_queue_and_returns_ assert not any(task_id in task_ids_after for task_id in should_be_cancelled) assert response.json() == [ { - "experiment_definition": { - "plan_name": "test", - "sample_id": "3", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_3", "id": "3", "data": {}}, + "experiment_definition": {"name": "test", "id": "3", "data": {}}, }, "id": "3", "status": "Cancelled", @@ -641,13 +659,14 @@ async def test_cancel_all_tasks_removes_all_queued_tasks_from_queue_and_returns_ } ], "position": None, + "kind": "Experiment", }, { - "experiment_definition": { - "plan_name": "test", - "sample_id": "4", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_4", "id": "4", "data": {}}, + "experiment_definition": {"name": "test", "id": "4", "data": {}}, }, "id": "4", "status": "Cancelled", @@ -668,6 +687,7 @@ async def test_cancel_all_tasks_removes_all_queued_tasks_from_queue_and_returns_ } ], "position": None, + "kind": "Experiment", }, ] @@ -676,11 +696,11 @@ def test_get_task_by_position_returns_expected_task(test_client: TestClient): response = test_client.get("/queue/1") assert response.status_code == 200 assert response.json() == { - "experiment_definition": { - "plan_name": "test", - "sample_id": "3", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_3", "id": "3", "data": {}}, + "experiment_definition": {"name": "test", "id": "3", "data": {}}, }, "id": "3", "status": "Queued", @@ -701,6 +721,7 @@ def test_get_task_by_position_returns_expected_task(test_client: TestClient): } ], "position": 1, + "kind": "Experiment", } @@ -708,11 +729,11 @@ def test_get_task_by_id_returns_expected_task(test_client: TestClient): response = test_client.get("/tasks/3") assert response.status_code == 200 assert response.json() == { - "experiment_definition": { - "plan_name": "test", - "sample_id": "3", - "params": {}, + "experiment": { + "name": "test_experiment", "instrument_session": "", + "sample": {"name": "test_8_3", "id": "3", "data": {}}, + "experiment_definition": {"name": "test", "id": "3", "data": {}}, }, "id": "3", "status": "Queued", @@ -733,6 +754,7 @@ def test_get_task_by_id_returns_expected_task(test_client: TestClient): } ], "position": 1, + "kind": "Experiment", } @@ -782,8 +804,7 @@ class SomeError(QueueError): ... "/queue", json=[ { - "plan_name": "add_tasks", - "sample_id": "1", + "name": "add_tasks", "params": {"time": 10}, "instrument_session": "abc", } diff --git a/tests/unit_tests/test_queue.py b/tests/unit_tests/test_queue.py index 761127a..64c912a 100644 --- a/tests/unit_tests/test_queue.py +++ b/tests/unit_tests/test_queue.py @@ -15,7 +15,14 @@ from daq_queuing_service.plugins.construct_task_request import ( construct_blueapi_call_list, ) -from daq_queuing_service.task import ExperimentDefinition, Status, Task +from daq_queuing_service.task import ( + Experiment, + ExperimentDefinition, + Sample, + Status, + Task, + TaskKind, +) from daq_queuing_service.task_queue.queue import ( TaskQueue, TaskWithPosition, @@ -33,10 +40,17 @@ def make_new_task(id_str: str): return Task( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id=id_str, params={}, instrument_session="" - ), id=id_str, + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", + id=id_str, + data={}, + ), + sample=Sample(name="test_sample", id=id_str, data={}), + ), ) @@ -265,8 +279,13 @@ async def test_get_queue_only_returns_tasks_in_queue( result = await task_queue_with_history.get_queue() assert result == [ TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="2", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="2", data={} + ), + sample=Sample(name="test_8_2", id="2", data={}), ), id="2", status=Status.IN_PROGRESS, @@ -283,10 +302,16 @@ async def test_get_queue_only_returns_tasks_in_queue( ) ], position=0, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="3", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="3", data={} + ), + sample=Sample(name="test_8_3", id="3", data={}), ), id="3", status=Status.QUEUED, @@ -303,10 +328,16 @@ async def test_get_queue_only_returns_tasks_in_queue( ) ], position=1, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="4", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="4", data={} + ), + sample=Sample(name="test_8_4", id="4", data={}), ), id="4", status=Status.QUEUED, @@ -323,6 +354,7 @@ async def test_get_queue_only_returns_tasks_in_queue( ) ], position=2, + kind=TaskKind.EXPERIMENT, ), ] @@ -334,8 +366,13 @@ async def test_get_history_only_returns_tasks_in_history( result = await task_queue_with_history.get_history() assert result == [ TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="0", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="0", data={} + ), + sample=Sample(name="test_8_0", id="0", data={}), ), id="0", status=Status.ERROR, @@ -358,10 +395,16 @@ async def test_get_history_only_returns_tasks_in_history( ) ], position=None, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="1", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="1", data={} + ), + sample=Sample(name="test_8_1", id="1", data={}), ), id="1", status=Status.COMPLETE, @@ -378,6 +421,7 @@ async def test_get_history_only_returns_tasks_in_history( ) ], position=None, + kind=TaskKind.EXPERIMENT, ), ] @@ -390,8 +434,13 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( result = await task_queue_with_history.get_tasks() assert result == [ TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="0", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="0", data={} + ), + sample=Sample(name="test_8_0", id="0", data={}), ), id="0", status=Status.ERROR, @@ -414,10 +463,16 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( ) ], position=None, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="1", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="1", data={} + ), + sample=Sample(name="test_8_1", id="1", data={}), ), id="1", status=Status.COMPLETE, @@ -434,10 +489,16 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( ) ], position=None, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="2", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="2", data={} + ), + sample=Sample(name="test_8_2", id="2", data={}), ), id="2", status=Status.IN_PROGRESS, @@ -454,10 +515,16 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( ) ], position=0, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="3", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="3", data={} + ), + sample=Sample(name="test_8_3", id="3", data={}), ), id="3", status=Status.QUEUED, @@ -474,10 +541,16 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( ) ], position=1, + kind=TaskKind.EXPERIMENT, ), TaskWithPosition( - experiment_definition=ExperimentDefinition( - plan_name="test", sample_id="4", instrument_session="" + experiment=Experiment( + name="test_experiment", + instrument_session="", + experiment_definition=ExperimentDefinition( + name="test", id="4", data={} + ), + sample=Sample(name="test_8_4", id="4", data={}), ), id="4", status=Status.QUEUED, @@ -494,6 +567,7 @@ async def test_get_tasks_returns_tasks_in_queue_and_history( ) ], position=2, + kind=TaskKind.EXPERIMENT, ), ] diff --git a/tests/unit_tests/test_task.py b/tests/unit_tests/test_task.py index cfc6d1a..b67fe9c 100644 --- a/tests/unit_tests/test_task.py +++ b/tests/unit_tests/test_task.py @@ -2,7 +2,7 @@ from blueapi.service.model import TaskRequest from daq_queuing_service.blueapi_interaction.blueapi_call import BlueapiCall, CallStatus -from daq_queuing_service.task import ExperimentDefinition, Status, Task +from daq_queuing_service.task import Status, Task @pytest.mark.parametrize( @@ -36,8 +36,6 @@ def test_task_status_derived_correctly_from_call_statuses( ) for status in blueapi_call_statuses ], - experiment_definition=ExperimentDefinition( - plan_name="", sample_id="", instrument_session="" - ), + experiment=TaskRequest(name="", instrument_session=""), ) assert task.status == expected_task_status