Generic pyAML measurements

[gubaidulinvadim]

The aim of this discussion is to identify measurement components that are present in all standard measurements and the level of customisation that is necessary. Please provide your input, based on this input, a suitable implementation can be proposed.

As part of the pyAML project, we aim to standardise how measurements are conducted, logged, and reused across different control modes (live, model, shadow, etc.). The goal is to create a flexible, reusable, and maintainable framework for measurements—such as orbit response matrices, tune corrections, and dispersion measurements, etc. Robustness, error handling, and user customisation should be ensured.
This discussion is based on the pyAML User Interface Specification (Chapter 6) and recent feedback from the dedicated working group (https://github.com/orgs/python-accelerator-middle-layer/discussions/41).

Use Case Definition

We want to enable users to:

• Run standardised measurements (e.g., ORM, tune correction) across different control modes.
• Customise measurement behaviour via callbacks (e.g., error handling, intermediate checks).
• Ensure measurements are reproducible, with metadata and state management.
• Measurement log generation
• Handle interruptions (e.g., Ctrl+C) gracefully and restore initial states.
• Reusability of common parts between different measurements (Chromaticity and Dispersion measurement, for example)
• Support real-time feedback (e.g., progress bars, interactive plotting).

Possible generic stages/components of any measurement are:

1. Initial data loading (devices to use, response matrices loaded, etc.)
2. Metadata
3. Saving the initial state / final state and going back
4. General function to take data + metadata
5. Measurement logging, possible progress bar, callbacks
6. Optional intermediate checks for when to record data or interrupt measurement
7. Saving of measurement log
8. Measurement output saving
9. Measurement output analysis
10. Interactive plotting

[TeresiaOlsson]

These are the features I have come up with so far that we want at HZB. Depending on what happens with the options for bluesky compatible devices maybe not all need to be available in the standard measurement for pyAML. For us it would be fine to eventually implement separate bluesky-based applications for some measurements and make them available as part of the ecosystem for those labs who want features that aren't available in the standard.

Top priority:

These two are top priority for us because without it there is no point for us to test in the control room since the data won't be useful and the applications will anyway need to be rewritten when these missing features from the specification is implemented.

• Option to choose if measurement should be run in hardware or physics units without requirement to have to configure the other mode unless is is necessary for the specific application. For example, LOCO will require to have configured a model and conversion factors but most applications don't need this if you have simple magnets and just want to run in hardware units.
• No sleeps at application level. The devices should tell the application when a change is finished and if it was successful or timed out. This should control the execution of the application.

General features:

• Save data for each step in the measurement process and not just the final output so you can check what happened and debug if required.
• Separation of measurement data and analysis so you easily can rerun the analysis if required.
• Save data to a standard format including metadata.
• Be able to define configuration signals which are read and saved as part of the metadata but not as part of the measurement.
• Option to save to file or to database.
• Be able to define checkpoints which allows you to pause and resume a measurement at safe positions.
• Be able to cancel a measurement and the initial settings are automatically restored.
• Possibility to define separate setup and restore stages for devices which require changing of configuration and other preparations before a measurement.
• Live plotting which can be turned on/off as desired.
• Save to elog if desired.
• Streaming updates if desired to be able to monitor a measurement when not in the control room.

[JeanLucPons]

In addition, I add that we would like to:

Having callback (or generator) to:

• Add user specific additional task to be performed after or before measurement.
• Implement scan progress (including stage data) for graphical applications , servers
• Interrupt the process (by hand or programmatically), a graphical application can have an "Abort" button.
• Save user specific additional data at speficic location (file system, DB etc)
• User specific Logging

It is important to define what is a Stage:

• i.e.: Orbit averaging in separate stages ?

It is import to define the way of working:

• Separate scan and analysis: I mean that the scan process is not aware of what is performed and it is up to the user to load tuning or analysis tools with appropriate data afterwards ?
• Having dedicated measurement tools such as: Dispersion, Chroma , ORM ?

I would like to end with a clear interface such as:
Here is a naive example:

   
class Stage(ABC):    
    @abstractmethod
    run(self):...
        
class ORMStageConfig(BaseModel):
    step: int
    action: int
    steerer: Magnet
    horbit: np.array
    vorbit: np.array
    .....
    
 case ORMStage(Stage):
    run():
       do_something()


class Measurement(ABC):

    # Run all stages
    @abstractmethod
    def run(self, callback:Callable): ...

    # Run stages
    @abstractmethod
    def run(self, stages: list[Stages], callback:Callable): ...

    # Retreive data
    @abstractmethod
    def get(self) -> Data: ...

   etc

class ORMMeasurementConfig(BaseModel):
    hsteerers: MagnetArray
    vsteerers: MagnetArray
    bpms: BPMArray
    

class ORMMeasurement(Measurement):

    def __init__(self, cfg:ORMMeasurementConfig):
       # Create Stages
       
     def get_stages() -> list[ORMStage]:
       # Return stages
       
     def run_stage() -> StageData:
       do_something()

   def run(self, callback:Callable): 
      # Run all
       
   def run(self, stages: list[ORMStage], callback:Callable): 
      # Run list of stages
 

[TeresiaOlsson]

I think if the user always has to load the analysis tool themselves afterwards that will quickly be annoying. But at the same time we want to be able to replace the analysis part and also sometimes only rerun that part. That's where I think @yhidaka s suggestion of flows become especially useful. To me it sounds like you have separated the different steps in pieces and then you can put as many pieces as you want together. So you could for example have a measurement flow that by default always includes a certain analysis step but if you want you can replace that piece with another or execute only the beginning or the end of the flow.

[TeresiaOlsson]

I also like the suggestion @yhidaka had of being able to switch out a device which produces a value from just reading a PV to a device which gets it from doing a measurement. He mentioned the tune as example but I think this is also valid for implementing a general response matrix measurement. In MML there is such a measurement which takes a set of actuators and detectors and the orbit and tune response matrices are just implemented as calling that general measurement with specific devices. The chromaticity response matrix however is implemented differently because it requires to measure the chromaticity at each step. But if the chromaticity measurement had been implemented in a way where it could be handled similarly as device the general response matrix measurement could also have been used for that instead of having a completely separate implementation to maintain.

There are likely also many other use cases where you would like to measure a response matrix but the detector is something that you can't directly get from the control system and need to measure.

[JeanLucPons]

Yes the idea is the same for all. You have a Tune object the returns tunex, tuney and in the same manner we should have a chromaticity object that should return chromax, chromay then in the Stage object you will find the measruement tool (i.e: tune monitor or chroma_monitor).
i.e.

class ChromaticityResponseStageConfig(BaseModel):
    step: int
    action: int
    chroma:  ChomaticityMonitor
    sextu: Magnet
    .....
    
 case ChromaticityResponseStage(Stage):
    run():
       
       sextu.strenght.set(...) # sextu exitation
       mat.append((chroma0-chroma.get(fit_dispersion=False))/dk) # chroma.get will launch a chroma scan
       sextu.strenght.set(...) # sextu restore
       
class  ChromaticityResponseMeasurementConfig(BaseModel):
    chroma: ChomaticityMonitor
    sextu: MagnetArray
    
class ChromaticityResponseMeasurement(Measurement):

    def __init__(self, cfg:ChromaticityResponseMeasurement):       
        

[pcsch]

We have previously built a measurement framework at KIT and I was asked for a brief description. The framework is intended to provide an easy entry point to create measurements that can be exposed via the control system (EPICS in our case). The idea is to allow people without much understanding of the control system to create measurements integrated in the control system (as IOC/device).

Structure

Measurements are divided into three main blocks implemented for each measurement:

• prepare: get reference values etc.
• measure: actual measurement logic
• post_measure: implement analysis of acquired data and potential "manual" clean up

The implementation of the measurement block is intentionally left to the user. Having too strict rules about how to implement/segment the measurement lifts the entry barrier for new people and we tried to keep the barrier as low as possible.

Actions and Data Acquisition

The framework works with Monitor (reading data) and Actuator (controlling stuff) objects which might use the control system but can also perform other tasks or combine multiple calls in one. They work by implementing subclasses for new/complex tasks (Some implementations for simple tasks are provided).

Reading data is done by a read_data method of the measurement base class. This takes (by default) all registered Monitors and Actuators and reads their data with configured metadata (monitors and actuators can be explicitly specified to only read a subset).
We opted to do it this way instead of leaving reading monitors individually to the user because like this we can put the data in an internal data store that is later used to generate the output hdf5 file in a predefined structure and store data even in case of problems.

Resetting and Error Handling

Each Actuator is responsible for resetting it's value when a measurement finishes, crashes or is aborted.
For this, the framework, at the beginning a prepare method for each Actuator is called and at the end (finished, crashed or aborted measurement), a clean_up method is called.
The clean_up routine is implemented in each Actuator type individually in order to allow complex reset routines that are not just resetting an initial value. (Although for the default EpicsActuator it's just that.)
The framework tries to capture as many exceptions as possible, gracefully aborts the measurements and resets the state.
What is very helpful is that even if the measurement is aborted or throws an exception, the data, that is gathered until then is still saved to disk. There is also an option to "stream" data to disk as it arrives that ensures it's saved even in case there is a hard python crash (segmentation fault, OOM, etc.).

Result Analysis and Output File

Analysis of the measurement is done in the post_measure method, intentionally separated from the measurement routine. The return value of the post_measure method, used for analysis, is a dictionary (possibly nested) that will be stored in a Results group in the final hdf5 file. All in-memory data (for the actual measurement and for the analysis result) is saved to a single hdf5 file with a predefined structure with metadata. In case the Monitors created files instead of in-memory data (some measurement systems produce GB/TB worth of data), all files are gathered and put into a single tar archive together with an hdf5 file of the rest of the data. In that case, the hdf5 file contains the filenames of the data files at the relevant places.
Metadata for Actuators and Monitors always contains a timestamp. This allows to cross check with other values in our database.

Interfaces and Configuration

In addition to the pure measurement and data handling routines, the framework implements configuration and output capabilities. Just by defining configuration and output options (including data type, description etc.) when implementing the measurement, without further code, it's possible to execute it via command line, via python, via a stand alone (auto generated) gui and as an EPICS IOC via the control system.
For the last one we also implemented automated panel building for our operator gui (CSS).
This makes it very quick (and does not require much knowledge) to implement a full package from measurement to IOC to panels and put it in front of operators in their familiar interface.
The framework implements an easy way to configure automated posting of a (definable) subset of the results to our elog instance.
There is general logging implemented in the framework and a logger is provided for logging during the measurement. The output is routed to the interface that is used to execute the measurement (command line/python/GUI/IOC).

Some important points are:

• Being able to abort measurements in an orderly manner is important, including resetting the system afterwards
• Data being saved by the framework and not the measurement ensures a uniform structure and allows centralised data handling
• Running the measurement as an IOC allows access from everywhere and enables archiving of results in the normal database used for other PVs
• Gracefully handling errors/exceptions/crashes avoids loss of data
• Too many restrictions on the implementation raise the entry barrier and new comers have a hard time adapting and might not use the system at all
• Handling resetting in Actuator objects allows reusable complex reset procedures
• Dividing measurement and analysis allows rerunning an analysis on previous data

[JeanLucPons]

Yes this is a point we need to address (rerunning analyis).
Typically all "fit" methods, of all analysis tools should be able to get input data from the user.
Yes of course, pyaml should be able to be used inside an IOC or a Tango server.
There is already an basic example of a Tango server for an ORM.