Proposal: Runtime YellowPages for PyAML (auto-derived, regex-capable, and property-accessible)

[gupichon]

Context / Motivation

In practice, elements are addressed through different naming conventions. PyAML already provides:

• unique element identity via ElementHolder (single source of truth),
• named arrays (get_elements("QForTune"), etc.),
• set-like operations on arrays (|, &, -),
• selection by wildcard and by field (e.g. "hardware_name:srmag/*").

Yet HLAs and user scripts frequently need a stable way to resolve logical families such as:

• HorizontalCorrectors
• TuneQuadrupoles
• Cell C05 BPMs
• All elements matching a hardware regex

The PyAML specification mentions a YellowPages-like concept and dataset-based resolution. We propose implementing YellowPages as a runtime-only, auto-derived resolution layer attached to ElementHolder, without introducing a new YAML schema or Pydantic model.

---

Goals

1. No new configuration schema (no Pydantic YellowPages model).
2. YellowPages is auto-derived from existing configuration (elements + arrays already loaded).
3. YellowPages is enrichable:
   • by models during post_init(),
   • by users at runtime (API).
4. Regex support for selecting families dynamically.
5. register() provides attribute/property access (ergonomic holder.<family>).

---

Proposed API

Attached to ElementHolder:

sr.live.yellow_pages.get("QForTune")     # returns ElementArray-like
sr.live.yellow_pages.get("re:^QF")      # regex on name
sr.live.yellow_pages.get("re:hardware_name:^srmag/")  # regex on a property

sr.live.yellow_pages.register("OrbitCorrectors", hcorr | vcorr)
sr.live.OrbitCorrectors                 # attribute access (property-like)
corr = sr.live.yellow_pages.get("OrbitCorrectors")

sr.live.yellow_pages.register_regex(
    "AllC05", r"^.*-C05.*$", field="name"
)
sr.live.AllC05                          # resolved dynamically

---

Resolution rules (predictable)

yellow_pages.get(key) resolves in this order:

1. User/model registry (explicit registrations)
2. Existing named arrays in ElementHolder (get_elements, get_magnets, get_bpms, …)
3. Builtins (ALL_ELEMENTS, ALL_MAGNETS, …)
4. Regex keys (if key starts with re:)

---

“Properties” requirement

When a user/model registers a family name, that name should be accessible like a property:

sr.live.yellow_pages.register("TuneQuads", sr.live.get_magnets("QForTune"))
sr.live.TuneQuads

To avoid injecting class-level descriptors, this can be implemented via:

• ElementHolder.__getattr__ delegating to YellowPages when a matching key exists.

This provides “property-like” access without impacting other holders or requiring metaprogramming.

---

Why this fits PyAML

• Reuses ElementHolder as the authoritative object index.
• Reuses existing array semantics (filtering + set ops).
• Provides a clean bridge for HLA naming conventions.
• Adds a minimal runtime layer similar in spirit to ACCML YellowPages, but PyAML-native.
• Keeps configuration stable and avoids introducing a parallel schema to maintain.

---

Open Questions

• Do we want namespacing/datasets (ops, commissioning, …) as layered registries?
• Do we allow registered names to shadow existing arrays, or require explicit override?
• Should regex families be cached or always resolved live?
• Should we allow declaring regex-based arrays directly in the configuration, for example:

arrays:
  - type: pyaml.array.regex
    name: BPMCell04
    regex: ^BPM-C04

This would make regex-derived groups first-class arrays at load time rather than runtime-only YellowPages entries.

Architectural distinction

There is a conceptual difference between:

• Runtime YellowPages resolution
  Dynamic, contextual, and possibly user- or model-defined. These families may depend on runtime state, peer (design/live), or HLA logic.

• Configuration-defined regex arrays
  Deterministic, static groups resolved during configuration loading. They behave like any other array type and become part of the ElementHolder’s intrinsic structure.

If both mechanisms are supported, we should clarify:

• Whether pyaml.array.regex is a true array type resolved at load time,
• Or whether it is syntactic sugar that internally registers a YellowPages entry,
• And how precedence is handled between configuration arrays and runtime registrations.

[JeanLucPons]

I don't understand why a YellowPages layer is required ?
Why don't add methods directly to ElementHolder, there are already some defined ?
Concerning properties, it will rely on the name you give (compatible with python property naming) and can be done in ElementHolder.add() methods.
I do not see real need for additionnal register methods.

[gupichon]

Yes, this still works in my proposal. I only want to clarify that array creation happens at the Accelerator level.

sr.live.yellow_pages.get("QForTune")     # returns an ElementArray-like object
sr.live.yellow_pages.get("re:^QF")       # regex on name
sr.live.yellow_pages.get("re:hardware_name:^srmag/")  # regex on a property

sr.live.yellow_pages.register("OrbitCorrectors", hcorr | vcorr)
sr.live.OrbitCorrectors                  # attribute-like access (property style)
sr.design.OrbitCorrectors                # attribute-like access (property style)
sr.error[0].OrbitCorrectors              # attribute-like access (property style)
corr_live = sr.live.yellow_pages.get("OrbitCorrectors")

sr.live.yellow_pages.register_regex(
    "AllC05", r"^.*-C05.*$", field="name"
)
# AllC05 exist now everywhere
sr.live.AllC05 
sr.design.AllC05 
sr.error[0].AllC05 
sr.error[1].AllC05 

Arrays are defined globally at the Accelerator level but retrieved contextually through each environment (live, design, error, etc.).

[JeanLucPons]

I agree that things can be improved and I would prefer to add additional methods to ElementHolder rather than adding a yellow_pages field.

For the main the main interesting thing of yellow_pages is for instance to be able to write : quads = yp.get_quad_for_tune(). However you get a list of names and then you have to convert to an array attached to a CS or a Simulator to access it.

which is planned to be implemente as quads = sr.(live|desing).tune.get_quad() (suggestion of @GamelinAl)

Note that yp.get_quad_for_tune() is a limitation for us as we can have several tune tuning tools with different name. So we have at least to extend it to yp.get_quad_for_tune(tune_adjust_name). In any case, we can add a catalog of random names associated to a unique name and create arrays, but we should also be able to work without.

[JeanLucPons]

sr.live.yellow_pages.get("QForTune")     # returns an ElementArray-like object

For me, this is equivalent to what we already have:

sr.live.get_magnets("QForTune")     # returns an ElementArray-like object

[JeanLucPons]

note that, today YellowPages are handled in PyAML as named list of string in the config. Adding a simple way to retrieve this list of names (if needed) from the config at the accelrator level is doable rather easily.

[gubaidulinvadim]

sr.live.yellow_pages.get("QForTune")     # returns an ElementArray-like object

For me, this is equivalent to what we already have:

sr.live.get_magnets("QForTune")     # returns an ElementArray-like object

I think in this case the regular expessions still don't work. We can get a named array that is defined in the config file (or was created in the code previously and attached to the element holder). I'd say it would be useful to retrieve some array on the fly and to be able to register it to the element holder. At the time also for the registration step, I could not find myself if I can attach to both sr.live and sr.design with a single method. I've always attached to both. (It could be a problem with me, not with pyAML)

For example, I imagine this proposal could help with simplifying the following code for the user

hcorr_names = [
    mag.get_name()
    for mag in [sr.live|sr.design].get_all_magnets()
    if mag.get_name().endswith("hcorrector")
]
vcorr_names = [
    mag.get_name()
    for mag in [sr.live|sr.design].get_all_magnets()
    if mag.get_name().endswith("vcorrector")
]

[sr.live|sr.design].fill_magnet_array("HCORR", hcorr_names)
[sr.live|sr.design].fill_magnet_array("VCORR", vcorr_names)

I hope it is easier to do something like (not valid pseudocode ahead)

vcorrector_array = yellowpage.get('*.vcorrector'); yellowpage.register(vcorrector_array)

Edit: In this vcorr/hcorr example the config file did not have an array defined. So, for me the faster way was to create it this way.
Edit2: I think it would even be useful if we could find elements by name regexp and register them as an array to all control modes. Each control mode, after all, is supposed to have the same elements. Maybe even there's an easy way to determine the correct type of an array by checking individual element types.

[TeresiaOlsson]

It will take a while for me to have a detailed look at it because I have a backlog of maintenance I need to do first with MML and our cluster :/

Anyway, for me the yellow pages should be one layer up. So not sr.live.yellow_pages because then I can't use the yellow pages to understand that there is a live mode (and maybe many other modes too). For me this functionality should be at the Accelerator level so I can use it to browse the configuration, dynamically change it and get the elements and arrays I want for any mode.

But I like get much better than get_magnets. In MML there is the same way to get devices independently of what type it is. For me get_magnets was confusing because then I need to learn separate commands if I want to get devices for RF, BPMs etc. I prefer the MML approach with just the same command independently of device type.

I think the name yellow pages is not good. I already heard several people who has no intuitive feeling for what that means. I think it would be better to call it something like registry, catalog or config_manager. Some name that is easier to understand also for those who aren't familiar with old phone books where there was yellow pages.

Also, if there is sr.live.yellow_pages.get("^QF") can we also have sr.live.yellow_pages[^QF"] like in pyAT? Our users like that syntax a lot.

[gubaidulinvadim]

@JeanLucPons I think we all agree that sr.control_mode is very good and exactly what was specified as an interface. I agree that on the top level, maybe we can only have sr.get_some_names(), which is of questionable usefulness.

[TeresiaOlsson]

I know. I want the yellow pages to be able to give me that list of names. Otherwise how will the user know which various modes are available for them to use? I don't want to have to look in the yaml file or read the documentation to learn that there should be a live mode. I want to be able to load the configuration and ask yellow pages to list the different modes for me.

[gupichon]

Thanks for the feedback @TeresiaOlsson

Anyway, for me the yellow pages should be one layer up. So not sr.live.yellow_pages because then I can't use the yellow pages to understand that there is a live mode (and maybe many other modes too). For me this functionality should be at the Accelerator level so I can use it to browse the configuration, dynamically change it and get the elements and arrays I want for any mode.

The key point is that the objects retrieved by YellowPages are context-dependent. Therefore, they will not be the same in every mode (since the devices and/or the underlying control system may differ). However, I’ll try to find a way to address your problem. Could you provide some examples?

Also, if there is sr.live.yellow_pages.get("^QF") can we also have sr.live.yellow_pages[^QF"] like in pyAT? Our users like that syntax a lot.

Yes, if you want.

[TeresiaOlsson]

These are the steps I would do if I were a new user:

1. Load the configuration
2. Check the yellow pages and learn that there is live, design, twin etc.
3. Then think I want to use live and do something like sr.live.get("QF*) to get the elements.

[GamelinAl]

Anyway, for me the yellow pages should be one layer up. So not sr.live.yellow_pages because then I can't use the yellow pages to understand that there is a live mode (and maybe many other modes too). For me this functionality should be at the Accelerator level so I can use it to browse the configuration, dynamically change it and get the elements and arrays I want for any mode.

I am not sure about this. I agree that it's good to have a layer at the Accelerator level to describe the different modes but I think that the YellowPage (or whatever name) should returns object and to names.

But I like get much better than get_magnets. In MML there is the same way to get devices independently of what type it is. For me get_magnets was confusing because then I need to learn separate commands if I want to get devices for RF, BPMs etc. I prefer the MML approach with just the same command independently of device type.

I fully agree with this. This is quite confusing and unfriendly right now.

[TeresiaOlsson]

I think the information functionality of the yellow pages is more important than using it to extract the devices. At the workshop I saw that many people looked inside the yaml file to understand what had been configured, which modes where available, names of arrays etc. Then people got confused because the yaml file was long and difficult to browse. And that will also not work in the future when some labs instead of the yaml file reads the config directly from a database.

So we need some functionality which allows the user to see and understand what the configuration is so they later can make use if it. That's how I understood the purpose of the yellow pages. Like the phone book where I use it to look up who I want to call but not actually to call them. But maybe that's only how I understood it?

[gubaidulinvadim]

The long config file is our fault. It was just the config of all configurable elements of the SOLEIL II lattice, which has a lot of everything. Not the best for reading a config file, but it is useful if you want to play around with many different elements.

[TeresiaOlsson]

I think the long yaml file wasn't the problem. In the future I imagine a situation where the yaml file lives somewhere completely different in a separate repository on gitlab and the users will never go there and look at it. Maybe they will not even have permissions because they are a PhD student coming from another lab to do joint experiments with no access to our gitlab. Then they need some functionality to understand what our configuration is and a clear entry point for what to call to see it.

I think in MML this is clear. If I go to another lab where I know little about the machine, I at least know I can do getao and getad and I can learn all I need to start writing a script to do a measurement on that machine. It's that type of functionality I felt we were missing and why we had to look in the yaml file.

[kparasch]

Personally, I believe there is great potential in yellow pages if they can provide something like this:

print(sr.yellow_pages)

>>> Accelerators:
>>>     live
>>>     design
>>>     errors[0-20]
>>>     model
>>>     ...
>>> Tools:
>>>     DEFAULT_ORBIT_CORRECTION (pyaml.tuning_tools.orbit)
>>>     DEFAULT_TUNE_CORRECTION (pyaml.tuning_tools.tune)
>>>     DEFAULT_ORBIT_RESPONSE_MATRIX (pyaml.tuning_tools.orbit_response_matrix)
>>>     LOCO_ORBIT_RESPONSE_MATRIX (pyaml.tuning_tools.orbit_response_matrix)
>>>     ...
>>> Diagnostics:
>>>     DEFAULT_BETATRON_TUNE_MONITOR (pyaml.diagnostics.tune_monitor)
>>>     SPARE_BETATRON_TUNE_MONITOR (pyaml.diagnostics.tune_monitor)
>>>     Bunch_current_monitor (pyaml.diagnostics.current)
>>>     ...
>>> Arrays:
>>>     BPM (pyaml.arrays.bpm_array)
>>>     HCORR (pyaml.arrays.magnet_array)
>>>     VCORR (pyaml.arrays.magnet_array)
>>>     HCORR_LOCO (pyaml.arrays.magnet_array)
>>>     VCORR_LOCO (pyaml.arrays.magnet_array)
>>>     ...

[TeresiaOlsson]

Yes! That's exactly what I would want.

[TeresiaOlsson]

Then when I know that there are all of these things, I want the functionality to be able to browse what is the detailed configuration of a specific tool/array/device and if I discover something is wrong I can change it to test and debug without having to first go to the yaml file, change there and reload the configuration because that will be a too slow workflow.

[TeresiaOlsson]

As it was already mentioned. Without specifying sr.live or sr.design you can only get a list of names. this is what accml yp does. As already said, this sr.design, sr.live, sr.errors is the root of specification and impact all the design of PyAML. no switchtolive() or switchtosimu() as in MML. PyAML is built in an Object oriented style.

@JeanLucPons In case you don't already know because @simoneliuzzo also doesn't seem to know this? switch2online and switch2sim are just utility functions in MML that switches the default mode of all devices. You can use all modes simultaneously in MML too if you want. You just need to specify the mode flag in getpv. If you don't it uses the default mode you have specified for that device. The hardware to physics units work the same. There is a default mode which you can switch for all devices using the utility functions switch2hardware and switch2physics but you can always put the unit flag in getpv if you want to use another mode without switching. As I have understood it the sr.live in pyAML is just for user-friendliness for users who use IDEs but doesn't add any functionality compared to what is already in MML.

[JeanLucPons]

No sr.live sr.design sr.errors was rather a consequence of simulated commissioning and of LOCO.

[TeresiaOlsson]

I meant that the syntax could also have been sr('live'),sr('design') and sr('errors') which would be similar to the mode flag in MML.

[simoneliuzzo]

I think sr.live sr.design sr.errors is a rather nice approach.
It allows to switch the script globally, but also to access any control_mode at any time. I find this approach more flexible and user friendly that the switch2** features of MML. For me it is an improvement of pyAML compared to MML.
Also writing sr.control_mode is more pythonic than sr('control_mode').

[GamelinAl]

Thanks @gupichon for the proposition.

Here my very "user-oriented" view on the subject, I will try to describe the problem I would like to solve and explain why I think the present implementation does not provide this.

The problem

As a user, I would like to have an easy and direct access to:

1. The most general "master" ElementArray with everything inside (where I can used all the nice ElementArray feature of regex, sorting, ....)
2. The default arrays used by pyAML to do measurements (i.e. quads_for_tune, h_correctors, ...)
3. The user-specific arrays defined at configuration level (i.e. BPMs_in_Cell18, ...)
4. Instances of specific pyAML devices (Tune, ORM, Orbit, ... i.e. everything which is not a Magnet nor a BPM)
5. Possibly user-specific (named) arrays defined at runtime (if for this a specific "register" action is needed then it's fully fine for me)

These arrays should be directly exposed as properties and not obtained using their string name, so:

sr.live.yellow_pages.get("QForTune")     # not good ! 
sr.live.yellow_pages.QForTune     # perfect, I don't need to remember this particular name with caps, I can do tab completion

A few other important points :

• The way we access the arrays should be same on live and on design.
• In the measurements and in objects, we should access and use the same default arrays via the same interface.
  • quads = sr.live.tune.get_quad()is a reference to sr.live.yellow_pages.QForTune
  • If we modify a default array in yellow_pages, the modification is used in measurements.

If this can be done with ElementHolder or a new object I don't know but I think we should go toward this.

Why the present implementation needs to be improved

When you are a fresh pyAML user, most probably you never looked at the configuration (somebody set it up for you) so you don't know or remember the array names.

Initialization is fine:

sr = Accelerator.load("config.yaml") 
control_mode = sr.design

Now how do I get my "master array" with everything so I can use the sorting features? I needed help from Vadim to figure it out ...

names = [
    ele.get_name()
    for ele in control_mode.get_all_elements()
]
control_mode.fill_element_array("master", names)
master = control_mode.get_elements("master")

This "master" array is the basis from which you can create new array on the fly, so for me it should be easily available as property.

How can I access an predefined array in configuration? I need to know both the array name "HCOR" and the "type" of the array

hcor = control_mode.get_elements("HCOR") # does not work
hcor = control_mode.get_magnets("hcor") # does not work
hcor = control_mode.get_magnets("HCORR") # works

If it is accessible as property somewhere it is much easier.

Also there are already 29 (live|design).get_something methods exposed to the user ... I would say that's already too much and either we need something to provide a more convenient access (like YellowPages) or a more generic get method which does not discriminate if you want single/multiple element or element type.

[JeanLucPons]

I add that you have also to consider needs of developers and think about maintenance.
We (developer) work with tools that do not have constructed object (like in IPython).
We have to find best trade off.

[JeanLucPons]

I add that all additional layer need to be strongly typed.
For instance:
sr.design.tuning will be a "TuningToolHolder" confining the explicit list of default tuning tools and some additionnal get methods to get the non standard ones.

[TeresiaOlsson]

I don't think we should have a yellow pages field. That syntax is very confusing for me and I think we already have too many layers of attributes. That might work for the users who use tab completion but based on my experience many new Python users don't use this. Instead they print out objects to learn what is available inside.

For me the yellow pages is a completely separate class which has the functionality to manage the configuration and helping the users know which devices/arrays/tools that are available for them to use. I would rather do like in @kparasch suggestion and call the yellow pages on the Accelerator object to have an easy way to look inside and browse all the different layers of the accelerator object.

[gubaidulinvadim]

@JeanLucPons I don't really think anyone insists on having it as a separate layer. Let's see what Guillaume will propose today and if your requirements are met.

[gupichon]

I’m writing it right now and updating it while reading the latest comments.

[TeresiaOlsson]

@gupichon Can you perhaps explain a bit more what this means? If I understand it correctly it is a question about different ways the arrays could be defined and implemented?

There is a conceptual difference between:

Runtime YellowPages resolution
Dynamic, contextual, and possibly user- or model-defined. These families may depend on runtime state, peer (design/live), or HLA logic.

Configuration-defined regex arrays
Deterministic, static groups resolved during configuration loading. They behave like any other array type and become part of the ElementHolder’s intrinsic structure.

[TeresiaOlsson]

• If you define an array by regex in the configuration file, the expression is resolved once and only the resulting list of IDs is kept.

  • If you define it at runtime, the regex is preserved and the list is computed whenever needed. This allows you to define the array and then add elements to the control mode afterward:

    1. Load a basic configuration with some devices
    2. Create an array by regex or wildcards
       (array_manager.create_array(array_name="Cell04", expression="*-C04-*"))
    3. Add additional devices
    4. All devices matching the regex are present in the Cell04 array

After these discussions, I’m no longer sure this mechanism is useful. Please let me know what you think.

Ah, thank you. Then I understand. Actually, I like both of those things. The first one I would 100% use because it would make it much easier to write the configuration for both BESSY II and MLS. The second one I think our users would use because step i and ii sounds similar to how the wildcards work in pyAT which we use a lot. I have never thought about that it could be dynamic like in step iii and iv but I think if that was possible we probably would have some use case where it would be useful.

[TeresiaOlsson]

For me in this discussion, the 3 importants point are:

* YellowPages as @kparasch suggests (a dictionary that contains the config accessible at the accelerator level with customized repr).

* SubHolder (such as TuningToolHolder) as aggregates of ElementHolder to separate between various kind of object.

* get() methods with regexp() of fnmatch() at every SubHolder levels.

What is the purpose of the subholders? I didn't fully understand that in the discussion. Is it just for an easy way to get all tuning tools, all magnets etc. Similar to how you in pyAT can filter by type?

[JeanLucPons]

This will be handle by get("wildcard") functions that will return arrays.
I recall that to construct array you need to specify the Holder or to make it available from the Holder.
For configuration we will also integrate wildard to be able to write:

arrays:
  - type: pyaml.arrays.magnet
    name: QForTune
    elements:
      - QD*
      - QF*

and you will be able to do the same by code as already documented.

[JeanLucPons]

What is the purpose of the subholders? I didn't fully understand that in the discussion. Is it just for an easy way to get all tuning tools, all magnets etc. Similar to how you in pyAT can filter by type?

Yes

[TeresiaOlsson]

Yes, that point is about array creation; it is complementary to YellowPages. Initially, I wanted to merge the two concepts, which does not seem to be such a good idea after all…

I think separate the two concepts is a good idea but there could still be a common entry point for it for the users. In my very simplified demonstration of what I would want to replace the MML AO with that I showed once after looking at how the users at MAX IV use it, my thinking was that the Accelerator object would serve as this entry point. So you create the accelerator object and then that object also have the functionality to browse the configuration, change it, extract the devices, arrays etc. Then there is no separate yellow pages the user needs to know about since the functionality is hidden inside that class. Only the developers need to know that there is a yellow pages which is handling that functionality.

I still have this old idea in this repository https://github.com/python-accelerator-middle-layer/aml but after some more thinking about it I think it was too simplified and merged too many concerns into one class. Better would be to have separate objects inside the class which are responsible for the different functionalities. Like for example one config_manager, one catalog, one device_creator etc. But I still like the idea that there is the Accelerator which groups all of these functionalities together for the user. I think one reason why accml was so difficult to understand was because there was no such common grouping as an entry point for the user. So a new user doesn't know where to start and becomes confused.

[gupichon]

OK, many thanks to all for your contributions.
I will make a new proposal tomorrow morning. I have some ideas to satisfy everyone.

[gupichon]

Consolidated proposals

Following the discussion, the perspective is now the following:

YellowPages

• YellowPages becomes an Accelerator-level object.

• It is identifier-based only (no object return).

• It allows exploring:

  • available control modes
  • available arrays
  • element IDs contained in arrays

• It should also be accessible from control modes for convenience, so an HLA can simply use a control-mode object if sufficient:

  sr.live.yellow_pages

  (forwarding to the parent Accelerator).

---

Array creation

Arrays are created at the Accelerator level to ensure consistency across all control modes.

Creation supports:

• explicit list of IDs
• wildcards (*, ?)
• regex

All control modes receive the same arrays.

Open discussion: optionally allow ElementHolder.create_array(...), but delegate to the parent Accelerator if present.

---

Configuration extension

Allow declaring arrays in the configuration using (simplified examples):

arrays:
  - name: BPMCell04
    wildcards: "BPM*-C04*"
  - name: BPMCell04_regex
    regex: "^BPM-C04"
  - name: MyExplicitArray
    ids: ["BPM_C04-01", "BPM_C04-02"]
  - name: OrbitCorrectors
    expr: "HCORR | VCORR"
  - name: Cell04Quads
    expr: "(QF* | QD*) & re:.*-C04-.*"
  - name: ActiveQuads
    expr: "AllQuads - DisabledQuads"

Supported mechanisms:

• explicit list of IDs
• wildcards (*, ?)
• regex
• set operations on arrays (|, &, -, optionally parentheses)

This extends the existing ID-based array definition mechanism to allow declarative, composable array definitions directly in configuration.

---

ElementHolder API enrichment

Extend ElementHolder with enhanced search capabilities and simplified access methods:

• exact ID
• wildcards
• regex
• type

Add:

holder.get(query: str)
holder[query]  # alias

Returns:

• a single Element (exact match)
• or an ElementArray (multiple matches)

---

Typed sub-holders in ElementHolder

We could introduce strongly typed sub-holders under each control mode (e.g. tune, diagnostics, orbit):

tuning = sr.design.tuning  # -> TuningToolHolder
quads = sr.live.tune.get_quads()

These sub-holders would expose a clear, typed API instead of relying on generic access patterns.

Open question:

• How are they populated (configuration, model hooks, facility modules)?

---

Key features proposed (summary)

• Accelerator-level consistency,
• Identifier-based YellowPages exploration,
• Extended search capabilities,
• Regex/wildcard array definition both at runtime and in configuration.

The APIs and configuration examples above are intentionally simplified. The exact syntax and detailed behavior should be discussed and refined at the start of the implementation phase in a dedicated issue.

[gubaidulinvadim]

@simoneliuzzo , I don't think there are arrays of different types in these examples. One can always make an ElementArray just for some simple operations, such as identifying corrector magnets or BPMs in a particular cell. This functionality is already implemented. There's no point in trying to make a quadrupole-sextupole array because we should act on quadrupoles and sextupoles separately.

@JeanLucPons In examples of @GamelinAl this mixed-element array is only Cell1, no? It simply contains all elements in a Cell1. We already have possiblity to do that in pyAML, it's an ElementArray. It is not very useful as it just hold all the different elements. The only use so far was to test Set-like operations (& + - |) to isolate elements in a particular cell. I do not think a lot is necessary beyong that.

It's simply this in the config file:

type: pyaml.arrays.element
  name: Cell1
  elements:
  - BPM_001
  - SH1_COR_001
  - SH1_COR_001.sextupole
  - SH1_COR_001.hcorrector
  - SH1_COR_001.vcorrector
  - OH1_QCORROCT_1_QT_001
  - OH1_QCORROCT_1_QT_001.octupole
  - OH1_QCORROCT_1_QT_001.quadrupole
  - OH1_QCORROCT_1_QT_001.skewquad
  - SH2_HCOR_001
  - ...

[JeanLucPons]

Yes as i said, this is already done (set operations of array of elements). So no problem with that.

[gupichon]

@GamelinAl

• regarding Array creation, the proposal is just for "named arrays" ? It will still be possible to create "on the fly unnamed" sub-array, correct?

Yes. By an API in the ElementHolder (as a search result) or as a result of an operation (&, |, -,...)

• If I have these arrays defined in live mode, I can do Cell1_BPMs = Cell1 & BPMs. No need to define it at the Accelerator level?

It’s an unnamed array, so the same answer as before applies. If you want to have it at the Accelerator level, you can (through the API or via configuration), but it is not mandatory.

• regarding ElementHolder, can we also agree to expose the defined arrays as properties of the ElementHolder?

For me it's ok but the naming must respect python limitations obviously.

• Regarding the typed sub-holders, I am not sure to understand fully the consequences of this choice ...

  • For example this means a ORM object will not be held at the ElementHolder but only at the SubElementHolder level?

Honestly, I think this point needs to be clarified in a separate issue. I’m not sure users should be concerned about this matter, as long as it can be used where needed.

• Can we still have arrays which contains different types? For example BPM and Magnets?

It has been answer already. Yes, for convenience.

[JeanLucPons]

@GamelinAl
The SubHolder is not yet fully specified.
Concerning ORM, the problem is that you need several.

For instance one for OrbitCorr and one for LOCO. The 2 ones can be different. For instance, at ESRF, the one we use for LOCO is done on a subset of steerers otherwise it would take too long. This is why I would prefer to have 1 ORM per tuning tools with different config. We can keep at the top level but it will be confusing.
I would really prefer to have:

mat = sr.design.tuning.orbit.orm.measure(...)
# and
mat = sr.live.tuning.loco.orm.measure(...)

Which is more object oriented than:

mat = sr.live.orm.measure( my_setup_for_loco )
# or
mat = sr.desing.orm.measure( my_setup_for_orbitcorr )

[GamelinAl]

Ok, thanks for the answers, this is clear for me now and I think the proposal should solve my issues.

[gubaidulinvadim]

@gupichon @JeanLucPons I think it is OK to proceed to creating specific issues and continuing some discussion there whenever necessary. Do you agree?

• YellowPages issue (name lookup and Kostas' example)
• ElementHolder API / SubHolders (2 issues if necessary)

I would personally even say that the ElementHolder API is a more important issue. But it can be started with YellowPages as @JeanLucPons suggested.

[gupichon]

Yes, I think we can proceed with creating the issue.
I may include the Accelerator API and configuration evolutions as part of ‘ElementHolder API / SubHolders’ and perhaps name it ‘Array Management Enhancements’ or something similar. Or make more than one issue as you suggest...

YellowPages can be tricky. Everyone agrees on the services it should provide, but there is no obvious way to feed it.

We can proceed as you prefer.

[JeanLucPons]

Please make issue point by point otherwise code review will be impossible:

• YellowPages (as said, a simple dictionary with readable repr that give an overview of the setup @kparasch example is nice, i would also add size of arrays, thing like that)
• ElementHolder API & SubHolders
• WildCarded configuration

[JeanLucPons]

I create the issue for ElementHolder API #199

[gupichon]

Issue for YellowPages: #200. I haven’t described a precise interface yet; I focused on covering the requirements in this first draft. I’ll update the issue description as the discussion progresses.

[gupichon]

At least: #202 for wildcarded configuration