[WIP] XAS specialization using subclasses

[mretegan]

Another possibility for having fields that are acquisition mode dependent is to use subclasses as suggested here: nexusformat#1352 (comment)

This is a reasonable alternative, as it is unlikely that we will be able to define acquisition modes that will be reused by other techniques (the alternative option proposed above and implemented here: #6).

The two options were discussed in the NIAC https://www.nexusformat.org/Telco_20260211.html

[maurov]

@mretegan @woutdenolf @newville

My understanding of the NIAC minutes of Telco 20260211 is that both options are OK. As said many times, my position is to go for NeXus base classes representing the experimental data collection modes, as I wrote in the famous shared Google document long time ago. This solution has the strong advantage that then the base classes can be reused by other application definitions, like XMCD and other techniques. I do not understand why we are still hesitating on this. Please, let's move on.

As a first start, I propose implementing the basic modes that represent most of the XAS and other techniques' data:

• NXtrans: transmission, valid for any technique and any wavelength measuring sample absorption;
• NXtfy: total fluorescence yield;
• NXpfy: partial fluorescence yield;
• NXherfd: particular case of partial fluorescence yield with a high resolution spectrometer;

As an alternative, for the fluorescence yield (in view of the electron yield or the optical yield), we may adopt the subclass approach:

• NXfy base class:
  • NXfy_total
  • NXfy_partial
  • NXfy_herfd

But I think this is just a complication and I would just go for the first approach of base classes for each experimental collection mode.

[newville]

@maurov @mretegan Thanks (and sorry for the delay). I'm OK with either approach. I see the modes as mostly informative, as they suggest (but do not necessarily require) changes in the processing and analysis. But those steps are not really fixed anyway, so mode is mostly a "type hint".

[mretegan]

Thank you both for your input. I think that to have a complete picture, it will make sense to finish both in parallel and to create a few HDF5 examples. I will start working on this tomorrow.

[maurov]

@maurov @mretegan Thanks (and sorry for the delay). I'm OK with either approach. I see the modes as mostly informative, as they suggest (but do not necessarily require) changes in the processing and analysis. But those steps are not really fixed anyway, so mode is mostly a "type hint".

Hi @newville thanks for your feedback. To me, the modes are more than just informative. They tell exactly what is "mu" and how it was measured. For example, Fe K-edge XAS "mu" of Fe2O3 measured in transmission is a different thing than Fe K-edge XAS "mu" of Fe2O3 measured in HERFD. Furthermore, the "minimum required metadata" for transmission are not the same as HERFD.

[maurov]

Thank you both for your input. I think that to have a complete picture, it will make sense to finish both in parallel and to create a few HDF5 examples. I will start working on this tomorrow.

@mretegan for me it is very difficult to read the .nxdl.xml files directly. Would it be possible to have an automatic build of the documentation on the ESRF gitlab server? Or link here two HDF5 files generated following the two approaches. By the way, I do not think that our decision should be based on HDF5 readability, as most likely are our software tools that will read the HDF5 file, not humans.

[newville]

@maurov I would be cautious about being overly strict here.

Yes, data measured in different modes are different, and processing/analysis may want to do different steps based on the mode. And the mode should be stated.

And, yes, HERFD really ought to state energy analyzed (but that is also a trusted value), but is NeXuS going to say that a file is invalid if it states mode="HERFD" but does not correctly spell "analyzed energy" in every group?

[mretegan]

Thank you both for your input. I think that to have a complete picture, it will make sense to finish both in parallel and to create a few HDF5 examples. I will start working on this tomorrow.

@mretegan for me it is very difficult to read the .nxdl.xml files directly. Would it be possible to have an automatic build of the documentation on the ESRF gitlab server? Or link here two HDF5 files generated following the two approaches. By the way, I do not think that our decision should be based on HDF5 readability, as most likely are our software tools that will read the HDF5 file, not humans.

We have something set up, but it builds the main branch, and every time we want to switch, we need to update the CI file https://gitlab.esrf.fr/hdf5/nexus/nxxas.

You can also build locally. Go to the nexus_definitions folder and run (I use uv, but vanilla pip should work):

uv venv --python 3.12
source .venv/bin/activate
uv pip install -r requirements.txt 
make clean; make local
firefox build/manual/build/html/classes/applications/NXxas_new.html

[newville]

@mretegan @maurov Thanks. I don't really disagree with any of the changes here.

But getting bogged down on how to spell different types of emissions seems like both a killer of motivation and a detail that does not dramatically change the downstream use of the normalized mu data. Yes, it is helpful metadata, and can sometimes be important for comparing data. But detection modes can vary and evolve, and you may not know every possible data collection mode, and metadata can be messy, incomplete, or wrong. Still, an nxXAS group with normalized mu(E) is useful.

Please get this merged, and let us start using this. The danger that data collection and analysis applications will define their own HDF5 format and see no need to support this one is very real.

[newville]

@mretegan Why is Iref removed? For many people, the unbreakable coupling of an XAS measurement with a reference channel is vital. This really needs to be supported.

[maurov]

@mretegan Why is Iref removed? For many people, the unbreakable coupling of an XAS measurement with a reference channel is vital. This really needs to be supported.

@newville the idea is to substitute Iref with a ref subgroup that will be itself of NXXas type. In fact, it may happen that the "energy reference spectrum" is not measured as simply the "measured beam intensity after a reference foil, (= Iref)", but measured separately and in another mode/conditions. A typical case (even if it is less common) would be measuring a reference foil with an absorption edge close to the element/edge of interest in fluorescence mode and/or in a separate scan (e.g. for laboratory instruments).

[newville]

@mretegan Yes, of course, a separate spectrum can act as a reference signal.

Many people and many beamlines also often collect a reference spectrum in the same scan. This is more than an additional reference spectrum - it is unambiguously the same energy scan measuring multiple spectra simultaneously, and cannot be separated from the original spectrum.

And, yes, many beamlines at modern facilities have sufficiently stable energies and do not require this. But there are older spectra and older beamlines that do really need this.