Issues with the settimecontext Function

[Merlencer]

Affiliation
ENN ERI

Question
The server side uses version 7.49 on Ubuntu 20.04 system, while the client side uses version 7.92 on Windows 11 system.
First issue: The adaxuv001 node stores data as (-1, 1.999998, 0.000002). When using the settimecontext function, calling settimecontext(0D0, 0.1D0, 1E-3) returns results as follows: According to the preset expectation, the first returned value should be the data at 0 seconds—what is the reason for this discrepancy?

Second issue: After uploading the time axis to the node via an array, the settimecontext function becomes ineffective. Support for this functionality will be added in subsequent versions.

[mwinkel-dev]

Hi @Merlencer -- My initial hunch is that this is a round-off issue, however I might be wrong. I will see if I can reproduce this issue later today.

[mwinkel-dev]

Hi @Merlencer -- Am I correct that the adaxuv001 node is a "segmented" signal sampled at 500 kHz for three seconds (-1.0 to 2.0)? If so, does the settimecontext() work at the beginning of the signal (e.g., -1.0 to -0.9, downsampled at a 0.001 interval)?

Regarding the time range you specified, 0.0 to 0.1 downsampled to a 0.001 delta, the output you posted is not what I would expect. So something is awry. I will do some experiments to see what might be causing that output.

[mwinkel-dev]

Hi @Merlencer -- I have reproduced the problem using a 4K samples/second segmented signal. It is not displaying the first time nor the last time, plus also only returns "midpoint" times (i.e., in the center of the specified time step).

This is definitely not round off error. Thus my initial conjecture was wrong.

I will talk to the developer(s) that designed the segmented records to find out the reason for this behavior. (When downsampling, there are several possible choices: beginning of the interval, end of the interval, midpoint, or nearest point. Presumably there was a reason for choosing to use the midpoint.)

[Merlencer]

Hi @Merlencer -- Am I correct that the adaxuv001 node is a "segmented" signal sampled at 500 kHz for three seconds (-1.0 to 2.0)? If so, does the settimecontext() work at the beginning of the signal (e.g., -1.0 to -0.9, downsampled at a 0.001 interval)?

Regarding the time range you specified, 0.0 to 0.1 downsampled to a 0.001 delta, the output you posted is not what I would expect. So something is awry. I will do some experiments to see what might be causing that output.

I have another question: Suppose I have a node a. In my Python program, I first delete the node’s data and then upload new data to it, but the datafile keeps growing larger and larger. What is the reason for this? Additionally, does MDSplus support data querying across different versions? Does it have a feature similar to IMAS run number for versioning?

[mwinkel-dev]

Hi @Merlencer -- Thanks for the screenshot.

Here are answers to the new questions.

DATAFILE: When a plasma pulse ends on a tokamak, the data in the digitizers needs to be quickly stored. Furthermore, the physicists and operators need to quickly analyze the results so they can make adjustments for the next plasma pulse. MDSplus was thus designed for fast write performance and rapid inter-pulse analysis. It is faster to mark data as "no longer valid" than to actually delete it and compress the file. Thus to boost write performance, MDSplus just appends the new data to the end of the datafile. Thus, as you noticed, the datafile always grows.

Note though that the mdstcl utility does have a COMPRESS command that can be used to do the actual deletion and shrink the datafile. For example, if you need to share a large tree with a colleague in another country, you could first use the COMPRESS command to reduce the file size, and then upload the tree to a "cloud" account that is shared with your collaborator.
https://www.mdsplus.org/index.php/Documentation:Reference:TCL_0#COMPRESS

VERSIONS: Do you mean different versions of MDSplus? Or "versioned data"?

• Yes, new versions of MDSplus can read data from trees written with old versions of MDSplus.
• A common way to handle versioned data is simply to have different nodes in the tree. For example, "raw_data", "calibrated", "analyzed_1", and "analyze_2". When the shot is run, the digitizer's data is recorded in "raw_data" and a formula in "calibrated" applies the calibration constants. Later on, a Python program is run to analyze the calibrated data and writes the results to "analyzed_1". Another researcher wants the analysis done differently so runs a second Python program that reads the calibrated data and writes "analyze_2". This same approach can be used to store results from multiple simulation runs.
• Yes, MDSplus also supports "versioned nodes". This is a seldom used feature.
  https://www.mdsplus.org/index.php/Documentation:Users:Versions

[mwinkel-dev]

This is another page describing versioned nodes.

https://www.mdsplus.org/index.php?title=Documentation:Tutorial:Using%20Data%20Versioning&open=101583194294748512767&page=Documentation%2FTutorials%2FUsing+Data+Versioning

[mwinkel-dev]

Hi @Merlencer -- When dealing with segmented records and the settimecontext() function, note that it uses a default routine to do the downsampling. That routine supports these different sampling modes: Average, MinMax, Interpolation, Closest, and Previous. The default mode is Average, which does indeed report data at the midpoint of the "delta" time interval used for resampling.

Also, it is possible to supply you own resampling function (see the very end of this page).
https://www.mdsplus.org/index.php/LongPulseExtensions

I have not experimented to see if these features are available through the API that MDSplus provides for Python.