spotfire type information can be lost in an unexpected manner

[tibco-slolla]

Type information set through set_spotfire_types can be lost in unexpected ways through operations on the data.

For example:

import pandas as pd
import spotfire

data = pd.DataFrame({
    "A": [1, 2, 3],
    "B": [4.4, 5.5, 6.6],
    "C": ["apple", "banana", "cherry"]
})

type_info_orig = {
    "A": "LongInteger",
    "B": "Real",
    "C": "String"
}

spotfire.set_spotfire_types(data, type_info_orig)

type_info_1 = spotfire.get_spotfire_types(data)
print(f"{type_info_1=}")

# Operations like the following cause
# a loss of stored Spotfire types
# even though we wouldn't expect it.

# data.reset_index(drop=True, inplace=True) # -- example 1
# data.fillna(0, inplace=True) # -- example 2
# data.dropna(inplace=True) # -- example 3

type_info_2 = spotfire.get_spotfire_types(data)
print(f"{type_info_2=}")

pd.testing.assert_series_equal(type_info_1, type_info_2)

This is probably due to the use of pandas.DataFrame.attrs to store the information.

Note

attrs is marked as an experimental feature that might change without warning.