Skip to content

[Python] test_categorical_order_survives_roundtrip fails on wheels with Pandas Deprecation #50579

Description

@raulcd

Describe the bug, including details regarding any error messages, version, and platform.

Several nightly wheels:

Have failed during the last nightlies with:

 __________________ test_categorical_order_survives_roundtrip ___________________

    @pytest.mark.pandas
    def test_categorical_order_survives_roundtrip():
        # ARROW-6302
>       df = pd.DataFrame({"a": pd.Categorical(
            ["a", "b", "c", "a"], categories=["b", "c", "d"], ordered=True)})

/tmp/arrow-HEAD.ZW7lN/venv-wheel-3.12-manylinux_2_28_x86_64/lib64/python3.12/site-packages/pyarrow/tests/parquet/test_pandas.py:474: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/arrow-HEAD.ZW7lN/venv-wheel-3.12-manylinux_2_28_x86_64/lib64/python3.12/site-packages/pandas/core/arrays/categorical.py:504: in __init__
    codes = _get_codes_for_values(values, dtype.categories)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

values = <ArrowStringArray>
['a', 'b', 'c', 'a']
Length: 4, dtype: str
categories = Index(['b', 'c', 'd'], dtype='str')

    def _get_codes_for_values(
        values: Index | Series | ExtensionArray | np.ndarray,
        categories: Index,
    ) -> np.ndarray:
        """
        utility routine to turn values into codes given the specified categories
    
        If `values` is known to be a Categorical, use recode_for_categories instead.
        """
        codes = categories.get_indexer_for(values)
        wrong = (codes == -1) & ~isna(values)
        if wrong.any():
>           warnings.warn(
                "Constructing a Categorical with a dtype and values containing "
                "non-null entries not in that dtype's categories is deprecated "
                "and will raise in a future version.",
                Pandas4Warning,
                stacklevel=find_stack_level(),
            )
E           pandas.errors.Pandas4Warning: Constructing a Categorical with a dtype and values containing non-null entries not in that dtype's categories is deprecated and will raise in a future version.

/tmp/arrow-HEAD.ZW7lN/venv-wheel-3.12-manylinux_2_28_x86_64/lib64/python3.12/site-packages/pandas/core/arrays/categorical.py:3045: Pandas4Warning

The one investigated seems to be using:

Downloading pandas-3.0.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (10.9 MB)

Component(s)

Python

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions