feat: Add is_into_lazyframe, export missing functionalities, fix SQLFrame bug#3613
feat: Add is_into_lazyframe, export missing functionalities, fix SQLFrame bug#3613FBruzzesi wants to merge 25 commits into
is_into_lazyframe, export missing functionalities, fix SQLFrame bug#3613Conversation
|
downstream marimo is affected. I will investigate later today |
|
|
||
| def is_into_dataframe(native_dataframe: Any | IntoDataFrameT) -> TypeIs[IntoDataFrameT]: | ||
| """Check whether `native_dataframe` can be converted to a narwhals.stable.v1.DataFrame.""" | ||
| from narwhals.stable.v1 import DataFrame |
There was a problem hiding this comment.
Thoughts on pulling DataFrame from v1 here? Should we do the same for v2?
There was a problem hiding this comment.
I'm not sure actually.
Usually I'd say yes, but from_native converts between versions now. So maybe there's not a benefit to the version distinction?
| if is_narwhals_dataframe(frame) or is_narwhals_lazyframe(frame): | ||
| from narwhals._utils import qualified_type_name | ||
|
|
||
| caller = sys._getframe(1).f_code.co_name |
There was a problem hiding this comment.
Well aren't you a fancy boi 😄
dangotbanned
left a comment
There was a problem hiding this comment.
Thanks @FBruzzesi for powering through this
| or is_ibis_table(native_dataframe) | ||
| or is_duckdb_relation(native_dataframe) |
There was a problem hiding this comment.
i'm confused, doesn't the docstring say that these aren't accepted here?
There was a problem hiding this comment.
🤨
There was a problem hiding this comment.
well it says
arbitrary objects implementing the
__dataframe__interchange
protocolnarwhals.stable.v1.typing.DataFrameLikeare accepted
byv1.from_native(..., eager_or_interchange_only=True)but are not
recognised by this function
so why are they being special-cased?
based on the previous thread, i thought we were preseving v1 behaviour for is_into_dataframe
There was a problem hiding this comment.
I thought we were on the same page to fix the example in #3613 (comment)
import narwhals.stable.v1 as nw
nw.dependencies.is_into_dataframe(duckdb_obj) # False
isinstance(nw.from_native(duckdb_obj), nw.DataFrame) # Truethis requires to change is_into_dataframe runtime behavior for ibis and duckdb
There was a problem hiding this comment.
Yeah I was thinking the same.
The issue was with dask and __dataframe__.
I'm not sure what the purpose of a v1 is_into_dataframe is if it can't test any of the unique types it describes?
narwhals/narwhals/stable/v1/typing.py
Lines 28 to 29 in 98b4c8c
There was a problem hiding this comment.
sorry i wasn't clear
if there's no need to change it, i'd prefer to keep existing behaviour than to modify anything in v1
(and if it did need changing, it should be a separate pr labelled as bug fix)
There was a problem hiding this comment.
I understand the feeling of
if there's no need to change it, i'd prefer to keep existing behaviour than to modify anything in v1
yet to me this seems like a clear bug. Since we are touching every dependency file anyway, I thought to address it.
(and if it did need changing, it should be a separate pr labelled as bug fix)
In my defence the PR always had both enhancement and fix labels, and my first self comment was exactly on this very change to bring the focus on it #3613 (comment)
There was a problem hiding this comment.
true, thanks for looking into it. if it's ok, i'd still prefer to leave as-is, i'm concerned about unforeseen knock-on effects here not captured by ci or downstream tests
also true 🙌 but from a "do we need to revert this?" perspective, it's much easier if it's separate
| # TODO(Unassigned): For duckdb and ibis backends: | ||
| # * is_into_dataframe(native_frame) returns False | ||
| # * nw_v1.from_native(native_frame) returns a narwhals.stable.v1.DataFrame | ||
| # * Therefore is_into_dataframe(nw_v1.from_native(native_frame)) returns True | ||
| # See discussion https://github.com/narwhals-dev/narwhals/pull/3613#discussion_r3288440039 |
There was a problem hiding this comment.
@MarcoGorelli I added this comment to track the behavior after rolling back the changes



Description
Small late night rant: everything always turn out bigger than what it looks like. End of rant
As per title
is_into_lazyframedependenciesmodulesis_into_lazyframe(sqlframe_object) -> True_warn_if_narwhals_*by getting the calling function name viacaller = sys._getframe(1).f_code.co_nameand replaceis_pandas_dataframehardcoded value@dangotbanned I read your comment in #3612 when I was already down this rabbit hole 😭
What type of PR is this? (check all applicable)
Related issues
is_into_lazyframe#3612from_nativesupports #3530