Problem
The Column struct supports comparison via Rust operator overloading (>, <, >=, <=), but lacks explicit method equivalents like .gt(), .lt(), .ge(), .le().
PySpark, the Go client, and the Scala client all provide these as methods. They are essential for method chaining patterns:
// This doesn't work:
df.filter(col("age").gt(lit(27)))
// Workaround requires wrapping in parentheses:
df.filter(col("age") > lit(27)) // only works if filter accepts Column directly
// Or using string expressions:
df.filter("age > 27")Proposed solution
Add comparison methods to Column:
gt(other) — greater thanlt(other) — less thange(other) / gte(other) — greater than or equalle(other) / lte(other) — less than or equal
These should use the same UnresolvedFunction pattern as the existing eq method, calling the appropriate Spark comparison functions.
Problem
The
Columnstruct supports comparison via Rust operator overloading (>,<,>=,<=), but lacks explicit method equivalents like.gt(),.lt(),.ge(),.le().PySpark, the Go client, and the Scala client all provide these as methods. They are essential for method chaining patterns:
Proposed solution
Add comparison methods to
Column:gt(other)— greater thanlt(other)— less thange(other)/gte(other)— greater than or equalle(other)/lte(other)— less than or equalThese should use the same
UnresolvedFunctionpattern as the existingeqmethod, calling the appropriate Spark comparison functions.