Breakage in sparse structarray broadcasting

[jishnub]

The following works on StructArray v0.6.13, but is broken on v0.6.14:

On v0.6.3

julia> using StructArrays, SparseArrays

julia> StructArray{ComplexF64}((spzeros(2,2), spzeros(2,2))) - spzeros(ComplexF64,2,2)
2×2 Matrix{ComplexF64}:
 0.0+0.0im  0.0+0.0im
 0.0+0.0im  0.0+0.0im

On v0.6.4

julia> StructArray{ComplexF64}((spzeros(2,2), spzeros(2,2))) - spzeros(ComplexF64,2,2)
ERROR: MethodError: no method matching similar(::Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}, ::Type{Float64}, ::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}})
Closest candidates are:
  similar(::StructArray, ::Type, ::Tuple{Union{Integer, Base.OneTo}, Vararg{Union{Integer, Base.OneTo}}}) at ~/.julia/packages/StructArrays/F5fDf/src/structarray.jl:297
  similar(::StructArray, ::Type, ::Tuple{Union{Integer, AbstractUnitRange}, Vararg{Union{Integer, AbstractUnitRange}}}) at ~/.julia/packages/StructArrays/F5fDf/src/structarray.jl:297
  similar(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{N}}, ::Type{ElType}, ::Any) where {N, ElType} at broadcast.jl:212
  ...
Stacktrace:
  [1] similar(bc::Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}, #unused#::Type{Float64})
    @ Base.Broadcast ./broadcast.jl:211
  [2] (::StructArrays.var"#71#72"{Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}})(T::Type)
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/structarray.jl:530
  [3] macro expansion
    @ ~/.julia/packages/StructArrays/F5fDf/src/utils.jl:24 [inlined]
  [4] map_params_as_tuple(f::StructArrays.var"#71#72"{Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}}, #unused#::Type{NamedTuple{(:re, :im), Tuple{Float64, Float64}}})
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/utils.jl:23
  [5] map_params(f::StructArrays.var"#71#72"{Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}}, #unused#::Type{NamedTuple{(:re, :im), Tuple{Float64, Float64}}})
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/utils.jl:21
  [6] buildfromschema(initializer::StructArrays.var"#71#72"{Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}}, #unused#::Type{ComplexF64}, #unused#::Type{NamedTuple{(:re, :im), Tuple{Float64, Float64}}})
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/utils.jl:47
  [7] buildfromschema(initializer::StructArrays.var"#71#72"{Base.Broadcast.Broadcasted{SparseArrays.HigherOrderFns.SparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}}, #unused#::Type{ComplexF64})
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/utils.jl:35
  [8] similar(bc::Base.Broadcast.Broadcasted{StructArrays.StructArrayStyle{SparseArrays.HigherOrderFns.SparseMatStyle, 2}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, SparseMatrixCSC{ComplexF64, Int64}}}, #unused#::Type{ComplexF64})
    @ StructArrays ~/.julia/packages/StructArrays/F5fDf/src/structarray.jl:530
  [9] copy
    @ ./broadcast.jl:885 [inlined]
 [10] materialize
    @ ./broadcast.jl:860 [inlined]
 [11] broadcast_preserving_zero_d
    @ ./broadcast.jl:849 [inlined]
 [12] -(A::StructArray{ComplexF64, 2, NamedTuple{(:re, :im), Tuple{SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}}}, CartesianIndex{2}}, B::SparseMatrixCSC{ComplexF64, Int64})
    @ Base ./arraymath.jl:8
 [13] top-level scope
    @ REPL[4]:1

My versioninfo:

julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161e (2022-11-14 20:14 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores
Environment:
  JULIA_EDITOR = subl

[jishnub]

Bisected to 1afecf4 (#215)

CC: @N5N3

[N5N3]

Emm, I dont think it's a good idea to return a StructSparseArray here.
The duplicated index storage seems pretty bad.
So perhaps a SparseArray is a better choice?

[piever]

@jishnub do you have a sense of what is the correct fix? Should one simply return a SparseArray (as in #260, IIUC) or a StructArray whose columns are sparse arrays?

I might be inclined towards the fix in #260 (if the storage type is unsupported, avoid the StructArray wrapper altogether), but I confess I'm not entirely sure (for views, we do the other way, you get a StructArray of SubArrays).

[jishnub]

I'm not totally sure either, but returning a SparseArray sounds reasonable to me.