Can't compute ForwardDiff hessian

[bclyons12]

I was testing the master version of Interpolations.jl on my code and ran into an error inside of an optimizer with autodifferentiation. It looks like I can't take the ForwardDiff.hessian of a 2D cubic spline interpolation any more. Below is a minimal example, which works fine on Interpolations.jl v0.15 but fails on master (for either ForwardDiff v0.10 or v1). It looks like some issue with nested Duals.

import Pkg; Pkg.activate(temp=true)
Pkg.add(name="Interpolations", rev="master")
Pkg.add(name="ForwardDiff")
import Interpolations, ForwardDiff

N = 11
x = range(1.0, 3.0, N)
y = range(-3.0, 3.0, N)
u = rand(N, N)
uitp = Interpolations.cubic_spline_interpolation((x, y), u; extrapolation_bc=Interpolations.Line())

f = x -> uitp(x[1], x[2])
x = [π/3, -π/2]

ForwardDiff.hessian(f, x)

which gives:

Activating new project at `[/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV](http://localhost:8888/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV)`
    Updating git-repo `[https://github.com/JuliaMath/Interpolations.jl.git`](https://github.com/JuliaMath/Interpolations.jl.git%60)
   Resolving package versions...
    Updating `[/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Project.toml](http://localhost:8888/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Project.toml)`
  [a98d9a8b] + Interpolations v0.16.0 `[https://github.com/JuliaMath/Interpolations.jl.git#master`](https://github.com/JuliaMath/Interpolations.jl.git#master%60)
    Updating `[/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Manifest.toml](http://localhost:8888/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Manifest.toml)`
  [79e6a3ab] + Adapt v4.3.0
  [13072b0f] + AxisAlgorithms v1.1.0
  [d360d2e6] + ChainRulesCore v1.25.1
  [34da2185] + Compat v4.16.0
  [a98d9a8b] + Interpolations v0.16.0 `[https://github.com/JuliaMath/Interpolations.jl.git#master`](https://github.com/JuliaMath/Interpolations.jl.git#master%60)
  [6fe1bfb0] + OffsetArrays v1.17.0
⌅ [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.4.3
  [c84ed2f1] + Ratios v0.4.5
  [ae029012] + Requires v1.3.1
  [90137ffa] + StaticArrays v1.9.13
  [1e83bf80] + StaticArraysCore v1.4.3
  [efce3f68] + WoodburyMatrices v1.0.0
  [56f22d72] + Artifacts v1.11.0
  [ade2ca70] + Dates v1.11.0
  [8ba89e20] + Distributed v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.11.0
  [a63ad114] + Mmap v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [1a1011a3] + SharedArrays v1.11.0
  [6462fe0b] + Sockets v1.11.0
  [2f01184e] + SparseArrays v1.11.0
  [fa267f1f] + TOML v1.0.3
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [4536629a] + OpenBLAS_jll v0.3.27+1
  [bea87d4a] + SuiteSparse_jll v7.7.0+0
  [8e850b90] + libblastrampoline_jll v5.11.0+0
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
   Resolving package versions...
    Updating `[/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Project.toml](http://localhost:8888/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Project.toml)`
  [f6369f11] + ForwardDiff v1.0.1
    Updating `[/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Manifest.toml](http://localhost:8888/private/var/folders/nt/ct_lf2n94_1c21908mmbx55w0000gq/T/jl_n8v5VV/Manifest.toml)`
  [bbf7d656] + CommonSubexpressions v0.3.1
  [163ba53b] + DiffResults v1.1.0
  [b552c78f] + DiffRules v1.15.1
  [ffbed154] + DocStringExtensions v0.9.4
  [f6369f11] + ForwardDiff v1.0.1
  [92d709cd] + IrrationalConstants v0.2.4
  [692b3bcd] + JLLWrappers v1.7.0
  [2ab3a3ac] + LogExpFunctions v0.3.29
  [1914dd2f] + MacroTools v0.5.16
  [77ba4419] + NaNMath v1.1.3
  [276daf66] + SpecialFunctions v2.5.1
  [efe28fd5] + OpenSpecFun_jll v0.5.6+0
  [05823500] + OpenLibm_jll v0.8.5+0
MethodError: no method matching ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}(::Float64, ::ForwardDiff.Partials{2, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}})
The type `ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}` exists, but no method is defined for this combination of argument types when trying to construct it.

Closest candidates are:
  ForwardDiff.Dual{T, V, N}(::Number) where {T, V, N}
   @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl:78](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl#line=77)
  ForwardDiff.Dual{T, V, N}(::Any) where {T, V, N}
   @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl:77](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl#line=76)
  ForwardDiff.Dual{T, V, N}(::V, ::ForwardDiff.Partials{N, V}) where {T, V, N}
   @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl:17](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/dual.jl#line=16)
  ...


Stacktrace:
  [1] apply_partials(x_dual::ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}, val::Float64)
    @ InterpolationsForwardDiffExt [~/.julia/packages/Interpolations/5GvP0/ext/InterpolationsForwardDiffExt.jl:19](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/Interpolations/5GvP0/ext/InterpolationsForwardDiffExt.jl#line=18)
  [2] _broadcast_getindex_evalf
    @ ./broadcast.jl:678 [inlined]
  [3] _broadcast_getindex
    @ ./broadcast.jl:651 [inlined]
  [4] (::Base.Broadcast.var"#17#18"{Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(InterpolationsForwardDiffExt.apply_partials), Tuple{Tuple{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}}, Tuple{Float64, Float64}}}})(k::Int64)
    @ Base.Broadcast ./broadcast.jl:1102
  [5] ntuple
    @ ./ntuple.jl:49 [inlined]
  [6] copy
    @ ./broadcast.jl:1102 [inlined]
  [7] materialize
    @ ./broadcast.jl:872 [inlined]
  [8] maybe_clamp(::Interpolations.NeedsCheck, itp::Interpolations.BSplineInterpolation{Float64, 2, OffsetArrays.OffsetMatrix{Float64, Matrix{Float64}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, xs::Tuple{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}})
    @ InterpolationsForwardDiffExt [~/.julia/packages/Interpolations/5GvP0/ext/InterpolationsForwardDiffExt.jl:12](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/Interpolations/5GvP0/ext/InterpolationsForwardDiffExt.jl#line=11)
  [9] maybe_clamp(itp::Interpolations.BSplineInterpolation{Float64, 2, OffsetArrays.OffsetMatrix{Float64, Matrix{Float64}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, xs::Tuple{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}})
    @ Interpolations [~/.julia/packages/Interpolations/5GvP0/src/Interpolations.jl:450](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/Interpolations/5GvP0/src/Interpolations.jl#line=449)
 [10] ScaledInterpolation
    @ [~/.julia/packages/Interpolations/5GvP0/src/scaling/scaling.jl:81](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/Interpolations/5GvP0/src/scaling/scaling.jl#line=80) [inlined]
 [11] (::Interpolations.Extrapolation{Float64, 2, Interpolations.ScaledInterpolation{Float64, 2, Interpolations.BSplineInterpolation{Float64, 2, OffsetArrays.OffsetMatrix{Float64, Matrix{Float64}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}}}, Interpolations.BSpline{Interpolations.Cubic{Interpolations.Line{Interpolations.OnGrid}}}, Interpolations.Line{Nothing}})(::ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}, ::ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2})
    @ Interpolations [~/.julia/packages/Interpolations/5GvP0/src/extrapolation/extrapolation.jl:54](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/Interpolations/5GvP0/src/extrapolation/extrapolation.jl#line=53)
 [12] (::var"#1#2")(x::Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}})
    @ Main ./In[1]:12
 [13] vector_mode_dual_eval!
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/apiutils.jl:24](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/apiutils.jl#line=23) [inlined]
 [14] vector_mode_gradient(f::var"#1#2", x::Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}}})
    @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/gradient.jl:98](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/gradient.jl#line=97)
 [15] gradient
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/gradient.jl:20](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/gradient.jl#line=19) [inlined]
 [16] #107
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl:17](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl#line=16) [inlined]
 [17] vector_mode_dual_eval!
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/apiutils.jl:24](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/apiutils.jl#line=23) [inlined]
 [18] vector_mode_jacobian(f::ForwardDiff.var"#107#108"{var"#1#2", ForwardDiff.HessianConfig{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}}}}, x::Vector{Float64}, cfg::ForwardDiff.JacobianConfig{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}}})
    @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/jacobian.jl:129](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/jacobian.jl#line=128)
 [19] jacobian
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/jacobian.jl:22](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/jacobian.jl#line=21) [inlined]
 [20] hessian
    @ [~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl:18](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl#line=17) [inlined]
 [21] hessian(f::var"#1#2", x::Vector{Float64}, cfg::ForwardDiff.HessianConfig{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}, 2}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 2}}})
    @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl:15](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl#line=14)
 [22] hessian(f::var"#1#2", x::Vector{Float64})
    @ ForwardDiff [~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl:15](http://localhost:8888/lab/tree/FUSE/playground/~/.julia/packages/ForwardDiff/Wq9Wb/src/hessian.jl#line=14)
 [23] top-level scope
    @ In[1]:15

[bclyons12]

@mkitti I saw a new version got released, so this issue is present in v0.16 now.

[mkitti]

This issue is present in v0.16 now.

[mkitti]

@ajwheeler any ideas about what broke here?

[ajwheeler]

I will look into this today.

[ajwheeler]

This is happening for scaled interpolants only. It also only happens for hessians, not gradients. The culprit is my recursive implementation of apply_partials, which I think I've now fixed in #628.

Sorry I didn't catch this.