Skip to content

Rewrite HybridPSO L-BFGS with custom Strong Wolfe line search#90

Draft
AdityaPandeyCN wants to merge 5 commits intoSciML:mainfrom
AdityaPandeyCN:aditya/line_search
Draft

Rewrite HybridPSO L-BFGS with custom Strong Wolfe line search#90
AdityaPandeyCN wants to merge 5 commits intoSciML:mainfrom
AdityaPandeyCN:aditya/line_search

Conversation

@AdityaPandeyCN
Copy link
Copy Markdown
Contributor

Checklist

  • Appropriate tests were added
  • Any code changes were done in a way that does not break public API
  • All documentation related to code changes were updated
  • The new code follows the
    contributor guidelines, in particular the SciML Style Guide and
    COLPRAC.
  • Any new documentation only uses public API

Additional context

Replaced SimpleNonlinearSolve dependency with custom implementation:

  • Strong Wolfe line search (Nocedal & Wright Algorithm 3.5)
  • Zoom phase (Nocedal & Wright Algorithm 3.6)
  • L-BFGS two-loop recursion (Nocedal & Wright Algorithm 7.4)
  • Cubic interpolation for step length selection

Signed-off-by: AdityaPandeyCN <adityapand3y666@gmail.com>
Signed-off-by: AdityaPandeyCN <adityapand3y666@gmail.com>
@AdityaPandeyCN
Copy link
Copy Markdown
Contributor Author

@ChrisRackauckas Can you have a look?

Signed-off-by: AdityaPandeyCN <adityapand3y666@gmail.com>
Signed-off-by: AdityaPandeyCN <adityapand3y666@gmail.com>
Signed-off-by: AdityaPandeyCN <adityapand3y666@gmail.com>
@utkarsh530
Copy link
Copy Markdown
Member

I think we can make this more modular here. First, https://github.com/SciML/LineSearch.jl exists, so any line-search implementation should go there. I believe those line searches are GPU compatible. If not, we can have our own LS here. But the core (L)-BFGS algorithm should be simple, and I am not sure why a complete custom implementation will be better. NonlinearSolve.jl seems to have been a more modular solution, unless it wasn't performant enough.

@AdityaPandeyCN
Copy link
Copy Markdown
Contributor Author

I was assuming this is the Linesearch repo https://github.com/JuliaNLSolvers/LineSearches.jl whole time which is not gpu-compatible and implemented it from scratch here :(

@AdityaPandeyCN AdityaPandeyCN marked this pull request as draft April 3, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants