BFGS can loop infinitely when trust radius hits minimum on rejected steps

[yannnbingz]

The related PR can be found here: #8

Issue:

In relaxation.py, the BFGS optimizer can enter an infinite loop when consecutive steps are rejected on a shallow potential energy surface. The root cause is in _compute_new_position(): when a step is rejected (energy_new > energy_old) and the parabolic backtrack gives trust_radius >= trust_radius_min (line 448).
This erases the counter that tracks whether the BFGS history has already been reset, preventing the termination condition (tr_min_hit > 1) from ever being reached.
The problematic infinite loop follows this pattern:

Step	SCF result	What happens	tr_min_hit
N	E_new > E_old	Rejected. Backtrack gives tr < tr_min. Since tr_min_hit == 0, sets it to 1, resets Hessian, takes steepest-descent step.	1
N+1	E_new > E_old	Rejected. Backtrack gives tr ≥ tr_min. Code sets tr_min_hit = 0 ← bug erases counter.	0
N+2	E_new > E_old	Rejected. Backtrack gives tr < tr_min. Since tr_min_hit == 0, sets it to 1, resets Hessian again.	1 instead of 2
→ N+ ...	Same rejection, same backtrack → tr_min_hit = 0 again. Cycle repeats indefinitely.	...

The convergence escape hatch requires tr_min_hit to reach 2 (two consecutive tr < tr_min hits), but the intervening rejected step with tr ≥ tr_min always resets the counter to 0.

This is triggered on systems with shallow or flat excited-state PES where many consecutive BFGS steps are rejected.

Proposed Fix:

In _compute_new_position(), inside the step-rejection branch, replace self.tr_min_hit = 0 with pass when trust_radius >= trust_radius_min:

# Before (line 447-448):
            else:
                self.tr_min_hit = 0

# After:
            else:
                pass

This preserves the counter across rejected steps so the termination condition can be reached:

Step	Before fix	After fix
N	tr < tr_min → tr_min_hit = 1, reset Hessian	Same
N+1	tr ≥ tr_min → tr_min_hit = 0 (erased)	tr ≥ tr_min → tr_min_hit = 1 (preserved)
N+2	tr < tr_min → tr_min_hit = 1, reset again → ∞ loop	tr < tr_min → tr_min_hit = 2 → terminates

The counter is still properly reset to 0 whenever a step is accepted (energy decreases), both in _compute_new_position() (line 476) and _compute_trust_radius() (line 651), so normal BFGS operation is unaffected.

Related Issue:

Separately, _compute_trust_radius() (line 640) calls exit() when tr_min_hit reaches 2 on an accepted step. This terminates the Python process without saving the final geometry or running cleanup. Replacing exit() with allowing the main loop to check conv_bfgs (which is already set via tr_min_hit > 1 at line 356) would give a graceful shutdown instead.

Fix:

# Before (lines 637–640):
            if self.tr_min_hit == 1:
                self._log("history already reset at previous step: stopping")
                self.tr_min_hit = 2
                exit()

# After:
            if self.tr_min_hit == 1:
                self._log("history already reset at previous step: stopping")
                self.tr_min_hit = 2

[vyu16]

Thank you for reporting the issue. While I believe I understand the problem you described, could you still provide a minimal example that reproduces it? Thanks!

[yannnbingz]

Sure. Attached are the example files, and their output behaviour is as follows:

example.zip

tr_min = 0.0002

SCF cycle	BFGS step	Energy (Ry)	Result	trust_radius	tr_min_hit
...
9	4→5	-645.70973907	accepted	0.00446	0 (reset on accept)
...
13	5	-645.70973617	rejected, tr=0.000112 < tr_min	→ reset Hessian	1
14	5	-645.70970042	rejected, tr=0.00153 ≥ tr_min	← bug: tr_min_hit=0	0
15	5	-645.70972587	rejected, tr=0.00054 ≥ tr_min		0
16	5	-645.70973251	rejected, tr=0.000169 < tr_min	→ reset Hessian again	1
17	5	-645.70970042	rejected, tr=0.00153 ≥ tr_min	← bug: tr_min_hit=0	0
18	5	-645.70972587	rejected , tr=0.00054 ≥ tr_min		0
19	5	-645.70973251	rejected, tr=0.000169 < tr_min	→ reset Hessian again	1
20	5	-645.70970042	rejected, tr=0.00153 ≥ tr_min	← bug: tr_min_hit=0	0
21	5	-645.70972587	rejected, tr=0.00054 ≥ tr_min		0
...			identical 3-step cycle repeats forever

One can observe that SCF cycles 13 - 15 & 16 - 18 & 19 - 21 are repeating in 3-step cycle.
tr < tr_min hitted multiple time (eg, step-13, 16, 19) but tr_min_hit is always <= 1, hence tr_min_hit == 2 and termination is never happening.

Software version:

Quantum ESPRESSO (PWSCF) v.7.4.1
WEST (WBSE) v. 6.2.1

[vyu16]

Thanks for the files! The TDDFT code actually doesn't support metaGGA or smearing. Sorry for not making this clear. We're adding an explicit check to stop the code with an error message in such cases.

Do you have a reproducer that doesn't use metaGGA and smearing?

[yannnbingz]

Please try this example (it may not be the most physically sensible input, but it demonstrates the pattern):

example_v_B_hBN.zip

Starting from SCF cycle 28, the optimizer is stuck in an infinite 3-step cycle with bfgs_steps = 11 frozen.

SCF cycle	BFGS step	Energy (Ry)	trust_radius	tr_min_hit
28	11	-1704.515	0.000184 (< tr_min) → resets Hessian	1
29	11	-1704.510	0.003735 (≥ tr_min) ← bug	resets to 0
30	11	-1704.514	0.000740 (≥ tr_min)	0
31	11	-1704.515	0.000184 (< tr_min) → resets Hessian again	1
32	11	-1704.510	0.003735 ← bug	resets to 0 again
...