Skip to content

Tests: Limit max iterations in optimize#219

Merged
g-braeunlich merged 6 commits intomainfrom
tests-limit-max-iter
Feb 12, 2026
Merged

Tests: Limit max iterations in optimize#219
g-braeunlich merged 6 commits intomainfrom
tests-limit-max-iter

Conversation

@g-braeunlich
Copy link
Collaborator

@g-braeunlich g-braeunlich commented Feb 5, 2026 •
edited by markfuge
Loading

Description

  • Make upper limit for optimize iterations configurable in problems
  • Limit the number in tests (default: 20, can be changed by using ENGIBENCH_MAX_ITER env var)

Fixes #215

Type of change

Please delete options that are not relevant.

  • Documentation only change (no code changed)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • I have run the pre-commit checks with pre-commit run --all-files
  • I have run ruff check . and ruff format
  • I have run mypy .
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Reviewer Checklist:

  • The content of this PR brings value to the community. It is not too specific to a particular use case.
  • The tests and checks pass (linting, formatting, type checking). For a new problem, double check the github actions workflow to ensure the problem is being tested.
  • The documentation is updated.
  • The code is understandable and commented. No large code blocks are left unexplained, no huge file. Can I read and understand the code easily?
  • There is no merge conflict.
  • The changes are not breaking the existing results (datasets, training curves, etc.). If they do, is there a good reason for it? And is the associated problem version bumped?
  • For a new problem, has the dataset been generated with our slurm script so we can re-generate it if needed? (This also ensures that the problem is running on the HPC.)
  • For bugfixes, it is a robust fix and not a hacky workaround.

@g-braeunlich g-braeunlich mentioned this pull request Feb 5, 2026
Closed
2 tasks
@g-braeunlich g-braeunlich self-assigned this Feb 5, 2026
@g-braeunlich g-braeunlich force-pushed the tests-limit-max-iter branch 2 times, most recently from d28a518 to 57e8845 Compare February 5, 2026 15:56
@g-braeunlich g-braeunlich requested a review from markfuge February 5, 2026 16:07
markfuge
markfuge reviewed Feb 5, 2026
View reviewed changes
tests/test_problem_implementations.py Outdated
or problem_class.__module__.startswith("engibench.problems.airfoil")
or problem_class.__module__.startswith("engibench.problems.heatconduction")
):
if problem_id(problem_class) in {"power_electronics", "airfoil", "heatconduction2d", "heatconduction3d"}:
Copy link
Member

@markfuge markfuge Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why do we skip the checks for airfoil and heat conduction? I can understand power electronics, but the other ones all have optimize methods I believe

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 6, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Afaik, airfoil was skipped as it took too long. For heatconduction, I dont remember. However as I removed the blacklisting for heatconduction, I ran into the following error:

TypeError: object of type 'numpy.float64' has no len()

That happens here, on the line

assert len(optistep.obj_values) == expected_obj_count

Copy link
Member

@markfuge markfuge Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sounds like we need to fix this then. Are you able to identify it, or should @MiladHB take a look?

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really seems that optistep.obj_values is a scalar and not a np.array. But here I am really unsure it that is intended or a bug

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MiladHB Any comment on this?

Copy link
Contributor

@MiladHB MiladHB Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If n_iter > 1, then OptiStep should be an array with length greater than one. Whether it is represented as a 1D array (n_iter,) or a 2D array (n_iter, 1) depends on how we define OptiStep consistently across other problems.

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Not a problem specific question, so question back to @markfuge

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more context: all other problems return a 2D array. This includes beams2 which also only has one objective.

Copy link
Member

@markfuge markfuge Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g-braeunlich I think returning a 2D array makes sense here and we should make heat conduction compatible with this. Bonus is that if we ever have a multi-objective problem then the output format is already compatible, as one can add additional columns.

Anything else you need my input on to resolve here?

Copy link
Collaborator Author

@g-braeunlich g-braeunlich Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Then there is nothing else I need to know.

markfuge
markfuge approved these changes Feb 5, 2026
View reviewed changes
Copy link
Member

@markfuge markfuge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this is fine @g-braeunlich see comment on why we are skipping airfoil and heat conduction in the optimization tests. If there is a good reason for this then happy to merge as is.

@g-braeunlich
fix(problems): return an array for optistep.obj_values in heatconduction
a554c3f 
BREAKING CHANGE: For heatconduction2d and heatconduction3d,
in the `history` return value, the `obj_values` attribute of the items
is now no longer a scalar but an array.
@g-braeunlich g-braeunlich merged commit 489d4de into main Feb 12, 2026
14 checks passed
@g-braeunlich g-braeunlich deleted the tests-limit-max-iter branch February 12, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markfuge markfuge markfuge approved these changes

@MiladHB MiladHB MiladHB left review comments

Assignees

@g-braeunlich g-braeunlich

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Limit number of optimization iterations for running GitHub Action tests

3 participants

@g-braeunlich @markfuge @MiladHB