[issue-88] fix: Warn On Failure não é oracle, não reprova o teste (#88)#90
Merged
Conversation
Run Keyword And Warn On Failure estava agrupado com Continue On Failure em is_verification, recursando no keyword interno e contando como oracle. Mas Warn On Failure só loga um WARN e o teste segue verde: a asserção interna nunca reprova o teste. Mesmo problema do Run Keyword And Ignore Error (swallow). Separado: Continue On Failure mantém a recursão (oracle se o inner for); Warn On Failure retorna False sempre (não é oracle). Assim o teste cuja única verificação está dentro de Warn On Failure passa a cair em C2b, e um user keyword nomeado como verificador com esse wrapper vira R2 (hollow oracle). Regressões distinguindo os dois wrappers em tests/test_scanner.py. Closes #88
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
vinicq
added a commit
that referenced
this pull request
Jul 2, 2026
…e test (#90) Run Keyword And Warn On Failure was grouped with Continue On Failure in is_verification, recursing into the inner keyword and counting as an oracle. But Warn On Failure only logs a WARN and the test stays green: the inner assertion never fails the test. Same problem as Run Keyword And Ignore Error (swallow). Split: Continue On Failure keeps the recursion (an oracle if the inner one is); Warn On Failure always returns False (not an oracle). A test whose only verification sits inside Warn On Failure now falls to C2b, and a user keyword named as a verifier wrapped this way becomes R2 (hollow oracle). Regression tests distinguishing the two wrappers in tests/test_scanner.py. Closes #88
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
O que mudou
Run Keyword And Warn On Failureestava agrupado comContinue On Failureemis_verification(scanner.py), recursando no keyword interno e contando como oracle. Mas a semântica difere:Continue On Failure: a falha é registrada e o teste falha no fim. Oracle válido - mantém a recursão.Warn On Failure: a falha vira só um WARN e o teste segue verde. A asserção interna nunca reprova o teste - não é oracle, agora retornaFalsesempre.Consequências: um teste cuja única verificação está sob
Warn On Failurepassa a cair em C2b; um user keyword nomeado como verificador com esse wrapper vira R2 (hollow oracle).Testes
Regressões distinguindo os dois wrappers em
tests/test_scanner.py:Run Keyword And Warn On Failure Should Be Equal ${1} ${2}-> C2bRun Keyword And Continue On Failure Should Be Equal-> continua oracle (sem C2b)test_no_r2_warn_on_failure_around_assertionajustado (agora R2, era o teste que codificava o bug).Suíte completa: 240 verdes.
Closes #88