feat: improve pipeline log extraction for catchError and parallel failures#99
Open
lidiams96 wants to merge 10 commits intojenkinsci:mainfrom
Open
feat: improve pipeline log extraction for catchError and parallel failures#99lidiams96 wants to merge 10 commits intojenkinsci:mainfrom
lidiams96 wants to merge 10 commits intojenkinsci:mainfrom
Conversation
…lures - Strategy 1 now multi-collects: walks the FlowGraph and accumulates logs from ALL ErrorAction+LogAction nodes (not just the first match) using a HashSet to deduplicate origins. This captures parallel failures such as multiple RSpec pod crashes in a single pass. - Strategy 2 (new): WarningAction walk, runs only when Strategy 1 finds nothing. Looks for step nodes enclosed by a BlockStartNode with a WarningAction worse than SUCCESS (e.g. catchError with stageResult:FAILURE). Requires >= 2 lines matching the error pattern to avoid returning CI infrastructure steps ahead of the actual failing step log. - Strategy 3 (error pattern scan) now always runs as a supplement after Strategies 1 and 2, filling the remaining maxLines budget with deduplicated lines matching error patterns from the full console log. Previously it only ran as a final fallback. This fixes the catchError+sh(returnStatus:true)+ error() pattern where error() has an ErrorAction but no LogAction. - Added ERROR_PATTERN constant with universal keywords (error, exception, failed, fatal) and ERROR_CONTEXT_LINES = 5 context lines around each match. - Added getErrorPatternLines() method that reads the full build log, strips ConsoleNote annotations, marks matching lines with surrounding context, and returns up to maxLines results. - Extended PipelineLogExtractorTest with 5 integration tests covering: Strategy 1 standard failure, the catchError+returnStatus pattern (Strategy 3), early errors in large logs (Strategy 3), multi-error builds (Strategies 1 and 3), and an end-to-end test verifying the AI provider receives the inner catchError content. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
eec6a39 to
22999ea
Compare
Strategy 3 calls run.getLogInputStream() which may return null in some environments (e.g. when mocked). A null InputStream passed to InputStreamReader throws NullPointerException, which is not caught by the existing catch(IOException). Add an explicit null check to return an empty list instead. Also add the missing getLogInputStream() stub in testNullFlowExecutionFallsBackToBuildLog so the mock correctly models a build where the log stream is available but empty, allowing Strategy 3 to run without error and fall through to the getLog() fallback. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Run.getLogInputStream() is @nonnull per the Jenkins API contract, so the null check introduced in the previous commit is redundant and is correctly flagged by SpotBugs as RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE. Restore the original try-with-resources structure. The test stub (when(mockRun.getLogInputStream()).thenReturn(InputStream.nullInputStream())) remains to satisfy Mockito's default behaviour, which returns null for unstubbed methods regardless of @nonnull annotations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Improves PipelineLogExtractor so the plugin captures more accurate failure logs in complex pipelines (parallel failures and catchError + sh(returnStatus:true) + error() patterns), reducing reliance on the last-N-lines fallback and better surfacing real error output for AI analysis.
Changes:
- Update pipeline FlowGraph extraction to collect logs from multiple failing nodes and add a WarningAction-based walk.
- Add an always-on console-log error-pattern supplement to fill remaining
maxLines. - Add new Jenkins integration tests covering key extraction scenarios and an end-to-end verification via
TestProvider.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java |
Implements multi-strategy log extraction (multi-collect, WarningAction walk, pattern-scan supplement) and deduping. |
src/test/java/io/jenkins/plugins/explain_error/PipelineLogExtractorTest.java |
Adds integration + end-to-end tests validating improved extraction behavior in realistic Pipeline scenarios. |
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/jenkins/plugins/explain_error/PipelineLogExtractor.java
Outdated
Show resolved
Hide resolved
src/test/java/io/jenkins/plugins/explain_error/PipelineLogExtractorTest.java
Show resolved
Hide resolved
src/test/java/io/jenkins/plugins/explain_error/PipelineLogExtractorTest.java
Show resolved
Hide resolved
- Strategy 1: pass remaining budget to readLimitedLog() instead of always using maxLines, so later failure nodes still get log lines when earlier ones partially fill the budget - getErrorPatternLines(): replace load-all-then-scan with a streaming approach using a bounded circular pre-context buffer; avoids loading the full console log into memory for large builds - Strategy 3 dedup: change Set.add() to Set.contains() so repeated occurrences of the same error line (e.g. retries) are preserved when they appear in different contexts, not just deduplicated globally - Lower Strategy 3 completion log from INFO to FINE to reduce noise in Jenkins logs - Add @EnabledOnOs(OS.UNIX) to all Pipeline sh-step tests so they are skipped on Windows CI rather than failing with an unsupported step error Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
JUnit Jupiter 6.0.1 removed the OS.UNIX constant (only AIX, FREEBSD, LINUX, MAC, OPENBSD, SOLARIS, WINDOWS, OTHER remain). Replace all @EnabledOnOs(OS.UNIX) annotations with @DisabledOnOs(OS.WINDOWS) which is the correct equivalent for skipping sh-based Pipeline tests on Windows. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ion test - Expand ERROR_PATTERN to match plural forms: errors? and exceptions? so that common log lines like "2 errors found" or "exceptions thrown" are captured by Strategies 2 and 3 (Strategy 2 filter requires ≥2 error-pattern matches, so plurals directly affect its sensitivity) - Add strategy2_catchErrorWithWarningAction_extractsStepLog integration test: verifies that a direct sh failure inside catchError with buildResult/stageResult:'FAILURE' (which gives the BlockStartNode a WarningAction) is captured by Strategy 1 or 2, protecting against regressions in the WarningAction walk code path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ctor Address 19 coverage annotations reported by the Code Coverage check: - Remove dead code: accumulated trim (L237-238) was unreachable because the budget-tracking loop guarantees accumulated.size() <= maxLines. - strategy3_contextBuffer_evictsOldestLinesWhenBufferFull: covers L164/L165 (contextBuffer eviction when >ERROR_CONTEXT_LINES consecutive non-error lines appear before the first error pattern). - strategy3_maxLines_contextFlushExhaustsBudgetBeforeErrorLine: covers L149 false branch (while-loop exits because result.size() >= maxLines during pre-context flush) and L153 false branch (error line not added once budget is full). - strategy3_ioExceptionInGetLogInputStream_doesNotPropagate: covers L169-171 (IOException catch block in getErrorPatternLines — exception swallowed, falls back to run.getLog()). - strategy1_budgetExhausted_walkerBreaksAndStrategy3Skipped: covers L222 (walker break when remainingLines <= 0) and L282 false branch (Strategy 3 skipped entirely when budget is already zero). - strategy2_shSucceedsWithErrorOutput_capturedViaWarningActionWalk: covers L252-L270 (Strategy 2 body) by forcing Strategy 1 to return empty: sh succeeds (no ErrorAction) but prints >=2 error-pattern lines, then error() throws (ErrorAction but no LogAction) inside catchError(stageResult:FAILURE) so the block gets a WarningAction. Strategy 2 finds the sh LogAction. - strategy1_twoDirectFailures_primaryNodeIdSetByFirstVisitedNode: covers L232 false branch (primaryNodeId already set when second error node is visited, reverse walker order). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Code simplifications (dead code removal): - Remove Strategy 2 (WarningAction walk): never fires in practice because Jenkins sets WarningAction on the error step's FlowNode, not on the catchError BlockStartNode — Strategy 3 covers the same patterns via full console log scan. Removes unused BlockStartNode, WarningAction and Result imports. - Remove redundant `if (result.size() < maxLines)` guard inside `else if (futureContextRemaining > 0)`: L141 already guarantees result.size() < maxLines before any line is processed, making the false branch structurally unreachable. - Remove `if (stepLog == null || stepLog.isEmpty()) continue`: readLimitedLog never returns null, and addAll([]) is a safe no-op that avoids the need to cover an isEmpty() branch that cannot be triggered in practice. New tests for remaining uncovered branches: - strategy1_parallelFailure_seenOriginIdsPreventsDuplication: parallel block propagates the same exception to multiple FlowNodes; the second visit hits the seenOriginIds.contains() branch (L226). - strategy1_directErrorStep_logActionIsNullSkipsToStrategy3: direct error() step has ErrorAction but no LogAction; Strategy 1 hits the logAction==null branch (L228) and falls through to Strategy 3. - strategy3_manyPatternLines_budgetBreakStopsLoop: 20 error-pattern lines with maxLines=5; the dedup loop hits budget<=0 break (L285). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- L176: remove empty <p> tag in Javadoc (JavaDoc check warning)
- L242: refactor Strategy 3 dedup loop from for+break to
stream().filter().limit().forEach(), eliminating the partially-covered
break branch
- L216/L218: extract resolveOrigin() as a package-private delegate so
tests can override it via Mockito spy without needing static mocking.
Add three targeted tests:
* strategy1_resolveOriginNull_originNullBranchCovered (origin == null)
* strategy1_sameOriginVisitedTwice_seenOriginIdsBranchCovered
(seenOriginIds.contains())
* strategy1_originWithoutLogAction_logNullBranchCovered (logAction == null)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The current
PipelineLogExtractorhas two limitations that cause it to miss real errors in production pipelines:Single-match Strategy 1: The FlowGraph walk returns on the first
ErrorAction+LogActionnode found. In pipelines with parallel failures (e.g. multiple test runner pods crashing simultaneously), only the first error is captured and subsequent failures are dropped.catchError + sh(returnStatus:true) + error()pattern not detected: This common pipeline pattern is invisible to Strategy 1.sh(returnStatus:true)does not throw so it has noErrorAction. The subsequenterror()step records anErrorActionbut has noLogAction(it only carries the message string, not the sh output). Strategy 1 finds theerror()node, skips it (noLogAction), and falls back torun.getLog(maxLines)— which returns the last N lines of the full console log and misses the actual error output when the build has many subsequent steps.Solution
Strategy 1 — multi-collect: Instead of returning on the first
ErrorAction+LogActionmatch, accumulate logs from all matching nodes up tomaxLines, using aHashSetto deduplicate by origin node ID (reverse-chronological walk may visit the same origin via multiple FlowGraph paths).Strategy 2 (new) — WarningAction walk: Runs only when Strategy 1 finds nothing. Walks the FlowGraph looking for step nodes with a
LogActionenclosed by aBlockStartNodecarrying aWarningActionworse thanSUCCESS. This handles pipeline variants wherecatchErrorrecords aWarningActionon the block's start node rather than anErrorActionon the failing step. Requires ≥ 2 lines matching the error pattern to avoid capturing CI infrastructure steps (e.g. a curl command whose JSON payload happens to contain "failed") ahead of the actual failing step log.Strategy 3 — always-on supplement: The error pattern scan previously ran only as a final fallback when all other strategies produced nothing. It now always runs after Strategies 1 and 2, filling the remaining
maxLinesbudget with lines from the full console log that match error keywords (error,exception,failed,fatal) plus ±5 lines of surrounding context. Lines already captured by Strategies 1 or 2 are skipped viaHashSetdeduplication. This reliably surfaces thecatchError+sh(returnStatus:true)+error()pattern and catches errors buried early in large build logs.Alternatives
catchError+error()case where Strategy 1 finds anErrorActionnode with no usable log. The supplement approach avoids redundant I/O when Strategies 1/2 already filled the budget and adds no overhead when Strategy 3 has nothing new to contribute.Additional Context
Validated against a real production Jenkins pipeline (complex parallel build with thousands of tests across multiple parallel groups, ~30,000 line build log):
catchErrorblockThe 5 new integration tests in
PipelineLogExtractorTestcover all three strategies and include an end-to-end test that verifies the AI provider receives the innercatchErrorcontent rather than the generic fallback log.