FCT2-17944 FCT2-17945 FCT2-17946 : Adressing the Gaps in postmann collection#452
FCT2-17944 FCT2-17945 FCT2-17946 : Adressing the Gaps in postmann collection#452kmshrajcps wants to merge 8 commits into
Conversation
…nner + Newman Make the LCC Postman collection runnable end-to-end via both the Postman Collection Runner and Newman, and close the silent-pass bugs that let backend regressions slip through. Collection (LCCUserJourneyTests.postman_collection.json): - Rename journey-scoped requests with [ENC] / [ENM] prefixes so the Postman Runner's collection-wide name resolution can't bounce out of the journey folder mid-chain. - Delete the orphan root-level 8. Validate Transfer that duplicated the Copy/Move journey requests and broke Runner resolution. - Strip terminating setNextRequest(null) from setup and Copy/Move so the Runner can walk past the folder boundary; Newman's --folder flow is unaffected. - Drop seven redundant demo folders (1. Authentication, Case Registration, 2. Case Management, 3. Egress Integration, 4. NetApp Integration, 5. File Transfer, 6. Activity Logs) whose endpoints are already exercised by 0. E2E Setup or the three E2E journey folders. - Tighten E2E: Netapp to Egress Copy / 12. Poll Transfer Status to assert status === 'Completed' instead of !== 'Failed', so PartiallyCompleted no longer passes silently. - Replace 9. List Netapp Files' "latest by lastModified" pick with a deterministic name-match against egressCopyFileName, plus a constructed- path fallback for NetApp indexing lag. Bump take=50 -> take=500 so freshly-deposited files aren't hidden by accumulated test detritus. Run-E2E-Tests.ps1: - Change egressDestinationFolder from "4. Served Evidence/" (the upload folder, which guaranteed FileExists collision on every copy-back) to "3. Unused - disclosed/", aligning with the template default. Setup-EgressWorkspaceAndUpload.ps1: - Bump small-file post-upload verification window from 5x2s (10s total) to 20x3s (60s) so we more often capture Egress's canonical file ID instead of falling back to the upload-session ID, which trips up some downstream storage reads. Verified: both `Run-E2E-Tests.ps1 -SizeMB 10 -RegisterCase` and the default-mode equivalent pass all three E2E journeys end-to-end.
…lection Two reviewer-flagged regressions in the previous commit (9af1176): 1. egressServedEvidenceFolderId was being unset at the end of [ENC] 7b. List Files in Folder (Copy journey), which broke the full-Runner walk -- [ENM] 7b. List Files in Folder (Move journey) then fired with folder-id= empty and listed the workspace root instead of 4. Served Evidence/. Drop the unset in both journeys; the setup folder populates the variable once per top-level run and both journeys can read it without mid-flight cleanup. 2. 9. List Netapp Files now fails standalone -TestsToRun netapp-to-egress because that mode never uploads a Copy source and never runs the Egress -> NetApp Copy folder, so egressCopyFileName is unset and the deterministic name match hits the final else with a hard pm.expect.fail(). Restore a "latest by lastModified" fallback for the standalone path, and (since that file is usually the one a prior full-E2E run already round-tripped to Egress) uniquify egressDestinationFolder per standalone run so the copy-back can't FileExists-collide. Full E2E runs still take the deterministic branch and the original destination path. Verified: Run-E2E-Tests.ps1 -SizeMB 10 -RegisterCase ALL PASS (01:19) Run-E2E-Tests.ps1 -TestsToRun netapp-to-egress ALL PASS (00:17)
| $filesObj = $filesResult | ConvertFrom-Json | ||
| $uploadedFile = $filesObj | Where-Object { $_.name -eq $CurrentFileName -and $_.isFolder -eq $false } | ||
|
|
||
| if ($uploadedFile) { |
There was a problem hiding this comment.
Can we avoid relying on the upload ID fallback here and make sure we resolve the canonical Egress file ID before continuing?
In local testing the upload completes, but the verification polls 20 times and then falls back to $UploadId. That looks risky because the upload uses folder_path, while this check lists by folder_id=$FolderId, and $FolderId may be empty. The fallback can then pass an upload-session ID into later flows that expect a real file ID, which can cause 404 errors on move.
Could we either use the file ID from the completion response if Egress returns it, or resolve the target folder ID before polling?
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
Can we make the NetApp copy destination unique per collection run?
This request still writes the Egress-to-NetApp copy to {{netappFolderPath}}. If the imported collection is run again with the same generated environment, the same file name already exists in NetApp and the backend can return PartiallyCompleted with FileExists.
Example fix: use a run-scoped path such as Automation-Testing/e2e-<run-id>/ and use the same value for validate and initiate.
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
Can we fail the Move journey when the expected move source is missing instead of falling back to the latest files?
On a no-upload rerun, the original Move source has already been moved out of 4. Served Evidence/. This fallback can then pick the remaining Copy source and move/delete the wrong file while still reporting success.
Expected behavior: only use files matching the configured Move source, for example egressMoveFileNames. If they are not present, the test should fail or re-upload a Move source.
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
Can we make the NetApp-to-Egress destination unique per collection run?
This copy-back request still writes to the fixed {{egressDestinationFolder}}. After the first run, the file already exists under 3. Unused - disclosed/, so a second run with the same environment can return PartiallyCompleted with FileExists.
Example fix: set egressDestinationFolder to a per-run subfolder such as 3. Unused - disclosed/e2e-<run-id>/ before validate, then reuse that value for initiate.
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
Can we add copy-back coverage for the Move journey as well?
The current NetApp-to-Egress copy-back flow selects egressCopyFileName from Automation-Testing/. The Move journey writes to Automation-Testing-Move/, so the moved file is not copied back to Egress or asserted.
Suggested fix: add a separate Move copy-back folder, or parameterise this flow so it can copy back and verify the Move output too.
… unique destinations (PR review)
- Add E2E: Netapp Move to Egress Copy folder ([NME] 9-14) covering copy-back
of the moved file out of Automation-Testing-Move/ and verify the activity log.
- Wire netapp-move-to-egress into Run-E2E-Tests.ps1 (ValidateSet, folder map,
ordering after move).
- Uniquify NetApp + Egress destinations per collection run via ws-<workspaceId>/
suffix on netappFolderPath, netappMoveFolderPath, egressDestinationFolder
(applied at the prerequest of each 8/9/10. Validate so subsequent Initiate
reads the same value). Idempotent via /\/ws-[^/]+\/$/ guard.
- Replace silent "use latest" fallback in [ENC]/[ENM] 7b. List Files in Folder
with pm.expect.fail when configured egressFileNames are missing, so a
no-upload rerun fails loudly instead of moving/deleting the wrong file.
- Fix Move folder clone leaks: [NME] 10. Validate now sends
sourceRootFolderPath: {{netappMoveFolderPath}} (was netappFolderPath).
|
The README still describes the previous flow — there's no mention of the |
…ney-end + guard hardening, README (PR review)
Collection:
- 0. E2E Setup / 1. Get Azure AD Token stamps a fresh e2eRunId = {{$guid}};
all six uniquify blocks now compose per-run sub-folders
(Automation-Testing/e2e-run-<id>/) instead of per-workspace ws-<id>/.
A second Postman Runner pass restamps e2eRunId so Copy journeys are
replay-safe; Move journeys still hit a backend-imposed limitation
documented in the README.
- 14. and [NME] 14. Download Activity Log: added Final transfer status
assertion (matching [ENC]/[ENM] 12.); dropped setNextRequest(null) from
14. so full Runner passes fall through to [NME], kept it only on the
genuinely-last [NME] 14.
- [ENC]/[ENM] 7b. List Files in Folder: hardcoded per-folder prefix
('egressCopy' / 'egressMove') instead of regex-sniffing
pm.execution.location, so the Move journey can't silently grab the
Copy source set if the API is unavailable or a folder is renamed.
- Register Case prerequest guard: switched _registerCase === 'false' to
_registerCase !== 'true' so a missing/blank variable also skips
(matches the folder-level guard).
- All four Poll Transfer Status scripts: added MAX_POLL_ATTEMPTS = 60
cap on both the 404-retry and InProgress-retry paths; fails loudly
with the last seen status instead of looping to CI timeout.
- Back-direction (12. / [NME] 12.) Poll Transfer Status: switched the
partial-failure log from the non-existent response.fileStatuses to
failedItems[{sourcePath, errorCode}] so real failures actually log.
- [NME] 9. List Netapp Files: removed the constructed-path fallback
when the Move source isn't in the NetApp listing; now fails fast
with "Run the E2E: Egress to NetApp Move journey first (or use
-TestsToRun all)" so standalone netapp-move-to-egress runs don't
submit a transfer for a file that doesn't exist.
- Fixed Move-folder clone leaks ([NME] 9 log line, [NME] 10 validate
body sourceRootFolderPath: {{netappMoveFolderPath}}).
README:
- Documents the 0. E2E Setup folder, the once-per-run setup pass, and
the LCCTestEnvironment_post-setup.postman_environment.json export.
- Adds netapp-move-to-egress to the -TestsToRun docs.
- Adds an Execution Models section covering PowerShell+Newman, Postman
Runner (with an explicit can/can't table), and single-journey subsets.
- Adds Collection internals: per-journey egressCopyFile* / egressMoveFile*
variable table, replay-safety mechanics, and the Move journey replay
limitation.
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
The fail-fast here marks the test failed but doesn't stop the journey. Nothing calls pm.execution.setNextRequest(null) after the failing assertion, and Run-E2E-Tests.ps1 runs Newman without --bail, so [NME] 10 through [NME] 14 still execute. In a full Runner pass where the Move forward folder was filtered out, netappFilePath still holds the Copy journey's file (set by 9. List Netapp Files in the previous folder), so this journey submits a transfer for the wrong file. That's the cross-journey mix-up the per-journey split was built to prevent. It's a copy rather than a move, so nothing is deleted, and the run still ends up red, but we've fired an unwanted transfer and buried the real failure under downstream noise.
Can we add pm.execution.setNextRequest(null) straight after the failing test so the journey actually stops here?
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
MAX_POLL_ATTEMPTS = 60 works out at roughly 3 to 4 minutes per transfer: each poll costs the 2000ms busy-wait in the prerequest, plus Newman's --delay-request 1000, plus the request itself. The suite explicitly supports -SizeGB (default 1 GB) and -FileCount, and a transfer that size can legitimately take longer than 4 minutes, so runs that passed before this change could now fail at the cap rather than on anything real.
Could we read the cap from the environment with 60 as the default, e.g. parseInt(pm.environment.get('maxPollAttempts') || '60'), and let Run-E2E-Tests.ps1 scale it with the requested file size?
There was a problem hiding this comment.
Same constant at Line numbers 2635, 3500, 4348
| }, | ||
| { | ||
| "key": "Content-Type", | ||
|
|
There was a problem hiding this comment.
When the cap trips, neither the 404 branch nor the InProgress branch calls setNextRequest, so execution falls through to Verify Activity Log and Download Activity Log, which then fail as well ("Final transfer status is Completed"). One root cause shows up as three failures in the report. Worth ending the journey at the cap so the report points at the actual problem?
There was a problem hiding this comment.
Same pattern in the other three poll scripts
| # NetApp -> Egress copy-back destination. Must NOT equal the upload | ||
| # folder (`4. Served Evidence/`) or every run produces PartiallyCompleted | ||
| # with FileExists, because the file being copied back from NetApp is the | ||
| # one the upload step just put in `4. Served Evidence/`. Per issue #4, |
There was a problem hiding this comment.
This comment block still explains FileExists-avoidance through timestamped upload filenames ("Per issue #4 ... collisions there only occur if the same timestamped filename is reused"). The collection now appends e2e-run-<id>/ to egressDestinationFolder, and that subfolder is the actual guard. Can we update this block to describe the current mechanism?
|
I couldn't find anything that calls this script, so either fix works: add the per-journey variables (mirroring |
… with current mechanism (PR review) - Run-E2E-Tests.ps1: the egressDestinationFolder comment described FileExists-avoidance via timestamped upload filenames, but the actual guard is now the per-run e2e-run-<id>/ sub-folder appended by the collection's 10. / [NME] 10. Validate prerequest. Concise three-line comment reflects that. - README.md: "Speed" row in the Run Modes table relabelled the previous three-journey timings as four-journey without re-measuring. Replaced the precise minute counts with relative descriptions until we have fresh numbers.
…-fast, scale poll cap, retire dead script (PR review)
Address four PR-review comments on FCT2-17945.
[NME] 9. List Netapp Files stops on failure
The fail-fast branch (Move source missing) and the empty-folder branch
both marked the test failed but didn't end the journey, so [NME] 10-14
still ran -- and with a stale netappFilePath set by the previous Copy
back folder, this folder submitted a transfer for the wrong file. Now
each failure branch calls pm.execution.setNextRequest(null). Removed
the trailing unconditional setNextRequest('[NME] 10. ...') since
Postman/Newman advances sequentially by default on the success path.
MAX_POLL_ATTEMPTS is now configurable and scales with file size
MAX_POLL_ATTEMPTS = 60 worked out at ~3-4 min per transfer, but the
suite supports -SizeGB (default 1 GB) and -FileCount; a legitimate
large transfer could fail at the cap rather than on anything real.
All four poll scripts now read parseInt(pm.environment.get(
'maxPollAttempts') || '60'). Run-E2E-Tests.ps1 computes a scaled
value (60 base + 30 attempts per 100 MB above 100 MB, capped at 600
~= 30 min) and injects it via the env update step.
Cap-trip ends the journey
When the cap tripped, the 404 and InProgress branches marked the test
failed but didn't call setNextRequest, so Verify Activity Log and
Download Activity Log then also failed (transferStatus != 'Completed').
One root cause showed up as three failures. All four poll scripts now
call pm.execution.setNextRequest(null) after the cap-trip assertion in
both branches so the report points at the actual problem.
Retire update-env-multifile.ps1
The script only wrote the legacy egressFileName / egressFileNames, so
an env updated through it left egressCopyFile* / egressMoveFile* unset
-- the 7b scripts then read stale per-journey values from a previous
full run, silently running against the wrong source. Zero callers in
the repo (dead since FCT2-13364); deleted rather than carrying the
maintenance debt of mirroring Run-E2E-Tests.ps1's per-journey writes.
|
3 participants
close the silent-pass bugs that let backend regressions slip through.
PartiallyCompleted no longer passes silently.
Run-E2E-Tests.ps1:
Setup-EgressWorkspaceAndUpload.ps1:
Verified: both
Run-E2E-Tests.ps1 -SizeMB 10 -RegisterCaseand the default-mode equivalent pass all three E2E journeys end-to-end.