newman command failure & assesrtion failure

[murtyc]

Hi,

Firstly, thanks for the nice work and I've learned few things after working through this.
I have experienced 2 issues and my pipeline was successful after applying the fixes.
I can understand Issue 2 is kind of intentionally created to demonstrate the capability of CodeBuild reports. However it would be good to explain the steps to fix the test assertion failure.

Issue 1: newman command failed (refer below error)

Command did not exit successfully newman run PetStoreAPI.postman_collection.json --environment PetStoreAPIEnvironment.postman_environment.json -r junit exit status 1

Phase complete: BUILD State: FAILED
Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: newman run PetStoreAPI.postman_collection.json --environment PetStoreAPIEnvironment.postman_environment.json -r junit. Reason: exit status 1
Entering phase POST_BUILD

Fix: add cli in the command (refer below) in postman-newman-buildspec.yml
newman run PetStoreAPI.postman_collection.json
--environment PetStoreAPIEnvironment.postman_environment.json
-r cli,junit

Issue 2: Assertion failure for Is response time is less than 200ms. Reason: It's taking up to ~600ms
I've updated PetStoreAPI.postman_collection.json with 600 in 3 locations and copied the file to S3 bucket
pm.expect(pm.response.responseTime).to.be.below(600)

[premnhs]

Hi Murthy

Thanks for the above post. I m fairly new to aws and I managed to complete all the setup except this issue. I came across the same issue 1 of your post. When I updated according to your suggestion. I m still hitting the . Following is the error

[Container] [Date] Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: newman run testspa.json --environment testspa.json -r cli,junit. Reason: exit status 1

so not sure if I m missing anything else.

Any assistance would be helpful.

[GammaBlaze]

I have run into the same issue even after attempting @murtyc's fix. Please see the attached log extract frome Codebuild.
TestPetStoreAPI - Build Failure.txt

[ankitgupta404]

The addition of 'cli', does help resolve some of the test issues. if you notice the TestPetStoreAPI logs, you will see that the test execution is happening. the results of the same can be seen in Code build test report history tab.

You still see the error because there are 3 assertions which are failing related to response time. I believe this error "exit status 1" is a standard newman execution error in case of failed assertions , which is not ideal.

You can fix the response time errors by updating the expected response time in "PetStoreAPI.postman_collection.json" file