Build 55, 56, and 57 failed

[dartandrevinsky]

Stage:

e2e tests

Summary:

Test Suites: 1 failed, 1 total
Tests:       3 failed, 3 passed, 6 total
Snapshots:   0 total
Time:        102.683 s
Ran all test suites.

Logs:

FAIL  tests-ui/comprehensive.spec.js (101.913 s)
  Interaction with the entire FTGO UI application:
    00. Ground-zero tests. Browser capabilities
      ✓ Settings (6 ms)
      ✓ Navigation to Landing and a screenshot (2238 ms)
    10. Landing Page -> Restaurants List -> Menu Page
      ✓ Navigation to Landing (2059 ms)
      ✕ [landing page] Correct entry, submission, landing on Restaurants List (31570 ms)
      ✕ [restaurants list page] Navigation, picking correct restaurant, landing of Menu page (30177 ms)
      ✕ [restaurant menu page] Structure check, menu picking, going to checkout (30188 ms)

Log details:

  ● Interaction with the entire FTGO UI application: › 10. Landing Page -> Restaurants List -> Menu Page › [landing page] Correct entry, submission, landing on Restaurants List

    TimeoutError: waiting for selector `[data-testid^="button|submit|landing page form|"][disabled]` failed: timeout 30000ms exceeded

      54 | export function waitForSelector(page, sel, ...args) {
      55 |   console.log(`waitForSelector(page, '${ sel }')`);
    > 56 |   return page.waitForSelector(String(sel), ...args);
         |               ^
      57 | }
      58 |
      59 | export function waitForTimeout(page, msec) {

      at new WaitTask (../node_modules/puppeteer/src/common/DOMWorld.ts:780:28)
      at DOMWorld.waitForSelectorInPage (../node_modules/puppeteer/src/common/DOMWorld.ts:632:22)
      at Object.internalHandler.waitFor (../node_modules/puppeteer/src/common/QueryHandler.ts:78:19)
      at DOMWorld.waitForSelector (../node_modules/puppeteer/src/common/DOMWorld.ts:490:25)
      at Frame.waitForSelector (../node_modules/puppeteer/src/common/FrameManager.ts:1163:47)
      at Page.waitForSelector (../node_modules/puppeteer/src/common/Page.ts:2100:29)
      at waitForSelector (puppeteerExtensions.js:56:15)
      at Object.expectDisabled (helpers/index.js:28:14)
      at submitTheAddressAndTimeFormSuccessfully (pages/landing.js:27:48)

  ● Interaction with the entire FTGO UI application: › 10. Landing Page -> Restaurants List -> Menu Page › [restaurants list page] Navigation, picking correct restaurant, landing of Menu page

    TimeoutError: waiting for selector `[data-testid^="page|restaurants list|"]` failed: timeout 30000ms exceeded

      54 | export function waitForSelector(page, sel, ...args) {
      55 |   console.log(`waitForSelector(page, '${ sel }')`);
    > 56 |   return page.waitForSelector(String(sel), ...args);
         |               ^
      57 | }
      58 |
      59 | export function waitForTimeout(page, msec) {

      at new WaitTask (../node_modules/puppeteer/src/common/DOMWorld.ts:780:28)
      at DOMWorld.waitForSelectorInPage (../node_modules/puppeteer/src/common/DOMWorld.ts:632:22)
      at Object.internalHandler.waitFor (../node_modules/puppeteer/src/common/QueryHandler.ts:78:19)
      at DOMWorld.waitForSelector (../node_modules/puppeteer/src/common/DOMWorld.ts:490:25)
      at Frame.waitForSelector (../node_modules/puppeteer/src/common/FrameManager.ts:1163:47)
      at Page.waitForSelector (../node_modules/puppeteer/src/common/Page.ts:2100:29)
      at waitForSelector (puppeteerExtensions.js:56:15)
      at Object.ensurePresent (helpers/index.js:22:14)
      at fn (pages/restaurantsList.js:15:38)
      at cb (pages/utilities.js:79:22)
      at Object.browseForRestaurantWithMenuItems (pages/utilities.js:46:45)
      at afn (comprehensive.spec.js:111:39)
      at Promise.catch.testNumber.testNumber (testWrapper.js:64:17)
      at Object.<anonymous> (testWrapper.js:56:14)

  ● Interaction with the entire FTGO UI application: › 10. Landing Page -> Restaurants List -> Menu Page › [restaurant menu page] Structure check, menu picking, going to checkout

    TimeoutError: waiting for selector `[data-testid^="table|restaurant menu|"]` failed: timeout 30000ms exceeded

      54 | export function waitForSelector(page, sel, ...args) {
      55 |   console.log(`waitForSelector(page, '${ sel }')`);
    > 56 |   return page.waitForSelector(String(sel), ...args);
         |               ^
      57 | }
      58 |
      59 | export function waitForTimeout(page, msec) {

      at new WaitTask (../node_modules/puppeteer/src/common/DOMWorld.ts:780:28)
      at DOMWorld.waitForSelectorInPage (../node_modules/puppeteer/src/common/DOMWorld.ts:632:22)
      at Object.internalHandler.waitFor (../node_modules/puppeteer/src/common/QueryHandler.ts:78:19)
      at DOMWorld.waitForSelector (../node_modules/puppeteer/src/common/DOMWorld.ts:490:25)
      at Frame.waitForSelector (../node_modules/puppeteer/src/common/FrameManager.ts:1163:47)
      at Page.waitForSelector (../node_modules/puppeteer/src/common/Page.ts:2100:29)
      at waitForSelector (puppeteerExtensions.js:56:15)
      at Object.ensurePresent (helpers/index.js:22:14)
      at fn (pages/restaurantMenu.js:22:37)
      at cb (pages/utilities.js:79:22)
      at Object.checkStructure (pages/utilities.js:46:45)
      at afn (comprehensive.spec.js:117:38)
      at Promise.catch.testNumber.testNumber (testWrapper.js:64:17)
      at Object.<anonymous> (testWrapper.js:56:14)

[dartandrevinsky]

Having reached the green build, I spent some of my time and thought about what was holding me most during fixing this issue. And here are my thoughts (first) and solutions to them (they follow the first list):

1. one of the important points is that I am not that friendly with the shell commands and its options. I could’ve spent much less time if I knew the port (8080) wasn’t working. Like I described, I tried to put through 5000 and 8080 to my local machine to reproduce the testing environment exactly as it was on the CI server. I managed, however, to address my need with the 3rd-party tool, ngrok.

2. probably the most important thing that could be improved (but I don’t know yet how) was inability to identify a problem that occurred during the docker build.
   The command docker-compose up -d --build covers all output in the containers. I accidentally typed docker-compose (with no arguments) and saw the output from frontend and backend containers. And this was a deal-breaker. I would like to split this command (docker-compose up -d --build) unto two: the first - builds, the second outputs the containers’ output. I wonder if docker-compose up -d would suffice?

3. the third and most mysterious thing. I don’t know why npm run start for the backend server failed on the CI and was success everywhere else (my machine, my remote docker machine). Locally, I ensured that the node version was the same and still, the build on CI failed (to be precise, the time during the start of the backend server).

4. wait-for-services.sh script was set up to check only the frontend service. When I added the backend port - that’s when I realized there was a problem with it.

Thinking of solutions to the above points:

1. I probably need to have spent more time over docker documentation (of find a solid online docker course) to know there was a simpler solution to run a command to identify mapped ports, and started (or downed) containers right off.

2. the second point is the number one thing to address to improve debuggability of the problems that occur (and get eaten) during docker build time. Could you please suggest the commands/arguments to achieve that?

3. Honestly, I don’t know the reason behind it, so I can’t come up with a solution yet.

4. I need to address this script (wait-for-services.sh) to test both, front- and back-, ends, and to bail quickly if either is not working (presently, the script runs to timeout if the service is not responding.