From 3c76a870875c87919c80fcae124e44868ff5a518 Mon Sep 17 00:00:00 2001 From: Shelley Nason Date: Wed, 8 Apr 2026 09:52:48 -0400 Subject: [PATCH 1/2] Add automated tests to replace manual test suite --- Dockerfile | 2 +- features/404_tracking.feature | 21 +++ features/autotracker.feature | 85 ++++++++++ features/autotracker_download.feature | 23 --- features/banner_tracker.feature | 12 ++ features/basic_page_load.feature | 6 +- features/configuration.feature | 19 ++- features/gas4_functions.feature | 156 ++++++++++++++++++ features/gas_functions.feature | 39 +++++ features/parallel_tracker.feature | 45 +++++ features/pii_redaction.feature | 15 +- features/routing.feature | 24 +++ features/search_tracking.feature | 26 +++ features/support/dapconfig.js | 15 ++ .../support/step_definitions/browser_steps.js | 57 +++---- .../step_definitions/dataLayer_steps.js | 120 +++++++++----- .../step_definitions/interaction_steps.js | 7 + .../support/step_definitions/loading_steps.js | 30 +++- .../support/step_definitions/youtube_steps.js | 81 +++++++++ features/youtube_tracking.feature | 101 ++++++++++++ features/youtube_tracking_integration.feature | 19 +++ package-lock.json | 72 +------- package.json | 2 +- test_site/404.html | 13 ++ test_site/agency-tracking.html | 34 ++++ test_site/index.html | 8 +- test_site/not-found.html | 13 ++ test_site/youtube.html | 7 +- 28 files changed, 871 insertions(+), 181 deletions(-) create mode 100644 features/404_tracking.feature create mode 100644 features/autotracker.feature delete mode 100644 features/autotracker_download.feature create mode 100644 features/banner_tracker.feature create mode 100644 features/gas4_functions.feature create mode 100644 features/gas_functions.feature create mode 100644 features/parallel_tracker.feature create mode 100644 features/routing.feature create mode 100644 features/search_tracking.feature create mode 100644 features/support/step_definitions/youtube_steps.js create mode 100644 features/youtube_tracking.feature create mode 100644 features/youtube_tracking_integration.feature create mode 100644 test_site/404.html create mode 100644 test_site/agency-tracking.html create mode 100644 test_site/not-found.html diff --git a/Dockerfile b/Dockerfile index 6a3b402..5bbbd89 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,6 +1,6 @@ FROM nginx ARG DAP_ENV='dev' ENV DAP_ENV=${DAP_ENV} -COPY test_site Universal-Federated-Analytics-Min.js Federated.js.map /usr/share/nginx/html/ +COPY test_site Universal-Federated-Analytics-Min.js Universal-Federated-Analytics.js Federated.js.map /usr/share/nginx/html/ COPY nginx-test.conf.template /etc/nginx/conf.d/ RUN envsubst '${DAP_ENV}' < /etc/nginx/conf.d/nginx-test.conf.template > /etc/nginx/conf.d/default.conf diff --git a/features/404_tracking.feature b/features/404_tracking.feature new file mode 100644 index 0000000..fe1ce9a --- /dev/null +++ b/features/404_tracking.feature @@ -0,0 +1,21 @@ +Feature: DAP modifies page_location for 404 pages + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: Page title containing "404" sets page_location to a vpv404 path + When I load the test page "404.html" + Then DAP will set custom dimensions for the DAP property + | page_location | http://dap-test-site.local/vpv404/404.html | + + Scenario: Page title containing "not found" sets page_location to a vpv404 path + When I load the test page "not-found.html" + Then DAP will set custom dimensions for the DAP property + | page_location | http://dap-test-site.local/vpv404/not-found.html | + + Scenario: 404 page with a referrer appends the referrer to page_location + When I load the test site + And I click on link with href "/404.html" and wait for new page to load + Then DAP will set custom dimensions for the DAP property + | page_location | http://dap-test-site.local/vpv404/404.html/http://dap-test-site.local/ | diff --git a/features/autotracker.feature b/features/autotracker.feature new file mode 100644 index 0000000..ba70627 --- /dev/null +++ b/features/autotracker.feature @@ -0,0 +1,85 @@ +# DAP's GA4 property is configured to disable most enhanced measurement events. +# The DAP code takes over responsibility for tracking these events, rather than relying on gtag's implementation. +# This test feature verifies DAP's alternative implementation of outbound click and file download tracking. +Feature: Downloads and outbound link clicks are tracked when autotracking is enabled + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: Clicking an email link fires email_click when autotracking is enabled + Given DAP is configured with autotracking enabled + When I load the test site + And I click on element with selector "a[href='mailto:test@domain.com']" + Then a "email_click" event is sent to DAP with parameters + | link_text | mailto:[REDACTED_EMAIL] | + | link_domain | domain.com | + | outbound | true | + | interaction_type | Mouse Click | + + Scenario: Clicking an email link does not fire email_click when autotracking is disabled + Given DAP is configured with autotracking disabled + When I load the test site + And I click on element with selector "a[href='mailto:test@domain.com']" + Then no "email_click" event is sent to DAP + + Scenario: Clicking a telephone link fires telephone_click when autotracking is enabled + Given DAP is configured with autotracking enabled + When I load the test site + And I click on element with selector "a[href='tel:+1437-925-1855']" + Then a "telephone_click" event is sent to DAP with parameters + | link_text | Telephone [REDACTED_TEL] | + | interaction_type | Mouse Click | + + Scenario: Clicking a telephone link does not fire telephone_click when autotracking is disabled + Given DAP is configured with autotracking disabled + When I load the test site + And I click on element with selector "a[href='tel:+1437-925-1855']" + Then no "telephone_click" event is sent to DAP + + Scenario: Clicking an outbound link sends a click event when autotracking is enabled + Given DAP is configured with autotracking enabled + When I load the test site + And I click on element with selector "a[href='http://www.gsa.gov/travelpolicy']" + Then a "click" event is sent to DAP with parameters + | link_url | http://gsa.gov/travelpolicy | + | link_domain | gsa.gov | + | link_text | http://gsa.gov/travelpolicy | + | outbound | true | + | interaction_type | Mouse Click | + + Scenario: Clicking an outbound link does not fire event when autotracking is disabled + Given DAP is configured with autotracking disabled + When I load the test site + And I click on element with selector "a[href='http://www.gsa.gov/travelpolicy']" + Then no "click" event is sent to DAP + + Scenario: Clicking a file download link reports the download when autotracking is enabled + When I load the test site + And I click on a file to download it + Then a "file_download" event is sent to DAP with parameters + | file_name | /about.zip | + | file_extension | zip | + | link_text | /about.zip | + | link_id | internalDownload | + | link_url | http://dap-test-site.local/about.zip | + | link_domain | dap-test-site.local | + | interaction_type | Mouse Click | + + Scenario: Pressing Enter on a file download link reports the download when autotracking is enabled + When I load the test site + And I highlight and press Enter on a file to download it + Then a "file_download" event is sent to DAP with parameters + | file_name | /about.zip | + | file_extension | zip | + | link_text | /about.zip | + | link_id | internalDownload | + | link_url | http://dap-test-site.local/about.zip | + | link_domain | dap-test-site.local | + | interaction_type | Enter Key Keystroke | + + Scenario: File downloads are not tracked when autotracking is disabled + Given DAP is configured with autotracking disabled + When I load the test site + And I click on a file to download it + Then no "file_download" event is sent to DAP diff --git a/features/autotracker_download.feature b/features/autotracker_download.feature deleted file mode 100644 index 80a7425..0000000 --- a/features/autotracker_download.feature +++ /dev/null @@ -1,23 +0,0 @@ -Feature: Downloads are reported to DAP when autotracking is enabled - - Background: - Given I load an empty browser - And DAP is configured for agency "GSA" - - Scenario: User clicks to download file with autotracker on - Given DAP is configured with autotracking enabled - When I load the test site - And I click on a file to download it - Then the file download is reported to DAP with interaction type "Mouse Click" - - Scenario: User presses Enter to download file with autotracker on - Given DAP is configured with autotracking enabled - When I load the test site - And I highlight and press Enter on a file to download it - Then the file download is reported to DAP with interaction type "Enter Key Keystroke" - - Scenario: User clicks to download file with autotracker off - Given DAP is configured with autotracking disabled - When I load the test site - And I click on a file to download it - Then the file download is not reported to DAP diff --git a/features/banner_tracker.feature b/features/banner_tracker.feature new file mode 100644 index 0000000..a25a666 --- /dev/null +++ b/features/banner_tracker.feature @@ -0,0 +1,12 @@ +Feature: DAP tracks clicks on any 'Official website of the US government' banner + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: Clicking the USA banner button fires official_usa_site_banner_click + When I load the test site + And I click on element with selector "#banner-button" + Then a "official_usa_site_banner_click" event is sent to DAP with parameters + | link_text | Here's how you know | + | section | header | \ No newline at end of file diff --git a/features/basic_page_load.feature b/features/basic_page_load.feature index f28f98a..0289e26 100644 --- a/features/basic_page_load.feature +++ b/features/basic_page_load.feature @@ -8,12 +8,10 @@ Feature: Test the outgoing requests sent by a basic page with DAP code loaded Scenario: Loading the page with the DAP code without further action When I load the test site And I wait 5 seconds - Then there is a GA4 request - But there are no unexpected requests + Then there are no unexpected requests Scenario: Loading the page with the DAP code and clicking a button When I load the test site And I click on element with selector "#banner-button" And I wait 5 seconds - Then there is a GA4 request - But there are no unexpected requests + Then there are no unexpected requests diff --git a/features/configuration.feature b/features/configuration.feature index 4d58792..a1405b6 100644 --- a/features/configuration.feature +++ b/features/configuration.feature @@ -19,4 +19,21 @@ Feature: A site can load the DAP code with varying levels of customization Then DAP will set custom dimensions for the DAP property | agency | GSA | | site_topic | analytics | - | site_platform | cloud.gov | \ No newline at end of file + | site_platform | cloud.gov | + + Scenario: Load a DAP-enabled page and check the built-in customer dimensions + Given DAP is configured for agency "GSA" + When I load the test site + Then DAP will set custom dimensions for the DAP property + | protocol | http: | + | hostname_dimension | dap-test-site.local | + | using_parallel_tracker | no | + And DAP will set the "script_source" dimension to a string matching "https?:\/\/.*\/universal-federated-analytics-min.js" + And DAP will set the "version" dimension to a string matching "\d{8} v\d+\.\d+ - ga4" + + Scenario: Load a DAP-enabled page with a custom cookie timeout + Given DAP is configured for agency "GSA" + And DAP is configured with cookie timeout of 1 months + When I load the test site + Then DAP will set custom dimensions for the DAP property + | cookie_expires | 2628000 | \ No newline at end of file diff --git a/features/gas4_functions.feature b/features/gas4_functions.feature new file mode 100644 index 0000000..68e75d5 --- /dev/null +++ b/features/gas4_functions.feature @@ -0,0 +1,156 @@ +Feature: gas4() custom event API + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: gas4() sends a virtual page_view event + When I load the test site + And I execute script "window.gas4('page_view', {page_location: '/virtual/page', page_title: 'Virtual Page'})" + Then a "page_view" event is sent to DAP with parameters + | page_location | http://dap-test-site.local/virtual/page | + | page_title | Virtual Page | + + Scenario: gas4() sends a virtual page_view event setting default page title + When I load the test site + And I execute script "window.gas4('page_view', {page_location: '/virtual/page'})" + Then a "page_view" event is sent to DAP with parameters + | page_location | http://dap-test-site.local/virtual/page | + | page_title | DAP test site | + + Scenario: gas4() sends a file_download event + When I load the test site + And I execute script "window.gas4('file_download', {file_name: '/test.pdf', file_extension: 'pdf', link_text: 'Download Test PDF', link_id: 'download-link', link_url: 'https://example.gov/test.pdf', link_domain: 'example.gov', interaction_type: 'Mouse Click'})" + Then a "file_download" event is sent to DAP with parameters + | file_name | /test.pdf | + | file_extension | pdf | + | link_text | Download Test PDF | + | link_id | download-link | + | link_url | https://example.gov/test.pdf | + | link_domain | example.gov | + | interaction_type | Mouse Click | + + Scenario: gas4() sends a form_start event + When I load the test site + And I execute script "window.gas4('form_start', {form_id: 'contact-form', form_name: 'Contact Form', form_destination: '/thank-you', section: 'body'})" + Then a "form_start" event is sent to DAP with parameters + | form_id | contact-form | + | form_name | Contact Form | + | form_destination | /thank-you | + | section | body | + + Scenario: gas4() sends a form_progress event + When I load the test site + And I execute script "window.gas4('form_progress', {form_id: 'contact-form', percent_scrolled: '50'})" + Then a "form_progress" event is sent to DAP with parameters + | form_id | contact-form | + | percent_scrolled | 50 | + + Scenario: gas4() sends a form_submit event + When I load the test site + And I execute script "window.gas4('form_submit', {form_step: 3, form_submit_text: 'Submit'})" + Then a "form_submit" event is sent to DAP with parameters + | form_step | 3 | + | form_submit_text | Submit | + + Scenario: gas4() sends a content_view event + When I load the test site + And I execute script "window.gas4('content_view', {content_type: 'article', content_id: '123'})" + Then a "content_view" event is sent to DAP with parameters + | content_type | article | + | content_id | 123 | + + Scenario: gas4() sends a share event + When I load the test site + And I execute script "window.gas4('share', {method: 'twitter', content_type: 'page', item_id: '123'})" + Then a "share" event is sent to DAP with parameters + | method | twitter | + | content_type | page | + | item_id | 123 | + + Scenario: gas4() sends a social_click event + When I load the test site + And I execute script "window.gas4('social_click', {link_url: 'https://twitter.com', social_network: 'twitter'})" + Then a "social_click" event is sent to DAP with parameters + | link_url | https://twitter.com | + | social_network | twitter | + + Scenario: gas4() sends an image_click event + When I load the test site + And I execute script "window.gas4('image_click', {link_id: 'hero-image', link_url: '/about'})" + Then a "image_click" event is sent to DAP with parameters + | link_id | hero-image | + | link_url | /about | + + Scenario: gas4() sends a cta_click event + When I load the test site + And I click on element with selector "::-p-text(Learn More)" + Then a "cta_click" event is sent to DAP with parameters + | link_text | | + | link_domain | | + | link_url | | + | link_id | | + | link_classes | | + | outbound | | + | section |
| + + Scenario: gas4() sends a navigation_click event + When I load the test site + And I click on element with selector "a[href='#home']" + Then a "navigation_click" event is sent to DAP with parameters + | link_text | | + | link_domain | | + | link_url | | + | link_id | | + | link_classes | | + | outbound | | + | section |
| + | menu_type | | + + Scenario: gas4() sends a was_this_helpful_submit event + When I load the test site + And I execute script "window.gas4('was_this_helpful_submit', {selection: 'yes', section: 'footer'})" + Then a "was_this_helpful_submit" event is sent to DAP with parameters + | selection | yes | + | section | footer | + + Scenario: gas4() sends a faq_click event + When I load the test site + And I click on element with selector "button::-p-text(FAQ)" + Then a "faq_click" event is sent to DAP with parameters + | selection | selection 1 | + | section |
| + + Scenario: gas4() sends an accordion_click event + When I load the test site + And I click on element with selector "button::-p-text(Accordion)" + Then a "accordion_click" event is sent to DAP with parameters + | selection | selection 1 | + | section |
| + + Scenario: gas4() sends an error event + When I load the test site + And I execute script "window.gas4('error', {type: '404', url: '/missing-page'})" + Then a "error" event is sent to DAP with parameters + | type | 404 | + | url | /missing-page | + + Scenario: gas4() sends a filter event + When I load the test site + And I execute script "window.gas4('filter', {filter_selection: 'date', section: 'results'})" + Then a "filter" event is sent to DAP with parameters + | filter_selection | date | + | section | results | + + Scenario: gas4() sends a sort event + When I load the test site + And I execute script "window.gas4('sort', {sort_selection: 'newest', section: 'results'})" + Then a "sort" event is sent to DAP with parameters + | sort_selection | newest | + | section | results | + + Scenario: gas4() with an invalid event name falls back to dap_event + When I load the test site + And I execute script "window.gas4('not_a_valid_event', {some_param: 'value'})" + Then a "dap_event" event is sent to DAP with parameters + | some_param | value | diff --git a/features/gas_functions.feature b/features/gas_functions.feature new file mode 100644 index 0000000..4cacf0f --- /dev/null +++ b/features/gas_functions.feature @@ -0,0 +1,39 @@ +Feature: gas() custom event API + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: gas() sends a virtual pageview + When I load the test site + And I execute script "window.gas('send', 'pageview', '/virtual/page')" + Then a "page_view" event is sent to DAP with parameters + | page_location | http://dap-test-site.local/virtual/page | + | page_title | DAP test site | + + Scenario: gas() sends a virtual pageview with a custom title + When I load the test site + And I execute script "window.gas('send', 'pageview', '/virtual/page', 'My Custom Title')" + Then a "page_view" event is sent to DAP with parameters + | page_location | http://dap-test-site.local/virtual/page | + | page_title | My Custom Title | + + Scenario: gas() sends a custom event + When I load the test site + And I execute script "window.gas('send', 'event', 'test_category', 'test_action', 'test_label', 1)" + Then a "dap_event" event is sent to DAP with parameters + | event_category | test_category | + | event_action | test_action | + | event_label | test_label | + | event_value | 1 | + | non_interaction | false | + + Scenario: gas() sends a custom event with only category and action + When I load the test site + And I execute script "window.gas('send', 'event', 'only_category', 'only_action')" + Then a "dap_event" event is sent to DAP with parameters + | event_category | only_category | + | event_action | only_action | + | event_label | | + | event_value | 0 | + | non_interaction | false | diff --git a/features/parallel_tracker.feature b/features/parallel_tracker.feature new file mode 100644 index 0000000..ecae4c5 --- /dev/null +++ b/features/parallel_tracker.feature @@ -0,0 +1,45 @@ +Feature: A site can use the DAP code with their own parallel GA4 property + + Background: + Given I load an empty browser + + Scenario: Load a page with parallel GA4 property configured but no custom dimensions + Given DAP is configured for agency "HHS" + And DAP is configured for subagency "CDC" + And DAP is configured with parallel GA4 property "G-111111" + When I load the test site + Then DAP will set custom dimensions for the DAP property + | agency | HHS | + | subagency | CDC | + | using_parallel_tracker | pga4 | + And DAP will configure property "G-111111" without custom dimensions + + Scenario: Load a page with parallel GA4 property configured and custom dimensions enabled + Given DAP is configured for agency "HHS" + And DAP is configured for subagency "CDC" + And DAP is configured with parallel GA4 property "G-111111" + And DAP is configured to set custom dimensions on the parallel tracker + When I load the test site + Then DAP will set custom dimensions for the DAP property + | agency | HHS | + | subagency | CDC | + | using_parallel_tracker | pga4 | + And DAP will set custom dimensions for the property "G-111111" + | agency | HHS | + | subagency | CDC | + | protocol | http: | + | hostname_dimension | dap-test-site.local | + | using_parallel_tracker | pga4 | + + Scenario: Load a page with parallel GA4 property configured with site topic and site platform + Given DAP is configured for agency "HHS" + And DAP is configured with site topic "Health" + And DAP is configured with site platform "Cloud.gov" + And DAP is configured with parallel GA4 property "G-111111" + And DAP is configured to set custom dimensions on the parallel tracker + When I load the test site + Then DAP will set custom dimensions for the property "G-111111" + | site_topic | health | + | site_platform | cloud.gov | + + diff --git a/features/pii_redaction.feature b/features/pii_redaction.feature index 563c7c4..f7a3907 100644 --- a/features/pii_redaction.feature +++ b/features/pii_redaction.feature @@ -1,4 +1,4 @@ -Feature: DAP redacts PII +Feature: DAP redacts PII when appropriate Background: Given I load an empty browser @@ -53,14 +53,15 @@ Feature: DAP redacts PII | type | validation | | url | /contact/submit | - Scenario: PII redaction applies to parameters in config calls + Scenario: PII redaction applies to parameters in config calls for DAP property Given the page URL has query parameter "search" set to "user@example.com" When I load the test site - Then the config call for the DAP property has "page_location" containing "[REDACTED_EMAIL]" + Then DAP will set custom dimensions for the DAP property + | page_location | http://dap-test-site.local/?query=[REDACTED_EMAIL] | - Scenario: PII redaction applies to automatically collected events + Scenario: PII redaction applies to parameters in config calls for parallel tracking properties Given the page URL has query parameter "search" set to "user@example.com" - And I set the browser to intercept outbound requests + And DAP is configured with parallel GA4 property "G-111111" When I load the test site - And I wait 5 seconds - Then there is a GA4 request reporting event "page_view" with parameter "dl" containing "[REDACTED_EMAIL]" + Then DAP will set custom dimensions for the property "G-111111" + | page_location | http://dap-test-site.local/?query=[REDACTED_EMAIL] | diff --git a/features/routing.feature b/features/routing.feature new file mode 100644 index 0000000..b87e690 --- /dev/null +++ b/features/routing.feature @@ -0,0 +1,24 @@ +Feature: DAP sends custom events to the correct GA4 properties + + Scenario: DAP sends custom events to the DAP property and to parallel tracking properties + Given I load an empty browser + And DAP is configured for agency "GSA" + And DAP is configured with parallel GA4 property "G-1111111111" + When I load the test site + And I execute script "window.gas4('error', {type: '404', url: '/missing-page'})" + Then a "error" event is sent to DAP with parameters + | send_to | GSA_GA4_ENOR0,GSA_GA4_ENOR1 | + And the "G-9TNNMGP8WJ" property belongs to the "GSA_GA4_ENOR0" group + And the "G-1111111111" property belongs to the "GSA_GA4_ENOR1" group + + Scenario: DAP does not send custom events to any other GA4 properties on the page + Given I load an empty browser + And DAP is configured for agency "GSA" + When I load the test page "agency-tracking.html" + And I execute script "window.gas4('error', {type: '404', url: '/missing-page'})" + Then there are 1 "error" events sent to DAP + And a "error" event is sent to DAP with parameters + | send_to | GSA_GA4_ENOR0 | + And the "G-9TNNMGP8WJ" property belongs to the "GSA_GA4_ENOR0" group + And the "G-SITEOWN1" property does not belong to the "GSA_GA4_ENOR0" group + And the "G-SITEOWN2" property does not belong to the "GSA_GA4_ENOR0" group \ No newline at end of file diff --git a/features/search_tracking.feature b/features/search_tracking.feature new file mode 100644 index 0000000..bc71b41 --- /dev/null +++ b/features/search_tracking.feature @@ -0,0 +1,26 @@ +# DAP's GA4 property is configured to disable most enhanced measurement events. +# The DAP code takes over responsibility for tracking these events, rather than relying on gtag's implementation. +# This test feature verifies DAP's alternative implementation of site search tracking. +Feature: DAP reports site searches as view_search_results events + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + + Scenario: Page URL with q parameter sends a view_search_results event + Given the page URL has query parameter "q" set to "searchterm" + When I load the test site + Then a "view_search_results" event is sent to DAP with parameters + | search_term | searchterm | + + Scenario: Page URL with a custom search parameter sends a view_search_results event + Given DAP is configured with custom search parameter "mysearch" + And the page URL has query parameter "mysearch" set to "searchterm" + When I load the test site + Then a "view_search_results" event is sent to DAP with parameters + | search_term | searchterm | + + Scenario: Page URL with a non-search query parameter does not send a view_search_results event + Given the page URL has query parameter "mysearch" set to "searchterm" + When I load the test site + Then no "view_search_results" event is sent to DAP diff --git a/features/support/dapconfig.js b/features/support/dapconfig.js index 3937a89..0a361b1 100644 --- a/features/support/dapconfig.js +++ b/features/support/dapconfig.js @@ -1,15 +1,30 @@ +/** + * Represents all the ways a DAP-enabled page can alter DAP's configuration at load-time via query params in the DAP script tag. + */ class DAPConfig { agency; subagency; sitetopic; siteplatform; + sp; + yt; + pga4; + parallelcd; autotracker; cto; + /** + * At minimum, DAPConfig requires an agency to be specified. Other fields can be set as needed after instantiation. + * @param agency + */ constructor(agency) { this.agency = agency; } + /** + * Converts the configured fields of this DAPConfig instance into a query parameter string that can be appended to the DAP script URL. + * @returns {string} the query param string + */ toQueryParams() { const configuredFields = Object.entries(this).filter(entry => entry[1] !== undefined); return new URLSearchParams(configuredFields).toString(); diff --git a/features/support/step_definitions/browser_steps.js b/features/support/step_definitions/browser_steps.js index 088f6ed..a4dd4c2 100644 --- a/features/support/step_definitions/browser_steps.js +++ b/features/support/step_definitions/browser_steps.js @@ -3,6 +3,12 @@ import puppeteer from 'puppeteer'; import * as chai from 'chai' const expect = chai.expect; +chai.config.showDiff = true; +chai.config.truncateThreshold = 0; + +/** + * Returns a promise that resolves after the specified number of milliseconds. + */ function delay(milliseconds) { return new Promise((resolve) => { setTimeout(resolve, milliseconds); @@ -10,10 +16,14 @@ function delay(milliseconds) { } Given("I load an empty browser", async function () { - this.browser = await puppeteer.launch(); + this.browser = await puppeteer.launch({ + args: [ + '--host-resolver-rules=MAP dap-test-site.local 127.0.0.1:8080', + ] + }); this.page = await this.browser.newPage(); - if (process.env.VERBOSE == 'true') { + if (process.env.VERBOSE === 'true') { // Log page events to the node console this.page .on('console', message => @@ -24,6 +34,19 @@ Given("I load an empty browser", async function () { .on('requestfailed', request => console.log(`${request.failure().errorText} ${request.url()}`)) } + + // Google's gtag library treats the dataLayer as mostly but not entirely append only. + // For instance, gtag('config', 'G-XXXX', { 'groups': 'group1' }) configures the property with the specified groups, but removes the groups object from the dataLayer. + // So, tests must be run against a mock of dataLayer.push, not directly against the dataLayer itself. + await this.page.evaluateOnNewDocument(() => { + window.mockDataLayer = []; + window.dataLayer = window.dataLayer || []; + const originalPush = window.dataLayer.push; + window.dataLayer.push = function (argumentObj) { + window.mockDataLayer.push(structuredClone(Array.from(argumentObj))); + return originalPush.call(this, argumentObj); + } + }); }); Given("I set the browser to intercept outbound requests", async function () { @@ -45,40 +68,14 @@ When("I wait {int} seconds", async function (delaySeconds) { await delay(delaySeconds * 1000); }); -Then("there is a GA4 request", function () { - const ga4Request = this.requests.find(request => { - try { - const url = new URL(request.url); - return url.host === "www.google-analytics.com" && url.pathname === "/g/collect"; - } catch (e) { - return false; - } - }); - expect(ga4Request).to.exist; -}); - -Then("there is a GA4 request reporting event {string} with parameter {string} containing {string}", function (eventName, paramName, value) { - const ga4Request = this.requests.find(request => { - try { - const url = new URL(request.url); - return url.host === "www.google-analytics.com" - && url.pathname === "/g/collect" - && url.searchParams.has("en", eventName) - && url.searchParams.has(paramName) && url.searchParams.get(paramName).includes(value); - } catch (e) { - return false; - } - }); - expect(ga4Request).to.exist; -}); - Then("there are no unexpected requests", function () { const requestURLs = this.requests.map((request) => { return (new URL(request.url)).host; }); + // Should be no calls to www.youtube.com in default configuration (youtube tracking is opt-in) const allowedURLs = [ - "localhost:8080", + "dap-test-site.local", "d3vtlq0ztv2u27.cloudfront.net", "dap.digitalgov.gov", "www.googletagmanager.com", diff --git a/features/support/step_definitions/dataLayer_steps.js b/features/support/step_definitions/dataLayer_steps.js index 50a4f9b..440e29d 100644 --- a/features/support/step_definitions/dataLayer_steps.js +++ b/features/support/step_definitions/dataLayer_steps.js @@ -2,54 +2,92 @@ import { Then } from "@cucumber/cucumber"; import * as chai from 'chai' const expect = chai.expect; -Then("the config call for the DAP property has {string} containing {string}", async function (key, substring) { - const configCommand = await this.page.evaluate(() => { - return window.dataLayer.find(item => item[0] === 'config' && item[1] === "G-9TNNMGP8WJ"); - }); - expect(configCommand["2"][key]).to.include(substring); -}); +const TEST_PROPERTY_ID = "G-9TNNMGP8WJ"; -Then("DAP will set custom dimensions for the DAP property", async function (table) { - const configCommand = await this.page.evaluate(() => { - return window.dataLayer.find(item => item[0] === 'config' && item[1] === "G-9TNNMGP8WJ"); - }); - expect(configCommand["2"]).to.include(table.rowsHash()); -}); - -Then("the file download is reported to DAP with interaction type {string}", async function (interactionType) { - const event = await this.page.evaluate(() => { - return window.dataLayer.find(item => item[0] === 'event' && item[1] === 'file_download'); - }); - expect(event).to.deep.equal( - { - '0': 'event', - '1': 'file_download', - '2': { - "interaction_type": interactionType, - "send_to": 'GSA_GA4_ENOR0', - "event_name_dimension": 'file_download', - "file_extension": "zip", - "file_name": "/about.zip", - "link_domain": "localhost", - "link_id": "internalDownload", - "link_text": "/about.zip", - "link_url": "http://localhost:8080/about.zip", - } - } +/** + * Converts the string values from the Cucumber data table to match the types of the actual values in the DAP config/event objects. + */ +function convertDataTableTypesToMatchActual(table, actual) { + return Object.fromEntries( + Object.entries(table.rowsHash()).map(([k, v]) => { + const actualValue = actual[k]; + if (typeof actualValue === 'number') return [k, Number(v)]; + if (typeof actualValue === 'boolean') return [k, v === 'true']; + return [k, v]; + }) ); +} + +Then("DAP will set custom dimensions for the DAP property", async function (table) { + const configCommand = await this.page.evaluate((propertyId) => { + return window.mockDataLayer.find(item => item[0] === 'config' && item[1] === propertyId); + }, TEST_PROPERTY_ID); + expect(configCommand).to.exist; + const expected = convertDataTableTypesToMatchActual(table, configCommand["2"]); + expect(configCommand["2"]).to.containSubset(expected); }); -Then("the file download is not reported to DAP", async function () { - const event = await this.page.evaluate(() => { - return window.dataLayer.find(item => item[0] === 'event' && item[1] === 'file_download'); - }); - expect(event).to.be.undefined; +Then("DAP will set the {string} dimension to a string matching {string}", async function (key, regex) { + const configCommand = await this.page.evaluate((propertyId) => { + return window.mockDataLayer.find(item => item[0] === 'config' && item[1] === propertyId); + }, TEST_PROPERTY_ID); + expect(configCommand["2"][key]).to.match(new RegExp(regex)); +}); + +Then("DAP will set custom dimensions for the property {string}", async function (propertyId, table) { + const configCommand = await this.page.evaluate((propertyId) => { + return window.mockDataLayer.find(item => item[0] === 'config' && item[1] === propertyId); + }, propertyId); + expect(configCommand).to.exist; + const expected = convertDataTableTypesToMatchActual(table, configCommand["2"]); + expect(configCommand["2"]).to.containSubset(expected); +}); + +Then("DAP will configure property {string} without custom dimensions", async function (propertyId) { + const configCommand = await this.page.evaluate((propertyId) => { + return window.mockDataLayer.find(item => item[0] === 'config' && item[1] === propertyId); + }, propertyId); + expect(configCommand["2"]).to.not.include.keys("agency", "subagency", "script_source", "version", "protocol", "hostname_dimension", "using_parallel_tracker"); +}); + +Then("the {string} property belongs to the {string} group", async function (propertyId, groupName) { + const configEvents = await this.page.evaluate((propertyId) => { + return window.mockDataLayer.filter(item => item[0] === 'config' && item[1] === propertyId); + }, propertyId); + expect(configEvents).to.not.be.empty; + const groups = configEvents.flatMap(config => config[2]?.groups ?? []); + expect(groups).to.include(groupName); +}); + +Then("the {string} property does not belong to the {string} group", async function (property, group) { + const configEvents = await this.page.evaluate((id) => { + return window.mockDataLayer.filter(item => item[0] === 'config' && item[1] === id); + }, property); + expect(configEvents).to.not.be.empty; + const groups = configEvents.flatMap(config => config[2]?.groups ?? []); + expect(groups).to.not.include(group); }); Then("a {string} event is sent to DAP with parameters", async function (eventName, table) { - const event = await this.page.evaluate((name) => { - return window.dataLayer.find(item => item[0] === 'event' && item[1] === name); + const event = await this.page.evaluate((eventName) => { + return window.mockDataLayer.find(item => item[0] === 'event' && item[1] === eventName); }, eventName); expect(event).to.exist; - expect(event["2"]).to.include(table.rowsHash()); + const expected = convertDataTableTypesToMatchActual(table, event[2]); + expect(event["2"]).to.containSubset(expected); +}); + +Then("no {string} event is sent to DAP", async function (eventName) { + const event = await this.page.evaluate((eventName) => { + return window.mockDataLayer.find(item => item[0] === 'event' && item[1] === eventName); + }, eventName); + expect(event).to.be.undefined; }); + +Then("there are {int} {string} events sent to DAP", async function (count, eventName) { + const events = await this.page.evaluate((name) => { + return window.mockDataLayer.filter(item => item[0] === 'event' && item[1] === name); + }, eventName); + expect(events).to.have.lengthOf(count); +}); + diff --git a/features/support/step_definitions/interaction_steps.js b/features/support/step_definitions/interaction_steps.js index 3fbffe6..4acb962 100644 --- a/features/support/step_definitions/interaction_steps.js +++ b/features/support/step_definitions/interaction_steps.js @@ -15,4 +15,11 @@ When("I highlight and press Enter on a file to download it", async function () { When("I click on element with selector {string}", async function (elementSelector) { await this.page.locator(elementSelector).click(); +}); + +When("I click on link with href {string} and wait for new page to load", async function (href) { + await Promise.all([ + this.page.waitForNavigation(), + this.page.locator(`a[href='${href}']`).click() + ]); }); \ No newline at end of file diff --git a/features/support/step_definitions/loading_steps.js b/features/support/step_definitions/loading_steps.js index 3d9b488..e5e5c3f 100644 --- a/features/support/step_definitions/loading_steps.js +++ b/features/support/step_definitions/loading_steps.js @@ -18,6 +18,26 @@ Given("DAP is configured with site platform {string}", function (siteplatform) { this.dapConfig.siteplatform = siteplatform; }); +Given("DAP is configured with parallel GA4 property {string}", function (parallel_id) { + this.dapConfig.pga4 = parallel_id; +}); + +Given("DAP is configured to set custom dimensions on the parallel tracker", function () { + this.dapConfig.parallelcd = true; +}); + +Given("DAP is configured with cookie timeout of {int} months", function (months) { + this.dapConfig.cto = months; +}); + +Given("DAP is configured with custom search parameter {string}", function (param) { + this.dapConfig.sp = param; +}); + +Given("DAP is configured with YouTube tracking enabled", function () { + this.dapConfig.yt = true; +}); + Given("DAP is configured with autotracking enabled", function () { this.dapConfig.autotracker = true; }); @@ -32,5 +52,13 @@ Given("the page URL has query parameter {string} set to {string}", function (key }); When("I load the test site", async function () { - await this.page.goto(`http://localhost:8080?${this.dapConfig.toQueryParams()}&${new URLSearchParams(this.pageParams).toString()}`); + const params = new URLSearchParams(this.pageParams).toString(); + const url = `http://dap-test-site.local?${this.dapConfig.toQueryParams()}${params ? '&' + params : ''}`; + await this.page.goto(url); +}); + +When("I load the test page {string}", async function (path) { + const params = new URLSearchParams(this.pageParams).toString(); + const url = `http://dap-test-site.local/${path}?${this.dapConfig.toQueryParams()}${params ? '&' + params : ''}`; + await this.page.goto(url); }); diff --git a/features/support/step_definitions/youtube_steps.js b/features/support/step_definitions/youtube_steps.js new file mode 100644 index 0000000..71edde2 --- /dev/null +++ b/features/support/step_definitions/youtube_steps.js @@ -0,0 +1,81 @@ +import { Given, When } from "@cucumber/cucumber"; + +When("I wait for the YouTube iframe API to load", async function () { + await this.page.waitForFunction(() => typeof YT !== 'undefined' && YT.loaded); +}); + +When("I play YouTube video {string}", async function (videoId) { + await this.page.evaluate((id) => { + const iframe = document.querySelector('iframe[id="' + id + '"]'); + const origin = new URL(iframe.src).origin; + iframe.contentWindow.postMessage(JSON.stringify({ event: 'command', func: 'playVideo', args: [] }), origin); + }, videoId); +}); + +const mockYouTubeScript = ` + window._ytPlayers = {}; + + function MockPlayer(videoId, opts) { + this._videoId = videoId; + this._duration = 200; + this._currentTime = 0; + this._handlers = opts.events; + window._ytPlayers[videoId] = this; + if (opts.events.onReady) opts.events.onReady({ target: this }); + } + MockPlayer.prototype.getDuration = function() { return this._duration; }; + MockPlayer.prototype.getCurrentTime = function() { return this._currentTime; }; + MockPlayer.prototype.seekTo = function(seconds) { this._currentTime = seconds; }; + MockPlayer.prototype.getVideoData = function() { return { video_id: this._videoId, title: 'Test Video' }; }; + MockPlayer.prototype.getVideoUrl = function() { return 'https://www.youtube.com/watch?v=' + this._videoId; }; + + window.YT = { + loaded: true, + PlayerState: { ENDED: 0, PLAYING: 1, PAUSED: 2, BUFFERING: 3, CUED: 5 }, + Player: MockPlayer + }; + setTimeout(function() { + if (window.onYouTubeIframeAPIReady) window.onYouTubeIframeAPIReady(); + }, 200); +`; + +Given("I mock the YouTube IFrame API", async function () { + await this.page.setRequestInterception(true); + this.page.on('request', (request) => { + if (request.url().includes('youtube.com/iframe_api')) { + request.respond({ status: 200, contentType: 'application/javascript', body: mockYouTubeScript }); + } else { + request.continue(); + } + }); +}); + +When("I simulate YouTube video {string} playing", async function (videoId) { + await this.page.evaluate((id) => { + const player = window._ytPlayers[id]; + player._handlers.onStateChange({ target: player, data: window.YT.PlayerState.PLAYING }); + }, videoId); +}); + +When("I simulate YouTube video {string} pausing", async function (videoId) { + await this.page.evaluate((id) => { + const player = window._ytPlayers[id]; + player._handlers.onStateChange({ target: player, data: window.YT.PlayerState.PAUSED }); + }, videoId); +}); + + +When("I set the YouTube video {string} current time to {int}%", async function (videoId, percent) { + await this.page.evaluate((id, pct) => { + const player = window._ytPlayers[id]; + player.seekTo((pct / 100) * player.getDuration()); + }, videoId, percent); +}); + +When("I simulate YouTube video {string} completing", async function (videoId) { + await this.page.evaluate((id) => { + const player = window._ytPlayers[id]; + player._handlers.onStateChange({ target: player, data: window.YT.PlayerState.ENDED }); + }, videoId); +}); + diff --git a/features/youtube_tracking.feature b/features/youtube_tracking.feature new file mode 100644 index 0000000..84565a1 --- /dev/null +++ b/features/youtube_tracking.feature @@ -0,0 +1,101 @@ +# DAP's GA4 property is configured to disable most enhanced measurement events. +# The DAP code takes over responsibility for tracking these events, rather than relying on gtag's implementation. +# This test feature verifies DAP's alternative implementation of video engagement tracking. +Feature: DAP tracks YouTube video interactions + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + And DAP is configured with YouTube tracking enabled + And I mock the YouTube IFrame API + + Scenario: Starting a video sends a video_start event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + Then a "video_start" event is sent to DAP with parameters + | video_provider | youtube | + | video_id | 7b-fcRQ2Q7k | + | video_title | Test Video | + | video_percent | 0 | + | video_url | https://www.youtube.com/watch?v=7b-fcRQ2Q7k | + | video_duration | 200 | + | video_current_time | 0 | + + Scenario: Pausing a video sends a video_pause event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 20% + And I simulate YouTube video "7b-fcRQ2Q7k" pausing + Then a "video_pause" event is sent to DAP with parameters + | video_provider | youtube | + | video_id | 7b-fcRQ2Q7k | + | video_title | Test Video | + | video_percent | 20 | + | video_url | https://www.youtube.com/watch?v=7b-fcRQ2Q7k | + | video_duration | 200 | + | video_current_time | 40 | + + Scenario: Resuming a paused video sends a video_play event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 25% + And I simulate YouTube video "7b-fcRQ2Q7k" pausing + And I simulate YouTube video "7b-fcRQ2Q7k" playing + Then a "video_play" event is sent to DAP with parameters + | video_provider | youtube | + | video_id | 7b-fcRQ2Q7k | + | video_title | Test Video | + | video_percent | 25 | + | video_url | https://www.youtube.com/watch?v=7b-fcRQ2Q7k | + | video_duration | 200 | + | video_current_time | 50 | + + Scenario: Video progress is reported at milestone percentages + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 30% + And I wait 2 seconds + Then a "video_progress" event is sent to DAP with parameters + | video_provider | youtube | + | video_id | 7b-fcRQ2Q7k | + | video_title | Test Video | + | video_percent | 25 | + | video_url | https://www.youtube.com/watch?v=7b-fcRQ2Q7k | + | video_duration | 200 | + | video_current_time | 50 | + + Scenario: Completing a video sends a video_complete event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 100% + And I simulate YouTube video "7b-fcRQ2Q7k" completing + Then a "video_complete" event is sent to DAP with parameters + | video_provider | youtube | + | video_id | 7b-fcRQ2Q7k | + | video_title | Test Video | + | video_percent | 100 | + | video_url | https://www.youtube.com/watch?v=7b-fcRQ2Q7k | + | video_duration | 200 | + | video_current_time | 200 | + + Scenario: Replaying a completed video sends a new video_start event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 100% + And I simulate YouTube video "7b-fcRQ2Q7k" completing + And I set the YouTube video "7b-fcRQ2Q7k" current time to 0% + And I simulate YouTube video "7b-fcRQ2Q7k" playing + Then there are 2 "video_start" events sent to DAP + + Scenario: Playing a second video on the same page sends a second video_start event + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I simulate YouTube video "7b-fcRQ2Q7k" playing + And I simulate YouTube video "LcvYjkCBY28" playing + Then there are 2 "video_start" events sent to DAP \ No newline at end of file diff --git a/features/youtube_tracking_integration.feature b/features/youtube_tracking_integration.feature new file mode 100644 index 0000000..74de417 --- /dev/null +++ b/features/youtube_tracking_integration.feature @@ -0,0 +1,19 @@ +# DAP's YouTube tracker works by registering event listener callbacks for each of the page's YouTube videos using the YouTube iframe API. +# youtube_tracking.feature unit tests the registered callbacks to check that DAP responds correctly to various events. +# In contrast, this integration test confirms that the YouTube iframe API has actually registered DAP's callbacks +Feature: DAP successfully integrates with the YouTube iframe API + + Background: + Given I load an empty browser + And DAP is configured for agency "GSA" + And DAP is configured with YouTube tracking enabled + + # video_start would be a better test but onError is the only YouTube IFrame API event to trigger reliably in the CI environment + # In any case, this test confirms that DAP has successfully registered its callbacks with the YouTube iframe API + Scenario: Attempting to play a private YouTube video sends a video_error event to DAP + When I load the test page "youtube.html" + And I wait for the YouTube iframe API to load + And I play YouTube video "zt4t5kOHBig" + And I wait 3 seconds + Then a "video_error" event is sent to DAP with parameters + | videotitle | | diff --git a/package-lock.json b/package-lock.json index 32df0a5..f9a5c36 100644 --- a/package-lock.json +++ b/package-lock.json @@ -13,7 +13,7 @@ }, "devDependencies": { "@cucumber/cucumber": "^11.3.0", - "chai": "^5.1.1", + "chai": "^6.2.2", "eslint": "^9.8.0", "eslint-plugin-compat": "^6.0.0", "eslint-plugin-jsdoc": "^50.2.2", @@ -840,15 +840,6 @@ "dev": true, "license": "Python-2.0" }, - "node_modules/assertion-error": { - "version": "2.0.1", - "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", - "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==", - "dev": true, - "engines": { - "node": ">=12" - } - }, "node_modules/assertion-error-formatter": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/assertion-error-formatter/-/assertion-error-formatter-3.0.0.tgz", @@ -1140,19 +1131,13 @@ } }, "node_modules/chai": { - "version": "5.1.1", - "resolved": "https://registry.npmjs.org/chai/-/chai-5.1.1.tgz", - "integrity": "sha512-pT1ZgP8rPNqUgieVaEY+ryQr6Q4HXNg8Ei9UnLUrjN4IA7dvQC5JB+/kxVcPNDHyBcc/26CXPkbNzq3qwrOEKA==", + "version": "6.2.2", + "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz", + "integrity": "sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==", "dev": true, - "dependencies": { - "assertion-error": "^2.0.1", - "check-error": "^2.1.1", - "deep-eql": "^5.0.1", - "loupe": "^3.1.0", - "pathval": "^2.0.0" - }, + "license": "MIT", "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/chalk": { @@ -1172,15 +1157,6 @@ "url": "https://github.com/chalk/chalk?sponsor=1" } }, - "node_modules/check-error": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/check-error/-/check-error-2.1.1.tgz", - "integrity": "sha512-OAlb+T7V4Op9OwdkjmguYRqncdlx5JiofwOAUkmTF+jNdHwzTaTs4sRAGpzLF3oOz5xAyDGrPgeIDFQmDOTiJw==", - "dev": true, - "engines": { - "node": ">= 16" - } - }, "node_modules/chromium-bidi": { "version": "0.6.5", "resolved": "https://registry.npmjs.org/chromium-bidi/-/chromium-bidi-0.6.5.tgz", @@ -1361,15 +1337,6 @@ } } }, - "node_modules/deep-eql": { - "version": "5.0.2", - "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-5.0.2.tgz", - "integrity": "sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==", - "dev": true, - "engines": { - "node": ">=6" - } - }, "node_modules/deep-is": { "version": "0.1.4", "resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz", @@ -1944,15 +1911,6 @@ "node": "6.* || 8.* || >= 10.*" } }, - "node_modules/get-func-name": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/get-func-name/-/get-func-name-2.0.2.tgz", - "integrity": "sha512-8vXOvuE167CtIc3OyItco7N/dpRtBbYOsPsXCz7X/PMnlGjYjSGuZJgM1Y7mmew7BKf9BqvLX2tnOVy1BBUsxQ==", - "dev": true, - "engines": { - "node": "*" - } - }, "node_modules/get-stream": { "version": "5.2.0", "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz", @@ -2502,15 +2460,6 @@ "dev": true, "license": "MIT" }, - "node_modules/loupe": { - "version": "3.1.1", - "resolved": "https://registry.npmjs.org/loupe/-/loupe-3.1.1.tgz", - "integrity": "sha512-edNu/8D5MKVfGVFRhFf8aAxiTM6Wumfz5XsaatSxlD3w4R1d/WEKUTydCdPGbl9K7QG/Ca3GnDV2sIKIpXRQcw==", - "dev": true, - "dependencies": { - "get-func-name": "^2.0.1" - } - }, "node_modules/lower-case": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/lower-case/-/lower-case-2.0.2.tgz", @@ -2855,15 +2804,6 @@ "url": "https://github.com/sponsors/isaacs" } }, - "node_modules/pathval": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/pathval/-/pathval-2.0.0.tgz", - "integrity": "sha512-vE7JKRyES09KiunauX7nd2Q9/L7lhok4smP9RZTDeD4MVs72Dp2qNFVz39Nz5a0FVEW0BJR6C0DYrq6unoziZA==", - "dev": true, - "engines": { - "node": ">= 14.16" - } - }, "node_modules/pend": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz", diff --git a/package.json b/package.json index c3e2f53..4abfe1d 100644 --- a/package.json +++ b/package.json @@ -25,7 +25,7 @@ "homepage": "https://github.com/digital-analytics-program/gov-wide-code#readme", "devDependencies": { "@cucumber/cucumber": "^11.3.0", - "chai": "^5.1.1", + "chai": "^6.2.2", "eslint": "^9.8.0", "eslint-plugin-compat": "^6.0.0", "eslint-plugin-jsdoc": "^50.2.2", diff --git a/test_site/404.html b/test_site/404.html new file mode 100644 index 0000000..d523943 --- /dev/null +++ b/test_site/404.html @@ -0,0 +1,13 @@ + + + + + + 404 + + + +

404

+

The page you requested could not be found.

+ + diff --git a/test_site/agency-tracking.html b/test_site/agency-tracking.html new file mode 100644 index 0000000..3eea33c --- /dev/null +++ b/test_site/agency-tracking.html @@ -0,0 +1,34 @@ + + + + + + + Agency tracking test page + + + + + + + + + + + + + + +

Agency tracking test page

+ + diff --git a/test_site/index.html b/test_site/index.html index a167469..33df71d 100644 --- a/test_site/index.html +++ b/test_site/index.html @@ -2,6 +2,7 @@ + DAP test site @@ -25,10 +26,10 @@

Banner Click

An official website of the United States government

-

Here’s how you know

+

Here's how you know

@@ -136,6 +137,7 @@

(Same Domain & SubDomain) - Relative Path

/policy/about/
/poicy/about
/policy/about
+ /404.html

(Same Domain & SubDomain) - Absolute Path

https://www.dap-test-site.app.cloud.gov/about.php
@@ -159,7 +161,7 @@

(Same Domain & Different SubDomain - Level 2) - Absolute Path

External Links

(Different Domain) - Absolute Path

- http://www.gsa.gov/travelpolicy
+ http://www.gsa.gov/travelpolicy
http://www.gsa.gov/travelpolicy/home.aspx?id=12#top
http://www.domain.com/about.php
http://www.domain.com/about/ S1
diff --git a/test_site/not-found.html b/test_site/not-found.html new file mode 100644 index 0000000..f212d01 --- /dev/null +++ b/test_site/not-found.html @@ -0,0 +1,13 @@ + + + + + + Page Not Found + + + +

Page Not Found

+

The page you requested could not be found.

+ + diff --git a/test_site/youtube.html b/test_site/youtube.html index 5886dab..3995d01 100644 --- a/test_site/youtube.html +++ b/test_site/youtube.html @@ -2,6 +2,7 @@ + DAP test site - YouTube tracking @@ -14,11 +15,11 @@ src="https://www.youtube-nocookie.com/embed/7b-fcRQ2Q7k" frameborder="0" width="500" allowfullscreen="" alt="MedWatch Tips & Tools (January 2016)">

- +
From 117e46be83d8c7ff4eab953c86c8f02b3ae0ba3f Mon Sep 17 00:00:00 2001 From: Shelley Nason Date: Wed, 22 Apr 2026 13:57:35 -0400 Subject: [PATCH 2/2] Improve the testing README --- features/README.md | 117 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 100 insertions(+), 17 deletions(-) diff --git a/features/README.md b/features/README.md index f52698b..a57f705 100644 --- a/features/README.md +++ b/features/README.md @@ -3,16 +3,74 @@ The automated tests for the DAP code are implemented using [cucumber-js](https://github.com/cucumber/cucumber-js) and [puppeteer](https://pptr.dev/). -By default the tests use the local version of the DAP javascript files. Loading -the code into a test HTML page, performing various user actions, and testing -the behavior of the code in response to the user actions. +```mermaid +--- +title: Testing architecture +--- + +flowchart LR + Runner("`Test Runner
*Cucumber.js*`"):::local + Puppeteer("`Browser Automation
*Puppeteer*`"):::local + Browser(Chromium):::local + Site("`Test Website
*nginx in Docker*`"):::local + config_startup@{ shape: diamond, label: "Configurable at
website startup" } +subgraph code [DAP code] +DAP_Local(DAP code - local):::local +DAP_Staging(DAP code - staging):::remote +DAP_Prod(DAP code - production):::remote +end +GA4(DAP GA4 test property):::remote +subgraph Legend +legend_local(Running locally):::local +legend_remote(Running remotely):::remote +end + +classDef local fill:#f96 +classDef remote fill:#f9f + +Runner -->|drives| Puppeteer -->|controls| Browser -->|loads| Site +config_startup --> DAP_Local +config_startup --> DAP_Staging +config_startup --> DAP_Prod +Site-.->|sends events to| GA4 +Site -->|loads one of|config_startup +``` + +## Running the test site +The test site is a simple nginx server running in a Docker container. It serves static HTML pages that include the DAP code, allowing us to test the DAP code's behavior in a realistic environment. + +To start up the test site, run one of the `test-site-*` commands: + +```bash +npm run test-site-dev +``` +This particular command will start up the test site at http://localhost:8080/. All pages will be configured to use your local version of the DAP code by including the script tag: + +```angular2html + +``` + +You can also choose to run the test site configured to load a deployed version of the DAP code, either the staging or production version, by running `npm run test-site-stg` or `npm run test-site-prd`, respectively. + +## Configuring DAP in the test site +The DAP code is configurable at load time via query parameters. For instance, to enable DAP's YouTube tracking, a website +can load the DAP code with the `yt` query parameter set to `true`. + +```angular2html + +``` -Use the `DAP_ENV` environment variable to insert live versions of the code into -the test HTML page instead of the local version as described below. +The full list of configurable options is available in the [DAP Code Capabilities Summary](https://github.com/digital-analytics-program/gov-wide-code/wiki/DAP-Code-Capabilities-Summary). + +The test site is designed to pass through query parameters to the DAP code, allowing you to test different DAP configurations by changing the URL. +For instance, opening http://localhost:8080/youtube.html?yt=true&search=nasa will load the page for testing DAP's YouTube tracking with YouTube tracking enbabled. +Any query parameters that don't match one of DAP's configuration options will be treated as a normal query parameter (e.g. `search=nasa`). + +This capability is used extensively in the automated tests to check the DAP code's behavior in various configurations. ## Running the tests -Start up the test site at http://localhost:8080/: +Start up the test site using any of the `test-site-*` commands: ```bash npm run test-site-dev @@ -38,24 +96,49 @@ npm run cucumber:report Test report should be available in `output/test-results.html`. -## Configuring with environment variables +### Running the tests in verbose mode -### Verbose mode - -Print debugging information to stdout while running the tests: +Verbose mode logs events that occur within the browser during test execution. If you find yourself wishing that you could +watch what's happening in browser devtools during test execution, try verbose mode: ```bash VERBOSE=true npm run cucumber ``` -### Run tests against the live staging environment +## Testing approach +Here's a high-level overview of how browser activity is turned into analytics events by DAP and GA4. Google owns everything in this diagram except for the "DAP code" box: -```bash -DAP_ENV=staging npm run cucumber -``` +```mermaid +--- +title: DAP/GA4 processing +--- -### Run tests against the live production environment +flowchart LR +subgraph browser [Inside browser] +dap("`DAP code
*listens for browser events*`") +gtag("`gtag code
*listens for browser events*`") +end +collect("`Google endpoint
*https://www.google-analytics.com/g/collect*`") +data@{ shape: cyl, label: "3. GA4 event store" } -```bash -DAP_ENV=production npm run cucumber +dap-->|1. API call|gtag -->|2. POST|collect-->|insert events|data ``` +The basic testing strategy is to insert a test probe at one or more of the numbered points in the diagram and then confirm what the probe +sees once we generate some browser events. Which probe(s) should we use? +1. API call: gtag offers an official, [documented](https://developers.google.com/tag-platform/gtagjs/reference) Javascript API. As such, +we can run tests against a mocked version of the API and, assuming we correctly understand how the API works, have some confidence that passing tests ensure correct behavior. +These tests will be fast and fairly reliable (Puppeteer will always introduce some flakiness). +2. POST: Google Analytics does not treat the `collect` endpoint as a public API. It is not officially documented, both in terms of the structure of the requests to it and +in terms of what triggers those requests. It is true that many developer plugins/extensions have been built to interpret these requests but it would +be risky to build our test suite on top of reverse-engineered knowledge of an unofficial API. Tests would need a plugin of this type that was officially released by Google. +3. GA4 event store: For e2e tests, we could use the Google Analytics Data API to query for GA4 events that we expected to be received during the test run. + +The current test suite only uses probe 1. Since the gtag API documentation is actually not very thorough, we should probably add some +integration-type tests that use probes 2 and/or 3 to confirm that our understanding of the API is correct and that the events we expect to be generated are actually being received by Google Analytics. + + + + + + +