Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM nginx
ARG DAP_ENV='dev'
ENV DAP_ENV=${DAP_ENV}
COPY test_site Universal-Federated-Analytics-Min.js Federated.js.map /usr/share/nginx/html/
COPY test_site Universal-Federated-Analytics-Min.js Universal-Federated-Analytics.js Federated.js.map /usr/share/nginx/html/
COPY nginx-test.conf.template /etc/nginx/conf.d/
RUN envsubst '${DAP_ENV}' < /etc/nginx/conf.d/nginx-test.conf.template > /etc/nginx/conf.d/default.conf
21 changes: 21 additions & 0 deletions features/404_tracking.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Feature: DAP modifies page_location for 404 pages

Background:
Given I load an empty browser
And DAP is configured for agency "GSA"

Scenario: Page title containing "404" sets page_location to a vpv404 path
When I load the test page "404.html"
Then DAP will set custom dimensions for the DAP property
| page_location | http://dap-test-site.local/vpv404/404.html |

Scenario: Page title containing "not found" sets page_location to a vpv404 path
When I load the test page "not-found.html"
Then DAP will set custom dimensions for the DAP property
| page_location | http://dap-test-site.local/vpv404/not-found.html |

Scenario: 404 page with a referrer appends the referrer to page_location
When I load the test site
And I click on link with href "/404.html" and wait for new page to load
Then DAP will set custom dimensions for the DAP property
| page_location | http://dap-test-site.local/vpv404/404.html/http://dap-test-site.local/ |
117 changes: 100 additions & 17 deletions features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,74 @@
The automated tests for the DAP code are implemented using [cucumber-js](https://github.com/cucumber/cucumber-js)
and [puppeteer](https://pptr.dev/).

By default the tests use the local version of the DAP javascript files. Loading
the code into a test HTML page, performing various user actions, and testing
the behavior of the code in response to the user actions.
```mermaid
---
title: Testing architecture
---

flowchart LR
Runner("`Test Runner<br>*Cucumber.js*`"):::local
Puppeteer("`Browser Automation<br>*Puppeteer*`"):::local
Browser(Chromium):::local
Site("`Test Website<br>*nginx in Docker*`"):::local
config_startup@{ shape: diamond, label: "Configurable at<br>website startup" }
subgraph code [DAP code]
DAP_Local(DAP code - local):::local
DAP_Staging(DAP code - staging):::remote
DAP_Prod(DAP code - production):::remote
end
GA4(DAP GA4 test property):::remote
subgraph Legend
legend_local(Running locally):::local
legend_remote(Running remotely):::remote
end

classDef local fill:#f96
classDef remote fill:#f9f

Runner -->|drives| Puppeteer -->|controls| Browser -->|loads| Site
config_startup --> DAP_Local
config_startup --> DAP_Staging
config_startup --> DAP_Prod
Site-.->|sends events to| GA4
Site -->|loads one of|config_startup
```

## Running the test site
The test site is a simple nginx server running in a Docker container. It serves static HTML pages that include the DAP code, allowing us to test the DAP code's behavior in a realistic environment.

To start up the test site, run one of the `test-site-*` commands:

```bash
npm run test-site-dev
```
This particular command will start up the test site at http://localhost:8080/. All pages will be configured to use your local version of the DAP code by including the script tag:

```angular2html
<script async id="_fed_an_ua_tag" src="Universal-Federated-Analytics-Min.js"></script>
```

You can also choose to run the test site configured to load a deployed version of the DAP code, either the staging or production version, by running `npm run test-site-stg` or `npm run test-site-prd`, respectively.

## Configuring DAP in the test site
The DAP code is configurable at load time via query parameters. For instance, to enable DAP's YouTube tracking, a website
can load the DAP code with the `yt` query parameter set to `true`.

```angular2html
<script async type="text/javascript" id="_fed_an_ua_tag" src="https://dap.digitalgov.gov/Universal-Federated-Analytics-Min.js?agency=HHS&yt=true"></script>
```

Use the `DAP_ENV` environment variable to insert live versions of the code into
the test HTML page instead of the local version as described below.
The full list of configurable options is available in the [DAP Code Capabilities Summary](https://github.com/digital-analytics-program/gov-wide-code/wiki/DAP-Code-Capabilities-Summary).

The test site is designed to pass through query parameters to the DAP code, allowing you to test different DAP configurations by changing the URL.
For instance, opening http://localhost:8080/youtube.html?yt=true&search=nasa will load the page for testing DAP's YouTube tracking with YouTube tracking enbabled.
Any query parameters that don't match one of DAP's configuration options will be treated as a normal query parameter (e.g. `search=nasa`).

This capability is used extensively in the automated tests to check the DAP code's behavior in various configurations.

## Running the tests

Start up the test site at http://localhost:8080/:
Start up the test site using any of the `test-site-*` commands:

```bash
npm run test-site-dev
Expand All @@ -38,24 +96,49 @@ npm run cucumber:report

Test report should be available in `output/test-results.html`.

## Configuring with environment variables
### Running the tests in verbose mode

### Verbose mode

Print debugging information to stdout while running the tests:
Verbose mode logs events that occur within the browser during test execution. If you find yourself wishing that you could
watch what's happening in browser devtools during test execution, try verbose mode:

```bash
VERBOSE=true npm run cucumber
```

### Run tests against the live staging environment
## Testing approach
Here's a high-level overview of how browser activity is turned into analytics events by DAP and GA4. Google owns everything in this diagram except for the "DAP code" box:

```bash
DAP_ENV=staging npm run cucumber
```
```mermaid
---
title: DAP/GA4 processing
---

### Run tests against the live production environment
flowchart LR
subgraph browser [Inside browser]
dap("`DAP code<br>*listens for browser events*`")
gtag("`gtag code<br>*listens for browser events*`")
end
collect("`Google endpoint<br>*https://www.google-analytics.com/g/collect*`")
data@{ shape: cyl, label: "3. GA4 event store" }

```bash
DAP_ENV=production npm run cucumber
dap-->|1. API call|gtag -->|2. POST|collect-->|insert events|data
```
The basic testing strategy is to insert a test probe at one or more of the numbered points in the diagram and then confirm what the probe
sees once we generate some browser events. Which probe(s) should we use?
1. API call: gtag offers an official, [documented](https://developers.google.com/tag-platform/gtagjs/reference) Javascript API. As such,
we can run tests against a mocked version of the API and, assuming we correctly understand how the API works, have some confidence that passing tests ensure correct behavior.
These tests will be fast and fairly reliable (Puppeteer will always introduce some flakiness).
2. POST: Google Analytics does not treat the `collect` endpoint as a public API. It is not officially documented, both in terms of the structure of the requests to it and
in terms of what triggers those requests. It is true that many developer plugins/extensions have been built to interpret these requests but it would
be risky to build our test suite on top of reverse-engineered knowledge of an unofficial API. Tests would need a plugin of this type that was officially released by Google.
3. GA4 event store: For e2e tests, we could use the Google Analytics Data API to query for GA4 events that we expected to be received during the test run.

The current test suite only uses probe 1. Since the gtag API documentation is actually not very thorough, we should probably add some
integration-type tests that use probes 2 and/or 3 to confirm that our understanding of the API is correct and that the events we expect to be generated are actually being received by Google Analytics.







85 changes: 85 additions & 0 deletions features/autotracker.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# DAP's GA4 property is configured to disable most enhanced measurement events.
# The DAP code takes over responsibility for tracking these events, rather than relying on gtag's implementation.
# This test feature verifies DAP's alternative implementation of outbound click and file download tracking.
Feature: Downloads and outbound link clicks are tracked when autotracking is enabled

Background:
Given I load an empty browser
And DAP is configured for agency "GSA"

Scenario: Clicking an email link fires email_click when autotracking is enabled
Given DAP is configured with autotracking enabled
When I load the test site
And I click on element with selector "a[href='mailto:test@domain.com']"
Then a "email_click" event is sent to DAP with parameters
| link_text | mailto:[REDACTED_EMAIL] |
| link_domain | domain.com |
| outbound | true |
| interaction_type | Mouse Click |

Scenario: Clicking an email link does not fire email_click when autotracking is disabled
Given DAP is configured with autotracking disabled
When I load the test site
And I click on element with selector "a[href='mailto:test@domain.com']"
Then no "email_click" event is sent to DAP

Scenario: Clicking a telephone link fires telephone_click when autotracking is enabled
Given DAP is configured with autotracking enabled
When I load the test site
And I click on element with selector "a[href='tel:+1437-925-1855']"
Then a "telephone_click" event is sent to DAP with parameters
| link_text | Telephone [REDACTED_TEL] |
| interaction_type | Mouse Click |

Scenario: Clicking a telephone link does not fire telephone_click when autotracking is disabled
Given DAP is configured with autotracking disabled
When I load the test site
And I click on element with selector "a[href='tel:+1437-925-1855']"
Then no "telephone_click" event is sent to DAP

Scenario: Clicking an outbound link sends a click event when autotracking is enabled
Given DAP is configured with autotracking enabled
When I load the test site
And I click on element with selector "a[href='http://www.gsa.gov/travelpolicy']"
Then a "click" event is sent to DAP with parameters
| link_url | http://gsa.gov/travelpolicy |
| link_domain | gsa.gov |
| link_text | http://gsa.gov/travelpolicy |
| outbound | true |
| interaction_type | Mouse Click |

Scenario: Clicking an outbound link does not fire event when autotracking is disabled
Given DAP is configured with autotracking disabled
When I load the test site
And I click on element with selector "a[href='http://www.gsa.gov/travelpolicy']"
Then no "click" event is sent to DAP

Scenario: Clicking a file download link reports the download when autotracking is enabled
When I load the test site
And I click on a file to download it
Then a "file_download" event is sent to DAP with parameters
| file_name | /about.zip |
| file_extension | zip |
| link_text | /about.zip |
| link_id | internalDownload |
| link_url | http://dap-test-site.local/about.zip |
| link_domain | dap-test-site.local |
| interaction_type | Mouse Click |

Scenario: Pressing Enter on a file download link reports the download when autotracking is enabled
When I load the test site
And I highlight and press Enter on a file to download it
Then a "file_download" event is sent to DAP with parameters
| file_name | /about.zip |
| file_extension | zip |
| link_text | /about.zip |
| link_id | internalDownload |
| link_url | http://dap-test-site.local/about.zip |
| link_domain | dap-test-site.local |
| interaction_type | Enter Key Keystroke |

Scenario: File downloads are not tracked when autotracking is disabled
Given DAP is configured with autotracking disabled
When I load the test site
And I click on a file to download it
Then no "file_download" event is sent to DAP
23 changes: 0 additions & 23 deletions features/autotracker_download.feature

This file was deleted.

12 changes: 12 additions & 0 deletions features/banner_tracker.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Feature: DAP tracks clicks on any 'Official website of the US government' banner

Background:
Given I load an empty browser
And DAP is configured for agency "GSA"

Scenario: Clicking the USA banner button fires official_usa_site_banner_click
When I load the test site
And I click on element with selector "#banner-button"
Then a "official_usa_site_banner_click" event is sent to DAP with parameters
| link_text | Here's how you know |
| section | header |
6 changes: 2 additions & 4 deletions features/basic_page_load.feature
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,10 @@ Feature: Test the outgoing requests sent by a basic page with DAP code loaded
Scenario: Loading the page with the DAP code without further action
When I load the test site
And I wait 5 seconds
Then there is a GA4 request
But there are no unexpected requests
Then there are no unexpected requests

Scenario: Loading the page with the DAP code and clicking a button
When I load the test site
And I click on element with selector "#banner-button"
And I wait 5 seconds
Then there is a GA4 request
But there are no unexpected requests
Then there are no unexpected requests
19 changes: 18 additions & 1 deletion features/configuration.feature
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,21 @@ Feature: A site can load the DAP code with varying levels of customization
Then DAP will set custom dimensions for the DAP property
| agency | GSA |
| site_topic | analytics |
| site_platform | cloud.gov |
| site_platform | cloud.gov |

Scenario: Load a DAP-enabled page and check the built-in customer dimensions
Given DAP is configured for agency "GSA"
When I load the test site
Then DAP will set custom dimensions for the DAP property
| protocol | http: |
| hostname_dimension | dap-test-site.local |
| using_parallel_tracker | no |
And DAP will set the "script_source" dimension to a string matching "https?:\/\/.*\/universal-federated-analytics-min.js"
And DAP will set the "version" dimension to a string matching "\d{8} v\d+\.\d+ - ga4"

Scenario: Load a DAP-enabled page with a custom cookie timeout
Given DAP is configured for agency "GSA"
And DAP is configured with cookie timeout of 1 months
When I load the test site
Then DAP will set custom dimensions for the DAP property
| cookie_expires | 2628000 |
Loading
Loading