This repo combines several automated testing patterns and frameworks in Python: Behavior-Driven Development with natural language tests written in Gherkin, Data-Driven Development, RobotFramework, and RPA Framework.
Visit the app/features/ folder and examine the .feature files, then visit the steps/ subdirectory and examine the corresponding .py files.
A good demo test for this repo is app/features/templates/wikipedia_search.feature.j2, which is the template used to generate app/features/wikipedia_search.feature. This test grabs a list of strings from app/test_data/wikipedia_searches.xlsx and searches for all of them on Wikipedia.
To generate tests from jinja templates, run template.py
To run tests, run behave.
To run a specific test scenario, run behave -n "<scenario name>"
The project includes a Nix flake for reproducible development environment and testing.
nix develop # Set up environment
nix run .#test # Run tests# Install dependencies
pip install -r requirements.txt
# Run tests (runner.sh handles template generation)
cd app && ./runner.sh- Page object models collect functions + locators for interacting with common web pages through Selenium
- Page class files are located in subdirs of app/page_data/ as .py files
- Page object locator files are located in subdirs of app/page_data/ as .yaml files.
- Page object images are located in subdirs of app/page_data/ as .png files
Page object images are stored for use with OCR/image recognition (provided by RobotFramework).
- Tests (.feature files) are located in app/features
- Some tests are generated from jinja templates stored in app/features/templates
- These are tables with values to use in test scenarios
- Located in subdirs of app/test_data/ as .yaml or .xlsx files
- These are Python files with functions corresponding to Gherkin steps found in .feature tests
- Located in the app/features/steps
I made a container for running tests in, so that I wasn't constrained by my development environment.
Example of using podman:
podman build --tag qa -f ./Containerfile
podman run -d -p 4444:4444 -p 7900:7900 -p 5999:5999 localhost/qa:latest
On MacOS you can install podman with brew.
You can also use Podman Desktop for a GUI.
brew install --cask podman-desktop
On M1/M2 macs, run podman build with: --arch=amd64 to prevent segfaults.
Currently using Sphinx.
Example:
$ behave -f sphinx.steps -o source/
$ cd docs
$ make html
One could also use: https://pypi.org/project/cucutags/
- Read this short tutorial on using behave for behavioral-driven testing: https://behave.readthedocs.io/en/stable/tutorial.html
- Selenium is wrapped by RobotFramework which simplifies many aspects of using Selenium
- RobotFramework is accessed via the rpaframework Python library, documented here: https://rpaframework.org/
- Understand how data driven testing works, example here: https://www.guru99.com/data-driven-testing.html
# Install dependencies manually
pip install -r requirements.txt
# For using playwright:
rfbrowser init
# For selenium
npm install chromedriver geckodriver
- Use good grammar
- Use present tense
- Use third person
- Use subject-predicate phrases
- Aim for a simple, terse flow
I am exploring mixing behavior-driven testing with data-driven testing.
In data driven testing it is common to seperate the test data from test automation logic. This added modularity allows for rapid development of tests. It is common to store test variables / parameters inside of excel, and then run tests based on all possible values of the parameters defined in the excel sheet.
Two additional concepts I am implementing:
- test data can be defined in yaml instead of excel
- web page data such as selenium element locators / images can be defined in an excel or yaml file
This makes maintaining selenium tests easier because it collects all data about web pages together in a singular place, rather than it being scattered throughout various test scripts. It allows you to specify multiple ways for the automation to locate page elements. It allows for interacting with pages following DRY, and for fixing many broken tests by correcting an element locator just once.
I use both Behave and RobotFramework in this project, which means you can write tests as either .robot or Gherkin (.feature) files. Both of these syntaxes are closer to pure English, which allows management/customers/non-coders to participate in writing tests and defining expected product behavior. Both of these frameworks also support behavior-driven development. RobotFramework is especially flexible, allowing you to compose tests in multiple styles.