Adding visual regression testing with BackstopJS

.design-system-version

@@ -1 +1 @@
-67.0.5
+70.0.12

.gitignore

@@ -5,4 +5,7 @@
 env
 templates/components
 templates/layout
-node_modules
+node_modules
+backstop_data/bitmaps_test/
+backstop_data/html_report/
+backstop.json

.pre-commit-config.yaml

@@ -1,4 +1,9 @@
 repos:
+  - repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+      - id: isort
+        args: ['--profile=black']
   - repo: https://github.com/ambv/black
     rev: 24.4.2
     hooks:
@@ -15,7 +20,4 @@ repos:
     rev: 7.1.0
     hooks:
       - id: flake8
-  - repo: https://github.com/pycqa/isort
-    rev: 5.13.2
-    hooks:
-      - id: isort
+        args: ['--max-line-length=88']

Makefile

@@ -13,3 +13,15 @@ format-python:
 
 format:
 	npx prettier --write .
+
+generate-backstopjs:
+	python3 -m generate_backstop_config
+
+reference:
+	backstop reference
+
+test:
+	backstop test
+
+approve:
+	backstop approve

README.md

@@ -42,3 +42,19 @@ npm install
 ## Running the Application
 
 For running this application, run `make run` which first executes `scripts/load_release.sh` script that downloads the Design System macros zip file from the github release and unzips them into a templates folder. Then, `flask --app application run ` renders all the example components as displayed in the Design System at `http://127.0.0.1:5000`. The CSS and JS are pulled in at runtime from the CDN.
+
+## Visual Tests
+
+For the visual regression tests BackstopJS is used. To install BackstopJS run:
+
+```
+npm install -g backstopjs
+```
+
+Note-: Make sure to install npm package([link](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)) before before installing BackstopJS.
+
+To utilize BackstopJS, the workflow is as follows:
+
+`make generate-backstopjs`: This python file generates `backstop.json` file which has the config for BackstopJS
+
+`make test`: This runs the tests which creates a set of screenshots, compares them with reference screenshots and shows any changes in the visual report (`backstop_data/html_report`). Note-: Make sure to keep the local server running before executing this step.

application.py

@@ -1,13 +1,70 @@
-from flask import Flask, render_template
+import os
+
+import frontmatter
+from flask import Flask, render_template, render_template_string, send_from_directory
 from jinja2 import ChainableUndefined
 
 app = Flask(__name__)
+
+
+def setAttributes(dictionary, attributes):
+    for key in attributes:
+        dictionary[key] = attributes[key]
+    return dictionary
+
+
+app.jinja_env.filters["setAttributes"] = setAttributes
 app.jinja_env.undefined = ChainableUndefined
 
 
+@app.route("/<path:filename>")
+def generate_images(filename):
+    return send_from_directory("static/", filename)
+
+
 @app.route("/")
-def hello_world():
-    return render_template("index.html", param="Hello world")
+def index():
+    root_directory = "templates/components"
+    directories = {
+        directory
+        for directory in os.listdir(root_directory)
+        for file in os.listdir(os.path.join(root_directory, directory))
+        if file.startswith("example")
+    }
+    return render_template("index.html", example_files=sorted(directories))
+
+
+@app.route("/components/<component_name>")
+def component(component_name):
+    example_files = [
+        file
+        for file in os.listdir(f"templates/components/{component_name}/")
+        if file.startswith("example")
+    ]
+    return render_template(
+        "component-examples-list.html",
+        example_files=sorted(example_files),
+        component_name=component_name,
+    )
+
+
+@app.route("/components/<component_name>/<filename>")
+def example(component_name, filename):
+    try:
+        with open(f"templates/components/{component_name}/{filename}", "r") as content:
+            content = frontmatter.load(content)
+        if "layout" in content.metadata:
+            template = content.content
+        else:
+            template = (
+                "{% extends 'layout/_template.njk' %}"
+                + "{% block body %}<div class='ons-u-p-m'>"
+                + content.content
+                + "</div>{% endblock %}"
+            )
+        return render_template_string(template)
+    except FileNotFoundError:
+        return "File not found"
 
 
 if __name__ == "__main__":