diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml index 8e75f096..0eb87705 100644 --- a/.github/workflows/run-tests.yml +++ b/.github/workflows/run-tests.yml @@ -1,7 +1,7 @@ # This workflow installs required Python dependencies and then runs the available tests. # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python -name: Run Acceptance and Unit tests +name: Run Acceptance and Unit tests (with and without visualisation dependencies) on: pull_request: @@ -21,10 +21,18 @@ jobs: uses: actions/setup-python@v3 with: python-version: "3.10" # Only the oldest supported Python version is included here - - name: Install dependencies - run: | - python -m pip install --upgrade pip # upgrade pip to latest version - pip install . # install pyproject.toml dependencies - excludes optional dependencies (such as visualisation) - - name: Run tests + + - name: Install dependencies (without extra visualisation dependencies) run: | - python run_tests.py + python -m pip install --upgrade pip # upgrade pip to latest version + pip install . # no additional [visualisation] dependencies + + - name: Run tests (without visualisation dependencies) + run: python run_tests.py + + - name: Install extra visualisation dependencies + run: pip install ".[visualisation]" # extra [visualisation] dependencies in pyproject.toml + + - name: Run Tests (with visualisation dependencies) + run: python run_tests.py + diff --git a/README.md b/README.md index 6c3e9309..89594176 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ The recommended installation method is using [pip](http://pip-installer.org) After installation include `robotmbt` as library in your robot file to get access to the new functionality. To run your test suite model-based, use the __Treat this test suite model-based__ keyword as suite setup. Check the _How to model_ section to learn how to make your scenarios suitable for running model-based. -``` +```robotframework *** Settings *** Library robotmbt Suite Setup Treat this test suite model-based @@ -50,7 +50,8 @@ Modelling can be done directly from [Robot framework](https://robotframework.org Consider these two scenarios: -``` +```robotframework +*** Test Cases *** Buying a postcard When you buy a new postcard then you have a blank postcard @@ -63,7 +64,8 @@ Preparing for a birthday party Mapping the dependencies between scenarios is done by annotating the steps with modelling info. Modelling info is added to the documentation of the step as shown below. Regular documentation can still be added, as long as `*model info*` starts on a new line and a white line is included after the last `:OUT:` expression. -``` +```robotframework +*** Keywords *** you buy a new postcard [Documentation] *model info* ... :IN: None @@ -117,7 +119,8 @@ All example scenarios naturally contain data. This information is embedded in th #### Step argument modifiers -``` +```robotframework +*** Test Cases *** Personalising a birthday card Given there is a birthday card when Johan writes their name on the birthday card @@ -126,7 +129,8 @@ Personalising a birthday card The above scenario uses the name `Johan` to create a concrete example. But now suppose that from a testing perspective `Johan` and `Frederique` are part of the same equivalence class. Then the step `Frederique writes their name on the birthday card` would yield an equally valid scenario. This can be achieved by adding a modifier (`:MOD:`) to the model info of the step. The format of a modifier is a Robot argument to which you assign a list of options. The modifier updates the argument value to a randomly chosen value from the specified options. -``` +```robotframework +*** Keywords *** ${person} writes their name on the birthday card [Documentation] *model info* ... :MOD: ${person}= [Johan, Frederique] @@ -138,7 +142,8 @@ ${person} writes their name on the birthday card When constructing examples, they often express relations between multiple actors, where each actor can appear in multiple steps. This makes it important to know how modifiers behave when there are multiple modifiers in a scenario. -``` +```robotframework +*** Test Cases *** Addressing a birthday card Given Tannaz is having their birthday and Johan has a birthday card @@ -148,7 +153,8 @@ Addressing a birthday card Have a look at the when-step above. We will assume the model already contains a domain term with two properties: `birthday.celebrant = Tannaz` and `birthday.guests = [Johan, Frederique]`. -``` +```robotframework +*** Keywords *** ${sender} writes the address of ${receiver} on the birthday card [Documentation] *model info* ... :MOD: ${sender}= birthday.guests @@ -175,10 +181,12 @@ It is not possible to add new options to an existing example value. Any constrai It is possible for a step to keep the same options. The special `.*` notation lets you keep the available options as-is. Preceding steps must then supply the possible options. Some steps can, or must, deal with multiple independent sets of options that must not be mixed, because the expected results should differ. Suppose you have a set of valid and invalid passwords. You might be reluctant to include the superset of these as options to an authentication step. Instead, you can use `:MOD: ${password}= .*` as the modifier for that step. Like in the when-step for this scenario: -``` -Given 'secret' is too weak a password -When user tries to update their password to 'secret' -then the password is rejected +```robotframework +*** Test Cases *** +Reject password + Given 'secret' is too weak a password + When user tries to update their password to 'secret' + then the password is rejected ``` In a then-step, modifiers behave slightly different. In then-steps no new option constraints are accepted for an argument. Its value must already have been determined during the given- and when-steps. In other words, regardless of the actual modifier, the expression behaves as if it were `.*`. The exception to this is when a then-step signals the first use of a new example value. In that case the argument value from the original scenario text is used. @@ -193,12 +201,47 @@ For now, variable data considers strict equivalence classes only. This means tha By default, trace generation is random. The random seed used for the trace is logged by _Treat this test suite model-based_. This seed can be used to rerun the same trace, if no external random factors influence the test run. To activate the seed, pass it as argument: -``` +```robotframework Treat this test suite model-based seed=eag-etou-cxi-leamv-jsi ``` Using `seed=new` will force generation of a new reusable seed and is identical to omitting the seed argument. To completely bypass seed generation and use the system's random source, use `seed=None`. This has even more variation but does not produce a reusable seed. +### Graphs + +A graph can be included in the log file to visualise how scenarios are linked. This helps in understanding a test suite's structure and reveals alternative paths that did not make it into the final trace. + +To enable graph generation, some extra dependencies must be installed: `pip install robotframework-mbt[visualisation]` + +Generate the graph by setting the graph style for the model-based suite. The graph will be included in the Robot log file as part of the keyword's logging. + +```robotframework +Treat this test suite Model-based graph=scenario +``` + +Available graph styles: + +* scenario + * Compact view: Each scenario is shown as one node. +* scenario-delta-value + * Expanded view: Scenarios can become multiple nodes if they affect system state in different ways. + +#### Exporting and importing graph data + +Graph data can be stored by setting an output directory using argument `export_graph_data`. This createss a json file, named after the test suite, in the selected folder. Any accessable path can be used. For your convenience, Robot Framework offers an [automatic variable](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#automatic-variables) `${OUTPUT_DIR}` that points to this run's output directory. The `export_graph_data` argument can be used independent of the `graph` argument. + +```robotframework +Treat this test suite Model-based export_graph_data=${OUTPUT_DIR} +``` + +To recreate a graph from previously exported graph data, use: + +```robotframework +Show model graph from exported file json_file_path= graph_style=scenario +``` + +This will draw a graph from the exported file, without the need to rerun the test suite. It is possible to select a different graph style than was used during the test run. If no graph style is selected, then the scenario graph style is used. + ### Option management If you want to set configuration options for use in multiple test suites without having to repeat them, the keywords __Set model-based options__ and __Update model-based options__ can be used to configure RobotMBT library options. _Set_ takes the provided options and discards any previously set options. _Update_ allows you to modify existing options or add new ones. Reset all options by calling _Set_ without arguments. Direct options provided to __Treat this test suite model-based__ take precedence over library options and affect only the current test suite. diff --git a/atest/robotMBT tests/07__processor_options/random_seeds/02__reusing_seed_reproduces_trace.robot b/atest/robotMBT tests/07__processor_options/random_seeds/02__reusing_seed_reproduces_trace.robot index 28e5e180..d0e2b737 100644 --- a/atest/robotMBT tests/07__processor_options/random_seeds/02__reusing_seed_reproduces_trace.robot +++ b/atest/robotMBT tests/07__processor_options/random_seeds/02__reusing_seed_reproduces_trace.robot @@ -5,7 +5,7 @@ Documentation This suite uses the `seed` argument to reproduce a previously ... indicated by the number string. Suite Setup Run keywords Set suite variable ${trace} ${empty} ... AND Treat this test suite Model-based seed=aqmou-eelcuu-sniu-ugsyek-jyhoor -Suite Teardown Should be equal ${trace} 6930142758 +Suite Teardown Should be equal ${trace} 6879523014 Library robotmbt *** Test Cases *** diff --git a/atest/robotMBT tests/07__processor_options/random_seeds/03__retrace_with_refinement.robot b/atest/robotMBT tests/07__processor_options/random_seeds/03__retrace_with_refinement.robot index fc39b74a..5461c2d8 100644 --- a/atest/robotMBT tests/07__processor_options/random_seeds/03__retrace_with_refinement.robot +++ b/atest/robotMBT tests/07__processor_options/random_seeds/03__retrace_with_refinement.robot @@ -9,7 +9,7 @@ Documentation This suite uses the `seed` argument to reproduce a previously ... traces possible of varying length. Suite Setup Run keywords Set suite variable ${trace} ${empty} ... AND Treat this test suite Model-based seed=xou-pumj-ihj-oibiyc-surer -Suite Teardown Should be equal ${trace} B1A5BY3B4BX6B2 +Suite Teardown Should be equal ${trace} B3AY2A5A6B4BX1 Library robotmbt *** Test Cases *** diff --git a/atest/robotMBT tests/07__processor_options/random_seeds/04__retrace_with_step_modifiers.robot b/atest/robotMBT tests/07__processor_options/random_seeds/04__retrace_with_step_modifiers.robot index 3ffdfe43..d3a4a52c 100644 --- a/atest/robotMBT tests/07__processor_options/random_seeds/04__retrace_with_step_modifiers.robot +++ b/atest/robotMBT tests/07__processor_options/random_seeds/04__retrace_with_step_modifiers.robot @@ -6,7 +6,7 @@ Documentation This suite uses the `seed` argument to reproduce a previously ... included, the modifers will number the scenarios at random. The reproduced trace ... must match for both the scenario order and the data order. Suite Setup Treat this test suite Model-based seed=iulr-vih-esycu-eyl-yfa -Suite Teardown Should be equal ${trace} H6G3E5I4D9F8J2B1A7C0 +Suite Teardown Should be equal ${trace} H2G3C1E7I5B9F0A4D6J8 Library robotmbt *** Test Cases *** diff --git a/atest/robotMBT tests/07__processor_options/random_seeds/05__retrace_combined.robot b/atest/robotMBT tests/07__processor_options/random_seeds/05__retrace_combined.robot index cfbca3b2..5b23e5f9 100644 --- a/atest/robotMBT tests/07__processor_options/random_seeds/05__retrace_combined.robot +++ b/atest/robotMBT tests/07__processor_options/random_seeds/05__retrace_combined.robot @@ -15,8 +15,8 @@ Documentation This suite uses the `seed` argument to reproduce a previously ... a data choice by using step modifiers. The lower level includes a path choice. ... Both low-level scenarios are equally valid, the only difference is that the data ... choice is included either once or twice. -Suite Setup Treat this test suite Model-based seed=gujuqt-iakm-oexo-xnu-huba -Suite Teardown Should be equal ${trace} ATQQPRSPRPXY +Suite Setup Treat this test suite Model-based seed=kece-zwu-eihho-yli-rbixx +Suite Teardown Should be equal ${trace} APPSRTTPPXY Library robotmbt *** Test Cases *** diff --git a/atest/robotMBT tests/10__visualisation/01__scenario_graph/__init__.robot b/atest/robotMBT tests/10__visualisation/01__scenario_graph/__init__.robot new file mode 100644 index 00000000..3b161c11 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/01__scenario_graph/__init__.robot @@ -0,0 +1,3 @@ +*** Settings *** +Suite Setup Clear prior exports ${OUTPUT_DIR}${/}run_model_with_graph.json +Library ../graph_checker.py diff --git a/atest/robotMBT tests/10__visualisation/01__scenario_graph/run_model_with_graph.robot b/atest/robotMBT tests/10__visualisation/01__scenario_graph/run_model_with_graph.robot new file mode 100644 index 00000000..10c438c4 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/01__scenario_graph/run_model_with_graph.robot @@ -0,0 +1,29 @@ +*** Settings *** +Documentation This suite runs model-based with the graphing option enabled. The checks for the contents +... of the graph are delegated to the second robot file in the same folder. Export/import +... functionality is used to gain access to the graph contents. +Suite Setup Treat this test suite Model-based graph=scenario export_graph_data=${OUTPUT_DIR} +Resource ../../../resources/birthday_cards_data_variation.resource +Library robotmbt + +*** Test Cases *** +Buying a card + Given Jonathan is having their birthday + and Douwe is a friend of Jonathan + and Diogo is a friend of Jonathan + and Tycho is a friend of Jonathan + and Thomas is a friend of Jonathan + When Douwe buys a birthday card + then there is a blank birthday card available + +Someone writes their name on the card + Given there is a birthday card + and Diogo's name is not yet on the birthday card + when Diogo writes their name on the birthday card + then the birthday card has 'Diogo' written on it + +At least 3 people can write their name on the card + Given the birthday card has 2 different names written on it + and Tycho's name is not yet on the birthday card + when Tycho writes their name on the birthday card + then the birthday card has 3 different names written on it diff --git a/atest/robotMBT tests/10__visualisation/01__scenario_graph/verify_model_graph.robot b/atest/robotMBT tests/10__visualisation/01__scenario_graph/verify_model_graph.robot new file mode 100644 index 00000000..dca6d27b --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/01__scenario_graph/verify_model_graph.robot @@ -0,0 +1,33 @@ +*** Settings *** +Documentation This suite takes the graph generated in the first suite in this folder +... and checks the content's properties. +Library ../graph_checker.py +Library Collections +Suite Setup Import graph data from ${OUTPUT_DIR}${/}run_model_with_graph.json + +*** Test Cases *** +Scenarios are nodes in the graph + VAR @{expected_nodes} start + ... Buying a card + ... Someone writes their name on the card + ... At least 3 people can write their name on the card + ${node_count}= Number of graph nodes + Should be equal ${node_count} ${4} + @{all_nodes}= List of node titles + Lists should be equal ${all_nodes} ${expected_nodes} ignore_order=True + +Dependent nodes are connected by directed edges + @{successors}= All successors to node Buying a card + VAR @{next_node} Someone writes their name on the card + Should be equal ${successors} ${next_node} + @{successors}= All successors to node Someone writes their name on the card + Should not contain ${successors} Buying a card + +The dependency order for nodes is top-down + ${first_pos}= Vertical position of node Buying a card + ${second_pos}= Vertical position of node Someone writes their name on the card + Should be true ${second_pos} < ${first_pos} + +Repeating scenarios have a self-looping edge + @{successors}= All successors to node Someone writes their name on the card + Should contain ${successors} Someone writes their name on the card diff --git a/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/__init__.robot b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/__init__.robot new file mode 100644 index 00000000..3b161c11 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/__init__.robot @@ -0,0 +1,3 @@ +*** Settings *** +Suite Setup Clear prior exports ${OUTPUT_DIR}${/}run_model_with_graph.json +Library ../graph_checker.py diff --git a/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/run_model_with_graph.robot b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/run_model_with_graph.robot new file mode 100644 index 00000000..9cb0b3eb --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/run_model_with_graph.robot @@ -0,0 +1,29 @@ +*** Settings *** +Documentation This suite runs model-based with the graphing option enabled. The checks for the contents +... of the graph are delegated to the second robot file in the same folder. Export/import +... functionality is used to gain access to the graph contents. +Suite Setup Treat this test suite Model-based graph=scenario-delta-value export_graph_data=${OUTPUT_DIR} +Resource ../../../resources/birthday_cards_data_variation.resource +Library robotmbt + +*** Test Cases *** +Buying a card + Given Jonathan is having their birthday + and Douwe is a friend of Jonathan + and Diogo is a friend of Jonathan + and Tycho is a friend of Jonathan + and Thomas is a friend of Jonathan + When Douwe buys a birthday card + then there is a blank birthday card available + +Someone writes their name on the card + Given there is a birthday card + and Diogo's name is not yet on the birthday card + when Diogo writes their name on the birthday card + then the birthday card has 'Diogo' written on it + +At least 3 people can write their name on the card + Given the birthday card has 2 different names written on it + and Tycho's name is not yet on the birthday card + when Tycho writes their name on the birthday card + then the birthday card has 3 different names written on it diff --git a/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/verify_model_graph.robot b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/verify_model_graph.robot new file mode 100644 index 00000000..fcb351b8 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/02__scenario-delta-value_graph/verify_model_graph.robot @@ -0,0 +1,36 @@ +*** Settings *** +Documentation This suite takes the graph generated in the first suite in this folder +... and checks how the content's properties for the scenario-delta-value +... graph are different compared to the scenario graph. +Library ../graph_checker.py +Library Collections +Suite Setup Import graph data from ${OUTPUT_DIR}${/}run_model_with_graph.json graph_type=scenario-delta-value + +*** Test Cases *** +Repeated scenario with different states become separate nodes + [Documentation] In the standard scenario graph, the repeated scenario would show up + ... as a self-looping edge. In the scenario-delta-value variant, also + ... state changes are taken into account, meaning that the two variants + ... will each get their own node. + ${node_count}= Number of graph nodes + Should be true ${node_count} > ${4} + @{all_nodes}= List of node titles + Should contain ${all_nodes} Someone writes their name on the card + Should contain ${all_nodes} Someone writes their name on the card (rep 2) + +Full node text contains state info + [Documentation] Next to the scenario name, the node text for scenario-delta-value graphs + ... also contians information about the model state. 'Buying a card' + ... initialises the modeland contains all properties in their initial state. + ... 'host' and 'names' are properties being tracked by the model. + ${full_text}= Full node text of node Buying a card + Should contain ${full_text} host + Should contain ${full_text} names + +Full node text only contains changed information + [Documentation] When someone writes their name on the card, then the host or celebrant + ... does not change. The node for this scenario should only show the state + ... that changed during this scenario. + ${full_text}= Full node text of node Someone writes their name on the card (rep 2) + Should contain ${full_text} names + Should not contain ${full_text} host diff --git a/atest/robotMBT tests/10__visualisation/03__verify_importing_graphs.robot b/atest/robotMBT tests/10__visualisation/03__verify_importing_graphs.robot new file mode 100644 index 00000000..3750bfdd --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/03__verify_importing_graphs.robot @@ -0,0 +1,39 @@ +*** Settings *** +Documentation This suite takes the graph generated earlier in the visualisation suite +... and uses it to check import functionality. +Library robotmbt +Library graph_checker.py +Library Collections + +*** Variables *** +${prior_export} ${OUTPUT_DIR}${/}run_model_with_graph.json + +*** Test Cases *** +Import as any graph type + [Documentation] The data that is stored during export is always the compelte data set. This + ... has the advantage that different graph types can be reconstructed from the + ... data without rerunning the model. + Import graph data from ${prior_export} graph_type=scenario + Show model graph from exported file ${prior_export} graph_style=scenario + ${node_count}= Number of graph nodes + Should be equal ${node_count} ${4} + Import graph data from ${prior_export} graph_type=scenario-delta-value + Show model graph from exported file ${prior_export} graph_style=scenario-delta-value + ${node_count}= Number of graph nodes + Should be true ${node_count} > ${4} + +Future major version bump imports are rejected + ${future_file}= Modify export file with future major version number ${prior_export} + Run keyword and expect error *incompatible RobotMBT version + ... Import graph data from ${future_file} + +Future minor verion bump imports are accepted + ${future_file}= Modify export file with future minor version number ${prior_export} + Import graph data from ${future_file} + ${node_count}= Number of graph nodes + Should be equal ${node_count} ${4} + +Ill-formed files are rejected for import + ${corrupted_file}= Corrupt export file ${prior_export} + Run keyword and expect error *could not be loaded as RobotMBT graph data + ... Import graph data from ${corrupted_file} diff --git a/atest/robotMBT tests/10__visualisation/__init__.robot b/atest/robotMBT tests/10__visualisation/__init__.robot new file mode 100644 index 00000000..ac5d9fd4 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/__init__.robot @@ -0,0 +1,8 @@ +*** Settings *** +Suite Setup Skip if optional visualisation is not installed +Library graph_checker.py + +*** Keywords *** +Skip if optional visualisation is not installed + ${partial_installation}= Graphing dependencies missing + Skip If ${partial_installation} Visualisation dependencies not installed. Please read the README for information on how to do this. diff --git a/atest/robotMBT tests/10__visualisation/graph_checker.py b/atest/robotMBT tests/10__visualisation/graph_checker.py new file mode 100644 index 00000000..cb5c3160 --- /dev/null +++ b/atest/robotMBT tests/10__visualisation/graph_checker.py @@ -0,0 +1,110 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +import os +from typing import Literal +from robot.api.deco import library, keyword +try: + from robotmbt.visualise.visualiser import Visualiser + from robotmbt.visualise.networkvisualiser import NetworkVisualiser +except ImportError: + Visualiser = None + + +@library(scope='SUITE', auto_keywords=True) +class GraphChecker: + def graphing_dependencies_missing(self) -> bool: + return Visualiser is None + + def clear_prior_exports(self, file_path): + if os.path.exists(file_path): + os.remove(file_path) + if os.path.exists(file_path): + raise OSError(f"Removing previous export file '{file_path}' failed") + + def import_graph_data_from(self, file_path: str, graph_type: str = 'scenario'): + self.visualiser = Visualiser() + self.visualiser.load_from_file(file_path) + self.graph = self.visualiser._get_graph(graph_type) + self.nxgraph = self.graph.networkx + + @property + def to_node(self) -> dict[str, str]: + """scenario name to node id converter""" + node_dict = {self._label_to_scenario(node['label']): id for id, node in self.nxgraph.nodes().items()} + if len(node_dict) < self.number_of_graph_nodes(): + print("Warning: Repeated scenario names present in graph") + return node_dict + + @property + def from_node(self) -> dict[str, str]: + """node id to scenario name converter""" + return {id: self._label_to_scenario(node['label']) for id, node in self.nxgraph.nodes().items()} + + def number_of_graph_nodes(self) -> int: + return len(self.nxgraph.nodes()) + + def list_of_node_titles(self) -> list[str]: + return list(self.to_node) + + def full_node_text_of_node(self, node_title: str) -> str: + node = self.to_node[node_title] + return self.nxgraph.nodes()[node]['label'] + + def all_successors_to_node(self, node_title: str) -> list[str]: + return [self.from_node[node] for node in self.nxgraph.successors(self.to_node[node_title])] + + def vertical_position_of_node(self, node_title: str) -> int: + nxvisual = NetworkVisualiser(self.graph, 'dummy') + return nxvisual.node_dict[self.to_node[node_title]].y + + @keyword("Modify export file with future ${bump} version number") + def modify_export_file_with_future_version_number(self, bump: Literal['major', 'minor'], file_path: str) -> str: + with open(file_path, "r") as f: + contents = f.read() + new_file = file_path.replace('.json', '_future.json') + new_version = '2.0.0' if bump == 'major' else '1.1.1' + with open(new_file, "w") as f: + f.write(contents.replace('"ROBOTMBT_MODEL_VERSION": "1.0.0"', f'"ROBOTMBT_MODEL_VERSION": "{new_version}"')) + return new_file + + def corrupt_export_file(self, file_path: str) -> str: + with open(file_path, "r") as f: + contents = f.read() + new_file = file_path.replace('.json', '_corrupt.json') + with open(new_file, "w") as f: + f.write(contents) + f.seek(0) + f.write('corrupted-'*20) + return new_file + + @staticmethod + def _label_to_scenario(labeltext: str) -> str: + return labeltext.split("\n\n")[0].replace('\n', ' ') diff --git a/pyproject.toml b/pyproject.toml index 25e5cd49..264b66e2 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta" [project] name = "robotframework-mbt" -version = "0.10.0" +version = "0.11.0" description = "Model-Based Testing in Robot framework with test case generation" readme = "README.md" authors = [{ name = "Johan Foederer", email = "github@famfoe.nl" }] @@ -23,8 +23,11 @@ dependencies = [ ] requires-python = ">=3.10" -[tool.setuptools] -packages = ["robotmbt"] +[tool.setuptools.packages.find] +include = ["robotmbt*"] [project.urls] Homepage = "https://github.com/JFoederer/robotframeworkMBT" + +[project.optional-dependencies] +visualisation = ["networkx", "bokeh", "grandalf", "jsonpickle"] diff --git a/robotmbt/__init__.py b/robotmbt/__init__.py index 15f9b3e0..bcdfcc7b 100644 --- a/robotmbt/__init__.py +++ b/robotmbt/__init__.py @@ -40,4 +40,4 @@ class robotmbt(SuiteReplacer): """ -__version__ = VERSION +__version__: str = VERSION diff --git a/robotmbt/modeller.py b/robotmbt/modeller.py index 6b793557..2f29f191 100644 --- a/robotmbt/modeller.py +++ b/robotmbt/modeller.py @@ -34,12 +34,13 @@ from robot.api import logger from robot.utils import is_list_like +from robot.errors import TimeoutExceeded # Raised by Robot in case of keyword timeout from .modelspace import ModelSpace -from .steparguments import StepArgument, StepArguments +from .steparguments import StepArgument, StepArguments, ArgKind from .substitutionmap import SubstitutionMap from .suitedata import Scenario, Step -from .tracestate import TraceState +from .tracestate import TraceState, TraceSnapShot def try_to_fit_in_scenario(candidate: Scenario, tracestate: TraceState): @@ -80,6 +81,8 @@ def process_scenario(scenario: Scenario, model: ModelSpace) -> tuple[Scenario, S else: return None, None, dict(fail_msg=f"Unable to insert scenario {scenario.src_id}, " f"{scenario.name}, due to step '{step}': [{expr}] is False") + except TimeoutExceeded: + raise except Exception as err: return None, None, dict(fail_msg=f"Unable to insert scenario {scenario.src_id}, " f"{scenario.name}, due to step '{step}': [{expr}] {err}") @@ -129,6 +132,8 @@ def handle_refinement_exit(inserted_refinement: Scenario, tracestate: TraceState try: if tracestate.model.process_expression(expr, refinement_tail.steps[1].args) is False: break + except TimeoutExceeded: + raise except Exception: break else: @@ -172,7 +177,7 @@ def generate_scenario_variant(scenario: Scenario, model: ModelSpace) -> Scenario modded_arg, constraint = _parse_modifier_expression(expr, step.args) if step.args[modded_arg].is_default: continue - if step.args[modded_arg].kind in [StepArgument.EMBEDDED, StepArgument.POSITIONAL, StepArgument.NAMED]: + if step.args[modded_arg].kind in [ArgKind.EMBEDDED, ArgKind.POSITIONAL, ArgKind.NAMED]: org_example = step.args[modded_arg].org_value if step.gherkin_kw == 'then': constraint = None # No new constraints are processed for then-steps @@ -193,7 +198,7 @@ def generate_scenario_variant(scenario: Scenario, model: ModelSpace) -> Scenario else: options = None subs.substitute(org_example, options) - elif step.args[modded_arg].kind == StepArgument.VAR_POS: + elif step.args[modded_arg].kind == ArgKind.VAR_POS: if step.args[modded_arg].value: modded_varargs = model.process_expression(constraint, step.args) if not is_list_like(modded_varargs): @@ -202,7 +207,7 @@ def generate_scenario_variant(scenario: Scenario, model: ModelSpace) -> Scenario # change the number of arguments in the list, making it impossible to decide which values to # match and which to drop and/or duplicate. step.args[modded_arg].value = modded_varargs - elif step.args[modded_arg].kind == StepArgument.FREE_NAMED: + elif step.args[modded_arg].kind == ArgKind.FREE_NAMED: if step.args[modded_arg].value: modded_free_args = model.process_expression(constraint, step.args) if not isinstance(modded_free_args, dict): @@ -211,6 +216,8 @@ def generate_scenario_variant(scenario: Scenario, model: ModelSpace) -> Scenario step.args[modded_arg].value = modded_free_args else: raise AssertionError(f"Unknown argument kind for {modded_arg}") + except TimeoutExceeded: + raise except Exception as err: logger.debug(f"Unable to insert scenario {scenario.src_id}, {scenario.name}, due to modifier\n" f" In step {step}: {err}") @@ -234,12 +241,12 @@ def generate_scenario_variant(scenario: Scenario, model: ModelSpace) -> Scenario if step.args[modded_arg].is_default: continue org_example = step.args[modded_arg].org_value - if step.args[modded_arg].kind in [StepArgument.EMBEDDED, StepArgument.POSITIONAL, StepArgument.NAMED]: + if step.args[modded_arg].kind in [ArgKind.EMBEDDED, ArgKind.POSITIONAL, ArgKind.NAMED]: step.args[modded_arg].value = subs.solution[org_example] return scenario -def _parse_modifier_expression(expression: str, args: tuple[str]) -> tuple[str, str]: +def _parse_modifier_expression(expression: str, args: StepArguments) -> tuple[str, str]: if expression.startswith('${'): for var in args: if expression.casefold().startswith(var.arg.casefold()): @@ -251,7 +258,7 @@ def _parse_modifier_expression(expression: str, args: tuple[str]) -> tuple[str, raise ValueError(f"Invalid argument substitution: {expression}") -def rewind(tracestate: TraceState, drought_recovery: bool = False) -> Scenario: +def rewind(tracestate: TraceState, drought_recovery: bool = False) -> TraceSnapShot | None: if tracestate[-1].remainder and tracestate.highest_part(tracestate[-1].remainder.src_id) > 1: # When rewinding an 'in between' part, rewind both the part and the refinement tracestate.rewind() diff --git a/robotmbt/modelspace.py b/robotmbt/modelspace.py index 51f9edec..e5cac2a9 100644 --- a/robotmbt/modelspace.py +++ b/robotmbt/modelspace.py @@ -31,6 +31,7 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import copy +from typing import Any from .steparguments import StepArguments @@ -41,11 +42,11 @@ class ModellingError(Exception): class ModelSpace: def __init__(self, reference_id=None): - self.ref_id = str(reference_id) - self.std_attrs = [] - self.props = dict() - self.values = dict() # For using literals without having to use quotes (abc='abc') - self.scenario_vars = [] + self.ref_id: str = str(reference_id) + self.std_attrs: list[str] = [] + self.props: dict[str, RecursiveScope | ModelSpace] = dict() + self.values: dict[str, Any] = dict() # For using literals without having to use quotes (abc='abc') + self.scenario_vars: list[RecursiveScope] = [] self.std_attrs = dir(self) def __repr__(self): @@ -57,7 +58,7 @@ def copy(self): def __eq__(self, other): return self.get_status_text() == other.get_status_text() - def add_prop(self, name): + def add_prop(self, name: str): if name == 'scenario': raise ModellingError(f"scenario is a reserved attribute.") if name in self.props or name in self.values: @@ -65,7 +66,7 @@ def add_prop(self, name): self.props[name] = ModelSpace(name) setattr(self, name, self.props[name]) - def del_prop(self, name): + def del_prop(self, name: str): if name == 'scenario': raise ModellingError(f"scenario is a reserved attribute and cannot be removed.") if name not in self.props: @@ -91,7 +92,7 @@ def end_scenario_scope(self): else: self.props.pop('scenario') - def process_expression(self, expression, step_args=StepArguments()): + def process_expression(self, expression: str, step_args: StepArguments = StepArguments()) -> Any: expr = step_args.fill_in_args(expression.strip(), as_code=True) if self._is_new_vocab_expression(expr): self.add_prop(self._vocab_term(expr)) @@ -131,7 +132,7 @@ def process_expression(self, expression, step_args=StepArguments()): return result - def __handle_attribute_error(self, err): + def __handle_attribute_error(self, err: AttributeError): if isinstance(err.obj, str) and err.obj in self.values: # This situation occurs when using e.g. 'foo.bar' in the model before calling 'new foo'. # The NameError on foo is handled by adding its alias, which results in an AttributeError @@ -139,7 +140,7 @@ def __handle_attribute_error(self, err): raise ModellingError(f"{err.obj} used before definition") raise ModellingError(f"{err.name} used before assignment") - def __add_alias(self, missing_name, step_args): + def __add_alias(self, missing_name: str, step_args: StepArguments): if missing_name == 'scenario': raise ModellingError("Accessing scenario scope while there is no scenario active.\n" "If you intended this to be a literal, please use quotes ('scenario' or \"scenario\").") @@ -152,18 +153,18 @@ def __add_alias(self, missing_name, step_args): self.values[missing_name] = value @staticmethod - def _is_new_vocab_expression(expression): + def _is_new_vocab_expression(expression: str) -> bool: return expression.lower().startswith('new ') and len(expression.split()) == 2 @staticmethod - def _is_del_vocab_expression(expression): + def _is_del_vocab_expression(expression: str) -> bool: return expression.lower().startswith('del ') and len(expression.split()) == 2 @staticmethod - def _vocab_term(expression): + def _vocab_term(expression: str) -> str: return expression.split()[-1] - def get_status_text(self): + def get_status_text(self) -> str: status = str() scenario_attrs = [] for p in self.props: @@ -197,12 +198,12 @@ class RecursiveScope: def __init__(self, outer): super().__setattr__('_outer_scope', outer) - def __getattr__(self, attr): + def __getattr__(self, attr: str): if hasattr(super().__getattribute__('_outer_scope'), attr): return getattr(self._outer_scope, attr) return super().__getattribute__(attr) - def __setattr__(self, attr, value): + def __setattr__(self, attr: str, value): if hasattr(self._outer_scope, attr): setattr(self._outer_scope, attr, value) else: @@ -214,3 +215,17 @@ def __iter__(self): def __bool__(self): return any(True for _ in self) + + def __eq__(self, other): + self_set = set([(attr, getattr(self, attr)) for attr in dir(self._outer_scope) + dir(self) + if not attr.startswith('__') and attr != '_outer_scope']) + other_set = set([(attr, getattr(other, attr)) for attr in dir(other._outer_scope) + dir(other) + if not attr.startswith('__') and attr != '_outer_scope']) + return self_set == other_set + + def __str__(self): + res = "{" + for k, v in self: + res += f"{k}={v}, " + res += "}" + return res diff --git a/robotmbt/steparguments.py b/robotmbt/steparguments.py index 3f9d918c..e9b78daa 100644 --- a/robotmbt/steparguments.py +++ b/robotmbt/steparguments.py @@ -30,7 +30,9 @@ # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +from enum import Enum, auto from keyword import iskeyword +from typing import Any import builtins @@ -38,7 +40,7 @@ class StepArguments(list): def __init__(self, iterable=[]): super().__init__(item.copy() for item in iterable) - def fill_in_args(self, text, as_code=False): + def fill_in_args(self, text: str, as_code: bool = False) -> str: result = text for arg in self: sub = arg.codestring if as_code else str(arg.value) @@ -52,49 +54,51 @@ def __getitem__(self, key): return super()[key] @property - def modified(self): + def modified(self) -> bool: return any([arg.modified for arg in self]) +class ArgKind(Enum): + EMBEDDED = auto() + POSITIONAL = auto() + VAR_POS = auto() + NAMED = auto() + FREE_NAMED = auto() + UNKNOWN = auto() + + class StepArgument: - # kind list - EMBEDDED = 'EMBEDDED' - POSITIONAL = 'POSITIONAL' - VAR_POS = 'VAR_POS' - NAMED = 'NAMED' - FREE_NAMED = 'FREE_NAMED' - - def __init__(self, arg_name, value, kind=None, is_default=False): - self.name = arg_name - self.org_value = value - self.kind = kind # one of the values from the kind list - self._value = None - self._codestr = None + def __init__(self, arg_name: str, value: Any, kind: ArgKind = ArgKind.UNKNOWN, is_default: bool = False): + self.name: str = arg_name + self.org_value: Any = value + self.kind: ArgKind = kind + self._value: Any = None + self._codestr: str | None = None self.value = value # is_default indicates that the argument was not filled in from the scenario. This # argment's value is taken from the keyword's default as provided by Robot. - self.is_default = is_default + self.is_default: bool = is_default @property - def arg(self): + def arg(self) -> str: return "${%s}" % self.name @property - def value(self): + def value(self) -> Any: return self._value @value.setter - def value(self, value): - self._value = value + def value(self, value: Any): + self._value: Any = value self._codestr = self.make_codestring(value) self.is_default = False @property - def modified(self): + def modified(self) -> bool: return self.org_value != self.value @property - def codestring(self): + def codestring(self) -> str | None: return self._codestr def copy(self): @@ -106,18 +110,18 @@ def __str__(self): return f"{self.name}={self.value}" @staticmethod - def make_codestring(text): + def make_codestring(text: Any) -> str: codestr = str(text) if codestr.title() in ['None', 'True', 'False']: return codestr.title() try: float(codestr) - except: + except (TypeError, ValueError, OverflowError, ZeroDivisionError): codestr = StepArgument.make_identifier(codestr) return codestr @staticmethod - def make_identifier(s): + def make_identifier(s: Any) -> str: _s = str(s).replace(' ', '_') if _s.isidentifier(): return f"{_s}_" if iskeyword(_s) or _s in dir(builtins) else _s diff --git a/robotmbt/substitutionmap.py b/robotmbt/substitutionmap.py index 55419702..961bbbc3 100644 --- a/robotmbt/substitutionmap.py +++ b/robotmbt/substitutionmap.py @@ -31,6 +31,7 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import random +from typing import Any class SubstitutionMap: @@ -56,16 +57,16 @@ def copy(self): new.solution = self.solution.copy() return new - def substitute(self, example_value, constraint): + def substitute(self, example_value: str, constraint: list[Any]): self.solution = {} if example_value in self.substitutions: self.substitutions[example_value].add_constraint(constraint) else: self.substitutions[example_value] = Constraint(constraint) - def solve(self): + def solve(self) -> dict[str, str]: self.solution = {} - solution = dict() + solution: dict[str, str] = dict() substitutions = self.copy().substitutions unsolved_subs = list(substitutions) subs_stack = [] @@ -112,16 +113,17 @@ def solve(self): class Constraint: - def __init__(self, constraint): + def __init__(self, constraint: list[Any]): try: # Keep the items in optionset unique. Refrain from using Python sets # due to non-deterministic behaviour when using random seeding. - self.optionset = list(dict.fromkeys(constraint)) - except: - self.optionset = None + self.optionset: list[Any] | None = list(dict.fromkeys(constraint)) + except TypeError: + self.optionset: list[Any] | None = None if not self.optionset or isinstance(constraint, str): raise ValueError(f"Invalid option set for initial constraint: {constraint}") - self.removed_stack = [] + + self.removed_stack: list[str | Placeholder] = [] def __repr__(self): return f'Constraint([{", ".join([str(e) for e in self.optionset])}])' @@ -132,14 +134,14 @@ def __iter__(self): def copy(self): return Constraint(self.optionset) - def add_constraint(self, constraint): + def add_constraint(self, constraint: list[Any] | None): if constraint is None: return self.optionset = [opt for opt in self.optionset if opt in constraint] if not len(self.optionset): raise ValueError('No options left after adding constraint') - def remove_option(self, option): + def remove_option(self, option: str): try: self.optionset.remove(option) self.removed_stack.append(option) diff --git a/robotmbt/suitedata.py b/robotmbt/suitedata.py index 113c6c49..6bbb7d61 100644 --- a/robotmbt/suitedata.py +++ b/robotmbt/suitedata.py @@ -31,27 +31,33 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import copy +from typing import Literal + +from robot.errors import TimeoutExceeded # Raised by Robot in case of keyword timeout +from robot.running.arguments.argumentspec import ArgumentSpec from robot.running.arguments.argumentvalidator import ArgumentValidator +from robot.running.keywordimplementation import KeywordImplementation import robot.utils.notset -from .steparguments import StepArgument, StepArguments +from .steparguments import StepArgument, StepArguments, ArgKind +from .substitutionmap import SubstitutionMap class Suite: - def __init__(self, name, parent=None): - self.name = name - self.filename = '' - self.parent = parent - self.suites = [] - self.scenarios = [] - self.setup = None # Can be a single step or None - self.teardown = None # Can be a single step or None + def __init__(self, name: str, parent=None): + self.name: str = name + self.filename: str = '' + self.parent: Suite | None = parent + self.suites: list[Suite] = [] + self.scenarios: list[Scenario] = [] + self.setup: Step | None = None # Can be a single step or None + self.teardown: Step | None = None # Can be a single step or None @property - def longname(self): + def longname(self) -> str: return f"{self.parent.longname}.{self.name}" if self.parent else self.name - def has_error(self): + def has_error(self) -> bool: return ((self.setup.has_error() if self.setup else False) or any([s.has_error() for s in self.suites]) or any([s.has_error() for s in self.scenarios]) @@ -65,22 +71,22 @@ def steps_with_errors(self): class Scenario: - def __init__(self, name, parent=None): - self.name = name + def __init__(self, name: str, parent: Suite | None = None): + self.name: str = name # Parent scenario is kept for easy searching, processing and referencing # after steps and scenarios have been potentially moved around - self.parent = parent - self.setup = None # Can be a single step or None - self.teardown = None # Can be a single step or None - self.steps = [] - self.src_id = None - self.data_choices = {} + self.parent: Suite | None = parent + self.setup: Step | None = None # Can be a single step or None + self.teardown: Step | None = None # Can be a single step or None + self.steps: list[Step] = [] + self.src_id: int | None = None + self.data_choices: SubstitutionMap = SubstitutionMap() @property - def longname(self): + def longname(self) -> str: return f"{self.parent.longname}.{self.name}" if self.parent else self.name - def has_error(self): + def has_error(self) -> bool: return ((self.setup.has_error() if self.setup else False) or any([s.has_error() for s in self.steps]) or (self.teardown.has_error() if self.teardown else False)) @@ -96,7 +102,7 @@ def copy(self): duplicate.data_choices = self.data_choices.copy() return duplicate - def split_at_step(self, stepindex): + def split_at_step(self, stepindex: int): """Returns 2 partial scenarios. With stepindex 0 the first part has no steps and all steps are in the last part. With @@ -113,29 +119,32 @@ def split_at_step(self, stepindex): class Step: - def __init__(self, steptext, *args, parent, assign=(), prev_gherkin_kw=None): + def __init__(self, steptext: str, *args, parent: Suite | Scenario, assign: tuple[str] = (), + prev_gherkin_kw: Literal['given', 'when', 'then'] | None = None): # org_step is the first keyword cell of the Robot line, including step_kw, # excluding positional args, excluding variable assignment. - self.org_step = steptext + self.org_step: str = steptext # org_pn_args are the positional and named arguments as parsed # from the Robot text ('posA' , 'posB', 'named1=namedA') - self.org_pn_args = args - self.parent = parent # Parent scenario for easy searching and processing. - self.assign = assign # For when a keyword's return value is assigned to a variable. Taken directly from Robot. + self.org_pn_args: tuple[str, ...] = args + self.parent: Suite | Scenario = parent # Parent scenario for easy searching and processing. + # For when a keyword's return value is assigned to a variable. + # Taken directly from Robot. + self.assign: tuple[str] = assign # gherkin_kw is one of 'given', 'when', 'then', or None for non-bdd keywords. self.gherkin_kw = self.step_kw if \ str(self.step_kw).lower() in ['given', 'when', 'then', 'none'] else prev_gherkin_kw - self.signature = None # Robot keyword with its embedded arguments in ${...} notation. - self.args = StepArguments() # embedded arguments list of StepArgument objects. - self.detached = False # Decouples StepArguments from the step text (refinement use case) + self.signature: str | None = None # Robot keyword with its embedded arguments in ${...} notation. + self.args: StepArguments = StepArguments() # embedded arguments list of StepArgument objects. + self.detached: bool = False # Decouples StepArguments from the step text (refinement use case) # model_info contains modelling information as a dictionary. The standard format is # dict(IN=[], OUT=[]) and can optionally contain an error field. - # IN and OUT are lists of Python evaluatable expressions. + # The values of IN and OUT are lists of Python evaluatable expressions. # The `new vocab` form can be used to create new domain objects. # The `vocab.attribute` form can then be used to express relations # between properties from the domain vocabulaire. # Custom processors can define their own attributes. - self.model_info = dict() + self.model_info: dict[str, str | list[str]] = dict() def __str__(self): return self.keyword @@ -152,26 +161,26 @@ def copy(self): cp.model_info = self.model_info.copy() return cp - def has_error(self): + def has_error(self) -> bool: return 'error' in self.model_info - def get_error(self): + def get_error(self) -> str | None: return self.model_info.get('error') @property - def full_keyword(self): + def full_keyword(self) -> str: """The full keyword text, quad space separated, including its arguments and return value assignment""" return " ".join(str(p) for p in (*self.assign, self.keyword, *self.posnom_args_str)) @property - def keyword(self): + def keyword(self) -> str: if not self.signature: return self.org_step s = f"{self.step_kw} {self.signature}" if self.step_kw else self.signature return self.args.fill_in_args(s) @property - def posnom_args_str(self): + def posnom_args_str(self) -> tuple[str, ...]: """A tuple with all arguments in Robot accepted text format ('posA' , 'posB', 'named1=namedA')""" if self.detached or not self.args.modified: return self.org_pn_args @@ -179,14 +188,14 @@ def posnom_args_str(self): for arg in self.args: if arg.is_default: continue - if arg.kind == arg.POSITIONAL: + if arg.kind == ArgKind.POSITIONAL: result.append(arg.value) - elif arg.kind == arg.VAR_POS: + elif arg.kind == ArgKind.VAR_POS: for vararg in arg.value: result.append(vararg) - elif arg.kind == arg.NAMED: + elif arg.kind == ArgKind.NAMED: result.append(f"{arg.name}={arg.value}") - elif arg.kind == arg.FREE_NAMED: + elif arg.kind == ArgKind.FREE_NAMED: for name, value in arg.value.items(): result.append(f"{name}={value}") else: @@ -194,38 +203,42 @@ def posnom_args_str(self): return tuple(result) @property - def gherkin_kw(self): + def gherkin_kw(self) -> Literal['given', 'when', 'then'] | None: return self._gherkin_kw @gherkin_kw.setter - def gherkin_kw(self, value): + def gherkin_kw(self, value: str | None): + """if value is type str, it must be a case insensitive variant of given, when, then""" self._gherkin_kw = value.lower() if value else None @property - def step_kw(self): + def step_kw(self) -> str | None: first_word = self.org_step.split()[0] return first_word if first_word.lower() in ['given', 'when', 'then', 'and', 'but'] else None @property - def kw_wo_gherkin(self): + def kw_wo_gherkin(self) -> str: """The keyword without its Gherkin keyword. I.e., as it is known in Robot framework.""" return self.keyword.replace(self.step_kw, '', 1).strip() if self.step_kw else self.keyword - def add_robot_dependent_data(self, robot_kw): + def add_robot_dependent_data(self, robot_kw: KeywordImplementation): """robot_kw must be Robot Framework's keyword object from Robot's runner context""" try: if robot_kw.error: raise ValueError(robot_kw.error) if robot_kw.embedded: - self.args = StepArguments([StepArgument(*match, kind=StepArgument.EMBEDDED) for match in - zip(robot_kw.embedded.args, robot_kw.embedded.parse_args(self.kw_wo_gherkin))]) + self.args = StepArguments([StepArgument(*match, kind=ArgKind.EMBEDDED) for match in + zip(robot_kw.embedded.args, + robot_kw.embedded.parse_args(self.kw_wo_gherkin))]) self.args += self.__handle_non_embedded_arguments(robot_kw.args) self.signature = robot_kw.name self.model_info = self.__parse_model_info(robot_kw._doc) + except TimeoutExceeded: + raise except Exception as ex: self.model_info['error'] = str(ex) - def __handle_non_embedded_arguments(self, robot_argspec): + def __handle_non_embedded_arguments(self, robot_argspec: ArgumentSpec) -> list[StepArgument]: result = [] p_args = [a for a in self.org_pn_args if '=' not in a or r'\=' in a] n_args = [a.split('=', 1) for a in self.org_pn_args if '=' in a and r'\=' not in a] @@ -236,19 +249,19 @@ def __handle_non_embedded_arguments(self, robot_argspec): for arg in robot_argspec: if not p_args or (arg.kind != arg.POSITIONAL_ONLY and arg.kind != arg.POSITIONAL_OR_NAMED): break - result.append(StepArgument(argument_names.pop(0), p_args.pop(0), kind=StepArgument.POSITIONAL)) + result.append(StepArgument(argument_names.pop(0), p_args.pop(0), kind=ArgKind.POSITIONAL)) robot_args.pop(0) if p_args and robot_args[0].kind == robot_args[0].VAR_POSITIONAL: - result.append(StepArgument(argument_names.pop(0), p_args, kind=StepArgument.VAR_POS)) + result.append(StepArgument(argument_names.pop(0), p_args, kind=ArgKind.VAR_POS)) free = {} for name, value in n_args: if name in argument_names: - result.append(StepArgument(name, value, kind=StepArgument.NAMED)) + result.append(StepArgument(name, value, kind=ArgKind.NAMED)) argument_names.remove(name) else: free[name] = value if free: - result.append(StepArgument(argument_names.pop(-1), free, kind=StepArgument.FREE_NAMED)) + result.append(StepArgument(argument_names.pop(-1), free, kind=ArgKind.FREE_NAMED)) for unmentioned_arg in argument_names: arg = next(arg for arg in robot_args if arg.name == unmentioned_arg) default_value = arg.default @@ -262,7 +275,7 @@ def __handle_non_embedded_arguments(self, robot_argspec): # but use different names in the method signature. Robot Framework implementation is incomplete for this # aspect and differs between library and user keywords. assert False, f"No default argument expected to be needed for '{unmentioned_arg}' here" - result.append(StepArgument(unmentioned_arg, default_value, kind=StepArgument.NAMED, is_default=True)) + result.append(StepArgument(unmentioned_arg, default_value, kind=ArgKind.NAMED, is_default=True)) return result @staticmethod @@ -278,7 +291,7 @@ def __validate_arguments(spec, positionals, nameds): # Use the Robot mechanism for validation to yield familiar error messages ArgumentValidator(spec).validate(p, n) - def __parse_model_info(self, docu): + def __parse_model_info(self, docu: str) -> dict[str, list[str]]: model_info = dict() mi_index = docu.find("*model info*") if mi_index == -1: diff --git a/robotmbt/suiteprocessors.py b/robotmbt/suiteprocessors.py index aa6b7f9d..4680540e 100644 --- a/robotmbt/suiteprocessors.py +++ b/robotmbt/suiteprocessors.py @@ -34,18 +34,25 @@ import random from robot.api import logger +from robot.errors import TimeoutExceeded from . import modeller from .modelspace import ModelSpace -from .suitedata import Suite +from .suitedata import Suite, Scenario from .tracestate import TraceState +try: + from .visualise.visualiser import Visualiser +except ImportError: + Visualiser = None + class SuiteProcessors: - def echo(self, in_suite): + @staticmethod + def echo(in_suite: Suite) -> Suite: return in_suite - def flatten(self, in_suite): + def flatten(self, in_suite: Suite) -> Suite: """ Takes a Suite as input and returns a Suite as output. The output Suite does not have any sub-suites, only scenarios. The scenarios do not have a setup. Any setup @@ -73,40 +80,131 @@ def flatten(self, in_suite): out_suite.suites = [] return out_suite - def process_test_suite(self, in_suite, *, seed='new'): + def process_test_suite(self, in_suite: Suite, *, seed: str | int | bytes | bytearray = 'new', + graph: str = '', export_graph_data: str = '') -> Suite: + visualiser = self._init_visualiser(in_suite.name) if graph or export_graph_data else None self.out_suite = Suite(in_suite.name) self.out_suite.filename = in_suite.filename self.out_suite.parent = in_suite.parent self._fail_on_step_errors(in_suite) self.flat_suite = self.flatten(in_suite) - for id, scenario in enumerate(self.flat_suite.scenarios, start=1): scenario.src_id = id - self.scenarios = self.flat_suite.scenarios[:] + self.scenarios: list[Scenario] = self.flat_suite.scenarios[:] logger.debug("Use these numbers to reference scenarios from traces\n\t" + "\n\t".join([f"{s.src_id}: {s.name}" for s in self.scenarios])) self._init_randomiser(seed) - self.shuffled = [s.src_id for s in self.scenarios] - random.shuffle(self.shuffled) # Keep a single shuffle for all TraceStates (non-essential) + self._prio_order = self._pre_explore_paths()[0] # a short trace without the need for repeating scenarios is preferred - tracestate = self._try_to_reach_full_coverage(allow_duplicate_scenarios=False) - + tracestate = self._try_to_reach_full_coverage(allow_duplicate_scenarios=False, randomise=False, + visualiser=visualiser) if not tracestate.coverage_reached(): logger.debug("Direct trace not available. Allowing repetition of scenarios") - tracestate = self._try_to_reach_full_coverage(allow_duplicate_scenarios=True) - if not tracestate.coverage_reached(): - raise Exception("Unable to compose a consistent suite") + tracestate = self._try_to_reach_full_coverage(allow_duplicate_scenarios=True, randomise=True, + visualiser=visualiser) + if graph: + self._write_visualisation(visualiser, graph) + if export_graph_data: + self._export_graph_data(visualiser, export_graph_data) + if not tracestate.coverage_reached(): + raise Exception("Unable to compose a consistent suite") - self.out_suite.scenarios = tracestate.get_trace() self._report_tracestate_wrapup(tracestate) + self.out_suite.scenarios = tracestate.get_trace() return self.out_suite - def _try_to_reach_full_coverage(self, allow_duplicate_scenarios): - tracestate = TraceState(self.shuffled) + def draw_graph_from_export_file(self, file_path: str, graph_style: str): + if visualiser := self._init_visualiser(): + visualiser.load_from_file(file_path) + logger.info(visualiser.generate_visualisation(graph_style), html=True) + else: + logger.info(f'Visualisation disabled due to initialisation failure.') + + def _pre_explore_paths(self) -> list[list[int]]: + id_list = [s.src_id for s in self.scenarios] + random.shuffle(id_list) # pre-shuffle to prevent scenario 1 from always getting first prio + tracestates = [] + for prio_id in id_list: + # For each available id, give it pole position in the prio order and randomise the rest. + # The randomised bit will give some good variation and the prio_id will always be tried + # first giving more (not all) insight in its dependencies. + scenarios = id_list[:] + scenarios.remove(prio_id) + random.shuffle(scenarios) + scenarios.insert(0, prio_id) + logger.debug(f"Exploring with prio order {scenarios}") + tracestate = TraceState(scenarios) + count = 0 + candidate_id = tracestate.next_candidate(retry=False, randomise=False) + while candidate_id is not None: + count += 1 + candidate = self._select_scenario_variant(candidate_id, tracestate) + if candidate: # No valid variant available in the current state + modeller.try_to_fit_in_scenario(candidate, tracestate) + else: + tracestate.reject_scenario(candidate_id) + candidate_id = tracestate.next_candidate(retry=False) + tracestates.append(tracestate) + logger.debug(f"Result trace: {tracestate.id_trace} " + f"of length {len(tracestate)} after exploring {count} options.") + + # first suggested prio-order: + # - all ids from src_id list of longest trace + # - but all unreached scenarios (over all traces) are moved to the front + # - and all unreached in this trace move into second place + unreached = self._unreached_scenarios(tracestates) + longest = self._longest_trace(tracestates) + first_suggestion = [id for id in longest.c_pool if id in unreached] + first_suggestion += [id for id in longest.c_pool if id not in first_suggestion and longest.count(id) == 0] + first_suggestion += [id for id in longest.c_pool if longest.count(id) > 0] + logger.debug(f"Prio order suggestion: {first_suggestion}") + + # second suggested prio-order: + # - Take the last trace that saw a scenario inserted that had not been reached in any of the prior + # traces. There may be a unique sequence in here to reach that scenario. + # - reordering to insert unreached sceanrio is the same as with the first suggested prio-order + latest_new = self._last_new_coverage(tracestates) + second_suggestion = [id for id in latest_new.c_pool if id in unreached] + second_suggestion += [id for id in latest_new.c_pool + if id not in second_suggestion and latest_new.count(id) == 0] + second_suggestion += [id for id in latest_new.c_pool if latest_new.count(id) > 0] + + suggestions = [first_suggestion] + if first_suggestion != second_suggestion: + logger.debug(f"2nd suggestion: {second_suggestion}") + suggestions.append(second_suggestion) + return suggestions + + @staticmethod + def _longest_trace(tracestate_list: list[TraceState]) -> TraceState: + lengths = [len(ts) for ts in tracestate_list] + return tracestate_list[lengths.index(max(lengths))] + + @staticmethod + def _last_new_coverage(tracestate_list: list[TraceState]) -> TraceState: + ids = [] + last_trace = tracestate_list[0] + for tracestate in tracestate_list: + for id in [int(float(long_id)) for long_id in tracestate.id_trace]: + if id not in ids: + ids.append(id) + last_trace = tracestate + return last_trace + + @staticmethod + def _unreached_scenarios(tracestate_list: list[TraceState]) -> list[int]: + total_coverage = dict().fromkeys(tracestate_list[0].c_pool, 0) + for ts in tracestate_list: + total_coverage = {k: total_coverage[k]+v for k, v in ts.c_pool.items()} + return [id for id in total_coverage if total_coverage[id] == 0] + + def _try_to_reach_full_coverage(self, allow_duplicate_scenarios: bool, randomise: bool = False, + visualiser: Visualiser = None) -> TraceState: + tracestate = TraceState(self._prio_order) while not tracestate.coverage_reached(): - candidate_id = tracestate.next_candidate(retry=allow_duplicate_scenarios) + candidate_id = tracestate.next_candidate(retry=allow_duplicate_scenarios, randomise=randomise) if candidate_id is None: # No more candidates remaining for this level if not tracestate.can_rewind(): break @@ -117,9 +215,11 @@ def _try_to_reach_full_coverage(self, allow_duplicate_scenarios): candidate = self._select_scenario_variant(candidate_id, tracestate) if not candidate: # No valid variant available in the current state tracestate.reject_scenario(candidate_id) + self._update_visualisation(visualiser, tracestate) continue previous_len = len(tracestate) modeller.try_to_fit_in_scenario(candidate, tracestate) + self._update_visualisation(visualiser, tracestate) self._report_tracestate_to_user(tracestate) if len(tracestate) > previous_len: logger.debug(f"last state:\n{tracestate.model.get_status_text()}") @@ -133,22 +233,24 @@ def _try_to_reach_full_coverage(self, allow_duplicate_scenarios): modeller.rewind(tracestate, drought_recovery=True) self._report_tracestate_to_user(tracestate) logger.debug(f"last state:\n{tracestate.model.get_status_text()}") + self._update_visualisation(visualiser, tracestate) + self._update_visualisation(visualiser, tracestate) return tracestate @staticmethod - def __last_candidate_changed_nothing(tracestate): + def __last_candidate_changed_nothing(tracestate: TraceState) -> bool: if len(tracestate) < 2: return False if tracestate[-1].id != tracestate[-2].id: return False return tracestate[-1].model == tracestate[-2].model - def _select_scenario_variant(self, candidate_id, tracestate): + def _select_scenario_variant(self, candidate_id: int, tracestate: TraceState) -> Scenario: candidate = self._scenario_with_repeat_counter(candidate_id, tracestate) candidate = modeller.generate_scenario_variant(candidate, tracestate.model or ModelSpace()) return candidate - def _scenario_with_repeat_counter(self, index, tracestate): + def _scenario_with_repeat_counter(self, index: int, tracestate: TraceState) -> Scenario: """ Fetches the scenario by index and, if this scenario is already used in the trace, adds a repetition counter to its name. @@ -161,8 +263,9 @@ def _scenario_with_repeat_counter(self, index, tracestate): return candidate @staticmethod - def _fail_on_step_errors(suite): + def _fail_on_step_errors(suite: Suite): error_list = suite.steps_with_errors() + if error_list: err_msg = "Steps with errors in their model info found:\n" err_msg += '\n'.join([f"{s.keyword} [{s.model_info['error']}] used in {s.parent.name}" @@ -170,24 +273,24 @@ def _fail_on_step_errors(suite): raise Exception(err_msg) @staticmethod - def _report_tracestate_to_user(tracestate): + def _report_tracestate_to_user(tracestate: TraceState): user_trace = f"[{', '.join(tracestate.id_trace)}]" logger.debug(f"Trace: {user_trace} Reject: {list(tracestate.tried)}") @staticmethod - def _report_tracestate_wrapup(tracestate): + def _report_tracestate_wrapup(tracestate: TraceState): logger.info("Trace composed:") for progression in tracestate: logger.info(progression.scenario.name) logger.debug(f"model\n{progression.model.get_status_text()}\n") @staticmethod - def _init_randomiser(seed): + def _init_randomiser(seed: str | int | bytes | bytearray | None): if isinstance(seed, str): seed = seed.strip() if str(seed).lower() == 'none': logger.info( - f"Using system's random seed for trace generation. This trace cannot be rerun. Use `seed=new` to generate a reusable seed.") + "Using system's random seed for trace generation. This trace cannot be rerun. Use `seed=new` to generate a reusable seed.") elif str(seed).lower() == 'new': new_seed = SuiteProcessors._generate_seed() logger.info(f"seed={new_seed} (use seed to rerun this trace)") @@ -197,7 +300,7 @@ def _init_randomiser(seed): random.seed(seed) @staticmethod - def _generate_seed(): + def _generate_seed() -> str: """Creates a random string of 5 words between 3 and 6 letters long""" vowels = ['a', 'e', 'i', 'o', 'u', 'y'] consonants = ['b', 'c', 'd', 'f', 'g', 'h', 'j', 'k', 'l', @@ -219,3 +322,47 @@ def _generate_seed(): words.append(string) seed = '-'.join(words) return seed + + @staticmethod + def _init_visualiser(name: str = '') -> Visualiser: + global Visualiser + if Visualiser is None: + Visualiser = False + logger.warn(f'Visualisation requested, but required dependencies are not installed. ' + 'Refer to the README on how to install these dependencies. ') + return Visualiser(name) if Visualiser else None + + @staticmethod + def _update_visualisation(visualiser, tracestate: TraceState): + if visualiser: + try: + visualiser.update_trace(tracestate) + except TimeoutExceeded: + raise + except Exception as e: + logger.debug(f'Could not update visualisation due to error!\n{e}') + + @staticmethod + def _write_visualisation(visualiser, graph_style: str): + if visualiser: + try: + text = visualiser.generate_visualisation(graph_style) + logger.info(text, html=True) + except TimeoutExceeded: + raise + except Exception as e: + logger.debug(f'Could not generate visualisation due to error!\n{e}') + else: + logger.info("Graph skipped due to prior failure") + + @staticmethod + def _export_graph_data(visualiser, export_dir): + if visualiser: + try: + file_name = visualiser.export_to_file(export_dir) + logger.info(f"Graph data stored in file: {file_name}") + except TimeoutExceeded: + raise + except Exception as e: + logger.info("Could not export visualisation due to failure.") + logger.debug(f"Export error:\n{e}") diff --git a/robotmbt/suitereplacer.py b/robotmbt/suitereplacer.py index 943b36ca..0c9daacb 100644 --- a/robotmbt/suitereplacer.py +++ b/robotmbt/suitereplacer.py @@ -30,31 +30,34 @@ # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -from .suitedata import Suite, Scenario, Step -from .suiteprocessors import SuiteProcessors +from collections.abc import Callable +from typing import Any + +import robot.model import robot.running.model as rmodel from robot.api import logger -from robot.api.deco import keyword +from robot.api.deco import library, keyword from robot.libraries.BuiltIn import BuiltIn + +from .suitedata import Suite, Scenario, Step +from .suiteprocessors import SuiteProcessors + Robot = BuiltIn() +@library(scope="GLOBAL", listener='SELF') class SuiteReplacer: - ROBOT_LIBRARY_SCOPE = 'GLOBAL' - ROBOT_LISTENER_API_VERSION = 3 - - def __init__(self, processor='process_test_suite', processor_lib=None): - self.ROBOT_LIBRARY_LISTENER = self - self.current_suite = None - self.robot_suite = None - self.processor_lib_name = processor_lib - self.processor_name = processor - self._processor_lib = None - self._processor_method = None - self.processor_options = {} + def __init__(self, processor: str = 'process_test_suite', processor_lib: str | None = None): + self.current_suite: robot.model.TestSuite | None = None + self.robot_suite: robot.model.TestSuite | None = None + self.processor_lib_name: str | None = processor_lib + self.processor_name: str = processor + self._processor_lib: SuiteProcessors | None | object = None + self._processor_method: Callable[..., Suite] | None = None + self.processor_options: dict[str, Any] = {} @property - def processor_lib(self): + def processor_lib(self) -> SuiteProcessors: if self._processor_lib is None: self._processor_lib = SuiteProcessors() if self.processor_lib_name is None \ else Robot.get_library_instance(self.processor_lib_name) @@ -110,7 +113,18 @@ def update_model_based_options(self, **kwargs): """ self.processor_options.update(kwargs) - def __process_robot_suite(self, in_suite, parent): + @keyword("Show model graph from exported file") + def show_graph(self, json_file_path: str, graph_style: str = 'scenario'): + """ + If your previously ran `Treat this test suite Model-based` with the option to export + graph data to file, then this keyword can be used to directly draw a graph from the + exported file, without the need to rerun the test suite. It is possible to select a + different graph style than was used during the test run. If no graph style is selected, + then the scenario graph style is used. + """ + SuiteProcessors().draw_graph_from_export_file(json_file_path, graph_style) + + def __process_robot_suite(self, in_suite: robot.model.TestSuite, parent: Suite | None) -> Suite: out_suite = Suite(in_suite.name, parent) out_suite.filename = in_suite.source @@ -152,11 +166,11 @@ def __process_robot_suite(self, in_suite, parent): out_suite.scenarios.append(scenario) return out_suite - def __clearTestSuite(self, suite): + def __clearTestSuite(self, suite: robot.model.TestSuite): suite.tests.clear() suite.suites.clear() - def __generateRobotSuite(self, suite_model, target_suite): + def __generateRobotSuite(self, suite_model: Suite, target_suite: robot.model.TestSuite): for subsuite in suite_model.suites: new_suite = target_suite.suites.create(name=subsuite.name) new_suite.resource = target_suite.resource @@ -185,9 +199,9 @@ def __generateRobotSuite(self, suite_model, target_suite): else: new_tc.body.create_keyword(name=step.keyword, assign=step.assign, args=step.posnom_args_str) - def _start_suite(self, suite, result): + def _start_suite(self, suite: Suite | None, result): self.current_suite = suite - def _end_suite(self, suite, result): + def _end_suite(self, suite: Suite | None, result): if suite == self.robot_suite: self.robot_suite = None diff --git a/robotmbt/tracestate.py b/robotmbt/tracestate.py index 959db914..cc7dbecd 100644 --- a/robotmbt/tracestate.py +++ b/robotmbt/tracestate.py @@ -30,27 +30,55 @@ # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +import random + +from robotmbt.modelspace import ModelSpace +from robotmbt.suitedata import Scenario + + +class TraceSnapShot: + def __init__(self, id: str, inserted_scenario: Scenario, model_state: ModelSpace, + remainder: Scenario | None = None, drought: int = 0): + self.id: str = id + self.scenario: Scenario = inserted_scenario + self.remainder: Scenario | None = remainder + self._model: ModelSpace = model_state.copy() + self.coverage_drought: int = drought + + def copy(self): + cp = TraceState(self.c_pool.keys()) + cp.c_pool.update(self.c_pool) + cp._tried = [triedlist[:] for triedlist in self._tried] + cp._snapshots = self._snapshots[:] + cp._open_refinements = self._open_refinements[:] + return cp + + @property + def model(self) -> ModelSpace: + return self._model.copy() + + class TraceState: def __init__(self, scenario_indexes: list[int]): - self.c_pool = {index: 0 for index in scenario_indexes} + self.c_pool: dict[int, int] = {index: 0 for index in scenario_indexes} if len(self.c_pool) != len(scenario_indexes): raise ValueError("Scenarios must be uniquely identifiable") - self._tried = [[]] # Keeps track of the scenarios already tried at each step in the trace - self._snapshots = [] # Keeps details for elements in trace - self._open_refinements = [] + self._tried: list[list[int]] = [[]] # Keeps track of the scenarios already tried at each step in the trace + self._snapshots: list[TraceSnapShot] = [] # Keeps details for elements in trace + self._open_refinements: list[int] = [] @property - def model(self): + def model(self) -> ModelSpace | None: """returns the model as it is at the end of the current trace""" return self._snapshots[-1].model if self._snapshots else None @property - def tried(self): + def tried(self) -> tuple[int, ...]: """returns the indices that were rejected or previously inserted at the current position""" return tuple(self._tried[-1]) @property - def coverage_drought(self): + def coverage_drought(self) -> int: """Number of scenarios since last new coverage""" return self._snapshots[-1].coverage_drought if self._snapshots else 0 @@ -62,43 +90,52 @@ def id_trace(self): def active_refinements(self): return self._open_refinements[:] + def copy(self): + cp = TraceState(self.c_pool.keys()) + cp.c_pool.update(self.c_pool) + cp._tried = [triedlist[:] for triedlist in self._tried] + cp._snapshots = self._snapshots[:] + cp._open_refinements = self._open_refinements[:] + return cp + def coverage_reached(self): return all(self.c_pool.values()) - def get_trace(self): + def get_trace(self) -> list[Scenario]: return [snap.scenario for snap in self._snapshots] - def next_candidate(self, retry=False): - for i in self.c_pool: - if i not in self._tried[-1] and not self.is_refinement_active(i) and self.count(i) == 0: - return i - if not retry: + def next_candidate(self, retry: bool = False, randomise: bool = False): + untried_candidates = [i for i in self.c_pool if i not in self._tried[-1] + and not self.is_refinement_active(i)] + uncovered_candidates = [i for i in untried_candidates if self.count(i) == 0] + + if uncovered_candidates: + return random.choice(uncovered_candidates) if randomise else uncovered_candidates[0] + elif not retry or not untried_candidates: return None - for i in self.c_pool: - if i not in self._tried[-1] and not self.is_refinement_active(i): - return i - return None + else: + return random.choice(untried_candidates) if randomise else untried_candidates[0] - def count(self, index): + def count(self, index: int) -> int: """ Count the number of times the index is present in the trace. unfinished partial scenarios are excluded. """ return self.c_pool[index] - def highest_part(self, index): + def highest_part(self, index: int) -> int: """ Given the current trace and an index, returns the highest part number of an ongoing refinement for the related scenario. Returns 0 when there is no refinement active. """ - for i in range(1, len(self.id_trace)+1): + for i in range(1, len(self.id_trace) + 1): if self.id_trace[-i] == f'{index}': return 0 if self.id_trace[-i].startswith(f'{index}.'): return int(self.id_trace[-i].split('.')[1]) return 0 - def is_refinement_active(self, index=None): + def is_refinement_active(self, index: int | None = None) -> bool: """ When called with an index, returns True if that scenario is currently being refined When index is ommitted, return True if any refinement is active @@ -108,21 +145,21 @@ def is_refinement_active(self, index=None): else: return self.highest_part(index) != 0 - def get_remainder(self, index): + def get_remainder(self, index: int) -> Scenario | None: """ When pushing a partial scenario, the remainder can be passed along for safe keeping. This method retrieves the remainder for the last part that was pushed. """ last_part = self.highest_part(index) - index = -self.id_trace[::-1].index(f'{index}.{last_part}')-1 + index = -self.id_trace[::-1].index(f'{index}.{last_part}') - 1 return self._snapshots[index].remainder - def reject_scenario(self, i_scenario): + def reject_scenario(self, i_scenario: int): """Trying a scenario excludes it from further cadidacy on this level""" self._tried[-1].append(i_scenario) - def confirm_full_scenario(self, index, scenario, model): - c_drought = 0 if self.c_pool[index] == 0 else self.coverage_drought+1 + def confirm_full_scenario(self, index: int, scenario: Scenario, model: ModelSpace): + c_drought = 0 if self.c_pool[index] == 0 else self.coverage_drought + 1 self.c_pool[index] += 1 if self.is_refinement_active(index): id = f"{index}.0" @@ -133,9 +170,9 @@ def confirm_full_scenario(self, index, scenario, model): self._tried.append([]) self._snapshots.append(TraceSnapShot(id, scenario, model, drought=c_drought)) - def push_partial_scenario(self, index, scenario, model, remainder=None): + def push_partial_scenario(self, index: int, scenario: Scenario, model: ModelSpace, remainder=None): if self.is_refinement_active(index): - id = f"{index}.{self.highest_part(index)+1}" + id = f"{index}.{self.highest_part(index) + 1}" else: id = f"{index}.1" self._tried[-1].append(index) @@ -143,10 +180,10 @@ def push_partial_scenario(self, index, scenario, model, remainder=None): self._tried.append([]) self._snapshots.append(TraceSnapShot(id, scenario, model, remainder, self.coverage_drought)) - def can_rewind(self): + def can_rewind(self) -> bool: return len(self._snapshots) > 0 - def rewind(self): + def rewind(self) -> TraceSnapShot | None: id = self._snapshots[-1].id index = int(id.split('.')[0]) self._snapshots.pop() @@ -172,16 +209,3 @@ def __getitem__(self, key): def __len__(self): return len(self._snapshots) - - -class TraceSnapShot: - def __init__(self, id, inserted_scenario, model_state, remainder=None, drought=0): - self.id = id - self.scenario = inserted_scenario - self.remainder = remainder - self._model = model_state.copy() - self.coverage_drought = drought - - @property - def model(self): - return self._model.copy() diff --git a/robotmbt/version.py b/robotmbt/version.py index ea301cb7..f0f816c1 100644 --- a/robotmbt/version.py +++ b/robotmbt/version.py @@ -1 +1 @@ -VERSION = '0.10.0' +VERSION: str = '0.11.0' diff --git a/robotmbt/visualise/__init__.py b/robotmbt/visualise/__init__.py new file mode 100644 index 00000000..c7ec90cd --- /dev/null +++ b/robotmbt/visualise/__init__.py @@ -0,0 +1,29 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/robotmbt/visualise/graphs/README.md b/robotmbt/visualise/graphs/README.md new file mode 100644 index 00000000..dd0b1e94 --- /dev/null +++ b/robotmbt/visualise/graphs/README.md @@ -0,0 +1,74 @@ +# How to: Creating new graphs + +Extending the functionality of the visualiser with new graph types can result in better insights into created tests. The visualiser makes use of an abstract graph class that makes it easy to create new graph types. + +To create a new graph type, create an instance of `robotmbt/visualise/graphs/AbstractGraph`, instantiating the abstract methods. Please place the graph under `robotmbt/visualise/graphs/`. + +**NOTE**: when manually altering the `networkx` field, ensure its IDs remain as a serializable and hashable type when the constructor finishes. + +As an example, we show the implementation of the scenario graph below. In this graph type, nodes represent scenarios encountered in exploration, and edges show the flow between these scenarios. +It does not enable tooltips. + +```python +class ScenarioGraph(AbstractGraph[ScenarioInfo, None]): + @staticmethod + def select_node_info(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> ScenarioInfo: + return trace[index][0] + + @staticmethod + def select_edge_info(pair: tuple[ScenarioInfo, StateInfo]) -> None: + return None + + @staticmethod + def create_node_description(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> str: + return '' + + @staticmethod + def create_node_label(info: ScenarioInfo) -> str: + return info.name + + @staticmethod + def create_edge_label(info: None) -> str: + return '' + + @staticmethod + def get_legend_info_final_trace_node() -> str: + return "Scenario (part of trace)" + + @staticmethod + def get_legend_info_other_node() -> str: + return "Scenario (not in trace)" + + @staticmethod + def get_legend_info_final_trace_edge() -> str: + return "path included in trace" + + @staticmethod + def get_legend_info_other_edge() -> str: + return "not selected for trace" + + @staticmethod + def get_tooltip_name() -> str: + return "" +``` + +Once you have created a new Graph class, you can direct the visualiser to use it when your type is selected. +Simply add your class to the `GRAPHS` dictionary in `robotmbt/visualise/visualiser.py` like the others. For our example: + +```python +GRAPHS = { + ... + 'scenario': ScenarioGraph, + ... +} +``` + +Now, when selecting your graph type (in our example `Treat this test suite Model-based graph=scenario`), your graph will get constructed! + +**NOTE**: You need the optional dependencies for visualisation to create graphs, e.g.: + +```bash +pip install .[visualisation] +``` + +**TIP**: [Python virtual environments](https://docs.python.org/3/library/venv.html) are an easy way to test with different Python version and sets of dependencies. diff --git a/robotmbt/visualise/graphs/__init__.py b/robotmbt/visualise/graphs/__init__.py new file mode 100644 index 00000000..c7ec90cd --- /dev/null +++ b/robotmbt/visualise/graphs/__init__.py @@ -0,0 +1,29 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/robotmbt/visualise/graphs/abstractgraph.py b/robotmbt/visualise/graphs/abstractgraph.py new file mode 100644 index 00000000..4214d0db --- /dev/null +++ b/robotmbt/visualise/graphs/abstractgraph.py @@ -0,0 +1,186 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from abc import ABC, abstractmethod +from typing import Generic, TypeVar + +from robotmbt.visualise.models import TraceInfo, ScenarioInfo, StateInfo +import networkx as nx + + +NodeInfo = TypeVar('NodeInfo') +EdgeInfo = TypeVar('EdgeInfo') + + +class AbstractGraph(ABC, Generic[NodeInfo, EdgeInfo]): + def __init__(self, info: TraceInfo): + """ + Note that networkx's ids have to be of a serializable and hashable type after construction. + """ + # The underlying storage - a NetworkX DiGraph + self.networkx: nx.DiGraph = nx.DiGraph() + + # Keep track of node IDs + self.ids: dict[str, tuple[NodeInfo, str]] = {} + + # Add the start node + self.networkx.add_node('start', label='start', description='') + self.start_node = 'start' + + # Add nodes and edges for all traces + for trace in info.all_traces: + for i in range(len(trace)): + if i > 0: + from_node = self._get_or_create_id( + self.select_node_info(trace, i - 1), + self.create_node_description(trace, i - 1)) + else: + from_node = 'start' + to_node = self._get_or_create_id( + self.select_node_info(trace, i), + self.create_node_description(trace, i)) + self._add_node(from_node) + self._add_node(to_node) + self._add_edge(from_node, to_node, + self.create_edge_label(self.select_edge_info(trace[i]))) + + # Set the final trace and add any missing nodes/edges + self.final_trace = ['start'] + for i in range(len(info.current_trace)): + if i > 0: + from_node = self._get_or_create_id( + self.select_node_info(info.current_trace, i - 1), + self.create_node_description(info.current_trace, i - 1)) + else: + from_node = 'start' + to_node = self._get_or_create_id( + self.select_node_info(info.current_trace, i), + self.create_node_description(info.current_trace, i)) + self.final_trace.append(to_node) + self._add_node(from_node) + self._add_node(to_node) + self._add_edge(from_node, to_node, + self.create_edge_label(self.select_edge_info(info.current_trace[i]))) + + def get_final_trace(self) -> list[str]: + """ + Get the final trace as ordered node ids. + Edges are subsequent entries in the list. + """ + return self.final_trace + + def _get_or_create_id(self, info: NodeInfo, description: str) -> str: + """ + Get the ID for a state that has been added before, or create and store a new one. + """ + for i in self.ids.keys(): + if self.ids[i][0] == info: + return i + + new_id = f"node{len(self.ids)}" + self.ids[new_id] = info, description + return new_id + + def _add_node(self, node: str): + """ + Add node if it doesn't already exist. + """ + if node not in self.networkx.nodes: + self.networkx.add_node( + node, label=self.create_node_label(self.ids[node][0]), description=self.ids[node][1]) + + def _add_edge(self, from_node: str, to_node: str, label: str): + """ + Add edge if it doesn't already exist. + If edge exists, update the label information + """ + if (from_node, to_node) in self.networkx.edges: + if label == '': + return + old_label = nx.get_edge_attributes(self.networkx, 'label')[ + (from_node, to_node)] + if label in old_label.split('\n'): + return + new_label = old_label + '\n' + label + attr = {(from_node, to_node): {'label': new_label}} + nx.set_edge_attributes(self.networkx, attr) + else: + self.networkx.add_edge( + from_node, to_node, label=label) + + @staticmethod + @abstractmethod + def select_node_info(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> NodeInfo: + """Select the info to use to compare nodes and generate their labels for a specific graph type.""" + + @staticmethod + @abstractmethod + def select_edge_info(pair: tuple[ScenarioInfo, StateInfo]) -> EdgeInfo: + """Select the info to use to generate the label for each edge for a specific graph type.""" + + @staticmethod + @abstractmethod + def create_node_description(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> str: + """Create the description to be shown in a tooltip for a node given the full trace and its index.""" + + @staticmethod + @abstractmethod + def create_node_label(info: NodeInfo) -> str: + """Create the label for a node given its chosen information.""" + + @staticmethod + @abstractmethod + def create_edge_label(info: EdgeInfo) -> str: + """Create the label for an edge given its chosen information.""" + + @staticmethod + @abstractmethod + def get_legend_info_final_trace_node() -> str: + """Get the information to include in the legend for nodes that appear in the final trace.""" + + @staticmethod + @abstractmethod + def get_legend_info_other_node() -> str: + """Get the information to include in the legend for nodes that do not appear in the final trace.""" + + @staticmethod + @abstractmethod + def get_legend_info_final_trace_edge() -> str: + """Get the information to include in the legend for edges that appear in the final trace.""" + + @staticmethod + @abstractmethod + def get_legend_info_other_edge() -> str: + """Get the information to include in the legend for edges that do not appear in the final trace.""" + + @staticmethod + @abstractmethod + def get_tooltip_name() -> str: + """Get the name/title of the tooltip.""" diff --git a/robotmbt/visualise/graphs/scenariodeltavaluegraph.py b/robotmbt/visualise/graphs/scenariodeltavaluegraph.py new file mode 100644 index 00000000..1fd3fd5f --- /dev/null +++ b/robotmbt/visualise/graphs/scenariodeltavaluegraph.py @@ -0,0 +1,89 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from robotmbt.modelspace import ModelSpace + +from robotmbt.visualise.graphs.abstractgraph import AbstractGraph +from robotmbt.visualise.models import ScenarioInfo, StateInfo + + +class ScenarioDeltaValueGraph(AbstractGraph[tuple[ScenarioInfo, set[tuple[str, str]]], None]): + """ + The Scenario-delta-Value graph keeps track of both the scenarios and state updates encountered. + Its nodes are scenarios together with the property assignments after the scenario has run. + Its edges represent steps in the trace. + """ + + @staticmethod + def select_node_info(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) \ + -> tuple[ScenarioInfo, set[tuple[str, str]]]: + if index == 0: + return trace[0][0], StateInfo(ModelSpace()).difference(trace[0][1]) + else: + return trace[index][0], trace[index-1][1].difference(trace[index][1]) + + @staticmethod + def select_edge_info(pair: tuple[ScenarioInfo, StateInfo]) -> None: + return None + + @staticmethod + def create_node_description(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> str: + return str(trace[index][1]).replace('\n', '
') + + @staticmethod + def create_node_label(info: tuple[ScenarioInfo, set[tuple[str, str]]]) -> str: + res = "" + for assignment in info[1]: + res += "\n\n"+assignment[0]+":"+assignment[1] + return f"{info[0].name}{res}" + + @staticmethod + def create_edge_label(info: None) -> str: + return '' + + @staticmethod + def get_legend_info_final_trace_node() -> str: + return "Scenario + effect on model (part of trace)" + + @staticmethod + def get_legend_info_other_node() -> str: + return "Scenario + effect on model (not in trace)" + + @staticmethod + def get_legend_info_final_trace_edge() -> str: + return "path included in trace" + + @staticmethod + def get_legend_info_other_edge() -> str: + return "not selected for trace" + + @staticmethod + def get_tooltip_name() -> str: + return "Full state" diff --git a/robotmbt/visualise/graphs/scenariograph.py b/robotmbt/visualise/graphs/scenariograph.py new file mode 100644 index 00000000..1b186f6c --- /dev/null +++ b/robotmbt/visualise/graphs/scenariograph.py @@ -0,0 +1,79 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from robotmbt.visualise.graphs.abstractgraph import AbstractGraph +from robotmbt.visualise.models import ScenarioInfo, StateInfo + + +class ScenarioGraph(AbstractGraph[ScenarioInfo, None]): + """ + The scenario graph is the most basic representation of trace exploration. + It represents scenarios as nodes, and the trace as edges. + """ + + @staticmethod + def select_node_info(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> ScenarioInfo: + return trace[index][0] + + @staticmethod + def select_edge_info(pair: tuple[ScenarioInfo, StateInfo]) -> None: + return None + + @staticmethod + def create_node_description(trace: list[tuple[ScenarioInfo, StateInfo]], index: int) -> str: + return '' + + @staticmethod + def create_node_label(info: ScenarioInfo) -> str: + return info.name + + @staticmethod + def create_edge_label(info: None) -> str: + return '' + + @staticmethod + def get_legend_info_final_trace_node() -> str: + return "Scenario (part of trace)" + + @staticmethod + def get_legend_info_other_node() -> str: + return "Scenario (not in trace)" + + @staticmethod + def get_legend_info_final_trace_edge() -> str: + return "path included in trace" + + @staticmethod + def get_legend_info_other_edge() -> str: + return "not selected for trace" + + @staticmethod + def get_tooltip_name() -> str: + return "" diff --git a/robotmbt/visualise/models.py b/robotmbt/visualise/models.py new file mode 100644 index 00000000..658d49bf --- /dev/null +++ b/robotmbt/visualise/models.py @@ -0,0 +1,311 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from typing import Any + +from robot.api import logger + +from robotmbt.modelspace import ModelSpace +from robotmbt.suitedata import Scenario + +import jsonpickle +import tempfile +import os + +DESIRED_NAME_LINE_LENGTH = 20 + + +class ScenarioInfo: + """ + This contains all information we need from scenarios, abstracting away from the actual Scenario class: + - name + - src_id + """ + + def __init__(self, scenario: Scenario): + self.name = self._split_name(scenario.name) + self.src_id = scenario.src_id + + def __str__(self): + return f"Scenario {self.src_id}: {self.name}" + + def __eq__(self, other): + return self.src_id == other.src_id + + @staticmethod + def _split_name(name: str) -> str: + """ + Split a name into separate lines where each line is as close to the desired line length as possible. + """ + # Split into words + words = name.split(" ") + + # If any word is longer than the desired length, use that as the desired length instead + # (otherwise, we will always get a line (much) longer than the desired length, while the other lines will + # be constrained by the desired length) + desired_length = DESIRED_NAME_LINE_LENGTH + for i in words: + if len(i) > desired_length: + desired_length = len(i) + + res = "" + line = words[0] + for i in words[1:]: + # If the previous line was fully appended, simply take the current word as the new line + if line == '\n': + line += i + continue + + app_len = len(line + ' ' + i) + + # If the word fully fits into the line, simply append it + if app_len < desired_length: + line = line + ' ' + i + continue + + app_diff = abs(desired_length - app_len) + curr_diff = abs(desired_length - len(line)) + + # If the current line is closer to the desired length, use that + if curr_diff < app_diff: + res += line + line = '\n' + i + # If the current line with the new word is closer to the desired length, use that + else: + res += line + ' ' + i + line = '\n' + + # Append the final line if it wasn't empty + if line != '\n': + res += line + + return res + + +class StateInfo: + """ + This contains all information we need from states, abstracting away from the actual ModelSpace class: + - domain + - properties + """ + + @classmethod + def _create_state_with_prop(cls, name: str, attrs: list[tuple[str, Any]]): + space = ModelSpace() + prop = ModelSpace() + for (key, val) in attrs: + prop.__setattr__(key, val) + space.props[name] = prop + return cls(space) + + def difference(self, new_state) -> set[tuple[str, str]]: + """ + new_state: the new StateInfo to be compared to the self. + returns: a set of tuples with properties and their assignment. + """ + old: dict[str, dict | str] = self.properties.copy() + new: dict[str, dict | str] = new_state.properties.copy() + temp = StateInfo._dict_deep_diff(old, new) + for key in temp.keys(): + res = "" + for k, v in sorted(temp[key].items()): + res += f"\n\t{k}={v}" + temp[key] = res + return set(temp.items()) # type inference goes wacky here + + @staticmethod + def _dict_deep_diff(old_state: dict[str, Any], new_state: dict[str, Any]) -> dict[str, Any]: + res = {} + for key in new_state.keys(): + if key not in old_state: + res[key] = new_state[key] + elif isinstance(old_state[key], dict): + diff = StateInfo._dict_deep_diff(old_state[key], new_state[key]) + if len(diff) != 0: + res[key] = diff + elif old_state[key] != new_state[key]: + res[key] = new_state[key] + return res + + def __init__(self, state: ModelSpace): + self.domain = state.ref_id + + # Extract all attributes/properties stored in the model space and store them in the temp dict + # Similar in workings to ModelSpace's get_status_text + temp = {} + for p in state.props: + temp[p] = {} + if p == 'scenario': + temp['scenario'] = dict(state.props['scenario']) + else: + for attr in dir(state.props[p]): + temp[p][attr] = getattr(state.props[p], attr) + + # Filter empty entries + self.properties = {} + for p in temp.keys(): + if len(temp[p]) > 0: + self.properties[p] = temp[p].copy() + + def __eq__(self, other): + return self.domain == other.domain and self.properties == other.properties + + def __str__(self): + res = "" + for p in self.properties: + if res != "": + res += "\n\n" + res += f"{p}:" + for k, v in self.properties[p].items(): + res += f"\n\t{k}={v}" + return res + + +class TraceInfo: + """ + This keeps track of all information we need from all steps in trace exploration: + - current_trace: the trace currently being built up, a list of scenario/state pairs in order of execution + - all_traces: all valid traces encountered in trace exploration, up until the point they could not go any further + - previous_length: used to identify backtracking + """ + ROBOTMBT_MODEL_VERSION = '1.0.0' + + def __init__(self, name: str = ''): + self.ROBOTMBT_MODEL_VERSION: str = TraceInfo.ROBOTMBT_MODEL_VERSION + self.model_name: str = name + self.current_trace: list[tuple[ScenarioInfo, StateInfo]] = [] + self.all_traces: list[list[tuple[ScenarioInfo, StateInfo]]] = [] + self.previous_length: int = 0 + self.pushed: bool = False + + def update_trace(self, scenario: ScenarioInfo | None, state: StateInfo, length: int): + """ + Updates TraceInfo instance with the information that a scenario has run resulting in the given state as the nth + scenario of the trace, where n is the value of the length parameter. If length is greater than the previous + length of the trace to be updated, adds the given scenario/state to the trace. If length is smaller than the + previous length of the trace, roll back the trace until the step indicated by length. + scenario: the scenario that has run. + state: the state after scenario has run. + length: the step in the trace the scenario occurred in. + """ + if length > self.previous_length: + # New state - push + self._push(scenario, state, length - self.previous_length) + self.previous_length = length + elif length < self.previous_length: + # Backtrack - pop + self._pop(self.previous_length - length) + self.previous_length = length + + # Sanity check + if len(self.current_trace) > 0: + self._sanity_check(scenario, state, 'popping') + else: + # No change - sanity check + if len(self.current_trace) > 0: + self._sanity_check(scenario, state, 'nothing') + + def _push(self, scenario: ScenarioInfo, state: StateInfo, n: int): + if n > 1: + logger.warn( + f"Pushing multiple scenario/state pairs at once to trace info ({n})! Some info might be lost!") + for _ in range(n): + self.current_trace.append((scenario, state)) + self.pushed = True + + def _pop(self, n: int): + if self.pushed: + self.all_traces.append(self.current_trace.copy()) + for _ in range(n): + self.current_trace.pop() + self.pushed = False + + def __repr__(self) -> str: + return f"TraceInfo(traces=[{[f'[{[self.stringify_pair(pair) for pair in trace]}]' for trace in self.all_traces]}], current=[{[self.stringify_pair(pair) for pair in self.current_trace]}])" + + def _sanity_check(self, scen: ScenarioInfo, state: StateInfo, after: str): + (prev_scen, prev_state) = self.current_trace[-1] + if prev_scen != scen: + logger.warn( + f'TraceInfo got out of sync after {after}\nExpected scenario: {prev_scen}\nActual scenario: {scen}') + if prev_state != state: + logger.warn( + f'TraceInfo got out of sync after {after}\nExpected state: {prev_state}\nActual state: {state}') + + def export_graph(self, dir: str = '', atest: bool = False) -> str | None: + encoded_instance = jsonpickle.encode(self) + name = self.model_name.lower().replace(' ', '_') + if atest: + ''' + temporary file to not accidentally overwrite an existing file + mkstemp() is not ideal but given Python's limitations this is the easiest solution + as temporary file, a different method, is problematic on Windows + https://stackoverflow.com/a/57015383 + ''' + fd, dir = tempfile.mkstemp() + with os.fdopen(fd, "w") as f: + f.write(encoded_instance) + return dir + + if dir[-1] != '/': + dir += '/' + + # create folders if they do not exist + if not os.path.exists(dir): + os.makedirs(dir) + + with open(f"{dir}{name}.json", "w") as f: + f.write(encoded_instance) + return None + + @staticmethod + def import_graph_from_file(file_path: str): + try: + with open(file_path, "r") as f: + traceinfo = jsonpickle.decode(f.read()) + traceinfo.ROBOTMBT_MODEL_VERSION # trigger AttributeError if non-robotmbt data + except OSError: + raise + except Exception: + raise TypeError(f"Contents from '{file_path}' could not be loaded as RobotMBT graph data") + + if str(traceinfo.ROBOTMBT_MODEL_VERSION).split('.')[0] != TraceInfo.ROBOTMBT_MODEL_VERSION.split('.')[0]: + # raise if major version differs + raise ValueError("Graph data is from an incompatible RobotMBT version") + return traceinfo + + @staticmethod + def stringify_pair(pair: tuple[ScenarioInfo, StateInfo]) -> str: + """ + Takes a pair of a scenario and a state and returns a string describing both. + pair: a tuple consisting of a scenario and a state. + returns: formatted string based on the given scenario and state. + """ + return f"Scenario={pair[0].name}, State={pair[1]}" diff --git a/robotmbt/visualise/networkvisualiser.py b/robotmbt/visualise/networkvisualiser.py new file mode 100644 index 00000000..beacfd4b --- /dev/null +++ b/robotmbt/visualise/networkvisualiser.py @@ -0,0 +1,679 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from bokeh.core.enums import PlaceType, LegendLocationType +from bokeh.core.property.vectorization import value +from bokeh.embed import file_html +from bokeh.models import ColumnDataSource, Rect, Text, ResetTool, SaveTool, WheelZoomTool, PanTool, Plot, Range1d, \ + Title, FullscreenTool, CustomJS, Segment, Arrow, NormalHead, Bezier, Legend, ZoomInTool, ZoomOutTool, HoverTool + +# grandalf used under EPL license +from grandalf.graphs import Vertex as GVertex, Edge as GEdge, Graph as GGraph +from grandalf.layouts import SugiyamaLayout + +from networkx import DiGraph + +from robotmbt.visualise.graphs.abstractgraph import AbstractGraph + +# Padding within the nodes between the borders and inner text +HORIZONTAL_PADDING_WITHIN_NODES: int = 5 +VERTICAL_PADDING_WITHIN_NODES: int = 5 + +# Colors for different parts of the graph +FINAL_TRACE_NODE_COLOR: str = '#CCCC00' +OTHER_NODE_COLOR: str = '#999989' +FINAL_TRACE_EDGE_COLOR: str = '#444422' +OTHER_EDGE_COLOR: str = '#BBBBAA' + +# Legend placement +# Alignment within graph ('center' is in the middle, 'top_right' is the top right, etc.) +LEGEND_LOCATION: LegendLocationType | tuple[float, float] = 'top_right' +# Where it appears relative to graph ('center' is within, 'below' is below, etc.) +LEGEND_PLACE: PlaceType = 'center' + +# Dimensions of the plot in the window +INNER_WINDOW_WIDTH: int = 720 +INNER_WINDOW_HEIGHT: int = 480 +OUTER_WINDOW_WIDTH: int = INNER_WINDOW_WIDTH + (280 if LEGEND_PLACE == 'left' or LEGEND_PLACE == 'right' else 30) +OUTER_WINDOW_HEIGHT: int = INNER_WINDOW_HEIGHT + (150 if LEGEND_PLACE == 'below' or LEGEND_PLACE == 'center' else 30) + +# Font sizes +MAJOR_FONT_SIZE: int = 16 +MINOR_FONT_SIZE: int = 8 + + +class Node: + """ + Contains the information we need to add a node to the graph. + """ + + def __init__(self, node_id: str, label: str, x: int, y: int, width: float, height: float, description: str): + self.node_id = node_id + self.label = label + self.x = x + self.y = y + self.width = width + self.height = height + self.description = description + + +class Edge: + """ + Contains the information we need to add an edge to the graph. + """ + + def __init__(self, from_node: str, to_node: str, label: str, points: list[tuple[float, float]]): + self.from_node = from_node + self.to_node = to_node + self.label = label + self.points = points + + +class NetworkVisualiser: + """ + A container for a Bokeh graph, which can be created from any abstract graph. + """ + + def __init__(self, graph: AbstractGraph, suite_name: str): + # Extract what we need from the graph + self.networkx: DiGraph = graph.networkx + self.final_trace = graph.get_final_trace() + self.start = graph.start_node + + # Set up a Bokeh figure + self.plot = Plot(width=OUTER_WINDOW_WIDTH, height=OUTER_WINDOW_HEIGHT) + self.plot.output_backend = "svg" + + # Create Sugiyama layout + nodes, edges = self._create_layout() + self.node_dict: dict[str, Node] = {} + for node in nodes: + self.node_dict[node.node_id] = node + + # Keep track of arrows in the graph for scaling + self.arrows = [] + + # Create the hover tool to show tooltips + tooltip_name = graph.get_tooltip_name() + if tooltip_name: + self.hover = HoverTool() + tooltips = f""" +
+ {tooltip_name} +
+ @description{{safe}} +
+
+ """ + self.hover.tooltips = tooltips + else: + self.hover = None + + # Add the nodes to the graph + self._add_nodes(nodes) + + # Add the edges to the graph + self._add_edges(nodes, edges) + + # Add a legend to the graph + self._add_legend(graph) + + # Add our features to the graph (e.g. tools) + self._add_features(suite_name) + + def generate_html(self): + """ + Generate HTML for the Bokeh graph. + """ + return file_html(self.plot, 'inline', "graph") + + def _add_nodes(self, nodes: list[Node]): + """ + Add the nodes to the graph in the form of Rect and Text glyphs. + """ + # The ColumnDataSources to store our nodes and edges in Bokeh's format + node_source: ColumnDataSource = ColumnDataSource( + {'id': [], 'x': [], 'y': [], 'w': [], 'h': [], 'color': [], 'description': []}) + node_label_source: ColumnDataSource = ColumnDataSource( + {'id': [], 'x': [], 'y': [], 'label': []}) + + # Add all nodes to the column data sources + for node in nodes: + _add_node_to_sources(node, self.final_trace, node_source, node_label_source) + + # Add the glyphs for nodes and their labels + node_glyph = Rect(x='x', y='y', width='w', height='h', fill_color='color') + node_glyph_renderer = self.plot.add_glyph(node_source, node_glyph) + + if self.hover is not None: + self.hover.renderers = [node_glyph_renderer] + + node_label_glyph = Text(x='x', y='y', text='label', text_align='left', text_baseline='middle', + text_font_size=f'{MAJOR_FONT_SIZE}pt', text_font=value("Courier New")) + node_label_glyph.tags = [f"scalable_text{MAJOR_FONT_SIZE}"] + self.plot.add_glyph(node_label_source, node_label_glyph) + + def _add_edges(self, nodes: list[Node], edges: list[Edge]): + """ + Add the edges to the graph in the form of Arrow layouts and Segment, Bezier, and Text glyphs. + """ + # The ColumnDataSources to store our edges in Bokeh's format + edge_part_source: ColumnDataSource = ColumnDataSource( + {'from': [], 'to': [], 'start_x': [], 'start_y': [], 'end_x': [], 'end_y': [], 'color': []}) + edge_arrow_source: ColumnDataSource = ColumnDataSource( + {'from': [], 'to': [], 'start_x': [], 'start_y': [], 'end_x': [], 'end_y': [], 'color': []}) + edge_bezier_source: ColumnDataSource = ColumnDataSource( + {'from': [], 'to': [], 'start_x': [], 'start_y': [], 'end_x': [], 'end_y': [], 'control1_x': [], + 'control1_y': [], 'control2_x': [], 'control2_y': [], 'color': []}) + edge_label_source: ColumnDataSource = ColumnDataSource({'from': [], 'to': [], 'x': [], 'y': [], 'label': []}) + + for edge in edges: + _add_edge_to_sources(nodes, edge, self.final_trace, edge_part_source, edge_arrow_source, edge_bezier_source, + edge_label_source) + + # Add the glyphs for edges and their labels + edge_part_glyph = Segment(x0='start_x', y0='start_y', x1='end_x', y1='end_y', line_color='color') + self.plot.add_glyph(edge_part_source, edge_part_glyph) + + arrow_layout = Arrow( + end=NormalHead(size=10, fill_color='color', line_color='color'), + x_start='start_x', y_start='start_y', + x_end='end_x', y_end='end_y', line_color='color', + source=edge_arrow_source + ) + self.plot.add_layout(arrow_layout) + self.arrows.append(arrow_layout) + + edge_bezier_glyph = Bezier(x0='start_x', y0='start_y', x1='end_x', y1='end_y', cx0='control1_x', + cy0='control1_y', cx1='control2_x', cy1='control2_y', line_color='color') + self.plot.add_glyph(edge_bezier_source, edge_bezier_glyph) + + edge_label_glyph = Text(x='x', y='y', text='label', text_align='center', text_baseline='middle', + text_font_size=f'{MINOR_FONT_SIZE}pt', text_font=value("Courier New")) + edge_label_glyph.tags = [f"scalable_text{MINOR_FONT_SIZE}"] + self.plot.add_glyph(edge_label_source, edge_label_glyph) + + def _create_layout(self) -> tuple[list[Node], list[Edge]]: + """ + Create the Sugiyama layout using grandalf. + """ + # Containers to convert networkx nodes/edges to the proper format. + vertices = [] + edges = [] + flips = [] + + # Extract nodes from networkx and put them in the proper format to be used by grandalf. + start = None + for node_id in self.networkx.nodes: + v = GVertex(node_id) + w, h = _calculate_dimensions(self.networkx.nodes[node_id]['label']) + v.view = NodeView(w, h) + vertices.append(v) + if node_id == self.start: + start = v + + # Calculate which edges need to be flipped to make the graph acyclic. + flip = _flip_edges([e for e in self.networkx.edges]) + + # Extract edges from networkx and put them in the proper format to be used by grandalf. + for (from_id, to_id) in self.networkx.edges: + from_node = _find_node(vertices, from_id) + to_node = _find_node(vertices, to_id) + e = GEdge(from_node, to_node) + e.view = EdgeView() + edges.append(e) + if (from_id, to_id) in flip: + flips.append(e) + + # Feed the info to grandalf and get the layout. + g = GGraph(vertices, edges) + sugiyama = SugiyamaLayout(g.C[0]) + + # Set specific margins as these values worked best in user-testing + # Space between nodes + sugiyama.xspace = 10 + sugiyama.yspace = 15 + # Default width for nodes with no set width (in this case only for edge routing) + sugiyama.dw = 2 + # Default height for nodes with no set height (in this case only for edge routing) + sugiyama.dh = 2 + + sugiyama.init_all(roots=[start], inverted_edges=flips) + sugiyama.draw() + + # Extract the information we need from the nodes and edges and return them in our format. + ns = [] + for v in g.C[0].sV: + node_id = v.data + label = self.networkx.nodes[node_id]['label'] + (x, y) = v.view.xy + (w, h) = _calculate_dimensions(label) + description = self.networkx.nodes[node_id]['description'] + ns.append(Node(node_id, label, x, -y, w, h, description)) + + es = [] + for e in g.C[0].sE: + from_id = e.v[0].data + to_id = e.v[1].data + label = self.networkx.edges[(from_id, to_id)]['label'] + points = [] + # invert y axis + for p in e.view.points: + points.append((p[0], -p[1])) + + es.append(Edge(from_id, to_id, label, points)) + + return ns, es + + def _add_features(self, suite_name: str): + """ + Add our features to the graph such as tools, titles, and JavaScript callbacks. + """ + self.plot.add_layout(Title(text=suite_name, align="center"), "above") + + # Add the different tools + wheel_zoom = WheelZoomTool() + self.plot.add_tools(ResetTool(), SaveTool(), + wheel_zoom, PanTool(), + FullscreenTool(), ZoomInTool(factor=0.4), ZoomOutTool(factor=0.4)) + self.plot.toolbar.active_scroll = wheel_zoom + + if self.hover: + self.plot.add_tools(self.hover) + + # Specify the default range - these values represent the aspect ratio of the actual view in the window + self.plot.x_range = Range1d(-INNER_WINDOW_WIDTH / 2, INNER_WINDOW_WIDTH / 2) + self.plot.y_range = Range1d(-INNER_WINDOW_HEIGHT, 0) + self.plot.x_range.tags = [{"initial_span": INNER_WINDOW_WIDTH}] + self.plot.y_range.tags = [{"initial_span": INNER_WINDOW_HEIGHT}] + + # A JS callback to scale text and arrows, and change aspect ratio. + resize_cb = CustomJS(args=dict(xr=self.plot.x_range, yr=self.plot.y_range, plot=self.plot, arrows=self.arrows), + code=f""" + // Initialize initial scale tag + if (!plot.tags || plot.tags.length === 0) {{ + plot.tags = [{{ + initial_scale: plot.inner_height / (yr.end - yr.start) + }}] + }} + + // Calculate current x and y span + const xspan = xr.end - xr.start; + const yspan = yr.end - yr.start; + + // Calculate inner aspect ratio and span aspect ratio + const inner_aspect = plot.inner_width / plot.inner_height; + const span_aspect = xspan / yspan; + + // Let span aspect ratio match inner aspect ratio if needed + if (Math.abs(inner_aspect - span_aspect) > 0.05) {{ + const xmid = xr.start + xspan / 2; + const new_xspan = yspan * inner_aspect; + xr.start = xmid - new_xspan / 2; + xr.end = xmid + new_xspan / 2; + }} + + // Calculate scale factor + const scale = (plot.inner_height / yspan) / plot.tags[0].initial_scale + + // Scale text + for (const r of plot.renderers) {{ + if (!r.glyph || !r.glyph.tags) continue + + if (r.glyph.tags.includes("scalable_text{MAJOR_FONT_SIZE}")) {{ + r.glyph.text_font_size = Math.floor({MAJOR_FONT_SIZE} * scale) + "pt" + }} + + if (r.glyph.tags.includes("scalable_text{MINOR_FONT_SIZE}")) {{ + r.glyph.text_font_size = Math.floor({MINOR_FONT_SIZE} * scale) + "pt" + }} + }} + + // Scale arrows + for (const a of arrows) {{ + if (!a.properties) continue; + if (!a.properties.end) continue; + if (!a.properties.end._value) continue; + if (!a.properties.end._value.properties) continue; + if (!a.properties.end._value.properties.size) continue; + if (!a.properties.end._value.properties.size._value) continue; + if (!a.properties.end._value.properties.size._value.value) continue; + if (a._base_end_size == null) + a._base_end_size = a.properties.end._value.properties.size._value.value; + a.properties.end._value.properties.size._value.value = a._base_end_size * Math.sqrt(scale); + a.change.emit(); + }}""") + + # Add the callback to the values that change when zooming/resizing. + self.plot.x_range.js_on_change("start", resize_cb) + self.plot.x_range.js_on_change("end", resize_cb) + self.plot.y_range.js_on_change("start", resize_cb) + self.plot.y_range.js_on_change("end", resize_cb) + self.plot.js_on_change("inner_width", resize_cb) + self.plot.js_on_change("inner_height", resize_cb) + + def _add_legend(self, graph: AbstractGraph): + """ + Adds a legend to the graph with the node/edge information from the given graph. + """ + empty_source = ColumnDataSource({'_': [0]}) + + final_trace_node_glyph = Rect(x='_', y='_', width='_', height='_', fill_color=FINAL_TRACE_NODE_COLOR) + final_trace_node = self.plot.add_glyph(empty_source, final_trace_node_glyph) + + other_node_glyph = Rect(x='_', y='_', width='_', height='_', fill_color=OTHER_NODE_COLOR) + other_node = self.plot.add_glyph(empty_source, other_node_glyph) + + final_trace_edge_glyph = Segment( + x0='_', x1='_', + y0='_', y1='_', line_color=FINAL_TRACE_EDGE_COLOR + ) + final_trace_edge = self.plot.add_glyph(empty_source, final_trace_edge_glyph) + + other_edge_glyph = Segment( + x0='_', x1='_', + y0='_', y1='_', line_color=OTHER_EDGE_COLOR + ) + other_edge = self.plot.add_glyph(empty_source, other_edge_glyph) + + legend = Legend(items=[(graph.get_legend_info_final_trace_node(), [final_trace_node]), + (graph.get_legend_info_other_node(), [other_node]), + (graph.get_legend_info_final_trace_edge(), [final_trace_edge]), + (graph.get_legend_info_other_edge(), [other_edge])], + location=LEGEND_LOCATION, orientation="vertical") + self.plot.add_layout(legend, LEGEND_PLACE) + + +class NodeView: + """ + A view of a node in the format that grandalf expects. + """ + + def __init__(self, width: float, height: float): + self.w, self.h = width, height + self.xy = (0, 0) + + +class EdgeView: + """ + A view of an edge in the format that grandalf expects. + """ + + def __init__(self): + self.points = [] + + def setpath(self, points: list[tuple[float, float]]): + self.points = points + + +def _find_node(nodes: list[GVertex], node_id: str): + """ + Find a node given its id in a list of grandalf nodes. + """ + for node in nodes: + if node.data == node_id: + return node + return None + + +def _get_connection_coordinates(nodes: list[Node], node_id: str) -> list[tuple[float, float]]: + """ + Get the coordinates where edges can connect to a node given its id. + These places are the middle of the left, right, top, and bottom edge, as well as the corners of the node. + """ + start_possibilities = [] + + # Get node from list + node = None + for n in nodes: + if n.node_id == node_id: + node = n + break + + # Left + start_possibilities.append((node.x - node.width / 2, node.y)) + # Right + start_possibilities.append((node.x + node.width / 2, node.y)) + # Bottom + start_possibilities.append((node.x, node.y - node.height / 2)) + # Top + start_possibilities.append((node.x, node.y + node.height / 2)) + # Left bottom + start_possibilities.append((node.x - node.width / 2, node.y - node.height / 2)) + # Left top + start_possibilities.append((node.x - node.width / 2, node.y + node.height / 2)) + # Right bottom + start_possibilities.append((node.x + node.width / 2, node.y - node.height / 2)) + # Right top + start_possibilities.append((node.x + node.width / 2, node.y + node.height / 2)) + + return start_possibilities + + +def _minimize_distance(from_pos: list[tuple[float, float]], to_pos: list[tuple[float, float]]) -> tuple[ + float, float, float, float]: + """ + Find a pair of positions that minimizes their distance. + """ + min_dist = -1 + fx, fy, tx, ty = 0, 0, 0, 0 + + # Calculate the distance between all permutations + for fp in from_pos: + for tp in to_pos: + distance = (fp[0] - tp[0]) ** 2 + (fp[1] - tp[1]) ** 2 + if min_dist == -1 or distance < min_dist: + min_dist = distance + fx, fy, tx, ty = fp[0], fp[1], tp[0], tp[1] + + # Return the permutation with the shortest distance + return fx, fy, tx, ty + + +def _add_edge_to_sources(nodes: list[Node], edge: Edge, final_trace: list[str], edge_part_source: ColumnDataSource, + edge_arrow_source: ColumnDataSource, edge_bezier_source: ColumnDataSource, + edge_label_source: ColumnDataSource): + """ + Add an edge between two nodes to the ColumnDataSources. + Contains all logic to set their color, find their attachment points, and do self-loops properly. + """ + in_final_trace = False + for i in range(len(final_trace) - 1): + if edge.from_node == final_trace[i] and edge.to_node == final_trace[i + 1]: + in_final_trace = True + break + + if edge.from_node == edge.to_node: + _add_self_loop_to_sources(nodes, edge, in_final_trace, edge_arrow_source, edge_bezier_source, edge_label_source) + return + + start_x, start_y = 0, 0 + end_x, end_y = 0, 0 + + # Add edges going through the calculated points + for i in range(len(edge.points) - 1): + start_x, start_y = edge.points[i] + end_x, end_y = edge.points[i + 1] + + # Collect possibilities where the edge can start and end + if i == 0: + from_possibilities = _get_connection_coordinates(nodes, edge.from_node) + else: + from_possibilities = [(start_x, start_y)] + + if i == len(edge.points) - 2: + to_possibilities = _get_connection_coordinates(nodes, edge.to_node) + else: + to_possibilities = [(end_x, end_y)] + + # Choose connection points that minimize edge length + start_x, start_y, end_x, end_y = _minimize_distance(from_possibilities, to_possibilities) + + if i < len(edge.points) - 2: + # Middle part of edge without arrow + edge_part_source.data['from'].append(edge.from_node) + edge_part_source.data['to'].append(edge.to_node) + edge_part_source.data['start_x'].append(start_x) + edge_part_source.data['start_y'].append(start_y) + edge_part_source.data['end_x'].append(end_x) + edge_part_source.data['end_y'].append(end_y) + edge_part_source.data['color'].append(FINAL_TRACE_EDGE_COLOR if in_final_trace else OTHER_EDGE_COLOR) + else: + # End of edge with arrow + edge_arrow_source.data['from'].append(edge.from_node) + edge_arrow_source.data['to'].append(edge.to_node) + edge_arrow_source.data['start_x'].append(start_x) + edge_arrow_source.data['start_y'].append(start_y) + edge_arrow_source.data['end_x'].append(end_x) + edge_arrow_source.data['end_y'].append(end_y) + edge_arrow_source.data['color'].append(FINAL_TRACE_EDGE_COLOR if in_final_trace else OTHER_EDGE_COLOR) + + # Add the label + edge_label_source.data['from'].append(edge.from_node) + edge_label_source.data['to'].append(edge.to_node) + edge_label_source.data['x'].append((start_x + end_x) / 2) + edge_label_source.data['y'].append((start_y + end_y) / 2) + edge_label_source.data['label'].append(edge.label) + + +def _add_self_loop_to_sources(nodes: list[Node], edge: Edge, in_final_trace: bool, edge_arrow_source: ColumnDataSource, + edge_bezier_source: ColumnDataSource, edge_label_source: ColumnDataSource): + """ + Add a self-loop edge for a node to the ColumnDataSources, consisting of a Beziér curve and an arrow. + """ + connection = _get_connection_coordinates(nodes, edge.from_node) + + right_x, right_y = connection[1] + + # Add the Bézier curve + edge_bezier_source.data['from'].append(edge.from_node) + edge_bezier_source.data['to'].append(edge.to_node) + edge_bezier_source.data['start_x'].append(right_x) + edge_bezier_source.data['start_y'].append(right_y + 5) + edge_bezier_source.data['end_x'].append(right_x) + edge_bezier_source.data['end_y'].append(right_y - 5) + edge_bezier_source.data['control1_x'].append(right_x + 25) + edge_bezier_source.data['control1_y'].append(right_y + 25) + edge_bezier_source.data['control2_x'].append(right_x + 25) + edge_bezier_source.data['control2_y'].append(right_y - 25) + edge_bezier_source.data['color'].append(FINAL_TRACE_EDGE_COLOR if in_final_trace else OTHER_EDGE_COLOR) + + # Add the arrow + edge_arrow_source.data['from'].append(edge.from_node) + edge_arrow_source.data['to'].append(edge.to_node) + edge_arrow_source.data['start_x'].append(right_x + 0.001) + edge_arrow_source.data['start_y'].append(right_y - 5.001) + edge_arrow_source.data['end_x'].append(right_x) + edge_arrow_source.data['end_y'].append(right_y - 5) + edge_arrow_source.data['color'].append(FINAL_TRACE_EDGE_COLOR if in_final_trace else OTHER_EDGE_COLOR) + + # Add the label + edge_label_source.data['from'].append(edge.from_node) + edge_label_source.data['to'].append(edge.to_node) + edge_label_source.data['x'].append(right_x + 25) + edge_label_source.data['y'].append(right_y) + edge_label_source.data['label'].append(edge.label) + + +def _add_node_to_sources(node: Node, final_trace: list[str], node_source: ColumnDataSource, + node_label_source: ColumnDataSource): + """ + Add a node to the ColumnDataSources. + """ + node_source.data['id'].append(node.node_id) + node_source.data['x'].append(node.x) + node_source.data['y'].append(node.y) + node_source.data['w'].append(node.width) + node_source.data['h'].append(node.height) + node_source.data['color'].append( + FINAL_TRACE_NODE_COLOR if node.node_id in final_trace else OTHER_NODE_COLOR) + node_source.data['description'].append(node.description) + + node_label_source.data['id'].append(node.node_id) + node_label_source.data['x'].append(node.x - node.width / 2 + HORIZONTAL_PADDING_WITHIN_NODES) + node_label_source.data['y'].append(node.y) + node_label_source.data['label'].append(node.label) + + +def _calculate_dimensions(label: str) -> tuple[float, float]: + """ + Calculate a node's dimensions based on its label and the given font size constant. + Assumes the font is Courier New. + """ + lines = label.splitlines() + width = 0 + for line in lines: + width = max(width, len(line) * (MAJOR_FONT_SIZE / 3 + 5)) + height = len(lines) * (MAJOR_FONT_SIZE / 2 + 9) * 1.37 - 9 + return width + 2 * HORIZONTAL_PADDING_WITHIN_NODES, height + 2 * VERTICAL_PADDING_WITHIN_NODES + + +def _flip_edges(edges: list[tuple[str, str]]) -> list[tuple[str, str]]: + """ + Calculate which edges need to be flipped to make a graph acyclic. + """ + # Step 1: Build adjacency list from edges + adj = {} + for u, v in edges: + if u not in adj: + adj[u] = [] + adj[u].append(v) + + # Step 2: Helper function to detect cycles + def dfs(node, visited, rec_stack, cycle_edges): + visited[node] = True + rec_stack[node] = True + + if node in adj: + for neighbor in adj[node]: + edge = (node, neighbor) + + if not visited.get(neighbor, False): + if dfs(neighbor, visited, rec_stack, cycle_edges): + cycle_edges.append(edge) + elif rec_stack.get(neighbor, False): + # Found a cycle, add the edge to the cycle_edges list + cycle_edges.append(edge) + + rec_stack[node] = False + return False + + # Step 3: Detect cycles + visited = {} + rec_stack = {} + cycle_edges = [] + + for node in adj: + if not visited.get(node, False): + dfs(node, visited, rec_stack, cycle_edges) + + # Step 4: Return the list of edges that need to be flipped + # In this case, the cycle_edges are the ones that we need to "break" by flipping + return cycle_edges diff --git a/robotmbt/visualise/visualiser.py b/robotmbt/visualise/visualiser.py new file mode 100644 index 00000000..da0f33a5 --- /dev/null +++ b/robotmbt/visualise/visualiser.py @@ -0,0 +1,112 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from robotmbt.modelspace import ModelSpace +from robotmbt.tracestate import TraceState +from robotmbt.visualise import networkvisualiser +from robotmbt.visualise.graphs.scenariodeltavaluegraph import ScenarioDeltaValueGraph +from robotmbt.visualise.graphs.abstractgraph import AbstractGraph +from robotmbt.visualise.graphs.scenariograph import ScenarioGraph +from robotmbt.visualise.models import TraceInfo, StateInfo, ScenarioInfo +import html + +GRAPHS = { + 'scenario': ScenarioGraph, + 'scenario-delta-value': ScenarioDeltaValueGraph, +} + + +class Visualiser: + """ + The Visualiser class bridges the different concerns to provide + a simple interface through which the graph can be updated, + and retrieved in HTML format. + """ + + def __init__(self, suite_name: str = "", trace_info: TraceInfo = None): + self.trace_info: TraceInfo = trace_info if trace_info is not None else TraceInfo(suite_name) + self.suite_name = suite_name + + def load_from_file(self, file_path: str): + """ + Imports a JSON encoding of a graph and reconstructs the graph from it. The reconstructed + graph overrides the current graph instance this method is called on. + file_path: the path to a previously exported graph. + """ + self.trace_info = TraceInfo.import_graph_from_file(file_path) + + def export_to_file(self, file_path: str): + self.trace_info.export_graph(file_path) + + def update_trace(self, trace: TraceState): + """ + Uses the new snapshots from trace to update the trace info. + Multiple new snapshots can be pushed or popped at once. + """ + trace_len = len(trace._snapshots) + # We don't have any information + if trace_len == 0: + self.trace_info.update_trace(None, StateInfo(ModelSpace()), 0) + + # New snapshots have been pushed + elif trace_len > self.trace_info.previous_length: + prev = self.trace_info.previous_length + r = trace_len - prev + # Extract all snapshots that have been pushed and update our trace info with their scenario/model info + for i in range(r): + snap = trace._snapshots[prev + i] + scenario = snap.scenario + model = snap.model + self.trace_info.update_trace(ScenarioInfo(scenario), StateInfo(model), prev + i + 1) + + # Snapshots have been removed + else: + snap = trace._snapshots[-1] + scenario = snap.scenario + model = snap.model + self.trace_info.update_trace(ScenarioInfo(scenario), StateInfo(model), trace_len) + + def _get_graph(self, graph_type) -> AbstractGraph | None: + if graph_type not in GRAPHS.keys(): + return None + + return GRAPHS[graph_type](self.trace_info) + + def generate_visualisation(self, graph_type: str) -> str: + """ + Finalize the visualisation. Exports the graph to JSON if requested, and generates HTML if requested. + The boolean signals whether the output is in HTML format or not. + """ + graph: AbstractGraph = self._get_graph(graph_type) + if graph is None: + raise ValueError(f"Unknown graph type: {graph_type}") + + html_bokeh = networkvisualiser.NetworkVisualiser(graph, self.trace_info.model_name).generate_html() + return f'' diff --git a/utest/test_steparguments.py b/utest/test_steparguments.py index 6c906ca6..5c73903f 100644 --- a/utest/test_steparguments.py +++ b/utest/test_steparguments.py @@ -31,7 +31,7 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import unittest -from robotmbt.steparguments import StepArgument, StepArguments +from robotmbt.steparguments import StepArgument, StepArguments, ArgKind class TestStepArgument(unittest.TestCase): @@ -94,7 +94,7 @@ def test_is_default_property(self): self.assertFalse(arg2.is_default) def test_copies_are_the_same(self): - arg1 = StepArgument('foo', 7, kind=StepArgument.NAMED, is_default=True) + arg1 = StepArgument('foo', 7, kind=ArgKind.NAMED, is_default=True) arg2 = arg1.copy() self.assertEqual(arg1.arg, arg2.arg) self.assertEqual(arg1.value, arg2.value) @@ -107,7 +107,7 @@ def test_copies_are_the_same(self): self.assertEqual(arg2.arg, '${foo}') self.assertEqual(arg2.value, 8) self.assertEqual(arg2.org_value, 7) - self.assertEqual(arg2.kind, StepArgument.NAMED) + self.assertEqual(arg2.kind, ArgKind.NAMED) self.assertEqual(arg2.is_default, False) def test_original_value_is_kept_when_copying(self): @@ -118,11 +118,11 @@ def test_original_value_is_kept_when_copying(self): self.assertEqual(arg2.value, 8) def test_copies_are_independent(self): - arg1 = StepArgument('foo', 7, StepArgument.POSITIONAL) + arg1 = StepArgument('foo', 7, ArgKind.POSITIONAL) arg1.value = 8 arg2 = arg1.copy() arg2.value = 13 - arg2.kind = StepArgument.NAMED + arg2.kind = ArgKind.NAMED self.assertEqual(arg2.value, 13) self.assertEqual(arg1.value, 8) self.assertEqual(arg1.org_value, arg2.org_value) diff --git a/utest/test_suitedata.py b/utest/test_suitedata.py index ef6e3866..2623703a 100644 --- a/utest/test_suitedata.py +++ b/utest/test_suitedata.py @@ -33,6 +33,7 @@ import unittest from unittest.mock import patch +from enum import Enum, auto from types import SimpleNamespace from robotmbt.suitedata import Suite, Scenario, Step @@ -285,16 +286,25 @@ def test_copies_are_independent(self): def test_exteranally_determined_attributes_are_copied_along(self): self.scenario.src_id = 7 - class Dummy: + class SubstitutionMap: def copy(self): return 'dummy' - self.scenario.data_choices = Dummy() + self.scenario.data_choices = SubstitutionMap() dup = self.scenario.copy() self.assertEqual(dup.src_id, self.scenario.src_id) self.assertEqual(dup.data_choices, 'dummy') +class ArgKind(Enum): + EMBEDDED = auto() + POSITIONAL = auto() + VAR_POS = auto() + NAMED = auto() + FREE_NAMED = auto() + + @patch('robotmbt.suitedata.ArgumentValidator') +@patch('robotmbt.suitedata.ArgKind', new=ArgKind) class TestSteps(unittest.TestCase): def setUp(self): self.steps = self.create_steps() @@ -397,33 +407,37 @@ def test_return_value_multi_assignment_is_part_of_the_full_keyword_text(self, mo def test_argument_with_default_is_omitted_from_keyword_when_not_mentioned_named(self, mock): step = Step(RobotKwStub.STEPTEXT, 'posA', 'pos2=posB', parent=None) - step.args = StubStepArguments([StubArgument(name='pos1', value='posA', is_default=False, kind='POSITIONAL'), - StubArgument(name='pos2', value='posB', is_default=False, kind='NAMED'), - StubArgument(name='named1', value='namedA', is_default=True, kind='NAMED')]) + step.args = StubStepArguments( + [StubArgument(name='pos1', value='posA', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='pos2', value='posB', is_default=False, kind=ArgKind.NAMED), + StubArgument(name='named1', value='namedA', is_default=True, kind=ArgKind.NAMED)]) self.assertTupleEqual(step.posnom_args_str, ('posA', 'pos2=posB')) self.assertEqual(step.full_keyword, f"{RobotKwStub.STEPTEXT} posA pos2=posB") def test_argument_with_default_is_omitted_from_keyword_when_not_mentioned_positional(self, mock): step = Step(RobotKwStub.STEPTEXT, 'posA', 'posB', parent=None) - step.args = StubStepArguments([StubArgument(name='pos1', value='posA', is_default=False, kind='POSITIONAL'), - StubArgument(name='pos2', value='posB', is_default=False, kind='POSITIONAL'), - StubArgument(name='named1', value='namedA', is_default=True, kind='POSITIONAL')]) + step.args = StubStepArguments( + [StubArgument(name='pos1', value='posA', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='pos2', value='posB', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='named1', value='namedA', is_default=True, kind=ArgKind.POSITIONAL)]) self.assertTupleEqual(step.posnom_args_str, ('posA', 'posB')) self.assertEqual(step.full_keyword, f"{RobotKwStub.STEPTEXT} posA posB") def test_argument_with_default_is_included_in_keyword_when_mentioned_named(self, mock): step = Step(RobotKwStub.STEPTEXT, 'posA', 'pos2=posB', 'named1=namedA', parent=None) - step.args = StubStepArguments([StubArgument(name='pos1', value='posA', is_default=False, kind='POSITIONAL'), - StubArgument(name='pos2', value='posB', is_default=False, kind='NAMED'), - StubArgument(name='named1', value='namedA', is_default=False, kind='NAMED')]) + step.args = StubStepArguments( + [StubArgument(name='pos1', value='posA', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='pos2', value='posB', is_default=False, kind=ArgKind.NAMED), + StubArgument(name='named1', value='namedA', is_default=False, kind=ArgKind.NAMED)]) self.assertTupleEqual(step.posnom_args_str, ('posA', 'pos2=posB', 'named1=namedA')) self.assertEqual(step.full_keyword, f"{RobotKwStub.STEPTEXT} posA pos2=posB named1=namedA") def test_argument_with_default_is_included_in_keyword_when_mentioned_positional(self, mock): step = Step(RobotKwStub.STEPTEXT, 'posA', 'posB', 'namedA', parent=None) - step.args = StubStepArguments([StubArgument(name='pos1', value='posA', is_default=False, kind='POSITIONAL'), - StubArgument(name='pos2', value='posB', is_default=False, kind='POSITIONAL'), - StubArgument(name='named1', value='namedA', is_default=False, kind='POSITIONAL')]) + step.args = StubStepArguments( + [StubArgument(name='pos1', value='posA', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='pos2', value='posB', is_default=False, kind=ArgKind.POSITIONAL), + StubArgument(name='named1', value='namedA', is_default=False, kind=ArgKind.POSITIONAL)]) self.assertTupleEqual(step.posnom_args_str, ('posA', 'posB', 'namedA')) self.assertEqual(step.full_keyword, f"{RobotKwStub.STEPTEXT} posA posB namedA") @@ -482,11 +496,7 @@ class StubStepArguments(list): class StubArgument(SimpleNamespace): - EMBEDDED = 'EMBEDDED' - POSITIONAL = 'POSITIONAL' - VAR_POS = 'VAR_POS' - NAMED = 'NAMED' - FREE_NAMED = 'FREE_NAMED' + pass if __name__ == '__main__': diff --git a/utest/test_tracestate.py b/utest/test_tracestate.py index 77c1462a..d3f77ff1 100644 --- a/utest/test_tracestate.py +++ b/utest/test_tracestate.py @@ -46,14 +46,14 @@ def test_completing_single_size_trace(self): ts = TraceState([1]) self.assertEqual(ts.next_candidate(), 1) self.assertIs(ts.coverage_reached(), False) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertIs(ts.coverage_reached(), True) self.assertEqual(ts.get_trace(), ['one']) def test_confirming_excludes_scenario_from_candidacy(self): ts = TraceState([1]) self.assertEqual(ts.next_candidate(), 1) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertIs(ts.next_candidate(), None) def test_trying_excludes_scenario_from_candidacy(self): @@ -65,7 +65,7 @@ def test_trying_excludes_scenario_from_candidacy(self): def test_scenario_still_excluded_from_candidacy_after_rewind(self): ts = TraceState([1]) self.assertEqual(ts.next_candidate(), 1) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) ts.rewind() self.assertIs(ts.next_candidate(), None) @@ -74,7 +74,7 @@ def test_candidates_come_in_order_when_accepted(self): candidates = [] for _ in range(3): candidates.append(ts.next_candidate()) - ts.confirm_full_scenario(candidates[-1], 'scenario', {}) + ts.confirm_full_scenario(candidates[-1], ScenarioStub(), ModelStub()) candidates.append(ts.next_candidate()) self.assertEqual(candidates, [10, 20, 30, None]) @@ -96,30 +96,30 @@ def test_rejected_scenarios_are_candidates_for_new_positions(self): ts.reject_scenario(1) for _ in range(3): candidates.append(ts.next_candidate()) - ts.confirm_full_scenario(candidates[-1], 'scenario', {}) + ts.confirm_full_scenario(candidates[-1], ScenarioStub(), ModelStub()) candidates.append(ts.next_candidate()) self.assertEqual(candidates, [2, 1, 3, None]) def test_previously_confirmed_scenarios_can_be_retried_if_no_new_candidates_exist(self): ts = TraceState(range(3)) first_candidate = ts.next_candidate(retry=True) - ts.confirm_full_scenario(first_candidate, 'one', {}) + ts.confirm_full_scenario(first_candidate, ScenarioStub('one'), ModelStub()) ts.reject_scenario(ts.next_candidate(retry=True)) ts.reject_scenario(ts.next_candidate(retry=True)) retry_candidate = ts.next_candidate(retry=True) self.assertEqual(first_candidate, retry_candidate) - ts.confirm_full_scenario(retry_candidate, 'one', {}) - ts.confirm_full_scenario(ts.next_candidate(retry=True), 'two', {}) + ts.confirm_full_scenario(retry_candidate, ScenarioStub('one'), ModelStub()) + ts.confirm_full_scenario(ts.next_candidate(retry=True), ScenarioStub('two'), ModelStub()) self.assertFalse(ts.coverage_reached()) - ts.confirm_full_scenario(ts.next_candidate(retry=True), 'three', {}) + ts.confirm_full_scenario(ts.next_candidate(retry=True), ScenarioStub('three'), ModelStub()) self.assertTrue(ts.coverage_reached()) self.assertEqual(ts.get_trace(), ['one', 'one', 'two', 'three']) def test_retry_can_continue_once_coverage_is_reached(self): ts = TraceState([1, 2, 3]) - ts.confirm_full_scenario(ts.next_candidate(retry=True), 'one', {}) - ts.confirm_full_scenario(ts.next_candidate(retry=True), 'two', {}) - ts.confirm_full_scenario(ts.next_candidate(retry=True), 'three', {}) + ts.confirm_full_scenario(ts.next_candidate(retry=True), ScenarioStub('one'), ModelStub()) + ts.confirm_full_scenario(ts.next_candidate(retry=True), ScenarioStub('two'), ModelStub()) + ts.confirm_full_scenario(ts.next_candidate(retry=True), ScenarioStub('three'), ModelStub()) self.assertTrue(ts.coverage_reached()) self.assertEqual(ts.next_candidate(retry=True), 1) ts.reject_scenario(1) @@ -130,14 +130,14 @@ def test_count_scenario_repetitions(self): ts = TraceState([1, 2]) first = ts.next_candidate() self.assertEqual(ts.count(first), 0) - ts.confirm_full_scenario(first, 'one', {}) + ts.confirm_full_scenario(first, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.count(first), 1) - ts.confirm_full_scenario(first, 'one', {}) + ts.confirm_full_scenario(first, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.count(first), 2) def test_rewind_single_available_scenario(self): ts = TraceState([1]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertIs(ts.coverage_reached(), True) self.assertIs(ts.can_rewind(), True) ts.rewind() @@ -148,21 +148,21 @@ def test_rewind_single_available_scenario(self): def test_rewind_returns_none_after_rewinding_last_step(self): ts = TraceState([1]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertIs(ts.rewind(), None) def test_traces_can_have_multiple_scenarios(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'foo', dict(a=1)) + ts.confirm_full_scenario(1, ScenarioStub('foo'), ModelStub(a=1)) self.assertIs(ts.coverage_reached(), False) - ts.confirm_full_scenario(2, 'bar', dict(b=2)) + ts.confirm_full_scenario(2, ScenarioStub('bar'), ModelStub(b=2)) self.assertIs(ts.coverage_reached(), True) self.assertEqual(ts.get_trace(), ['foo', 'bar']) def test_rewind_returns_snapshot_of_the_step_before(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'foo', dict(a=1)) - ts.confirm_full_scenario(2, 'bar', dict(b=2)) + ts.confirm_full_scenario(1, ScenarioStub('foo'), ModelStub(a=1)) + ts.confirm_full_scenario(2, ScenarioStub('bar'), ModelStub(b=2)) tail = ts.rewind() self.assertEqual(tail.id, '1') self.assertEqual(tail.scenario, 'foo') @@ -170,32 +170,32 @@ def test_rewind_returns_snapshot_of_the_step_before(self): def test_completing_size_three_trace(self): ts = TraceState(range(3)) - ts.confirm_full_scenario(ts.next_candidate(), 'one', {}) - ts.confirm_full_scenario(ts.next_candidate(), 'two', {}) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('one'), ModelStub()) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('two'), ModelStub()) self.assertIs(ts.coverage_reached(), False) - ts.confirm_full_scenario(ts.next_candidate(), 'three', {}) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('three'), ModelStub()) self.assertIs(ts.coverage_reached(), True) self.assertEqual(ts.get_trace(), ['one', 'two', 'three']) def test_completing_size_three_trace_after_reject(self): ts = TraceState(range(3)) first = ts.next_candidate() - ts.confirm_full_scenario(first, first, {}) + ts.confirm_full_scenario(first, ScenarioStub('one'), ModelStub()) rejected = ts.next_candidate() ts.reject_scenario(rejected) third = ts.next_candidate() - ts.confirm_full_scenario(third, third, {}) + ts.confirm_full_scenario(third, ScenarioStub('three'), ModelStub()) self.assertIs(ts.coverage_reached(), False) second = ts.next_candidate() self.assertEqual(rejected, second) - ts.confirm_full_scenario(second, second, {}) + ts.confirm_full_scenario(second, ScenarioStub('two'), ModelStub()) self.assertIs(ts.coverage_reached(), True) - self.assertEqual(ts.get_trace(), [first, third, second]) + self.assertEqual(ts.get_trace(), ['one', 'three', 'two']) def test_completing_size_three_trace_after_rewind(self): ts = TraceState(range(3)) first = ts.next_candidate() - ts.confirm_full_scenario(first, first, {}) + ts.confirm_full_scenario(first, ScenarioStub('one'), ModelStub()) reject2 = ts.next_candidate() ts.reject_scenario(reject2) reject3 = ts.next_candidate() @@ -205,13 +205,13 @@ def test_completing_size_three_trace_after_rewind(self): self.assertEqual(len(ts.get_trace()), 0) retry_first = ts.next_candidate() self.assertNotEqual(first, retry_first) - ts.confirm_full_scenario(retry_first, retry_first, {}) + ts.confirm_full_scenario(retry_first, ScenarioStub('two'), ModelStub()) retry_second = ts.next_candidate() - ts.confirm_full_scenario(retry_second, retry_second, {}) + ts.confirm_full_scenario(retry_second, ScenarioStub('one'), ModelStub()) retry_third = ts.next_candidate() - ts.confirm_full_scenario(retry_third, retry_third, {}) + ts.confirm_full_scenario(retry_third, ScenarioStub('three'), ModelStub()) self.assertIs(ts.coverage_reached(), True) - self.assertEqual(ts.get_trace(), [retry_first, retry_second, retry_third]) + self.assertEqual(ts.get_trace(), ['two', 'one', 'three']) def test_highest_part_when_index_not_present(self): ts = TraceState([1]) @@ -219,13 +219,13 @@ def test_highest_part_when_index_not_present(self): def test_highest_part_for_non_partial_sceanrio(self): ts = TraceState([1]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub(), ModelStub()) self.assertEqual(ts.highest_part(1), 0) def test_model_property_takes_model_from_tail(self): ts = TraceState(range(2)) - ts.confirm_full_scenario(ts.next_candidate(), 'one', dict(a=1)) - ts.confirm_full_scenario(ts.next_candidate(), 'two', dict(b=2)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('one'), ModelStub(a=1)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('two'), ModelStub(b=2)) self.assertEqual(ts.model, dict(b=2)) ts.rewind() self.assertEqual(ts.model, dict(a=1)) @@ -233,7 +233,7 @@ def test_model_property_takes_model_from_tail(self): def test_no_model_from_empty_trace(self): ts = TraceState([1]) self.assertIs(ts.model, None) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertIsNotNone(ts.model) ts.rewind() self.assertIs(ts.model, None) @@ -249,16 +249,16 @@ def test_rejected_scenarios_are_tried(self): def test_confirmed_scenario_is_tried_and_triggers_next_step(self): ts = TraceState([1]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.tried, ()) ts.rewind() self.assertEqual(ts.tried, (1,)) def test_can_iterate_over_tracestate_snapshots(self): ts = TraceState([1, 2, 3]) - ts.confirm_full_scenario(ts.next_candidate(), 'one', dict(a=1)) - ts.confirm_full_scenario(ts.next_candidate(), 'two', dict(b=2)) - ts.confirm_full_scenario(ts.next_candidate(), 'three', dict(c=3)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('one'), ModelStub(a=1)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('two'), ModelStub(b=2)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('three'), ModelStub(c=3)) for act, exp in zip(ts, ['1', '2', '3']): self.assertEqual(act.id, exp) for act, exp in zip(ts, ['one', 'two', 'three']): @@ -268,9 +268,9 @@ def test_can_iterate_over_tracestate_snapshots(self): def test_can_index_tracestate_snapshots(self): ts = TraceState([1, 2, 3]) - ts.confirm_full_scenario(ts.next_candidate(), 'one', dict(a=1)) - ts.confirm_full_scenario(ts.next_candidate(), 'two', dict(b=2)) - ts.confirm_full_scenario(ts.next_candidate(), 'three', dict(c=3)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('one'), ModelStub(a=1)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('two'), ModelStub(b=2)) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('three'), ModelStub(c=3)) self.assertEqual(ts[0].id, '1') self.assertEqual(ts[1].scenario, 'two') self.assertEqual(ts[2].model, dict(c=3)) @@ -279,38 +279,38 @@ def test_can_index_tracestate_snapshots(self): def test_adding_coverage_prevents_drought(self): ts = TraceState(range(3)) - ts.confirm_full_scenario(ts.next_candidate(), 'one', {}) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(ts.next_candidate(), 'two', {}) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('two'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(ts.next_candidate(), 'three', {}) + ts.confirm_full_scenario(ts.next_candidate(), ScenarioStub('three'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) def test_repeated_scenarios_increases_drought(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 1) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 2) def test_drought_is_reset_with_new_coverage(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 1) - ts.confirm_full_scenario(2, 'two', {}) + ts.confirm_full_scenario(2, ScenarioStub('two'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) def test_rewind_includes_drought_update(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub()) self.assertEqual(ts.coverage_drought, 1) - ts.confirm_full_scenario(2, 'two', {}) + ts.confirm_full_scenario(2, ScenarioStub('two'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) ts.rewind() self.assertEqual(ts.coverage_drought, 1) @@ -321,29 +321,29 @@ def test_rewind_includes_drought_update(self): class TestPartialScenarios(unittest.TestCase): def test_push_partial_does_not_complete_coverage(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertEqual(ts.get_trace(), ['part1']) self.assertIs(ts.coverage_reached(), False) def test_confirm_full_after_push_partial_completes_coverage(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertIs(ts.coverage_reached(), False) - ts.push_partial_scenario(1, 'part2', {}) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) self.assertIs(ts.coverage_reached(), False) - ts.confirm_full_scenario(1, 'remainder', {}) + ts.confirm_full_scenario(1, ScenarioStub('remainder'), ModelStub()) self.assertEqual(ts.get_trace(), ['part1', 'part2', 'remainder']) self.assertIs(ts.coverage_reached(), True) def test_scenario_unavailble_once_pushed_partial(self): ts = TraceState([1]) candidate = ts.next_candidate() - ts.push_partial_scenario(candidate, 'part1', {}) + ts.push_partial_scenario(candidate, ScenarioStub('part1'), ModelStub()) self.assertIs(ts.next_candidate(), None) def test_rewind_of_single_part(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertEqual(ts.get_trace(), ['part1']) self.assertIs(ts.can_rewind(), True) ts.rewind() @@ -351,9 +351,9 @@ def test_rewind_of_single_part(self): def test_rewind_all_parts(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertIs(ts.coverage_reached(), False) - ts.push_partial_scenario(1, 'part2', {}) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) self.assertIs(ts.coverage_reached(), False) self.assertEqual(ts.get_trace(), ['part1', 'part2']) self.assertIs(ts.next_candidate(), None) @@ -368,14 +368,14 @@ def test_rewind_all_parts(self): def test_partial_scenario_still_excluded_from_candidacy_after_rewind(self): ts = TraceState([1]) self.assertEqual(ts.next_candidate(), 1) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) ts.rewind() self.assertIs(ts.next_candidate(), None) def test_rewind_to_partial_scenario(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', dict(a=1)) - ts.push_partial_scenario(1, 'part2', dict(b=2)) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub(a=1)) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub(b=2)) snapshot = ts.rewind() self.assertEqual(snapshot.id, '1.1') self.assertEqual(snapshot.scenario, 'part1') @@ -383,8 +383,8 @@ def test_rewind_to_partial_scenario(self): def test_rewind_last_part(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'one', dict(a=1)) - ts.push_partial_scenario(2, 'part1', dict(b=2)) + ts.confirm_full_scenario(1, ScenarioStub('one'), ModelStub(a=1)) + ts.push_partial_scenario(2, ScenarioStub('part1'), ModelStub(b=2)) snapshot = ts.rewind() self.assertEqual(ts.get_trace(), ['one']) self.assertEqual(snapshot.id, '1') @@ -393,9 +393,9 @@ def test_rewind_last_part(self): def test_rewind_all_parts_of_completed_scenario_at_once(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', dict(a=1)) - ts.push_partial_scenario(1, 'part2', dict(b=2)) - ts.confirm_full_scenario(1, 'remainder', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub(a=1)) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub(b=2)) + ts.confirm_full_scenario(1, ScenarioStub('remainder'), ModelStub()) tail = ts.rewind() self.assertEqual(ts.get_trace(), []) self.assertIs(ts.next_candidate(), None) @@ -403,11 +403,11 @@ def test_rewind_all_parts_of_completed_scenario_at_once(self): def test_tried_entries_after_rewind(self): ts = TraceState([1, 2, 10, 11, 12, 20, 21]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) ts.reject_scenario(10) ts.reject_scenario(11) - ts.confirm_full_scenario(2, 'two', {}) - ts.push_partial_scenario(1, 'part2', {}) + ts.confirm_full_scenario(2, ScenarioStub('two'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) ts.reject_scenario(20) ts.reject_scenario(21) self.assertEqual(ts.tried, (20, 21)) @@ -422,29 +422,29 @@ def test_tried_entries_after_rewind(self): def test_highest_part_after_first_part(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertEqual(ts[-1].id, '1.1') self.assertEqual(ts.highest_part(1), 1) def test_highest_part_after_multiple_parts(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) - ts.push_partial_scenario(1, 'part2', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) self.assertEqual(ts[-1].id, '1.2') self.assertEqual(ts.highest_part(1), 2) def test_highest_part_after_completing_multiple_parts(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) - ts.push_partial_scenario(1, 'part2', {}) - ts.confirm_full_scenario(1, 'remainder', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) + ts.confirm_full_scenario(1, ScenarioStub('remainder'), ModelStub()) self.assertEqual(ts[-1].id, '1.0') self.assertEqual(ts.highest_part(1), 0) def test_highest_part_after_partial_rewind(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) - ts.push_partial_scenario(1, 'part2', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) self.assertEqual(ts.highest_part(1), 2) ts.rewind() self.assertEqual(ts.highest_part(1), 1) @@ -454,9 +454,9 @@ def test_highest_part_after_partial_rewind(self): def test_highest_part_is_0_when_no_refinement_is_ongoing(self): ts = TraceState([1]) self.assertEqual(ts.highest_part(1), 0) - ts.push_partial_scenario(1, 'part1', {}) - ts.push_partial_scenario(1, 'part2', {}) - ts.confirm_full_scenario(1, 'remainder', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub()) + ts.confirm_full_scenario(1, ScenarioStub('remainder'), ModelStub()) self.assertEqual(ts.highest_part(1), 0) ts.rewind() self.assertEqual(ts.highest_part(1), 0) @@ -465,36 +465,36 @@ def test_count_scenario_repetitions_with_partials(self): ts = TraceState(range(2)) first = ts.next_candidate() self.assertEqual(ts.count(first), 0) - ts.confirm_full_scenario(first, 'full', {}) + ts.confirm_full_scenario(first, ScenarioStub('full'), ModelStub()) self.assertEqual(ts.count(first), 1) - ts.push_partial_scenario(first, 'part1', {}) - ts.push_partial_scenario(first, 'part2', {}) + ts.push_partial_scenario(first, ScenarioStub('part1'), ModelStub()) + ts.push_partial_scenario(first, ScenarioStub('part2'), ModelStub()) self.assertEqual(ts.count(first), 1) - ts.confirm_full_scenario(first, 'remainder', {}) + ts.confirm_full_scenario(first, ScenarioStub('remainder'), ModelStub()) self.assertEqual(ts.count(first), 2) second = ts.next_candidate() - ts.push_partial_scenario(second, 'part1', {}) + ts.push_partial_scenario(second, ScenarioStub('part1'), ModelStub()) self.assertEqual(ts.count(second), 0) - ts.push_partial_scenario(second, 'part2', {}) - ts.confirm_full_scenario(second, 'remainder', {}) + ts.push_partial_scenario(second, ScenarioStub('part2'), ModelStub()) + ts.confirm_full_scenario(second, ScenarioStub('remainder'), ModelStub()) self.assertEqual(ts.count(second), 1) def test_partial_scenario_is_tried_without_finishing(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', {}) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub()) self.assertEqual(ts.tried, ()) ts.rewind() self.assertEqual(ts.tried, (1,)) def test_get_last_snapshot_by_index(self): ts = TraceState([1]) - ts.push_partial_scenario(1, 'part1', dict(a=1)) + ts.push_partial_scenario(1, ScenarioStub('part1'), ModelStub(a=1)) self.assertEqual(ts[-1].id, '1.1') self.assertEqual(ts[-1].scenario, 'part1') self.assertEqual(ts[-1].model, dict(a=1)) self.assertEqual(ts[-1].coverage_drought, 0) - ts.push_partial_scenario(1, 'part2', dict(b=2)) - ts.confirm_full_scenario(1, 'remainder', dict(c=3)) + ts.push_partial_scenario(1, ScenarioStub('part2'), ModelStub(b=2)) + ts.confirm_full_scenario(1, ScenarioStub('remainder'), ModelStub(c=3)) self.assertEqual(ts[-1].id, '1.0') self.assertEqual(ts[-1].scenario, 'remainder') self.assertEqual(ts[-1].model, dict(c=3)) @@ -502,16 +502,48 @@ def test_get_last_snapshot_by_index(self): def test_only_completed_scenarios_affect_drought(self): ts = TraceState([1, 2]) - ts.confirm_full_scenario(1, 'one full', {}) - ts.push_partial_scenario(1, 'one part1', {}) + ts.confirm_full_scenario(1, ScenarioStub('one full'), ModelStub()) + ts.push_partial_scenario(1, ScenarioStub('one part1'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) - ts.confirm_full_scenario(1, 'one remainder', {}) + ts.confirm_full_scenario(1, ScenarioStub('one remainder'), ModelStub()) self.assertEqual(ts.coverage_drought, 1) - ts.push_partial_scenario(2, 'two part1', {}) + ts.push_partial_scenario(2, ScenarioStub('two part1'), ModelStub()) self.assertEqual(ts.coverage_drought, 1) - ts.confirm_full_scenario(2, 'two remainder', {}) + ts.confirm_full_scenario(2, ScenarioStub('two remainder'), ModelStub()) self.assertEqual(ts.coverage_drought, 0) + def test_tracestates_can_be_copied(self): + ts = TraceState([1, 2, 3]) + ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(2, 'two', {}) + cp = ts.copy() + self.assertEqual(ts.c_pool, cp.c_pool) + self.assertEqual(ts.tried, cp.tried) + self.assertEqual(ts.active_refinements, cp.active_refinements) + self.assertEqual(ts[-1], cp[-1]) + + def test_tracestate_copies_are_independent(self): + ts = TraceState([1, 2, 3]) + ts.confirm_full_scenario(1, 'one', {}) + ts.confirm_full_scenario(2, 'two', {}) + cp = ts.copy() + cp.push_partial_scenario(3, 'three', {}) + self.assertNotEqual(ts.active_refinements, cp.active_refinements) + cp.confirm_full_scenario(3, 'two', {}) + self.assertEqual(len(cp), len(ts)+2) + self.assertNotEqual(ts.c_pool, cp.c_pool) + cp.rewind() + self.assertIn(3, cp.tried) + self.assertNotIn(3, ts.tried) + + +class ScenarioStub(str): + """Stub for suitedata.Scenario""" + + +class ModelStub(dict): + """Stub for modelspace.ModelSpace""" + if __name__ == '__main__': unittest.main() diff --git a/utest/test_visualise_models.py b/utest/test_visualise_models.py new file mode 100644 index 00000000..1a09f0b1 --- /dev/null +++ b/utest/test_visualise_models.py @@ -0,0 +1,197 @@ +# BSD 3-Clause License +# +# Copyright (c) 2026, T.B. Dubbeling, J. Foederer, T.S. Kas, D.R. Osinga, D.F. Serra e Silva, J.C. Willegers +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this +# list of conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, +# this list of conditions and the following disclaimer in the documentation +# and/or other materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +from types import SimpleNamespace +import unittest + +try: + from robotmbt.visualise.models import ScenarioInfo, StateInfo, TraceInfo + + VISUALISE = True +except ImportError: + VISUALISE = False + +if VISUALISE: + class TestScenarioInfo(unittest.TestCase): + def test_scenarioInfo_constructor(self): + scenariostub = SimpleNamespace(name='test', src_id=0) + si = ScenarioInfo(scenariostub) + self.assertEqual(si.name, 'test') + self.assertEqual(si.src_id, 0) + + def test_split_name_empty_string(self): + result = ScenarioInfo._split_name("") + self.assertEqual(result, "") + self.assertNotIn('\n', result) + + def test_split_name_single_short_word(self): + result = ScenarioInfo._split_name("Hello") + self.assertEqual(result, "Hello") + self.assertNotIn('\n', result) + + def test_split_name_single_exact_length_word(self): + exact_20 = "abcdefghijklmnopqrst" + result = ScenarioInfo._split_name(exact_20) + self.assertEqual(result, exact_20) + self.assertNotIn('\n', result) + + def test_split_name_single_long_word(self): + name = "ThisIsAReallyLongNameWithoutAnySpacesAtAll" + result = ScenarioInfo._split_name(name) + self.assertEqual(result, name) + self.assertNotIn('\n', result) + + def test_split_name_two_words_short(self): + result = ScenarioInfo._split_name("Hello World") + self.assertEqual(result, "Hello World") + self.assertNotIn('\n', result) + + def test_split_name_two_words_exceeds_limit(self): + name = "Supercalifragilistic Hello" + result = ScenarioInfo._split_name(name) + + self.assertEqual(result.replace('\n', ' '), name) + self.assertIn('\n', result) + + def test_split_name_multiple_words_need_split(self): + name = "This is a very long scenario name that should be split" + result = ScenarioInfo._split_name(name) + + self.assertEqual(result.replace('\n', ' '), name) + self.assertIn('\n', result) + self.assertLessEqual(max([len(line) for line in result.split('\n')]), 20) + + class TestStateInfo(unittest.TestCase): + def test_stateInfo_empty(self): + modelspacestub = SimpleNamespace(ref_id=None, props={}) + s = StateInfo(modelspacestub) + self.assertEqual(str(s), '') + + def test_stateInfo_prop_empty(self): + modelspacestub = SimpleNamespace(ref_id=None, props={}) + s = StateInfo(modelspacestub) + self.assertEqual(str(s), '') + + def test_stateInfo_prop_val(self): + modelspacestub = SimpleNamespace(ref_id=None, props=dict(prop1=SimpleNamespace(value=1))) + s = StateInfo(modelspacestub) + self.assertTrue('prop1:' in str(s)) + self.assertTrue('value=1' in str(s)) + + def test_stateInfo_prop_val_empty(self): + class EmptyProp: + def __dir__(self): + return {} + + modelspacestub = SimpleNamespace(ref_id=None, props=dict(prop1=SimpleNamespace(value=1), + prop2=EmptyProp())) + s = StateInfo(modelspacestub) + self.assertTrue('prop1:' in str(s)) + self.assertTrue('value=1' in str(s)) + self.assertFalse('prop2:' in str(s)) + + class TestTraceInfo(unittest.TestCase): + def test_trace_info_update_normal(self): + info = TraceInfo() + scenariostub = SimpleNamespace(name='test', src_id=0) + + self.assertEqual(len(info.current_trace), 0) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 1)]), 1) + + self.assertEqual(len(info.current_trace), 1) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 2)]), 2) + + self.assertEqual(len(info.current_trace), 2) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 3)]), 3) + + self.assertEqual(len(info.current_trace), 3) + self.assertEqual(len(info.all_traces), 0) + + def test_trace_info_update_backtrack(self): + info = TraceInfo() + scenariostub = SimpleNamespace(name='test', src_id=0) + + self.assertEqual(len(info.current_trace), 0) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 1)]), 1) + + self.assertEqual(len(info.current_trace), 1) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 2)]), 2) + + self.assertEqual(len(info.current_trace), 2) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 3)]), 3) + + self.assertEqual(len(info.current_trace), 3) + self.assertEqual(len(info.all_traces), 0) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 2)]), 2) + + self.assertEqual(len(info.current_trace), 2) + self.assertEqual(len(info.all_traces), 1) + self.assertEqual(len(info.all_traces[0]), 3) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 1)]), 1) + + self.assertEqual(len(info.current_trace), 1) + self.assertEqual(len(info.all_traces), 1) + self.assertEqual(len(info.all_traces[0]), 3) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 4)]), 2) + + self.assertEqual(len(info.current_trace), 2) + self.assertEqual(len(info.all_traces), 1) + self.assertEqual(len(info.all_traces[0]), 3) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 5)]), 3) + + self.assertEqual(len(info.current_trace), 3) + self.assertEqual(len(info.all_traces), 1) + self.assertEqual(len(info.all_traces[0]), 3) + + info.update_trace(ScenarioInfo(scenariostub), StateInfo._create_state_with_prop('prop', [('value', 6)]), 4) + + self.assertEqual(len(info.current_trace), 4) + self.assertEqual(len(info.all_traces), 1) + self.assertEqual(len(info.all_traces[0]), 3) + + +if __name__ == '__main__': + unittest.main()